Groundfact — S0 Feasibility Spike¶
Date: 2026-08-08 · Status: measured, not estimated · Supersedes: spec §5.1 phase table, §13 Phase-1 acceptance criteria
Every number below was measured live against the real APIs on 2026-08-08. Probe scripts are throwaway (scratchpad, not committed). This document exists because the spec and the three research reports assert coverage and sizing figures that nobody had checked.
1. Headline findings¶
- The database is ~200 MB globally, not multi-GB. The entire World Bank WDI pull for 14 comparator geos across 66 years is 2.6 MB in DuckDB. This invalidates the disk-pressure reasoning behind the plan's Amendment C, and Amendment C is withdrawn — see §5.
- Only 411 of 1,498 WDI series carry any Nepal value (27%). The spec treats World Bank as the backbone; it cannot carry the ≥1,500-series Phase-1 target alone.
- ADB KIDB's API is gone. Every documented path 404s. Drop it from Phase 1.
- FAOSTAT's REST API is down (HTTP 521); the bulk path works. Build the connector bulk-first.
- UNESCO UIS is live and rich (5,063 indicators, 20,034 Nepal records) — doc 1 flagged it unverified and deferred it to "Phase 1.5". Promote it to Phase 1; it partly replaces the WHO GHO capacity dropped for licensing.
- Nepal's own NSO portal is HTTP-only with no API, and the census site has a broken TLS chain. Phase-3 extraction is harder than spec §5.1 rows 13–15 assume.
2. Endpoint verification¶
| Source | Endpoint tested | Result | Action |
|---|---|---|---|
| World Bank v2 | api.worldbank.org/v2/country/NPL/indicator/... |
200, 1,498 WDI series | Use as spec'd |
| WB Data360 | data360api.worldbank.org/data360/data |
200, returns SDMX-shaped obs | Works today; still beta — keep as flex point |
| UN SDG | unstats.un.org/sdgapi/v1/... |
200, 713 series catalogue | Use as spec'd |
| FAOSTAT bulk | bulks-faostat.fao.org/production/datasets_E.json |
200, 69 datasets | Primary path |
| FAOSTAT FENIX API | fenixservices.fao.org/faostat/api/v1/... |
521 (origin down, repeated) | Do not depend on |
| ILOSTAT TOC | rplumber.ilo.org/metadata/toc/indicator/ |
200, 1,956 indicators | Use (note trailing /indicator/) |
| ILOSTAT SDMX | sdmx.ilo.org/rest/dataflow/ILO |
200, 7.3 MB dataflow list | Use |
| IMF SDMX 3.0 | api.imf.org/external/sdmx/3.0/structure/dataflow/IMF.STA |
200, 331 KB | Use as spec'd |
| IMF legacy | dataservices.imf.org/REST/SDMX_JSON.svc/... |
ConnectionError | Confirmed dead, as doc 1 warned |
| OWID grapher | ourworldindata.org/grapher/{slug}.csv + .metadata.json |
200 | Use; no index endpoint — needs owid-catalog |
| DHS | api.dhsprogram.com/rest/dhs/{data,indicators,surveys} |
200 | Use as spec'd |
| ADB KIDB | kidb.adb.org/api/v3/sdmx/... (+7 path variants) |
404 — SPA shell on every path; sdmx.adb.org does not resolve |
Drop from Phase 1 |
| ADB Data Library | data.adb.org/api/3/action/... |
403 Cloudflare bot wall | Drop / manual |
| HDX CKAN | data.humdata.org/api/3/action/package_search?fq=groups:npl |
200, 345 Nepal datasets | Use |
| HDX HAPI | hapi.humdata.org/api/v2/... |
403 without ident; 200 with base64 app_identifier |
Use; generate ident |
| UNESCO UIS | api.uis.unesco.org/api/public/data/indicators |
200, 20,034 Nepal records | Promote to Phase 1 |
| UNICEF SDMX | sdmx.data.unicef.org/ws/public/sdmxapi/rest/dataflow/UNICEF |
200 | Viable |
| NRB forex | nrb.org.np/api/forex/v1/rates |
200, 586 days, 22 currencies/day | Confirmed clean |
| Open Data Nepal | api.opendatanepal.com/api/3/action/... |
200 CKAN | Viable Phase 3 |
| NSO portal | data.nsonepal.gov.np |
443 refused, port 80 open, /api/3/action/* → 404 |
HTTP-only, not CKAN — scrape |
| Census Nepal | censusnepal.cbs.gov.np |
TLS 1.3 but cert chain unverifiable; 18 B over https | Needs explicit trust handling |
3. Measured Nepal coverage¶
| Source | Measurement | Value |
|---|---|---|
| World Bank WDI | series in WDI | 1,498 |
| World Bank WDI | series with ≥1 Nepal value | 411 (27.4%) |
| UN SDG | catalogue series | 713 |
| UN SDG | series available for Nepal | 523 (17 goals / 169 targets / 251 indicators) |
| DHS | indicators for Nepal | 3,555 |
| DHS | national data points | 15,977 |
| DHS | surveys | 6 (1996, 2001, 2006, 2011, 2016, 2022) |
| ILOSTAT | indicators in TOC | 1,956 (Nepal ref_area present, 1961–2030) |
| UNESCO UIS | indicator catalogue | 5,063 |
| UNESCO UIS | Nepal records | 20,034 |
| FAOSTAT | bulk datasets | 69 |
| HDX | Nepal-tagged datasets | 345 |
| NRB forex | days available | 586 (~19 months), 22 currencies/day |
Verdict on the §13 target. "≥1,500 Nepal-relevant series" is achievable, but not from the World Bank — WB contributes 411. The target depends on SDG (523), ILOSTAT, UIS and DHS. Keep the 1,500 figure; change the assumption about where it comes from, and sequence UIS earlier.
4. Measured size — the finding that changes the plan¶
Full WB WDI pull, all 411 Nepal-covered series × 14 geos (NPL IND BGD PAK LKA BTN MDV AFG CHN VNM
KHM GBR WLD SAS) × 1960–2025:
| Metric | Value |
|---|---|
| Non-null observations | 198,969 |
| Parquet (zstd) | 1.22 MB (6.1 bytes/obs) |
| DuckDB table | 2.63 MB (13.2 bytes/obs) |
Extrapolated: WB WDI at global scope (217 economies + ~48 aggregates ≈ 15×) is ~40 MB. All Phase-1 sources together, assuming they total 4× WB's volume, is ~160 MB.
The spec's [FLEX] MotherDuck 10 GB threshold and the plan's 4 GB current.db budget are both
off by roughly two orders of magnitude.
5. Revisions to the approved plan¶
Amendment C is withdrawn. It restricted ingestion to the comparator set to protect a disk budget that does not exist. Ingest globally from day one wherever a source supports it — it costs tens of megabytes and it makes "compare Nepal to any country" work without a re-ingest. This is a straight improvement to the demo.
Amendment A stands, but my stated reasoning was partly wrong. Disk on twakka01 was one of my
arguments and that argument is dead. The build still belongs off-box for two intact reasons:
FAOSTAT bulk ingestion churns 50–70 MB zips across 69 datasets, and the embeddings step pulls
sentence-transformers/torch (~2 GB of dependencies, CPU-heavy) onto a box already running ~19
containers. Shipping a ~200 MB .db is trivially cheap — which makes the off-box build easier
than planned, not harder.
Amendments B and D stand. B (drop CC BY-NC sources) is unaffected. D is now discharged: the numbers are measured and recorded above.
New: drop ADB from Phase 1 (spec §5.1 row 9, plan session S3). Its API no longer exists.
New: promote UNESCO UIS from "Phase 1.5" to Phase 1 (spec §5.1 row 11). It is live, rich, and back-fills education/health capacity lost by dropping WHO GHO.
New: FAOSTAT connector is bulk-first, not FENIX-API-first (spec §5.1 row 4).
Revised S3 connector list: unsdg, faostat (bulk), ilostat, imf, dhs, hdx, uis
— dropping adb and who, adding uis.
6. Still unverified¶
- IMF: dataflow structure confirmed live; an actual Nepal observation pull was not run.
- OWID: no index endpoint exists; volume depends on
owid-catalog, not yet installed. - ILOSTAT Nepal series count: ref_area confirmed present, per-series coverage not counted.
- Competitor probe: Open Data Nepal's "Ask AI" was confirmed reachable but not exercised. Still owed before homepage positioning is written (plan S11).
- Census 2021 / NSO extraction: reachability diagnosed, no table extraction attempted.