Commit Graph

12 Commits

Author SHA1 Message Date
Fabio 5a22e7dcbb Cities: 19,518 municipal Jurisdiction nodes (generated tree)
Regenerated data/jurisdictions with the municipal layer: 10,253 cities,
4,331 towns, 3,716 villages, 1,218 boroughs. Jurisdiction records
10,390 -> 29,908.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 10:32:56 -04:00
Fabio 00a184bb3c Legal corpus: U.S. Code titles 1–11 from pinned OLRC XML (11,050 sections)
Raw OLRC USLM XML zips @ release 119-100 (retrieved 2026-07-04 via
Atlas depot), ingested with the standard pipeline: raw snapshot ->
per-section OKF markdown -> manifest + checksums. Title 52 untouched.
LegalText: 171 -> 11,221. Titles 12-54 await a clean OLRC retry.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 09:52:37 -04:00
Fabio 1beeaab2db Graph: national county↔district↔legislator authority layer
The first edges in the repo. Every US county now links to the districts it
sits in, weighted by area overlap, and every legislator links to the district
they represent — county → district → representative, traversable.

Data (PostGIS, no Atlas dependency — computed directly from geometry):
- data/county_district_edges.jsonl — 16,328 weighted edges. area_weight =
  ST_Area(ST_Intersection(county, district))/ST_Area(county) over ST_MakeValid
  Census TIGER 2024 geometries, overlaps <0.5% dropped. Uses area-intersection,
  NOT centroid-in-polygon, so urban districts that carve through counties
  (e.g. GA CD-5 / Atlanta across Clayton+DeKalb+Fulton) are captured, not
  silently dropped. Per-county weights sum to ~1.0 per district type.

build.py:
- ~6,896 new district nodes: 435 CD (ACS demographics where available; sparse
  for the 8 states missing from acs_cd) + 4,927 state-house + 1,897 state-senate
  (geometry-only). GEOID/number-keyed, stable.
- county index.md gains a `districts:` block (frontmatter) + a ## Districts
  section (body links, so link-integrity checks them).
- legislators + US House members gain `represents:` pointing at their district
  node; edge stored once on the person, inverse left to the view.
- schemas/jurisdiction: classification enum + chamber/districts fields.

Honest gaps (logged, not hidden): ID + NH have no state-house geometry, so
their house districts get nodes (represents resolves) but no county edges;
CD nodes for acs_cd-missing states are demographically sparse.

Deterministic (byte-identical rebuild); validate.py passes 26,617 records,
link-integrity clean (every district/represents link resolves).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 22:28:08 -04:00
Fabio 199d8b37d3 Municipals: nest 1,823 city officeholders under their real counties
Pull Atlas's place→county crosswalk (Census TIGER 2024 places × county
geometries, 32,041 places) into the build as a committed raw input, and
use it to resolve municipal officeholders whose county was unknown.

- data/place_county_crosswalk.jsonl — new raw input (place_geoid → county_geoid)
- build.py — place-name+state → county_geoid → ACS county slug as a fallback
  in county_slug(); city_slug() generalized to non-FL bare place labels
- data/sources/sources.yaml — provenance entry for the crosswalk

_unresolved municipals: 1919 → 58. The remainder are genuine: CT planning
regions (post-county), county-level rows mislabeled municipal upstream,
consolidated city-counties (null label), Wikidata-QID placeholder names,
and ambiguous same-name places — left unresolved rather than guessed.

Join key is county_geoid (5-digit FIPS); the directory slug is taken from
the ACS county node so municipals nest under the exact existing county dir.
Deterministic (byte-identical rebuild); validate.py passes 19,721 records.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 20:23:57 -04:00
Fabio ad7607856d Legal corpus: seed U.S. Code Title 52 2026-07-04 19:49:09 -04:00
Fabio 7a2cc64362 Sync: officeholders v3 — fresh export from live Atlas DB (13,329 tenures)
Re-exported directly from Postgres (10.0.0.116), not Atlas's introspection,
which under-reported the roster (its v3 export had 11,473, missing the
restored Florida League of Cities municipal set). Live DB current tenures:
state 7,561 · municipal 4,009 · county 1,217 · federal 542.

- build.py now reads officeholders-v3.jsonl (was v2); dataset vintage 2026-07-04
- fixed one corrupt start_date at source-export (21021-12-03 -> 2021-12-03,
  Lorraine Borowski, Mayor of Decorah IA)
- scripts/build_viz.py: committed, deterministic generator for the Board's
  county_data.json / county_detail.json (previously built ad-hoc)
- generate_changelog.py: ignore files whose only change is the vintage stamp,
  so the diff reflects government change, not re-export bookkeeping

Person files 11,285 -> 13,329. Non-FL municipal officials (1,903) are in the
tree but not yet county-mappable (place->county crosswalk still NULL).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 18:56:37 -04:00
Fabio 316c4e563c Documentation: accurate README + full /docs tree
The README described intent; the repo is now real. Rewrote it to reflect
what actually exists — 17,506 validated files across four entity types,
the deterministic pipeline, the commands — and added the /docs tree:

  government-as-code.md  the philosophy: git primitives -> government
  data-model.md          entity types, IDs, file format, the tree, determinism
  sources.md             where every fact comes from, and how current
  roadmap.md             done / next / later, honest about what isn't built
  contributing.md        the disciplines, and how to add a source

Plus data/sources/sources.yaml — the machine-readable source registry.

Every count, path, and source claim is grounded in the current tree, not
the aspirational brief. Internal links verified; validation still clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 08:38:06 -04:00
Fabio 5b0712c31f Add elections + demographics layer: candidates and jurisdictions
Two new entity types, built deterministically from the raw exports:

  Candidate (2,494)     — 2026 federal candidates as files under
                          us/states/<st>/candidates/, with office,
                          district, party, incumbent/challenger stance,
                          campaign committee, FEC id.

  Jurisdiction (3,494)  — demographic nodes carrying ACS 2023 data:
                          3,131 counties (deduped from 3,231 mixed-format
                          fips keys) + 363 congressional districts.

This completes the county skeleton nationwide: every US county is now a
browsable node with population, income, poverty, race, education and
housing — even where no officeholders are mapped yet. In Florida the
county index.md merges into the existing dir and reports its officeholder
count alongside demographics (St. Lucie: 33 officeholders + ACS profile).

build.py now reads seven raw inputs in one deterministic pass
(byte-identical on rerun). The mirror now answers: who holds power, who
runs this institution, who is running, and what each place is made of.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 23:35:00 -04:00
Fabio 6b3249ecc0 Raw exports: FEC candidates + ACS demographics
Three new source datasets from Atlas, committed as-is before any build:

  fec_candidates.jsonl  2,494  2026 federal candidates (officeholders.fec_data)
  acs_county.jsonl      3,231  Census ACS 2023 by county (~100 dup keys to dedupe)
  acs_cd.jsonl            363  Census ACS 2023 by congressional district

A new axis for the mirror: who is running (candidates) and the
demographics of each place (ACS). Keyed on fec_id / state+district /
county_fips — no person links yet. Raw first; the build into Candidate
and Jurisdiction entities follows.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 23:30:05 -04:00
Fabio d6874ca8a6 Close the institutional gap: bodies, committees, leadership
The seat data answered 'who represents this place?' but not 'who runs
this institution?' — the Speaker of the House existed only as the rep
for LA-4. This adds the missing axis of institutional power.

New Body entities (233): the U.S. House, U.S. Senate, Executive Office,
49 committees, 181 subcommittees — each with its leadership.

Federal people (532/542) enriched with bioguide IDs, leadership roles,
and committee seats, reconciled from congress-legislators by name.
Mike Johnson is now Speaker of the House, not just LA-4.

Sources ingested directly into the repo (repo-canonical): Atlas
supplied bodies/leadership/committee_memberships via congress-legislators,
committed raw in data/. build.py supersedes convert_officeholders.py and
reads all four inputs in one deterministic pass (byte-identical on rerun).

The 10 unenriched federal records are a finding, not a bug: vacancies,
the President, and ~5 seats where Atlas's roster disagrees with the
authoritative current roster — model-vs-reality discrepancies to surface.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 20:51:07 -04:00
Fabio 1545fcf5ca Raw JSONL to structured tree: 11,285 entity files
One record, one file. The jurisdiction tree is now real:

  us/people/                          542 federal
  us/states/<xx>/people/            7,561 state
  us/states/fl/counties/<c>/people/ 1,058 county
  .../municipalities/<m>/people/    2,124 municipal (67 FL counties)

Each file is OKF markdown + YAML frontmatter: fixed key order,
field-level source provenance, Atlas UUIDs for round-tripping,
deterministic paths and slugs. Converter passes the determinism
test: two runs, byte-identical tree.

Generated by scripts/convert_officeholders.py from
data/officeholders-v2.jsonl (unchanged).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 18:08:31 -04:00
Fabio 710405d710 Raw Atlas export: officeholders v2 (11,285 records)
The only surviving copy of the dataset after the DB loss on
10.0.0.116 — committed as-is, before any transformation.
Federal (536), state (7,560), county (1,057), municipal (2,118)
current officeholders. Source: Atlas via /depot/exports,
generated 2026-07-03.

Raw first, refined later: the conversion into structured entity
files will itself be a visible diff.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 17:31:05 -04:00