Commit Graph

23 Commits

Author SHA1 Message Date
Fabio 6c9990a115 Docs: document the Board pipeline + Artifact redeploy step
Add `make board` and docs/board.md so the Board's rebuild-and-redeploy flow
is a documented step, not tribal knowledge. Covers why the county data is
inlined (self-contained Artifact / CSP), that build_viz.py injects it into
board.html, and that a live Artifact must be REPUBLISHED (editing the repo
file doesn't update a deployed Artifact). README links it in both the make
list and the docs index.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 21:12:05 -04:00
Fabio 3e62a1b44a Board: add page <title> (Republic OS — The Board)
The page had no <title>, so browser tabs / the Artifact fell back to the
filename. build_viz.py's injection only rewrites the const lines, so the
title is preserved across regenerations.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 21:07:33 -04:00
Fabio 12850196f9 Board: regenerate board.html data in-place; drive header from TOTALS
The Board is a self-contained Artifact, so its county data is INLINED
(const D / const CDETAIL) — the standalone county_*.json companions are not
read by it. build_viz.py was only writing those companions, so the rendered
Board kept showing the pre-crosswalk state (67 counties). Now build_viz.py
injects the fresh data straight into board.html's three marked const lines,
and the header stats read from an injected TOTALS object instead of being
hardcoded (so they can't go stale — the original 11,285 drift).

- scripts/build_viz.py — inject_board(): rewrite const D / CDETAIL / TOTALS
  in board.html; load candidates + acs_cd for the totals
- viz/board.html — header driven by TOTALS; inlined data refreshed
  (county drill-downs 67 → 1049 counties; e.g. Fulton GA now shows all 7
  metro mayors)

Deterministic: board.html is byte-identical on rerun. Verified live —
header 13,329/2,494/3,131/363, Fulton/Bastrop drill-downs render nested
municipals.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 20:55:43 -04:00
Fabio e37a897111 Board: fix stale officeholder total in header (11,285 → 13,329)
The "OFFICEHOLDERS" stat was hardcoded to the old v2 Person count; v3 has
13,329. Counties/candidates/districts were unaffected. (Follow-up worth
doing: emit these header totals from build_viz.py so they can't go stale.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 20:40:37 -04:00
Fabio 662ccecb26 Board: surface the newly-nested municipals in county drill-downs
build_viz.py was resolving officials to counties without the place→county
crosswalk, so only FL's 67 counties had rosters. Factor the resolver out of
build.py into a shared build.place_resolver() and use it in both, so the
Board and the tree agree by construction.

- scripts/build.py — extract place_resolver(); main() now calls it (tree
  output byte-identical, refactor is behavior-preserving)
- scripts/build_viz.py — resolve via the shared resolver
- viz/county_data.json, viz/county_detail.json — regenerated

County drill-downs: 67 → 1049 counties with rosters; officials mapped
5184/5226 (42 county-unresolvable, tree-only). Deterministic.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 20:36:42 -04:00
Fabio 199d8b37d3 Municipals: nest 1,823 city officeholders under their real counties
Pull Atlas's place→county crosswalk (Census TIGER 2024 places × county
geometries, 32,041 places) into the build as a committed raw input, and
use it to resolve municipal officeholders whose county was unknown.

- data/place_county_crosswalk.jsonl — new raw input (place_geoid → county_geoid)
- build.py — place-name+state → county_geoid → ACS county slug as a fallback
  in county_slug(); city_slug() generalized to non-FL bare place labels
- data/sources/sources.yaml — provenance entry for the crosswalk

_unresolved municipals: 1919 → 58. The remainder are genuine: CT planning
regions (post-county), county-level rows mislabeled municipal upstream,
consolidated city-counties (null label), Wikidata-QID placeholder names,
and ambiguous same-name places — left unresolved rather than guessed.

Join key is county_geoid (5-digit FIPS); the directory slug is taken from
the ACS county node so municipals nest under the exact existing county dir.
Deterministic (byte-identical rebuild); validate.py passes 19,721 records.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 20:23:57 -04:00
Fabio ad7607856d Legal corpus: seed U.S. Code Title 52 2026-07-04 19:49:09 -04:00
Fabio fe8f82ae30 Changelog: first real diff — v2→v3 government sync
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 18:56:58 -04:00
Fabio 7a2cc64362 Sync: officeholders v3 — fresh export from live Atlas DB (13,329 tenures)
Re-exported directly from Postgres (10.0.0.116), not Atlas's introspection,
which under-reported the roster (its v3 export had 11,473, missing the
restored Florida League of Cities municipal set). Live DB current tenures:
state 7,561 · municipal 4,009 · county 1,217 · federal 542.

- build.py now reads officeholders-v3.jsonl (was v2); dataset vintage 2026-07-04
- fixed one corrupt start_date at source-export (21021-12-03 -> 2021-12-03,
  Lorraine Borowski, Mayor of Decorah IA)
- scripts/build_viz.py: committed, deterministic generator for the Board's
  county_data.json / county_detail.json (previously built ad-hoc)
- generate_changelog.py: ignore files whose only change is the vintage stamp,
  so the diff reflects government change, not re-export bookkeeping

Person files 11,285 -> 13,329. Non-FL municipal officials (1,903) are in the
tree but not yet county-mappable (place->county crosswalk still NULL).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 18:56:37 -04:00
Fabio 092e54ae21 Board: drill down into each county
Click any county to open a detail panel: its demographics, then the
actual officeholders in the mirror — grouped by county government and by
municipality, each with name and role — and a link to the county's files
on GitHub.

Richest for Florida, mirrored to the municipal level: e.g. St. Lucie
opens to 33 officeholders (18 county — commission, sheriff, tax collector,
school board — plus Fort Pierce, Port St. Lucie, and St. Lucie Village).
Counties without officeholders yet show their demographic node and a note.

Detail data (viz/county_detail.json, 181 KB) extracted from the entity
tree: 67 FL counties, 1,058 county + 2,124 municipal officials. Modal
closes on X, backdrop, or Esc. Verified in-browser: click opens the
panel, names render First-Last, municipality sections and GitHub links
resolve.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 09:28:21 -04:00
Fabio 41ec4bf173 Board: eliminate the fog of war
Dropped the Coverage lens and its dark-unmapped framing. The Board is now
a pure demographic instrument: three lenses (median income, population,
poverty rate), every one coloring all 3,109 counties — no darkness, no
'not yet mapped', no fog-of-war copy. Rail shows Districts (363) instead
of 'counties lit'; the ledger describes the demographic lenses.

Also removed the load-in fade, which was flashing the map black on first
paint — it now renders fully colored immediately.

Verified in-browser: income (green) and poverty (amber) choropleths both
render full-country; lens switching and hover readout intact.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 09:20:49 -04:00
Fabio 4e0c559227 Board: fix the county colors
The county fills were technically correct but visually near-black: linear
min-max scaling crammed a right-skewed distribution into the dark end, so
the map read as empty. Fixes:

  - Rank (percentile) coloring — spread the full palette evenly across
    counties, the standard choropleth technique, so every lens shows rich
    variation instead of a few bright outliers on black.
  - Brighter per-lens ramps (income green, population teal, poverty amber).
  - Coverage amber lifted so Florida's 67 lit counties are unmistakable
    against the fog-of-war country.
  - Default lens is now Median income, so the map opens full of color;
    Coverage (the fog of war) is one click away.

Verified in-browser: full green income choropleth, Florida clearly lit in
coverage, hover readout prints each county's demographics + officeholders.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 09:11:44 -04:00
Fabio e126d3e374 The Board, v2: nationwide county map with lenses
The map grew with the data. The old Board showed coverage at the state
level with a Florida drill-down; now that every US county is a demographic
node, the Board renders all 3,109 contiguous counties directly, with
switchable lenses:

  Coverage       — the true fog of war: 67 FL counties lit (officeholders
                   mirrored), the other 3,000+ dark on the frontier.
  Population     — ACS 2023 choropleth, 33 to 9.8M.
  Median income  — k to k.
  Poverty rate   — 1.4% to 48%.

Hover any county for its full readout (demographics + officeholder count).
Self-contained: inlined county geometry (duplicate-point-collapsed),
state-border overlay, hand-rolled projection, event-delegated hover for
3,109 paths, vanilla JS — no external requests. Verified: projection
places counties correctly, 3,099/3,109 join to ACS data.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 08:52:21 -04:00
Fabio 316c4e563c Documentation: accurate README + full /docs tree
The README described intent; the repo is now real. Rewrote it to reflect
what actually exists — 17,506 validated files across four entity types,
the deterministic pipeline, the commands — and added the /docs tree:

  government-as-code.md  the philosophy: git primitives -> government
  data-model.md          entity types, IDs, file format, the tree, determinism
  sources.md             where every fact comes from, and how current
  roadmap.md             done / next / later, honest about what isn't built
  contributing.md        the disciplines, and how to add a source

Plus data/sources/sources.yaml — the machine-readable source registry.

Every count, path, and source claim is grounded in the current tree, not
the aspirational brief. Internal links verified; validation still clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 08:38:06 -04:00
Fabio 2e5197d1a3 The changelog engine: turn git diffs into a civic changelog
The diff is the product — now literally. generate_changelog.py reads a
git diff of the entity tree between any two refs and says, in plain
language, what changed in the government: entities added / modified /
removed, grouped by type and by jurisdiction, dated from the commit.

CHANGELOG.md is seeded from the real history so far:
  - initial load: +11,285 people across 51 states
  - institutional gap: +233 bodies, ~532 people enriched
  - elections + demographics: +2,494 candidates, +3,494 jurisdictions
    (TX +541, FL +396, CA +341 ...)

On an initial load every entity is an addition; on a re-sync of the same
source, only real government changes will surface — which is the whole
point. When nothing changed, it says so.

make changelog [BASE=.. HEAD=..]. No third-party deps.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 08:28:50 -04:00
Fabio aefa65f5b3 Validation: JSON Schemas + validator (the discipline, enforced)
The README claimed 'validated by JSON Schema' — now it is true.

/schemas holds a JSON Schema for each entity type (Person, Body,
Candidate, Jurisdiction): a stricter profile over OKF's permissive
base, tolerating unknown keys by design.

scripts/validate.py checks all 17,506 files with no third-party deps
(hand-rolled frontmatter parser + minimal schema engine):
  1. OKF conformance — every file has frontmatter with a non-empty type
  2. Schema conformance — required fields, property types, enums
  3. Link integrity — internal /-rooted links resolve

Non-zero exit on failure, so it is a CI gate. Verified with a negative
test: it catches bad enums and dead links. All 17,506 files pass today.

Makefile ties it together: make build / validate / check.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 23:43:56 -04:00
Fabio 5b0712c31f Add elections + demographics layer: candidates and jurisdictions
Two new entity types, built deterministically from the raw exports:

  Candidate (2,494)     — 2026 federal candidates as files under
                          us/states/<st>/candidates/, with office,
                          district, party, incumbent/challenger stance,
                          campaign committee, FEC id.

  Jurisdiction (3,494)  — demographic nodes carrying ACS 2023 data:
                          3,131 counties (deduped from 3,231 mixed-format
                          fips keys) + 363 congressional districts.

This completes the county skeleton nationwide: every US county is now a
browsable node with population, income, poverty, race, education and
housing — even where no officeholders are mapped yet. In Florida the
county index.md merges into the existing dir and reports its officeholder
count alongside demographics (St. Lucie: 33 officeholders + ACS profile).

build.py now reads seven raw inputs in one deterministic pass
(byte-identical on rerun). The mirror now answers: who holds power, who
runs this institution, who is running, and what each place is made of.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 23:35:00 -04:00
Fabio 6b3249ecc0 Raw exports: FEC candidates + ACS demographics
Three new source datasets from Atlas, committed as-is before any build:

  fec_candidates.jsonl  2,494  2026 federal candidates (officeholders.fec_data)
  acs_county.jsonl      3,231  Census ACS 2023 by county (~100 dup keys to dedupe)
  acs_cd.jsonl            363  Census ACS 2023 by congressional district

A new axis for the mirror: who is running (candidates) and the
demographics of each place (ACS). Keyed on fec_id / state+district /
county_fips — no person links yet. Raw first; the build into Candidate
and Jurisdiction entities follows.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 23:30:05 -04:00
Fabio 421c5b3553 The Board: fog-of-war coverage map (first visualization)
The one radical view whose data is fully real today (brief §4.5).
Renders every officeholder in the mirror as coverage: teal is the
federal + state tier (the roof, every state), amber is county +
municipal depth (the foundation, so far only Florida). Click Florida
to descend into its 67 counties, shaded by local officeholder count.

Self-contained: inlined US state + FL county geometry, hand-rolled
lng/lat projection, vanilla JS — no external requests, no build deps.
Instrument aesthetic: dark ground, mono readout, breathing amber to
invite the drill-down. viz/coverage.json is generated from the tree.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 21:00:52 -04:00
Fabio d6874ca8a6 Close the institutional gap: bodies, committees, leadership
The seat data answered 'who represents this place?' but not 'who runs
this institution?' — the Speaker of the House existed only as the rep
for LA-4. This adds the missing axis of institutional power.

New Body entities (233): the U.S. House, U.S. Senate, Executive Office,
49 committees, 181 subcommittees — each with its leadership.

Federal people (532/542) enriched with bioguide IDs, leadership roles,
and committee seats, reconciled from congress-legislators by name.
Mike Johnson is now Speaker of the House, not just LA-4.

Sources ingested directly into the repo (repo-canonical): Atlas
supplied bodies/leadership/committee_memberships via congress-legislators,
committed raw in data/. build.py supersedes convert_officeholders.py and
reads all four inputs in one deterministic pass (byte-identical on rerun).

The 10 unenriched federal records are a finding, not a bug: vacancies,
the President, and ~5 seats where Atlas's roster disagrees with the
authoritative current roster — model-vs-reality discrepancies to surface.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 20:51:07 -04:00
Fabio 1545fcf5ca Raw JSONL to structured tree: 11,285 entity files
One record, one file. The jurisdiction tree is now real:

  us/people/                          542 federal
  us/states/<xx>/people/            7,561 state
  us/states/fl/counties/<c>/people/ 1,058 county
  .../municipalities/<m>/people/    2,124 municipal (67 FL counties)

Each file is OKF markdown + YAML frontmatter: fixed key order,
field-level source provenance, Atlas UUIDs for round-tripping,
deterministic paths and slugs. Converter passes the determinism
test: two runs, byte-identical tree.

Generated by scripts/convert_officeholders.py from
data/officeholders-v2.jsonl (unchanged).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 18:08:31 -04:00
Fabio 710405d710 Raw Atlas export: officeholders v2 (11,285 records)
The only surviving copy of the dataset after the DB loss on
10.0.0.116 — committed as-is, before any transformation.
Federal (536), state (7,560), county (1,057), municipal (2,118)
current officeholders. Source: Atlas via /depot/exports,
generated 2026-07-03.

Raw first, refined later: the conversion into structured entity
files will itself be a visible diff.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 17:31:05 -04:00
Fabio 4e0cc2bf9e Founding documents: README and technical brief
Government as an operating system, git as its missing changelog.
The mirror architecture, influence as a computable tool, a visual
grammar for power, and the game layer that lights the map.

The diff is the product.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 17:30:44 -04:00