From 316c4e563c823f200744606e8290dc9aa5d9a528 Mon Sep 17 00:00:00 2001 From: Fabio Date: Sat, 4 Jul 2026 08:38:06 -0400 Subject: [PATCH] Documentation: accurate README + full /docs tree MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The README described intent; the repo is now real. Rewrote it to reflect what actually exists — 17,506 validated files across four entity types, the deterministic pipeline, the commands — and added the /docs tree: government-as-code.md the philosophy: git primitives -> government data-model.md entity types, IDs, file format, the tree, determinism sources.md where every fact comes from, and how current roadmap.md done / next / later, honest about what isn't built contributing.md the disciplines, and how to add a source Plus data/sources/sources.yaml — the machine-readable source registry. Every count, path, and source claim is grounded in the current tree, not the aspirational brief. Internal links verified; validation still clean. Co-Authored-By: Claude Fable 5 --- README.md | 64 +++++++++++++++++++--- data/sources/sources.yaml | 77 +++++++++++++++++++++++++++ docs/contributing.md | 56 ++++++++++++++++++++ docs/data-model.md | 105 +++++++++++++++++++++++++++++++++++++ docs/government-as-code.md | 56 ++++++++++++++++++++ docs/roadmap.md | 37 +++++++++++++ docs/sources.md | 41 +++++++++++++++ 7 files changed, 429 insertions(+), 7 deletions(-) create mode 100644 data/sources/sources.yaml create mode 100644 docs/contributing.md create mode 100644 docs/data-model.md create mode 100644 docs/government-as-code.md create mode 100644 docs/roadmap.md create mode 100644 docs/sources.md diff --git a/README.md b/README.md index f29a560062..cb197c0d4f 100644 --- a/README.md +++ b/README.md @@ -2,13 +2,63 @@ An open, version-controlled mirror of public power in the United States. -Government is already software: statutes are code, agencies are processes, elections change the maintainers, and a bill is a pull request against the law. What government has never had is a changelog. This repository is that changelog. +Government is already software: statutes are code, agencies are processes, elections change the maintainers, and a bill is a pull request against the law. What government has never had is a **changelog**. This repository is that changelog. -**The diff is the product.** Every file here mirrors the observable public state of a government entity — people, offices, bodies, meetings, organizations, money, issues — as deterministic, sourced, human-readable records. When government changes, the files change, and git makes the change visible, permanent, and inspectable. +**The diff is the product.** Every file here mirrors the observable public state of a government entity as a deterministic, sourced, human-readable record. When government changes, the files change, and git makes the change visible, permanent, and inspectable. -- **Founding document:** [Technical Brief — Government as Software](Technical%20Brief%20—%20Government%20as%20Software.md) -- **Format:** Markdown + YAML frontmatter ([OKF](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md)-conformant), validated by JSON Schema -- **Data:** raw source exports land in `data/` first; refinement into structured entity files is itself a visible diff -- **Substrate for:** [influence.tools](https://influence.tools) +## What's in here today -Three disciplines govern everything here: never overwrite, never rewrite history, always push off-box. +17,506 entity files, every one schema-validated: + +| Entity | Count | What it is | +|---|---:|---| +| **Person** | 11,285 | Current officeholders — federal, all 50 state legislatures, and Florida down to the municipal level. Federal members carry their bioguide ID, leadership roles, and committee seats. | +| **Body** | 233 | The institutions themselves — the U.S. House, the Senate, 49 committees, 181 subcommittees, each with its leadership. | +| **Candidate** | 2,494 | Everyone running for federal office in 2026, from FEC filings. | +| **Jurisdiction** | 3,494 | Every U.S. county (3,131) and 363 congressional districts, each carrying Census demographics — the nationwide skeleton, filled with data. | + +The mirror answers, with receipts: *who holds power, who runs the institution, who's running,* and *what each place is made of* — and, through git, *what changed.* + +## How it works + +``` +raw source exports deterministic build validated tree +data/*.jsonl ──► scripts/build.py ──► data/jurisdictions/** +(committed as-is) (7 inputs, one pass) (OKF markdown + YAML) +``` + +- **Format** — Markdown with YAML frontmatter, [OKF](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md)-conformant, validated by JSON Schema in [`/schemas`](schemas). Human-readable in any editor or on GitHub; machine-readable by any agent. +- **Deterministic** — `make build` twice yields a byte-identical tree. Entity files change only when the government changes, so every diff is signal. +- **Sourced** — every record carries field-level provenance and a confidence level. This repo is a *mirror*, never the authority; official sources remain the source of truth. +- **Raw-first** — source exports land in `data/*.jsonl` untouched before any transformation, so the origin is always recoverable. + +## Commands + +```bash +make build # regenerate the entity tree from the raw exports +make validate # OKF + schema + link-integrity checks on all 17,506 files +make changelog # what changed in the government since the last commit +make check # build, then validate — the full gate +``` + +Everything runs on the Python standard library. No dependencies to install. + +## Documentation + +- [Government as Code](docs/government-as-code.md) — the philosophy: why git, and what its primitives mean for government +- [Data Model](docs/data-model.md) — entity types, IDs, the file format, the jurisdiction tree +- [Sources](docs/sources.md) — where every fact comes from, and how current it is +- [Roadmap](docs/roadmap.md) — what exists, what's next +- [Contributing](docs/contributing.md) — the disciplines, and how to add a source +- [Technical Brief — Government as Software](Technical%20Brief%20—%20Government%20as%20Software.md) — the founding document +- [CHANGELOG.md](CHANGELOG.md) — the civic changelog, generated from git history + +## The disciplines + +Three rules govern everything here, and they are won by subtraction: + +1. **Never overwrite** — a change is a new commit, not a clobbered file. Deletes are diffs. +2. **Never rewrite history** — no force-push, no squash. The past is load-bearing. +3. **Always push off-box** — one copy is a single point of failure; more is not. + +Substrate for [influence.tools](https://influence.tools). diff --git a/data/sources/sources.yaml b/data/sources/sources.yaml new file mode 100644 index 0000000000..fc7d62eb82 --- /dev/null +++ b/data/sources/sources.yaml @@ -0,0 +1,77 @@ +# Republic OS — source registry. +# Machine-readable index of the public sources feeding the mirror. +# Prose companion: docs/sources.md + +sources: + - id: atlas + name: Atlas / influence.tools + url: https://influence.tools + type: aggregator + authority_level: aggregator + requires_api_key: false + refresh_frequency: on-demand + feeds: [Person] + raw: [officeholders-v2.jsonl] + notes: > + Draws from official upstreams (House Clerk XML, Senate.gov XML, + OpenStates, Census TIGER/Line, FL Cities partner API) and records the + specific upstream per field. Officeholders at federal, state, and + Florida county/municipal levels. + + - id: congress-legislators + name: unitedstates/congress-legislators + url: https://github.com/unitedstates/congress-legislators + type: dataset + authority_level: authoritative-community + requires_api_key: false + refresh_frequency: continuous + feeds: [Body, Person] + raw: [bodies.jsonl, leadership.jsonl, committee_memberships.jsonl] + notes: > + Structured YAML for chambers, committees, subcommittees, current + leadership, and committee membership. Keyed by bioguide ID. + + - id: fec + name: Federal Election Commission + url: https://www.fec.gov/data/ + type: api + authority_level: official + requires_api_key: true + refresh_frequency: daily + feeds: [Candidate] + raw: [fec_candidates.jsonl] + notes: 2026 federal candidate filings; keyed by fec_id and committee_id. + + - id: census-acs + name: Census American Community Survey (2023) + url: https://www.census.gov/programs-surveys/acs + type: dataset + authority_level: official + requires_api_key: false + refresh_frequency: annual + feeds: [Jurisdiction] + raw: [acs_county.jsonl, acs_cd.jsonl] + notes: County and congressional-district demographics; keyed by FIPS. + +# Prepared but not yet ingested — see docs/roadmap.md +planned: + - id: govinfo + name: GovInfo + url: https://www.govinfo.gov/ + authority_level: official + - id: federal-register + name: Federal Register + url: https://www.federalregister.gov/ + authority_level: official + - id: usaspending + name: USAspending.gov + url: https://www.usaspending.gov/ + authority_level: official + - id: legistar + name: Legistar / Granicus / CivicPlus (local agenda portals) + url: https://webapi.legistar.com/ + authority_level: official + - id: census-tiger + name: Census TIGER/Line (geometry) + url: https://www.census.gov/geographies/mapping-files.html + authority_level: official diff --git a/docs/contributing.md b/docs/contributing.md new file mode 100644 index 0000000000..b201eabb05 --- /dev/null +++ b/docs/contributing.md @@ -0,0 +1,56 @@ +# Contributing + +Republic OS is a mirror maintained mostly by agents, but its rules are simple enough that anyone can work in it. This page covers the disciplines that keep the mirror trustworthy and the practical steps for adding to it. + +## The disciplines + +Three rules govern everything, and they are won by subtraction — by declining to destroy: + +1. **Never overwrite.** A change is a new commit, not a clobbered file. Deletes are diffs. +2. **Never rewrite history.** No force-push, no squash. The past is load-bearing — it *is* the changelog. +3. **Always push off-box.** One copy is a single point of failure; more is not. + +And one rule that makes every diff meaningful: + +4. **The build is deterministic.** Entity files change only when the government changes. Never introduce pipeline noise (run timestamps, unstable ordering, reformatting) into an entity file. If `make build` run twice produces a diff, that is a bug. + +## The shape of a change + +Everything flows one direction: + +``` +official source ──► raw export (data/*.jsonl) ──► build.py ──► entity tree +``` + +- **Raw first.** A new source lands in `data/.jsonl` committed *as-is*, before any transformation. The origin must always be recoverable. +- **Build, don't hand-edit.** Entity files under `data/jurisdictions/` are generated. To change them, change the source or the generator — never edit a generated file by hand, or the next build will overwrite it (and you'll have created noise, not signal). +- **Validate before committing.** `make check` runs the build and then validates all files. It must pass. + +## Adding a source + +1. Obtain the data as JSONL (one JSON object per line) and commit it to `data/.jsonl` untouched. +2. Register it in [`data/sources/sources.yaml`](../data/sources/sources.yaml) and note it in [docs/sources.md](sources.md). +3. Extend [`scripts/build.py`](../scripts/build.py) to read it and emit entities. Follow the existing patterns: + - Emit frontmatter keys in a **fixed order**; sort all iteration. + - Carry **field-level `sources`** and a `confidence` level on every record. + - Use **path-based IDs** with lowercase slugs and stable government identifiers where available. + - Reconcile to existing entities by a shared key, or by normalized name where none exists. +4. If it's a new entity type, add a JSON Schema to [`/schemas`](../schemas) — a stricter profile over OKF (required fields, property types, enums), tolerating unknown keys. +5. Run `make check`. Fix anything it reports. +6. Commit with a message that describes the *civic* change, not just the code change (see the existing history for the tone). + +## Reading a change + +You do not need deep git knowledge to read the mirror — four commands cover it: + +```bash +git log -- # the timeline of any official, body, or place +git diff .. # what changed between two states +git show # one change, in full +git blame # where each fact came from +make changelog # the same, rendered as a plain-language civic changelog +``` + +## A note on truth + +When the mirror disagrees with an official record, the official record is right and the mirror has a bug. Report it as a finding — the gap between the model and reality is information, not embarrassment. This repository never claims to be the authority; it only makes the authority easier to read. diff --git a/docs/data-model.md b/docs/data-model.md new file mode 100644 index 0000000000..85d8db0131 --- /dev/null +++ b/docs/data-model.md @@ -0,0 +1,105 @@ +# Data Model + +## The file + +Every entity is one Markdown file with YAML frontmatter. The frontmatter is the machine-readable record; the body is the human-readable page. The same file serves a journalist reading it on GitHub, an editor in an Obsidian vault, and an AI agent parsing frontmatter. + +```markdown +--- +type: Person +title: "Mike Johnson" +role: "U.S. House of Representatives - LA-4" +party: "R" +level: "federal" +branch: "legislative" +state: "LA" +leadership: + - role: "Speaker of the House" + body: "H" + since: "2025-01-03" +ids: + person: "fac58269-6f4a-4706-8eb4-95a549c07a41" + bioguide: "J000299" +sources: + - field: tenure + source: "House Clerk XML 2026-06-10" + - field: roles + source: "congress-legislators (unitedstates project)" +confidence: official +tags: [officeholder, federal, legislative, la] +timestamp: "2026-06-20" +--- + +# Mike Johnson + +Current U.S. House of Representatives - LA-4 (Congressional District 4). + +## Leadership + +- Speaker of the House (since 2025-01-03) +``` + +The format conforms to the [Open Knowledge Format (OKF)](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md): only `type` is strictly required, unknown keys are tolerated, and the concept ID is the file path. Republic OS layers a stricter, schema-validated profile on top — see [`/schemas`](../schemas). + +## Entity types + +| `type` | Count | Home | Schema | +|---|---:|---|---| +| **Person** | 11,285 | `us/people/`, `us/states//…/people/` | [person.schema.json](../schemas/person.schema.json) | +| **Body** | 233 | `us/bodies/…` | [body.schema.json](../schemas/body.schema.json) | +| **Candidate** | 2,494 | `us/states//candidates/` | [candidate.schema.json](../schemas/candidate.schema.json) | +| **Jurisdiction** | 3,494 | county `index.md`, `us/states//districts/` | [jurisdiction.schema.json](../schemas/jurisdiction.schema.json) | + +- **Person** — an officeholder, filed by the jurisdiction they serve. Federal members are enriched with their bioguide ID, leadership roles, and committee seats. +- **Body** — an institution: a chamber, committee, or subcommittee. The "repos" of the government-as-code model. Each lists its leadership; subcommittees name their parent. +- **Candidate** — someone running for office, distinct from a current officeholder. Sourced from FEC filings. +- **Jurisdiction** — a place with a government, carrying Census demographics. A county's node is its directory `index.md`; a district's is a file under `districts/`. + +## The jurisdiction tree + +Geography is a hierarchy: every entity sits in exactly one place in the tree. The schema is **fractal** — a county commissioner and a U.S. senator are both `Person`; a school board and the House are both `Body` — so the same shapes repeat at every level. + +``` +data/jurisdictions/ + us/ + people/ federal officeholders (542) + bodies/ House, Senate, committees, subcommittees + house/committees/… + senate/committees/… + states/ + fl/ + people/ FL state legislators + candidates/ 2026 federal candidates from FL + districts/ congressional districts (ACS demographics) + counties/ + st-lucie/ + index.md the county as a Jurisdiction node + people/ county officeholders + municipalities/ + port-st-lucie/ + people/ city officeholders +``` + +The tree is **complete on the map, uneven on the ground**: every U.S. county exists as a node (from Census data), but officeholder depth currently reaches the municipal level only in Florida. An empty directory is not a failure — it is the frontier. + +A second axis — cross-cutting **issues** that link entities across the tree — is designed but not yet built. See the [roadmap](roadmap.md). + +## Identifiers + +- **Path is identity.** The concept ID is the file path (OKF): `us/states/fl/counties/st-lucie/people/anthony-bonna` is both the address and the ID. Paths use lowercase slugs and stable government identifiers where available. +- **Carried IDs** round-trip back to sources: Atlas UUIDs (`person`, `office`, `tenure`, `jurisdiction`), `bioguide` for federal members, `fec` for candidates, `fips` for counties. +- **Reconciliation** across sources is by normalized name where no shared key exists — for example, congress-legislators (bioguide-keyed) is joined onto Atlas officeholders (UUID-keyed) by name. + +## Determinism — why the diffs are trustworthy + +The build is deterministic by construction: sorted iteration, a fixed key order, one full rebuild per run. `make build` twice produces a byte-identical tree, verified by hashing every file. Entity files change **only when the government changes**, never because of pipeline noise (timestamps, ordering, formatting). This is what makes `make changelog` meaningful — a diff is always a real change, never an artifact. + +## Validation + +[`scripts/validate.py`](../scripts/validate.py) checks all 17,506 files with no third-party dependencies: + +1. **OKF conformance** — every file has frontmatter with a non-empty `type`. +2. **Schema conformance** — required fields, property types, and enums per entity type. +3. **Link integrity** — every internal `/`-rooted markdown link resolves to a real file. + +It exits non-zero on failure, so it doubles as a CI gate. Run it with `make validate`. diff --git a/docs/government-as-code.md b/docs/government-as-code.md new file mode 100644 index 0000000000..880ca84763 --- /dev/null +++ b/docs/government-as-code.md @@ -0,0 +1,56 @@ +# Government as Code + +## The premise + +Government is an operating system that has run for 250 years without version control. + +The analogy is not decorative — it maps almost mechanically: + +| Software | Government | +|---|---| +| Source code | Statutes, ordinances, charters, budgets | +| Running processes | Agencies, departments, courts | +| Maintainers with commit rights | Elected officials, appointed boards | +| A pull request | A bill, a rezoning petition, a budget amendment | +| Code review | Committee hearings, public comment periods | +| The merge vote | The floor vote, the commission vote | +| A release | An adopted budget, an enacted law | +| Changing the maintainers | An election | +| The changelog | **Did not exist. This repository is it.** | + +Everything on the right is public. Almost none of it is *legible*. The state of government is scattered across PDFs, agenda portals, filings, and meeting videos — published, technically, but not readable, not diffable, and above all not *watchable*. A change to a zoning code and a change to a codebase are the same category of event; only one of them triggers a notification. + +We cannot read government's source directly. But we can mirror its observable public state into files, and version the mirror. Git then gives us, for free, the tooling software has had for decades. + +## What git's primitives mean here + +| Git | In this repository | +|---|---| +| The files | The current state — who holds which office, what a place is made of, who's running. | +| A commit | An observed change, timestamped: "this member's committee assignment changed." | +| `git log` | The timeline of any official, body, or place — forever. | +| `git diff` | The change itself, in plain terms. This is the product. | +| `git blame` | The provenance of any single fact, line by line. | +| `git show` | One change, in full, on the permanent record. | + +You do not need to know git deeply to read this repository. Four commands cover it: `git log` (what happened), `git diff` (what changed), `git show` (one change in full), `git blame` (where a fact came from). The agents that maintain the mirror do the writing. + +## The diff is the product + +The honest caveat: the mirror's history starts the day we start watching. Git's value here is entirely forward-looking — from the first commit on, *nothing changes unnoticed*. Nobody can quietly pull an item off an agenda or revise a bio; that edit is itself a diff, timestamped and permanent. + +This is why the [changelog](../CHANGELOG.md) matters more than any single record. A directory of officials is useful; a running account of *what your government changed this week* is a different kind of instrument. The changelog is generated directly from git — nobody writes it, it falls out of the files changing. + +For this to work, the files must be **boring**. The build is deterministic: the same entity always at the same path, keys always in the same order, identical formatting every run. Volatile bookkeeping ("we checked, nothing changed") never touches the entity files — only real government changes produce a diff. This single discipline is most of the engineering. See [the data model](data-model.md) for how it's enforced. + +## Principles + +- **Government should be legible.** Structure is the precondition for understanding. +- **Official sources remain the authority.** This repository is a mirror, not the source of truth. Every record cites where it came from and how confident we are. +- **Changes should be visible.** The value is not the snapshot; it's the diff between snapshots. +- **Git preserves history.** A record deleted upstream becomes a diff here, not a hole. Civic memory that survives deletion is a byproduct of refusing to overwrite. +- **Agents need structured civic memory.** The mirror is built to be read and reasoned over by software as much as by people — stable IDs, simple schemas, field-level sources, machine-diffable format. + +## Not the source of truth + +Worth repeating, because it is the ethical center of the project: **Republic OS is a mirror.** When it disagrees with an official record, the official record is right and the mirror has a bug — which is itself a finding, surfaced as a diff. The goal is to make public power easier to read, not to become an alternative authority over it. diff --git a/docs/roadmap.md b/docs/roadmap.md new file mode 100644 index 0000000000..e7050bd097 --- /dev/null +++ b/docs/roadmap.md @@ -0,0 +1,37 @@ +# Roadmap + +An honest account of what exists, what's next, and what's further out. This repository will always be evolving; there is no "finished." + +## Done + +- **The substrate** — repo-canonical, OKF markdown + YAML frontmatter, JSON-Schema validated, deterministic build (byte-identical on rerun). +- **Four entity types, 17,506 files** — Person (11,285), Body (233), Candidate (2,494), Jurisdiction (3,494). +- **The federal government, structurally** — every member of Congress, the chambers, 49 committees and 181 subcommittees with their leadership, and every member's committee seats. +- **The nationwide county skeleton** — all 3,131 U.S. counties as demographic nodes, plus 363 congressional districts, from Census ACS. +- **The 2026 federal candidate field** — every FEC filing as a Candidate. +- **Florida to the ground** — officeholders down to the municipal level across all 67 counties. +- **The pipeline** — `make build` / `validate` / `changelog` / `check`, standard-library only. +- **The changelog engine** — git diffs rendered as a plain-language civic changelog. +- **The Board** — a self-contained coverage map ([`viz/board.html`](../viz/board.html)). + +## Next + +- **Self-updating mirror** — a GitHub Action running `make check` on every push, then a scheduled job that pulls a fresh Atlas export, rebuilds, and commits the changelog automatically. This is the point where the mirror maintains itself and the daily civic changelog becomes real (the first *re-sync* diff, where only genuine government changes surface). +- **The demographic choropleth** — the Board already has ACS data for every county; shade the map by population, income, and poverty, turning it from a coverage indicator into an instrument. +- **State and local bodies** — Bodies currently exist only at the federal level. Mint them for state legislatures and, following the Florida depth, county commissions and city councils. +- **Graph export** — a loader that walks the tree and builds the relationship graph (Memgraph) as a materialized view, so the repo can rehydrate the graph rather than the reverse. + +## Later + +- **The issues axis** — cross-cutting `Issue` entities that link people, bodies, and places across the jurisdiction tree, so the mirror answers "who, nationwide, is touching this issue?" and not only "who governs here?" +- **The money layer** — donations, contracts, and grants as sourced event files; the flows that make influence legible (the Sankey / "circuit-breaker" views from the brief). +- **Meetings and agendas** — the highest-value local data, and the hardest: scraped from Legistar / Granicus / CivicPlus portals and, where towns publish nothing in text, from meeting video. A new agenda item is a diff with a deadline — a mission. +- **Contribution as correction** — let people propose fixes to the mirror (a wrong phone number, a missed meeting) as pull requests, making the public literal contributors to their own civic record. + +## Non-goals + +Deliberately out of scope, to keep the mirror honest and buildable: + +- Becoming an authority. This is a mirror; official sources remain the source of truth. +- A heavy web application, database dependency, or graph-database setup in the core repository. The substrate stays flat files. +- Ingesting all of government at once. Depth follows attention, county by county. diff --git a/docs/sources.md b/docs/sources.md new file mode 100644 index 0000000000..6f6ef624bc --- /dev/null +++ b/docs/sources.md @@ -0,0 +1,41 @@ +# Sources + +Republic OS is a mirror. Every fact in it comes from a public source, and every record carries field-level provenance and a confidence level. This page lists the sources feeding the mirror today; the machine-readable registry is [`data/sources/sources.yaml`](../data/sources/sources.yaml). + +The pipeline is **repo-canonical**: raw source exports land in `data/*.jsonl` untouched, and [`scripts/build.py`](../scripts/build.py) generates the entity tree from them in one deterministic pass. Nothing is hand-edited into the tree. + +## Active sources + +| Source | Feeds | Raw file | Authority | +|---|---|---|---| +| **Atlas / influence.tools** | Officeholders — federal, all state legislatures, FL county + municipal | `officeholders-v2.jsonl` (11,285) | Aggregator | +| **congress-legislators** ([unitedstates project](https://github.com/unitedstates/congress-legislators)) | Bodies, leadership roles, committee memberships | `bodies.jsonl` (233), `leadership.jsonl` (28), `committee_memberships.jsonl` (3,879) | Community, authoritative | +| **FEC** (Federal Election Commission) | 2026 federal candidate filings | `fec_candidates.jsonl` (2,494) | Official | +| **Census ACS 2023** | County & congressional-district demographics | `acs_county.jsonl` (3,231), `acs_cd.jsonl` (363) | Official | + +Atlas itself draws from official upstreams — House Clerk XML, Senate.gov XML, OpenStates, Census TIGER/Line geometry, and the Florida Cities partner API — and records the specific upstream in each record's `sources` block. So a Person file's provenance names not just "Atlas" but the document behind the field (e.g. `tenure: House Clerk XML 2026-06-10`). + +## Provenance in every record + +Each entity's frontmatter carries a `sources` list and a `confidence` level: + +```yaml +sources: + - field: tenure + source: "House Clerk XML 2026-06-10" + - field: roles + source: "congress-legislators (unitedstates project)" +confidence: official +``` + +`confidence` is one of `official`, `reported`, `inferred`, or `unverified`. Everything in the mirror today is `official`. + +## Reconciliation and its findings + +Sources use different keys — Atlas uses internal UUIDs, congress-legislators uses bioguide IDs, FEC uses committee IDs, ACS uses FIPS codes. Where no shared key exists, records are reconciled by normalized name. Federal officeholders reconcile at ~99% (531 of 532 members matched to their leadership and committee data). + +Disagreements between sources are not errors to hide but **findings to surface**. Where Atlas's officeholder roster names a different person for a seat than the authoritative current roster does, the mismatch stands as a visible discrepancy — the gap between the model and reality is itself information. + +## Known sources not yet ingested + +Named in the [technical brief](../Technical%20Brief%20—%20Government%20as%20Software.md) and [roadmap](roadmap.md), not yet in the mirror: campaign finance flows and contracts (the "money" layer), local agenda portals (Legistar / Granicus / CivicPlus), the Federal Register, lobbying registrations, and property records. These are where the mirror grows next.