Docs: the legislation corpus — 50 states, enacted, bill-as-PR

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Fabio
2026-07-06 17:30:50 -04:00
parent 5a13be122b
commit c22d86450b
3 changed files with 54 additions and 2 deletions
+5 -1
View File
@@ -8,7 +8,7 @@ Government is already software: statutes are code, agencies are processes, elect
## What's in here today
105,743 markdown records, every one schema-validated:
302,874 markdown records, every one schema-validated:
| Entity | Count | What it is |
|---|---:|---|
@@ -17,6 +17,7 @@ Government is already software: statutes are code, agencies are processes, elect
| **Candidate** | 2,494 | Everyone running for federal office in 2026, from FEC filings. |
| **Jurisdiction** | 29,905 | Every U.S. county (3,131), all 435 congressional districts (Census demographics on 363), every state legislative district (4,927 house, 1,897 senate), and **every incorporated municipality in the country — 19,513 cities, towns, villages, and boroughs** — keyed by Census place GEOID, with officeholders attached where the mirror has them. |
| **LegalText** | 59,740 | **The complete United States Code** — every section of all 53 titles that carry content (Title 53 is reserved), from General Provisions to Wildlife, mirrored section-by-section from the official OLRC USLM XML at release point 119-100 (current through Public Law 119-100). |
| **Bill** | 197,131 | **Enacted legislation from all 50 states** — every bill and resolution the states enacted, each with its version chain (the diff), action timeline (the PR events), sponsors (the authors), and roll-call votes (the merge decisions), from the OpenStates bulk snapshot vintage 2026-07-01. If the U.S. Code is the deployed branch, these are the merged pull requests. |
And the axes now cross: **27,804 authority edges** link 17,031 U.S. Code sections to the executive offices they empower, so the Attorney General's node lists every one of the 2,386 sections that vests authority in it. Law and power are no longer separate piles.
@@ -28,6 +29,7 @@ The mirror answers, with receipts: *who holds power, who runs the institution, w
raw source exports deterministic build validated tree
data/*.jsonl ──► scripts/build.py ──► data/jurisdictions/**
data/legal/raw ──► scripts/ingest_*.py ──► legal/**
depot: legislation ──► ingest_legislation.py ──► legislation/**
(committed as-is) (one pass) (OKF markdown + YAML)
```
@@ -41,6 +43,7 @@ data/legal/raw ──► scripts/ingest_*.py ──► legal/**
```bash
make build # regenerate the entity tree from the raw exports
make legal-us-code # regenerate the U.S. Code legal corpus (TITLES="1 2 ..." overridable)
make legislation # ingest enacted state legislation from the depot export (LEG_STATES="fl ca" overridable)
make validate # OKF + schema + link-integrity checks on all records
make changelog # what changed in the government since the last commit
make board # rebuild the Board's inlined data from the tree
@@ -55,6 +58,7 @@ Everything runs on the Python standard library. No dependencies to install.
- [Data Model](docs/data-model.md) — entity types, IDs, the file format, the jurisdiction tree
- [Sources](docs/sources.md) — where every fact comes from, and how current it is
- [Legal Corpus](docs/legal-corpus.md) — deterministic legal text mirroring, raw snapshots, manifests, and checksums
- [Legislation Corpus](docs/legislation.md) — enacted state law across all 50 states, bill-as-pull-request, raw off-repo on the depot
- [The Board](docs/board.md) — the county map view, how its data is rebuilt, and how to redeploy the Artifact
- [Roadmap](docs/roadmap.md) — what exists, what's next
- [Contributing](docs/contributing.md) — the disciplines, and how to add a source
+47
View File
@@ -0,0 +1,47 @@
# The Legislation Corpus
Enacted state legislation, all 50 states — the *merged pull requests* of
American law, where the U.S. Code corpus is the deployed branch.
## What's here
197,131 records under `legislation/us/states/{st}/{session}/`, one file per
enacted bill or resolution, from the OpenStates bulk snapshot vintage
**2026-07-01**. Each record carries the bill's version chain (the diff),
action timeline (the PR events), sponsors (the authors), roll-call votes (the
merge decisions), and a `source_hash` over the raw record.
**"Enacted" includes joint and concurrent resolutions**, not only bills — the
`classification` frontmatter field records which. Some states resolve
prolifically (Tennessee's 22,173 enacted = 8,785 bills + 13,388 joint
resolutions); the mirror records what the legislature enacted and leaves
filtering to views.
A bill is a pull request against the law: filed, revised through committee
substitutes (the commits), voted on (the reviews), and — if it passes and is
signed — merged. Enacted bills are the ones that merged.
## Provenance
- Source: OpenStates bulk snapshot (`data.openstates.org`), aggregated from
official legislature sites. Confidence: `reported`.
- Raw JSONL (~1.1 GB, 50 files) lives off-repo on the depot at
`/exports/legislation/2026-07-01/`; the committed anchor is
[`data/sources/legislation_manifest_2026-07-01.json`](../data/sources/legislation_manifest_2026-07-01.json)
with per-state SHA-256, byte counts, and record counts.
- Rebuild: `make legislation` (requires the depot mount; `LEG_STATES="fl ca"`
and `LEG_VINTAGE=` overridable). Deterministic — rerun is byte-identical.
## Next vintages
A future OpenStates snapshot re-exported over this tree surfaces *only real
legislative change* as diffs — new bills, new actions, new enactments. The
difference between two vintages is the changelog of American state law.
## Not yet
No bill → sponsor edges to `Person` nodes yet — that needs name reconciliation
(the OpenStates `person_id` is not the repo's identity), a follow-on axis like
the committee-seat and authority edges. And no bill *text*: the version chain
records the diff points and links to each version's source document, but the
statutory text itself is not mirrored in this pass.
+2 -1
View File
@@ -5,13 +5,14 @@ An honest account of what exists, what's next, and what's further out. This repo
## Done
- **The substrate** — repo-canonical, OKF markdown + YAML frontmatter, JSON-Schema validated, deterministic build (byte-identical on rerun).
- **Five entity types, 105,743 files** — Person (13,329), Body (275), Candidate (2,494), Jurisdiction (29,905), LegalText (59,740).
- **Six entity types, 302,874 files** — Person (13,329), Body (275), Candidate (2,494), Jurisdiction (29,905), LegalText (59,740), Bill (197,131).
- **The federal government, structurally** — every member of Congress, the chambers, 49 committees and 181 subcommittees with their leadership, and every member's committee seats.
- **The nationwide county skeleton** — all 3,131 U.S. counties as demographic nodes, plus all 435 congressional districts (Census ACS demographics on 363).
- **The authority layer** — every state legislative district (4,927 house, 1,897 senate) linked county↔district↔legislator, nationwide.
- **The municipal layer** — every incorporated place in the country (19,513 cities, towns, villages, boroughs) as a first-class Jurisdiction node keyed by Census place GEOID, modelled like a county (place + government fused). 2,229 are filled with their officeholders today (2,217 GEOID-resolved off a 98.8% name→GEOID join); the rest are truthful skeleton stubs — "governed, not yet mirrored" — waiting for depth, the same discipline as the county skeleton.
- **The complete U.S. Code** — every section of all 53 populated titles (Title 53 is reserved), 59,740 per-section records, mirrored from the official OLRC USLM XML at release point 119-100, uniform edition end to end, with manifests and checksums.
- **The authority axis (v1)** — the executive branch minted as nodes from the Code's own enumerations (15 Cabinet departments from 5 U.S.C. § 101, 21 Level I offices from § 5312), and **27,804 edges** linking 17,031 sections to the offices they empower. `scripts/extract_authority.py``data/section_authority_edges.jsonl` (a committed input, like the district edges); reciprocal "empowered by" links render on each office node. Named references only for now (cabinet-level); relative refs ("the Secretary") and the agency long tail are next.
- **The legislation corpus** — 197,131 enacted bills and resolutions from all 50 states (OpenStates vintage 2026-07-01) as `Bill` records with version chains, sponsors, votes, and action timelines. Raw JSONL off-repo on the depot; the per-state SHA-256 manifest committed as the provenance anchor. The state-law "merged PRs" beside the U.S. Code's "deployed branch"; the next vintage's diff is the changelog of American state law.
- **The 2026 federal candidate field** — every FEC filing as a Candidate.
- **Florida to the ground** — officeholders down to the municipal level across all 67 counties.
- **The pipeline** — `make build` / `validate` / `changelog` / `check`, standard-library only.