Docs: the legislation corpus — 50 states, enacted, bill-as-PR
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -8,7 +8,7 @@ Government is already software: statutes are code, agencies are processes, elect
|
|||||||
|
|
||||||
## What's in here today
|
## What's in here today
|
||||||
|
|
||||||
105,743 markdown records, every one schema-validated:
|
302,874 markdown records, every one schema-validated:
|
||||||
|
|
||||||
| Entity | Count | What it is |
|
| Entity | Count | What it is |
|
||||||
|---|---:|---|
|
|---|---:|---|
|
||||||
@@ -17,6 +17,7 @@ Government is already software: statutes are code, agencies are processes, elect
|
|||||||
| **Candidate** | 2,494 | Everyone running for federal office in 2026, from FEC filings. |
|
| **Candidate** | 2,494 | Everyone running for federal office in 2026, from FEC filings. |
|
||||||
| **Jurisdiction** | 29,905 | Every U.S. county (3,131), all 435 congressional districts (Census demographics on 363), every state legislative district (4,927 house, 1,897 senate), and **every incorporated municipality in the country — 19,513 cities, towns, villages, and boroughs** — keyed by Census place GEOID, with officeholders attached where the mirror has them. |
|
| **Jurisdiction** | 29,905 | Every U.S. county (3,131), all 435 congressional districts (Census demographics on 363), every state legislative district (4,927 house, 1,897 senate), and **every incorporated municipality in the country — 19,513 cities, towns, villages, and boroughs** — keyed by Census place GEOID, with officeholders attached where the mirror has them. |
|
||||||
| **LegalText** | 59,740 | **The complete United States Code** — every section of all 53 titles that carry content (Title 53 is reserved), from General Provisions to Wildlife, mirrored section-by-section from the official OLRC USLM XML at release point 119-100 (current through Public Law 119-100). |
|
| **LegalText** | 59,740 | **The complete United States Code** — every section of all 53 titles that carry content (Title 53 is reserved), from General Provisions to Wildlife, mirrored section-by-section from the official OLRC USLM XML at release point 119-100 (current through Public Law 119-100). |
|
||||||
|
| **Bill** | 197,131 | **Enacted legislation from all 50 states** — every bill and resolution the states enacted, each with its version chain (the diff), action timeline (the PR events), sponsors (the authors), and roll-call votes (the merge decisions), from the OpenStates bulk snapshot vintage 2026-07-01. If the U.S. Code is the deployed branch, these are the merged pull requests. |
|
||||||
|
|
||||||
And the axes now cross: **27,804 authority edges** link 17,031 U.S. Code sections to the executive offices they empower, so the Attorney General's node lists every one of the 2,386 sections that vests authority in it. Law and power are no longer separate piles.
|
And the axes now cross: **27,804 authority edges** link 17,031 U.S. Code sections to the executive offices they empower, so the Attorney General's node lists every one of the 2,386 sections that vests authority in it. Law and power are no longer separate piles.
|
||||||
|
|
||||||
@@ -28,6 +29,7 @@ The mirror answers, with receipts: *who holds power, who runs the institution, w
|
|||||||
raw source exports deterministic build validated tree
|
raw source exports deterministic build validated tree
|
||||||
data/*.jsonl ──► scripts/build.py ──► data/jurisdictions/**
|
data/*.jsonl ──► scripts/build.py ──► data/jurisdictions/**
|
||||||
data/legal/raw ──► scripts/ingest_*.py ──► legal/**
|
data/legal/raw ──► scripts/ingest_*.py ──► legal/**
|
||||||
|
depot: legislation ──► ingest_legislation.py ──► legislation/**
|
||||||
(committed as-is) (one pass) (OKF markdown + YAML)
|
(committed as-is) (one pass) (OKF markdown + YAML)
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -41,6 +43,7 @@ data/legal/raw ──► scripts/ingest_*.py ──► legal/**
|
|||||||
```bash
|
```bash
|
||||||
make build # regenerate the entity tree from the raw exports
|
make build # regenerate the entity tree from the raw exports
|
||||||
make legal-us-code # regenerate the U.S. Code legal corpus (TITLES="1 2 ..." overridable)
|
make legal-us-code # regenerate the U.S. Code legal corpus (TITLES="1 2 ..." overridable)
|
||||||
|
make legislation # ingest enacted state legislation from the depot export (LEG_STATES="fl ca" overridable)
|
||||||
make validate # OKF + schema + link-integrity checks on all records
|
make validate # OKF + schema + link-integrity checks on all records
|
||||||
make changelog # what changed in the government since the last commit
|
make changelog # what changed in the government since the last commit
|
||||||
make board # rebuild the Board's inlined data from the tree
|
make board # rebuild the Board's inlined data from the tree
|
||||||
@@ -55,6 +58,7 @@ Everything runs on the Python standard library. No dependencies to install.
|
|||||||
- [Data Model](docs/data-model.md) — entity types, IDs, the file format, the jurisdiction tree
|
- [Data Model](docs/data-model.md) — entity types, IDs, the file format, the jurisdiction tree
|
||||||
- [Sources](docs/sources.md) — where every fact comes from, and how current it is
|
- [Sources](docs/sources.md) — where every fact comes from, and how current it is
|
||||||
- [Legal Corpus](docs/legal-corpus.md) — deterministic legal text mirroring, raw snapshots, manifests, and checksums
|
- [Legal Corpus](docs/legal-corpus.md) — deterministic legal text mirroring, raw snapshots, manifests, and checksums
|
||||||
|
- [Legislation Corpus](docs/legislation.md) — enacted state law across all 50 states, bill-as-pull-request, raw off-repo on the depot
|
||||||
- [The Board](docs/board.md) — the county map view, how its data is rebuilt, and how to redeploy the Artifact
|
- [The Board](docs/board.md) — the county map view, how its data is rebuilt, and how to redeploy the Artifact
|
||||||
- [Roadmap](docs/roadmap.md) — what exists, what's next
|
- [Roadmap](docs/roadmap.md) — what exists, what's next
|
||||||
- [Contributing](docs/contributing.md) — the disciplines, and how to add a source
|
- [Contributing](docs/contributing.md) — the disciplines, and how to add a source
|
||||||
|
|||||||
@@ -0,0 +1,47 @@
|
|||||||
|
# The Legislation Corpus
|
||||||
|
|
||||||
|
Enacted state legislation, all 50 states — the *merged pull requests* of
|
||||||
|
American law, where the U.S. Code corpus is the deployed branch.
|
||||||
|
|
||||||
|
## What's here
|
||||||
|
|
||||||
|
197,131 records under `legislation/us/states/{st}/{session}/`, one file per
|
||||||
|
enacted bill or resolution, from the OpenStates bulk snapshot vintage
|
||||||
|
**2026-07-01**. Each record carries the bill's version chain (the diff),
|
||||||
|
action timeline (the PR events), sponsors (the authors), roll-call votes (the
|
||||||
|
merge decisions), and a `source_hash` over the raw record.
|
||||||
|
|
||||||
|
**"Enacted" includes joint and concurrent resolutions**, not only bills — the
|
||||||
|
`classification` frontmatter field records which. Some states resolve
|
||||||
|
prolifically (Tennessee's 22,173 enacted = 8,785 bills + 13,388 joint
|
||||||
|
resolutions); the mirror records what the legislature enacted and leaves
|
||||||
|
filtering to views.
|
||||||
|
|
||||||
|
A bill is a pull request against the law: filed, revised through committee
|
||||||
|
substitutes (the commits), voted on (the reviews), and — if it passes and is
|
||||||
|
signed — merged. Enacted bills are the ones that merged.
|
||||||
|
|
||||||
|
## Provenance
|
||||||
|
|
||||||
|
- Source: OpenStates bulk snapshot (`data.openstates.org`), aggregated from
|
||||||
|
official legislature sites. Confidence: `reported`.
|
||||||
|
- Raw JSONL (~1.1 GB, 50 files) lives off-repo on the depot at
|
||||||
|
`/exports/legislation/2026-07-01/`; the committed anchor is
|
||||||
|
[`data/sources/legislation_manifest_2026-07-01.json`](../data/sources/legislation_manifest_2026-07-01.json)
|
||||||
|
with per-state SHA-256, byte counts, and record counts.
|
||||||
|
- Rebuild: `make legislation` (requires the depot mount; `LEG_STATES="fl ca"`
|
||||||
|
and `LEG_VINTAGE=` overridable). Deterministic — rerun is byte-identical.
|
||||||
|
|
||||||
|
## Next vintages
|
||||||
|
|
||||||
|
A future OpenStates snapshot re-exported over this tree surfaces *only real
|
||||||
|
legislative change* as diffs — new bills, new actions, new enactments. The
|
||||||
|
difference between two vintages is the changelog of American state law.
|
||||||
|
|
||||||
|
## Not yet
|
||||||
|
|
||||||
|
No bill → sponsor edges to `Person` nodes yet — that needs name reconciliation
|
||||||
|
(the OpenStates `person_id` is not the repo's identity), a follow-on axis like
|
||||||
|
the committee-seat and authority edges. And no bill *text*: the version chain
|
||||||
|
records the diff points and links to each version's source document, but the
|
||||||
|
statutory text itself is not mirrored in this pass.
|
||||||
+2
-1
@@ -5,13 +5,14 @@ An honest account of what exists, what's next, and what's further out. This repo
|
|||||||
## Done
|
## Done
|
||||||
|
|
||||||
- **The substrate** — repo-canonical, OKF markdown + YAML frontmatter, JSON-Schema validated, deterministic build (byte-identical on rerun).
|
- **The substrate** — repo-canonical, OKF markdown + YAML frontmatter, JSON-Schema validated, deterministic build (byte-identical on rerun).
|
||||||
- **Five entity types, 105,743 files** — Person (13,329), Body (275), Candidate (2,494), Jurisdiction (29,905), LegalText (59,740).
|
- **Six entity types, 302,874 files** — Person (13,329), Body (275), Candidate (2,494), Jurisdiction (29,905), LegalText (59,740), Bill (197,131).
|
||||||
- **The federal government, structurally** — every member of Congress, the chambers, 49 committees and 181 subcommittees with their leadership, and every member's committee seats.
|
- **The federal government, structurally** — every member of Congress, the chambers, 49 committees and 181 subcommittees with their leadership, and every member's committee seats.
|
||||||
- **The nationwide county skeleton** — all 3,131 U.S. counties as demographic nodes, plus all 435 congressional districts (Census ACS demographics on 363).
|
- **The nationwide county skeleton** — all 3,131 U.S. counties as demographic nodes, plus all 435 congressional districts (Census ACS demographics on 363).
|
||||||
- **The authority layer** — every state legislative district (4,927 house, 1,897 senate) linked county↔district↔legislator, nationwide.
|
- **The authority layer** — every state legislative district (4,927 house, 1,897 senate) linked county↔district↔legislator, nationwide.
|
||||||
- **The municipal layer** — every incorporated place in the country (19,513 cities, towns, villages, boroughs) as a first-class Jurisdiction node keyed by Census place GEOID, modelled like a county (place + government fused). 2,229 are filled with their officeholders today (2,217 GEOID-resolved off a 98.8% name→GEOID join); the rest are truthful skeleton stubs — "governed, not yet mirrored" — waiting for depth, the same discipline as the county skeleton.
|
- **The municipal layer** — every incorporated place in the country (19,513 cities, towns, villages, boroughs) as a first-class Jurisdiction node keyed by Census place GEOID, modelled like a county (place + government fused). 2,229 are filled with their officeholders today (2,217 GEOID-resolved off a 98.8% name→GEOID join); the rest are truthful skeleton stubs — "governed, not yet mirrored" — waiting for depth, the same discipline as the county skeleton.
|
||||||
- **The complete U.S. Code** — every section of all 53 populated titles (Title 53 is reserved), 59,740 per-section records, mirrored from the official OLRC USLM XML at release point 119-100, uniform edition end to end, with manifests and checksums.
|
- **The complete U.S. Code** — every section of all 53 populated titles (Title 53 is reserved), 59,740 per-section records, mirrored from the official OLRC USLM XML at release point 119-100, uniform edition end to end, with manifests and checksums.
|
||||||
- **The authority axis (v1)** — the executive branch minted as nodes from the Code's own enumerations (15 Cabinet departments from 5 U.S.C. § 101, 21 Level I offices from § 5312), and **27,804 edges** linking 17,031 sections to the offices they empower. `scripts/extract_authority.py` → `data/section_authority_edges.jsonl` (a committed input, like the district edges); reciprocal "empowered by" links render on each office node. Named references only for now (cabinet-level); relative refs ("the Secretary") and the agency long tail are next.
|
- **The authority axis (v1)** — the executive branch minted as nodes from the Code's own enumerations (15 Cabinet departments from 5 U.S.C. § 101, 21 Level I offices from § 5312), and **27,804 edges** linking 17,031 sections to the offices they empower. `scripts/extract_authority.py` → `data/section_authority_edges.jsonl` (a committed input, like the district edges); reciprocal "empowered by" links render on each office node. Named references only for now (cabinet-level); relative refs ("the Secretary") and the agency long tail are next.
|
||||||
|
- **The legislation corpus** — 197,131 enacted bills and resolutions from all 50 states (OpenStates vintage 2026-07-01) as `Bill` records with version chains, sponsors, votes, and action timelines. Raw JSONL off-repo on the depot; the per-state SHA-256 manifest committed as the provenance anchor. The state-law "merged PRs" beside the U.S. Code's "deployed branch"; the next vintage's diff is the changelog of American state law.
|
||||||
- **The 2026 federal candidate field** — every FEC filing as a Candidate.
|
- **The 2026 federal candidate field** — every FEC filing as a Candidate.
|
||||||
- **Florida to the ground** — officeholders down to the municipal level across all 67 counties.
|
- **Florida to the ground** — officeholders down to the municipal level across all 67 counties.
|
||||||
- **The pipeline** — `make build` / `validate` / `changelog` / `check`, standard-library only.
|
- **The pipeline** — `make build` / `validate` / `changelog` / `check`, standard-library only.
|
||||||
|
|||||||
Reference in New Issue
Block a user