From a document landing to a signal surviving.

Continuous polling of primary sources across jurisdictions, plus exchange and settlement feeds. Several terabytes a day arrive as documents rather than as rows: filings, disclosures, amendments, withdrawals. Ingest is idempotent and every artefact is stored as received, because a re-parse of the original is the only way to correct an extraction error six months later.

Fails as: a source changes its response shape and the parser returns empty rather than raising. The pipeline reports success and the dataset quietly stops growing. Every collector asserts it reached the end of the range rather than trusting a row count.

Documents are machine-read into typed fields with the position of each value retained, so a figure can be traced to the page it came from. Values are asserted against expected domains — a percentage that arrives as text, a flag encoded as a letter rather than a boolean, a date in an unexpected locale.

Fails as: an encoding assumption. A comparison against the wrong type returns zero rows and the dataset appears empty, which reads as an absence of signal rather than as a bug. We assert on observed values, never on the shape we expect.

The same operator appears under several legal names, several registration numbers and several spellings across registers. Resolution runs on deterministic identifiers first, then on blocked probabilistic matching over name, address and officer overlap, with every merge carrying its evidence and a confidence.

Fails as: over-merging, which is worse than under-merging. One entity split across three identifiers inflates apparent breadth, and breadth enters the fundamental law under a square root — so a resolution error propagates directly into an overstated information ratio.

Every record carries the date it was observed and the date it became true, so any universe can be rebuilt as it stood on a past date. Entities that later ceased to exist stay in the reconstruction.

Fails as: look-ahead. A field populated after the decision date, joined as though it had been available at the time. It is the commonest cause of a backtest that cannot be reproduced live, and it is invisible unless availability is checked per field rather than per table.

Candidate structures are found by scanning across the resolved graph for co-occurrence that exceeds its base rate: entities transacting in unusual sequence, control changes clustering in a sector, formation activity concentrating in a band. Compute matters here because the search is over the full universe rather than a sample, and because a candidate is only interesting once it has been tested against the rate it would appear at by chance.

Fails as: multiplicity. Enough scans over a corpus this size will surface structure by construction. Families of tests are declared before the search and controlled by false-discovery rate, which is the only defence that survives contact with a large dataset.

Surviving candidates are estimated with intervals rather than points, tested out-of-sample on holdout periods fixed before tuning, and instrumented for decay from the day they go live. Capacity is estimated alongside the effect: turnover, participation, and the liquidity of the names selected.

Fails as: a single in-sample fit reported as a result. A strategy tuned until it looks good on history has been fitted to the history and will decay on contact with the present.