Market surveillance: the rule asks for every guarantee

In everyday terms

A fake order to move the price.

Picture an auction where someone shouts a huge bid just so others think the item is worth more, then backs out before the hammer falls. In markets this has a name: spoofing. The trader places a large order they do not intend to execute, the price moves, they trade on the other side and cancel the fake order.

To catch this afterwards, the regulator needs three things: a record of every order, including cancelled ones; certainty that the record was not tampered with; and tools that sweep years of that record looking for patterns.

In the US this is written into the Consolidated Audit Trail rule (SEC Rule 613); in Europe, into MiFID II and the Market Abuse Regulation. They are not recommendations. They are mandates.

The order book over a few seconds. The large buy order appears, pulls the price up, the manipulator sells on the other side and the order vanishes before executing. Only the full record, including the cancellation, reveals the play.

The map

One requirement, one piece.

The same order flow feeds all five pieces at once. None of them replaces the bank's or broker's surveillance system: they sit underneath it, as the layer that proves what the system above claims.

The rule asks for	Piece	The guarantee, in one sentence
Tamper-proof archive	GIRDER	changing any record breaks the signature, and anyone can check
Detect every abuse	SIEVE	no order matching the pattern is left out
Withstand people trying to fool it	TRUSS	the count stays right even under attack
Keep 5 to 7 years	TUBE	smaller file, with a tolerance guaranteed on every value
Prove execution quality	CLAMP	the answer comes as a range certain to contain the truth

One flow, five guarantees. All five match the requirements of the rule.

The adversary adapts

The manipulator reads the manual.

Surveillance is not statistics on a calm day. Spoofers design their orders precisely so as not to trip detection. A counter that works well on honest data can be fooled by someone who knows how it counts.

We showed this in practice in TRUSS: a popular distinct-item counter, the one in Redis, was led to count 30,000 times fewer items than it should by a sequence built to fool it. The defence is simple to explain: a secret key per installation, which the attacker does not know and so cannot aim at.

In markets this means: the metric that detects abuse cannot be gamed by the abuser.

The same counter, two inputs. With ordinary data, it is right. With a sequence built by someone who knows its rule, it counts almost nothing. With a secret key, the crafted sequence stops working. Illustrative bars: in the measured case the count fell 30,000-fold, too small to show.

What we have measured

A whole day of trades, proven.

To measure the keeping and answering part, we used a real day of bitcoin trading (open data, no key) as a stand-in for equity data, which is paid and has the same structure: 638,484 trades.

TUBE stored the day 50 times smaller, with prices exact to the cent. It tied the best scientific compressor we use as a reference (SZ3) and came out 2.9 times smaller than gzip.

CLAMP answered "how many trades left the ±0.5% band around the open?" with the exact answer, 795, reading only 3 of 2,130 blocks (0.14% of the file). And the record was sealed in GIRDER, with its time certified by PLUMB.

Open the trades demo →

Three words from this page

Spoofing

Placing a large fake order to move the price and cancelling it before it executes.

Retention

How long the record must be kept and available to the regulator: here, 5 to 7 years.

Execution quality

Proving the client got the best reasonable price at the time of the trade.

Limits

Where this could be wrong.

It is a thesis

Each piece is built and measured in its own field, but the combination of all five has not been run on equity data. We call it a thesis until measured.

There are strong incumbents

Trading surveillance is an established market (Nasdaq SMARTS, NICE Actimize, SteelEye, Eventus). The realistic entry is as a proof layer underneath them, not as a replacement.

Bitcoin is not a stock

The measured day is crypto, used as a stand-in. The data structure is the same, but volume, trading hours and order types differ.

Complete search is not the fastest

In memory, approximate search is faster than SIEVE. Its value is proving nothing was left out and working with years of data on disk, not speed.