GRACE-FO · satellite C/10 Jun 2024/accelerometer, 1 Hz/worst 20-min window of the day
error of each reading 0ε
error in velocity 0ε
error in position 0ε

Every reading is within spec.
The sum is not.

It is the same error, measured three times. It never breaks the per-sample margin — and it still walks out to 353 thousand times that margin once someone integrates twice, which is exactly what a flight computer does to know where it is.

m/s²

Why the per-sample error misleads

Error-bounded compression promises one thing per reading: the value you get back is within ε of the original. That promise holds — in the green trace at the top, the error touches 1.00 ε and never crosses it.

The trouble is that the error is not random. The technique holds a constant value while the signal drifts away slowly, so each reading’s small error leans the same way for long stretches. Noise cancels when you add it up; bias accumulates.

We measured how much of it is bias rather than magnitude: we flipped a coin for the sign of each small error, keeping its size exactly the same. If size were the problem, nothing would change.

8.4–17.7×worse for anyone integrating than white
noise of the same amplitude

What follows in practice: “I compressed within the sensor’s noise, so it is harmless” is a false statement for any customer who adds readings up — and almost every customer does. Fuel, energy, emissions, distance, dose, volume.

m

The guarantee starts holding in metres

We rewrote the compressor to promise on the reconstructed quantity too — position and velocity — not just on the isolated reading. Two pieces, and the second one is what solved it.

piece 1

Plan instead of react

The compressor used to decide when to record on the spot. Now it decides ahead, placing its foothold in the middle of the movement instead of chasing it. Each recording lasts far longer.

4× less space

piece 2

The periodic receipt

Every so often the compressor measures the deviation it accumulated and writes it down — a few bytes. Whoever reads applies the receipts and the deviation stops compounding. The more frequent the receipt, the tighter the guarantee.

a dial, not a redesign

The comparison below is of the same aggregate, on the same real data: 48 stations, 166 thousand samples. The width is how much the answer can vary — the smaller, the sharper the certificate.

worst case of εwhat an ordinary query returns2.5857%prefix boundconstraining the compressor · +37% bytes0.0092%anchor, absolute valuethe published state of the art0.2109%anchor, signedmeasure the deviation and keep the SIGN0.0003%

The signed anchor is 27× tighter than constraining the compressor, using 20% less space. The only difference from the published state of the art is the sign: keeping the absolute value lets you bound, and errors add up; keeping the sign lets you correct, and they cancel.

µg

Does this matter next to the error the sensor already makes?

Before building anything, we wrote down the condition under which the idea would be dead: if our contribution were far smaller than the instrument’s own drift, there would be no product. The test reduces both to the same effect, in the unit of the datasheet.

the sensor's own drift = 1.0×15.3×10.4×2.57×0.79×0.019×1 min10 min1 h6 h24 h

In orbit the verdict changes with the horizon, and that was the surprise. On a drone we dominated at every duration; on a satellite, past ~6 h our contribution falls below the sensor’s own drift. For long-horizon products the compression error is negligible; for short horizons it rules, and that is where the receipt matters.

arcsec

Attitude: from quaternion component to arcsecond

Nobody contracts “0.001 on the k component”. They contract arcseconds of pointing. The bridge is a hard bound, not an estimate.

|Δq| ≤ ε on all 4 components ⇒ θ ≤ 2·arctan(2ε)

the worst case is attained, not estimated — which is why it can be signed

contract

0.362"

equal to the star camera’s own formal error

worst real error

0.347"

the bound is only 3.5% loose

space

14×

smaller than the same data lossless

And there is a finding that changes the recommendation: the mission’s gyroscope product delivers an already-integrated angle, not a rate. Archiving that angle is strictly better than archiving the quaternion — 3× fewer bytes with a 2× tighter contract. They do not substitute for each other, because the gyro angle drifts on its own and the quaternion is the corrected attitude; the right design is to keep the cheap angle and anchor it periodically with the star camera.

bytes

You pick the point on the curve

A full day of the GRACE-FO accelerometer, 86,400 samples. The horizontal axis is the guaranteed position error over one hour; the vertical, how many times smaller than the same data compressed losslessly.

41.6×6.1 µm90.2×0.15 mm205×2.7 mm500×17 mm← tighter guaranteemore compression →

Even at the most conservative point — a guaranteed error of 6 micrometres over one hour — the file is 41.6× smaller than the lossless equivalent. And the guarantee is not our assertion: the result is re-executed in exact arithmetic and checked against all three limits before being reported.

What we still do not know

  • The data does not hand you an ε. On GRACE-FO, two ways of estimating the tolerance from the data itself err in opposite directions — one measures the filter, the other measures what the filter removed. The tolerance has to come from a document.
  • Filtered queries barely benefit from the receipt. We measured why: out of 10 thousand readings, the 61 sitting at the threshold account for 78% of the uncertainty. The uncertainty is not in the values, it is in who counts — and a receipt about values does not touch that.
  • The pointing requirement we found is the wrong category. It is a control requirement — how well the spacecraft points — not knowledge, and not archive fidelity. Using it as a tolerance would be a pretty, invalid number.
  • We publish the refutations with equal billing. Two of our own ideas fell along the way: a more detailed receipt per value range (3 to 15× more space, almost no gain) and baking the correction into the samples themselves (it works, but delivers no guarantee).

Bring your own ε — or find out what it should be.

Every experiment on this page has its code committed next to the result, and runs on public data. Including the ones that refuted us.

Get in touch