OlmoEarth 1.3 Base
4 configurations testedQuality criteria
{ "frozenBeforeEmbeddings": true, "task": "same-acquisition patch embedding retrieval", "reference": "ESA Sentinel-2 L2A SCL majority class, nearest-resampled independently from the input bands", "classIds": { "water": 6, "vegetation": 4 }, "minimumLabelPurity": 0.7, "training": [ "water-train", "vegetation-train" ], "heldOut": [ "water-test", "vegetation-test" ], "metric": "cosine nearest prototype on spatial-mean student_location_tokens", "requiredAccuracy": 1, "requiredCpuMpsCosine": 0.999, "limitations": "Two test patches from one Seattle acquisition, not independent regional/year/domain validation or a land-cover classifier. Training/test windows are spatially non-overlapping. Broader retrieval quality and other sensor modes remain unmeasured." }
| Precision | Device | Result | |
|---|---|---|---|
FP32 | CPU · WASM | Passed | Very fast |
FP32 | CPU · WASM | Passed | Very fast |
FP32 | CPU · WASM | Passed | Very fast |
FP32 | CPU · WASM | Passed | Very fast |
Test setup & sample details
- Quality gate
- { "frozenBeforeEmbeddings": true, "task": "same-acquisition patch embedding retrieval", "reference": "ESA Sentinel-2 L2A SCL majority class, nearest-resampled independently from the input bands", "classIds": { "water": 6, "vegetation": 4 }, "minimumLabelPurity": 0.7, "training": [ "water-train", "vegetation-train" ], "heldOut": [ "water-test", "vegetation-test" ], "metric": "cosine nearest prototype on spatial-mean student_location_tokens", "requiredAccuracy": 1, "requiredCpuMpsCosine": 0.999, "limitations": "Two test patches from one Seattle acquisition, not independent regional/year/domain validation or a land-cover classifier. Training/test windows are spatially non-overlapping. Broader retrieval quality and other sensor modes remain unmeasured." }
- Tested
- 2026-11-16T17:44:31.446Z
- Environment
- Apple M3 Max/macOS27.2; exact pinned runtime; existing mirror active. Public API timings are scoped observations; retained standalone quality/speed evidence keeps original timestamps.
- Runtime
- onnxruntime-web 1.31.0-dev.20260914-8d85527a0
- Artifact revision
- 30ea64d178d08192578a5ba6dd0eeda93b29bec1
- Fixture
{ "file": "model-samples/olmoearth-v1-3/water-test-browser-input.json", "size": 53465, "sha1": "8b6b9b3103bb6d00dd74231fb365e6f234066732", "sha256": "513eea1bcc3af5aedb990b780b140f13cfc8f44b421fb288b6aa489d4c2eb336", "url": "https://assets.localai.tools/samples/v1/files/513eea1bcc3af5aedb990b780b140f13cfc8f44b421fb288b6aa489d4c2eb336/water-test-browser-input.json" }- Rerun when
- artifact revision; runtime version; normalization; fixture checksum; quality gate; backend behavior
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "wasm",
"status": "pass",
"quality": {
"passed": true,
"metric": "finite prototype embedding",
"score": 1,
"scope": "Reference prototype construction; not a held-out prediction."
},
"runtime": {
"durationMs": 42.625,
"loadMs": 4678.80000000447,
"inferenceMs": 42.75500000268221,
"version": "1.31.0-dev.20260914-8d85527a0",
"engine": "onnxruntime-web",
"artifactRevision": "30ea64d178d08192578a5ba6dd0eeda93b29bec1",
"grade": "good",
"totalMs": 4721.555000007153,
"pixelsPerSecond": 23950.415154619575,
"actualDevice": "wasm",
"actualPrecision": "fp32",
"neuralExecuted": true
}
},
{
"precision": "fp32",
"device": "wasm",
"status": "pass",
"quality": {
"passed": true,
"metric": "finite prototype embedding",
"score": 1,
"scope": "Reference prototype construction; not a held-out prediction."
},
"runtime": {
"durationMs": 22.42000000178814,
"loadMs": 4678.80000000447,
"inferenceMs": 22.44999999552965,
"version": "1.31.0-dev.20260914-8d85527a0",
"engine": "onnxruntime-web",
"artifactRevision": "30ea64d178d08192578a5ba6dd0eeda93b29bec1",
"grade": "good",
"totalMs": 4701.25,
"pixelsPerSecond": 45612.472169438915,
"actualDevice": "wasm",
"actualPrecision": "fp32",
"neuralExecuted": true
}
},
{
"precision": "fp32",
"device": "wasm",
"status": "pass",
"quality": {
"passed": true,
"metric": "held-out same-acquisition retrieval accuracy",
"score": 1,
"scope": "Narrow two-class same-acquisition suite; exact native output parity separately passed. Training prototypes are reference construction, not independent held-out cases."
},
"runtime": {
"durationMs": 19.814999997615814,
"loadMs": 4678.80000000447,
"inferenceMs": 19.83500000089407,
"version": "1.31.0-dev.20260914-8d85527a0",
"engine": "onnxruntime-web",
"artifactRevision": "30ea64d178d08192578a5ba6dd0eeda93b29bec1",
"grade": "good",
"totalMs": 4698.635000005364,
"pixelsPerSecond": 51625.913786430196,
"actualDevice": "wasm",
"actualPrecision": "fp32",
"neuralExecuted": true
}
},
{
"precision": "fp32",
"device": "wasm",
"status": "pass",
"quality": {
"passed": true,
"metric": "held-out same-acquisition retrieval accuracy",
"score": 1,
"scope": "Narrow two-class same-acquisition suite; exact native output parity separately passed. Training prototypes are reference construction, not independent held-out cases."
},
"runtime": {
"durationMs": 22.28999999910593,
"loadMs": 4678.80000000447,
"inferenceMs": 22.309999994933605,
"version": "1.31.0-dev.20260914-8d85527a0",
"engine": "onnxruntime-web",
"artifactRevision": "30ea64d178d08192578a5ba6dd0eeda93b29bec1",
"grade": "good",
"totalMs": 4701.109999999404,
"pixelsPerSecond": 45898.70014489202,
"actualDevice": "wasm",
"actualPrecision": "fp32",
"neuralExecuted": true
}
}
]