SigLIP 2 Base · 224
4 configurations testedQuality criteria
Finite scores in [0,1], unique labels, expected person label ranked first, and its score is at least 10 times the strongest negative.
| Precision | Device | Result | |
|---|---|---|---|
Q8 | GPU · WebGPUAccelerated | Passed | Moderate |
Q8 | CPU · WASM | Passed | Moderate |
FP16 | CPU · WASM | Failed | — |
FP16 | GPU · WebGPU | Failed | — |
Test setup & sample details
- Quality gate
- Finite scores in [0,1], unique labels, expected person label ranked first, and its score is at least 10 times the strongest negative.
- Tested
- 2026-11-04T16:56:20.968Z
- Environment
- Not recorded
- Runtime
- Transformers.js 4.2.0
- Artifact revision
- ba1f3b0843f24bc5417d38e19c37b287d719b2f4
- Fixture
{ "file": "modnet-portrait-pexels-5965592.jpg", "url": "https://f005.backblazeb2.com/file/hubters-webai-public/assets/benchmarks/modnet-portrait-pexels-5965592.jpg", "size": 78124, "sha1": "38212739a4a4f0be25d84863cf991d7fb6c6d818", "width": 1024, "height": 683, "license": "Pexels License", "labels": { "file": "siglip-base-patch16-224.json", "url": "https://f005.backblazeb2.com/file/hubters-webai-public/assets/benchmarks/siglip-base-patch16-224.json", "size": 460, "sha1": "fecc3379e545c31c5fc0dbcd8ca8777189f661e1" } }- Rerun when
- artifact or tokenizer revision change; runtime version change; fixture, labels, or quality-gate change; browser/backend behavior change
Per-case quality & failure details
[
{
"precision": "q8",
"device": "wasm",
"status": "pass",
"quality": {
"finite": true,
"unique": true,
"topLabel": "a portrait photo of a person",
"topToNegativeRatio": 24.848950108462567,
"passed": true,
"visualization": true
},
"runtime": {
"loadMs": 2752.0999999940395,
"inferenceMs": 1528.2000000178814,
"totalMs": 5549.9000000059605,
"grade": "good",
"imagesPerSecond": 0.6543646119541284,
"artifactRevision": "ba1f3b0843f24bc5417d38e19c37b287d719b2f4",
"engine": {
"name": "Transformers.js",
"version": "4.2.0"
}
}
},
{
"precision": "q8",
"device": "webgpu",
"status": "pass",
"quality": {
"finite": true,
"unique": true,
"topLabel": "a portrait photo of a person",
"topToNegativeRatio": 63.35618606026562,
"passed": true,
"visualization": true
},
"runtime": {
"loadMs": 2785.199999988079,
"inferenceMs": 1733.699999988079,
"totalMs": 5857.9000000059605,
"grade": "good",
"imagesPerSecond": 0.5768010613179189,
"artifactRevision": "ba1f3b0843f24bc5417d38e19c37b287d719b2f4",
"engine": {
"name": "Transformers.js",
"version": "4.2.0"
}
}
},
{
"precision": "fp16",
"device": "wasm",
"status": "fail",
"quality": "Not embedded",
"runtime": {
"loadMs": null,
"inferenceMs": null,
"totalMs": 4723,
"grade": "failed"
},
"failure": "Can't create a session. ERROR_CODE: 1, ERROR_MESSAGE: /mnt/vss/_work/1/s/onnxruntime/core/graph/graph_utils.cc:30 int onnxruntime::graph_utils::GetIndexFromName(const Node &, const std::string &, bool) itr != node_args.end() was false. Attempting to get index by a name which does not exist:InsertedPrecisionFreeCast_/vision_model/encoder/layers.0/layer_norm1/Constant_output_0for node: /vision_model/encoder/layers.0/layer_norm1/Mul/SimplifiedLayerNormFusion/\n"
},
{
"precision": "fp16",
"device": "webgpu",
"status": "fail",
"quality": {
"finite": true,
"unique": true,
"topLabel": "a portrait photo of a person",
"topToNegativeRatio": 4.696770068504018,
"passed": false,
"visualization": true
},
"runtime": {
"loadMs": 3273.2999999821186,
"inferenceMs": 1148.699999988079,
"totalMs": 5907.100000023842,
"grade": "good",
"imagesPerSecond": 0.8705493166278208,
"artifactRevision": "ba1f3b0843f24bc5417d38e19c37b287d719b2f4",
"engine": {
"name": "Transformers.js",
"version": "4.2.0"
}
},
"failure": "Quality or 15-second interactive usability gate failed"
}
]