Qwen3 Reranker 0.6B
51 configurations testedQuality criteria
{ "version": 1, "expectedTop1Accuracy": 1, "minimumPositiveLogitMargin": 0.5, "maximumCpuScoreDifference": 0.08, "allScoresFinite": true, "minimumUsablePairsPerSecond": 0.25, "goodPairsPerSecond": 2, "scope": "Four short authored factual/code/negation/Chinese retrieval queries with hard negatives, plus all-irrelevant stability; not broad retrieval, multilingual or safety certification. All complete rankings retained; no calibrated relevance threshold or unknown rejection.", "browserMaximumTokens": 2048 }
| Precision | Device | Result | |
|---|---|---|---|
FP16 | GPU · WebGPUAccelerated | Passed | Fast |
FP16 | GPU · WebGPUAccelerated | Passed | Fast |
FP16 | GPU · WebGPUAccelerated | Passed | Fast |
FP16 | GPU · WebGPUAccelerated | Passed | Fast |
FP16 | GPU · WebGPUAccelerated | Passed | Fast |
FP16 | GPU · WebGPUAccelerated | Passed | Fast |
FP16 | GPU · WebGPUAccelerated | Passed | Fast |
FP16 | GPU · WebGPUAccelerated | Passed | Fast |
FP16 | GPU · WebGPUAccelerated | Passed | Fast |
Q4 | GPU · WebGPUAccelerated | Passed | Fast |
Q4 | GPU · WebGPUAccelerated | Passed | Fast |
FP16 | GPU · WebGPUAccelerated | Passed | Fast |
Q8 | GPU · WebGPUAccelerated | Passed | Slow |
Q8 | GPU · WebGPUAccelerated | Passed | Slow |
Q8 | GPU · WebGPUAccelerated | Passed | Slow |
Q8 | CPU · WASM | Passed | Moderate |
Q8 | CPU · WASM | Passed | Moderate |
Q8 | CPU · WASM | Passed | Moderate |
Q4 | CPU · WASM | Passed | Slow |
Q4 | CPU · WASM | Passed | Slow |
FP16 | CPU · WASM | Failed | — |
Q4 | CPU · WASM | Failed | — |
Q4 | CPU · WASM | Failed | — |
Q4 | CPU · WASM | Failed | — |
Q4 | CPU · WASM | Failed | — |
Q4 | CPU · WASM | Failed | — |
Q4 | CPU · WASM | Failed | — |
Q4 | CPU · WASM | Failed | — |
Q4 | CPU · WASM | Failed | — |
Q4 | GPU · WebGPU | Failed | — |
Q4 | GPU · WebGPU | Failed | — |
Q4 | GPU · WebGPU | Failed | — |
Q4 | GPU · WebGPU | Failed | — |
Q4 | GPU · WebGPU | Failed | — |
Q4 | GPU · WebGPU | Failed | — |
Q4 | GPU · WebGPU | Failed | — |
Q4 | GPU · WebGPU | Failed | — |
Q8 | CPU · WASM | Failed | — |
Q8 | CPU · WASM | Failed | — |
Q8 | CPU · WASM | Failed | — |
Q8 | CPU · WASM | Failed | — |
Q8 | CPU · WASM | Failed | — |
Q8 | CPU · WASM | Failed | — |
Q8 | CPU · WASM | Failed | — |
Q8 | GPU · WebGPU | Failed | — |
Q8 | GPU · WebGPU | Failed | — |
Q8 | GPU · WebGPU | Failed | — |
Q8 | GPU · WebGPU | Failed | — |
Q8 | GPU · WebGPU | Failed | — |
Q8 | GPU · WebGPU | Failed | — |
Q8 | GPU · WebGPU | Failed | — |
Test setup & sample details
- Quality gate
- { "version": 1, "expectedTop1Accuracy": 1, "minimumPositiveLogitMargin": 0.5, "maximumCpuScoreDifference": 0.08, "allScoresFinite": true, "minimumUsablePairsPerSecond": 0.25, "goodPairsPerSecond": 2, "scope": "Four short authored factual/code/negation/Chinese retrieval queries with hard negatives, plus all-irrelevant stability; not broad retrieval, multilingual or safety certification. All complete rankings retained; no calibrated relevance threshold or unknown rejection.", "browserMaximumTokens": 2048 }
- Tested
- 2026-11-02T18:09:59.530Z
- Environment
- { "hardware": "Apple M3 Max", "os": "macOS27.2", "runtime": "Transformers.js4.2.0", "scope": "Isolated Chrome module workers; pinned local mirror loads" }
- Runtime
- ONNX Runtime Web + Transformers.js tokenizer 1.26.0-dev.20260416-b7804b056c / 4.2.0
- Artifact revision
- 21fb0d00576b976132dbe2acb651dd9639881ecb
- Fixture
{ "file": "model-samples/qwen3-reranker-capital.json", "url": "https://assets.localai.tools/samples/v1/files/fcb30f2647fd37f102ebd90d61b0fd0bcb050ce208ff9caef38f79144fff9358/qwen3-reranker-capital.json", "size": 356, "sha1": "202d7afe4b252011e63de03681b2849227d4a46b", "sha256": "fcb30f2647fd37f102ebd90d61b0fd0bcb050ce208ff9caef38f79144fff9358" }- Rerun when
- Exact original/ONNX artifact revision or tokenizer; Pinned runtime or relevant browser/backend/OS behavior; Fixture bytes/expected ranking or quality gate; Prompt/truncation/yes-no scoring/preprocessing
Per-case quality & failure details
[
{
"precision": "fp16",
"device": "wasm",
"status": "fail",
"quality": "Not embedded",
"runtime": {
"loadMs": null,
"inferenceMs": null,
"totalMs": 4142.109999999404,
"grade": "failed",
"engine": {
"name": "ONNX Runtime Web + Transformers.js tokenizer",
"version": "1.26.0-dev.20260416-b7804b056c / 4.2.0"
},
"acceleration": "cpu",
"accelerationDetails": "ONNX Runtime WebGPU with CPU-assigned shape operations reported by runtime; CPUtokenizer/ranking. Not full GPU delegation."
},
"failure": "Error: Can't create a session. ERROR_CODE: 6, ERROR_MESSAGE: std::bad_alloc"
},
{
"precision": "fp16",
"device": "webgpu",
"status": "pass",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 10.65625,
"cpuScoreDifference": 0.002395129930311657,
"passed": true
},
"runtime": {
"loadMs": 5080.954999998212,
"inferenceMs": 314.9150000065565,
"totalMs": 5395.870000004768,
"pairsPerSecond": 9.526380134123622,
"grade": "good",
"engine": {
"name": "ONNX Runtime Web + Transformers.js tokenizer",
"version": "1.26.0-dev.20260416-b7804b056c / 4.2.0"
},
"acceleration": "partial",
"accelerationDetails": "ONNX Runtime WebGPU with CPU-assigned shape operations reported by runtime; CPUtokenizer/ranking. Not full GPU delegation."
}
},
{
"precision": "fp16",
"device": "webgpu",
"status": "pass",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 2.8671875,
"cpuScoreDifference": 0.0036258450810306986,
"passed": true
},
"runtime": {
"loadMs": 0,
"inferenceMs": 196.82999999821186,
"totalMs": 196.82999999821186,
"pairsPerSecond": 15.241579027725724,
"grade": "excellent",
"engine": {
"name": "ONNX Runtime Web + Transformers.js tokenizer",
"version": "1.26.0-dev.20260416-b7804b056c / 4.2.0"
},
"acceleration": "partial",
"accelerationDetails": "ONNX Runtime WebGPU with CPU-assigned shape operations reported by runtime; CPUtokenizer/ranking. Not full GPU delegation."
}
},
{
"precision": "fp16",
"device": "webgpu",
"status": "pass",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 2.7578125,
"cpuScoreDifference": 0.020679166305295538,
"passed": true
},
"runtime": {
"loadMs": 0,
"inferenceMs": 184.46500000357628,
"totalMs": 184.46500000357628,
"pairsPerSecond": 16.263247770264485,
"grade": "excellent",
"engine": {
"name": "ONNX Runtime Web + Transformers.js tokenizer",
"version": "1.26.0-dev.20260416-b7804b056c / 4.2.0"
},
"acceleration": "partial",
"accelerationDetails": "ONNX Runtime WebGPU with CPU-assigned shape operations reported by runtime; CPUtokenizer/ranking. Not full GPU delegation."
}
},
{
"precision": "fp16",
"device": "webgpu",
"status": "pass",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 12.515625,
"cpuScoreDifference": 0.0005681657456095028,
"passed": true
},
"runtime": {
"loadMs": 0,
"inferenceMs": 181.6650000065565,
"totalMs": 181.6650000065565,
"pairsPerSecond": 16.513912971082632,
"grade": "excellent",
"engine": {
"name": "ONNX Runtime Web + Transformers.js tokenizer",
"version": "1.26.0-dev.20260416-b7804b056c / 4.2.0"
},
"acceleration": "partial",
"accelerationDetails": "ONNX Runtime WebGPU with CPU-assigned shape operations reported by runtime; CPUtokenizer/ranking. Not full GPU delegation."
}
},
{
"precision": "fp16",
"device": "webgpu",
"status": "pass",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": null,
"positiveLogitMargin": null,
"cpuScoreDifference": 0.000009399821355189384,
"passed": true
},
"runtime": {
"loadMs": 0,
"inferenceMs": 174.60500000417233,
"totalMs": 174.60500000417233,
"pairsPerSecond": 17.18163855518635,
"grade": "excellent",
"engine": {
"name": "ONNX Runtime Web + Transformers.js tokenizer",
"version": "1.26.0-dev.20260416-b7804b056c / 4.2.0"
},
"acceleration": "partial",
"accelerationDetails": "ONNX Runtime WebGPU with CPU-assigned shape operations reported by runtime; CPUtokenizer/ranking. Not full GPU delegation."
}
},
{
"precision": "fp16",
"device": "webgpu",
"status": "pass",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 10.65625,
"cpuScoreDifference": 0.002395129930311657,
"passed": true
},
"runtime": {
"loadMs": 0,
"inferenceMs": 180.75,
"totalMs": 180.75,
"pairsPerSecond": 16.597510373443985,
"grade": "excellent",
"engine": {
"name": "ONNX Runtime Web + Transformers.js tokenizer",
"version": "1.26.0-dev.20260416-b7804b056c / 4.2.0"
},
"acceleration": "partial",
"accelerationDetails": "ONNX Runtime WebGPU with CPU-assigned shape operations reported by runtime; CPUtokenizer/ranking. Not full GPU delegation."
}
},
{
"precision": "fp16",
"device": "webgpu",
"status": "pass",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 10.65625,
"cpuScoreDifference": 0.002395129930311657,
"passed": true
},
"runtime": {
"loadMs": 0,
"inferenceMs": 181.40999999642372,
"totalMs": 181.40999999642372,
"pairsPerSecond": 16.53712584785371,
"grade": "excellent",
"engine": {
"name": "ONNX Runtime Web + Transformers.js tokenizer",
"version": "1.26.0-dev.20260416-b7804b056c / 4.2.0"
},
"acceleration": "partial",
"accelerationDetails": "ONNX Runtime WebGPU with CPU-assigned shape operations reported by runtime; CPUtokenizer/ranking. Not full GPU delegation."
}
},
{
"precision": "fp16",
"device": "webgpu",
"status": "pass",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 10.65625,
"cpuScoreDifference": 0.002395129930311657,
"passed": true
},
"runtime": {
"loadMs": 0,
"inferenceMs": 180.34000000357628,
"totalMs": 180.34000000357628,
"pairsPerSecond": 16.63524453776482,
"grade": "excellent",
"engine": {
"name": "ONNX Runtime Web + Transformers.js tokenizer",
"version": "1.26.0-dev.20260416-b7804b056c / 4.2.0"
},
"acceleration": "partial",
"accelerationDetails": "ONNX Runtime WebGPU with CPU-assigned shape operations reported by runtime; CPUtokenizer/ranking. Not full GPU delegation."
}
},
{
"precision": "fp16",
"device": "webgpu",
"status": "pass",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 10.65625,
"cpuScoreDifference": 0.002395129930311657,
"passed": true
},
"runtime": {
"loadMs": 0,
"inferenceMs": 176.42500001192093,
"totalMs": 176.42500001192093,
"pairsPerSecond": 17.004392800324737,
"grade": "excellent",
"engine": {
"name": "ONNX Runtime Web + Transformers.js tokenizer",
"version": "1.26.0-dev.20260416-b7804b056c / 4.2.0"
},
"acceleration": "partial",
"accelerationDetails": "ONNX Runtime WebGPU with CPU-assigned shape operations reported by runtime; CPUtokenizer/ranking. Not full GPU delegation."
}
},
{
"precision": "fp16",
"device": "webgpu",
"status": "pass",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 10.65625,
"cpuScoreDifference": 0.002395129930311657,
"passed": true
},
"runtime": {
"loadMs": 0,
"inferenceMs": 180.6850000023842,
"totalMs": 180.6850000023842,
"pairsPerSecond": 16.603481196338457,
"grade": "excellent",
"engine": {
"name": "ONNX Runtime Web + Transformers.js tokenizer",
"version": "1.26.0-dev.20260416-b7804b056c / 4.2.0"
},
"acceleration": "partial",
"accelerationDetails": "ONNX Runtime WebGPU with CPU-assigned shape operations reported by runtime; CPUtokenizer/ranking. Not full GPU delegation."
}
},
{
"precision": "q4",
"device": "wasm",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 9.584003448486328,
"cpuScoreDifference": 0.11164901595208176,
"passed": false
},
"runtime": {
"loadMs": 23190.340000003576,
"inferenceMs": 3611.125,
"totalMs": 26801.465000003576,
"pairsPerSecond": 0.830766035515248,
"grade": "acceptable"
}
},
{
"precision": "q4",
"device": "wasm",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 5.395444869995117,
"cpuScoreDifference": 0.4155872787931898,
"passed": false
},
"runtime": {
"loadMs": 0,
"inferenceMs": 3132.934999987483,
"totalMs": 3132.934999987483,
"pairsPerSecond": 0.9575685419620853,
"grade": "acceptable"
}
},
{
"precision": "q4",
"device": "wasm",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 4.061840057373047,
"cpuScoreDifference": 0.08335508050595608,
"passed": false
},
"runtime": {
"loadMs": 0,
"inferenceMs": 2798.265000000596,
"totalMs": 2798.265000000596,
"pairsPerSecond": 1.0720928861274257,
"grade": "acceptable"
}
},
{
"precision": "q4",
"device": "wasm",
"status": "pass",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 11.408878326416016,
"cpuScoreDifference": 0.002398360141037781,
"passed": true
},
"runtime": {
"loadMs": 0,
"inferenceMs": 2577.6500000059605,
"totalMs": 2577.6500000059605,
"pairsPerSecond": 1.163850794325476,
"grade": "acceptable"
}
},
{
"precision": "q4",
"device": "wasm",
"status": "pass",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": null,
"positiveLogitMargin": null,
"cpuScoreDifference": 0.00002289330735190944,
"passed": true
},
"runtime": {
"loadMs": 0,
"inferenceMs": 2548.1299999952316,
"totalMs": 2548.1299999952316,
"pairsPerSecond": 1.1773339664795808,
"grade": "acceptable"
}
},
{
"precision": "q4",
"device": "wasm",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 9.584003448486328,
"cpuScoreDifference": 0.11164901595208176,
"passed": false
},
"runtime": {
"loadMs": 0,
"inferenceMs": 2489.314999997616,
"totalMs": 2489.314999997616,
"pairsPerSecond": 1.2051508145826757,
"grade": "acceptable"
}
},
{
"precision": "q4",
"device": "wasm",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 9.584003448486328,
"cpuScoreDifference": 0.11164901595208176,
"passed": false
},
"runtime": {
"loadMs": 0,
"inferenceMs": 2417.655000001192,
"totalMs": 2417.655000001192,
"pairsPerSecond": 1.24087183655175,
"grade": "acceptable"
}
},
{
"precision": "q4",
"device": "wasm",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 9.584003448486328,
"cpuScoreDifference": 0.11164901595208176,
"passed": false
},
"runtime": {
"loadMs": 0,
"inferenceMs": 2444.234999999404,
"totalMs": 2444.234999999404,
"pairsPerSecond": 1.2273778912423443,
"grade": "acceptable"
}
},
{
"precision": "q4",
"device": "wasm",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 9.584003448486328,
"cpuScoreDifference": 0.11164901595208176,
"passed": false
},
"runtime": {
"loadMs": 0,
"inferenceMs": 2427.130000010133,
"totalMs": 2427.130000010133,
"pairsPerSecond": 1.236027736457246,
"grade": "acceptable"
}
},
{
"precision": "q4",
"device": "wasm",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 9.584003448486328,
"cpuScoreDifference": 0.11164901595208176,
"passed": false
},
"runtime": {
"loadMs": 0,
"inferenceMs": 2418.7249999940395,
"totalMs": 2418.7249999940395,
"pairsPerSecond": 1.2403228973973448,
"grade": "acceptable"
}
},
{
"precision": "q4",
"device": "webgpu",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 9.584008693695068,
"cpuScoreDifference": 0.11164829555449807,
"passed": false
},
"runtime": {
"loadMs": 24059.989999994636,
"inferenceMs": 602.2550000101328,
"totalMs": 24662.24500000477,
"pairsPerSecond": 4.981278694157003,
"grade": "good"
}
},
{
"precision": "q4",
"device": "webgpu",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 5.395452499389648,
"cpuScoreDifference": 0.41558823130212474,
"passed": false
},
"runtime": {
"loadMs": 0,
"inferenceMs": 301.0049999952316,
"totalMs": 301.0049999952316,
"pairsPerSecond": 9.966611850459376,
"grade": "good"
}
},
{
"precision": "q4",
"device": "webgpu",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 4.061850547790527,
"cpuScoreDifference": 0.08335493237827685,
"passed": false
},
"runtime": {
"loadMs": 0,
"inferenceMs": 298.6600000113249,
"totalMs": 298.6600000113249,
"pairsPerSecond": 10.044867072544843,
"grade": "excellent"
}
},
{
"precision": "q4",
"device": "webgpu",
"status": "pass",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 11.4088773727417,
"cpuScoreDifference": 0.0023983768381609094,
"passed": true
},
"runtime": {
"loadMs": 0,
"inferenceMs": 276.5700000077486,
"totalMs": 276.5700000077486,
"pairsPerSecond": 10.84716346644954,
"grade": "excellent"
}
},
{
"precision": "q4",
"device": "webgpu",
"status": "pass",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": null,
"positiveLogitMargin": null,
"cpuScoreDifference": 0.000022893812156904004,
"passed": true
},
"runtime": {
"loadMs": 0,
"inferenceMs": 274.85999999940395,
"totalMs": 274.85999999940395,
"pairsPerSecond": 10.914647456910812,
"grade": "excellent"
}
},
{
"precision": "q4",
"device": "webgpu",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 9.584008693695068,
"cpuScoreDifference": 0.11164829555449807,
"passed": false
},
"runtime": {
"loadMs": 0,
"inferenceMs": 298.60999999940395,
"totalMs": 298.60999999940395,
"pairsPerSecond": 10.046549010434976,
"grade": "excellent"
}
},
{
"precision": "q4",
"device": "webgpu",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 9.584008693695068,
"cpuScoreDifference": 0.11164829555449807,
"passed": false
},
"runtime": {
"loadMs": 0,
"inferenceMs": 306.09999999403954,
"totalMs": 306.09999999403954,
"pairsPerSecond": 9.800718719563596,
"grade": "good"
}
},
{
"precision": "q4",
"device": "webgpu",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 9.584008693695068,
"cpuScoreDifference": 0.11164829555449807,
"passed": false
},
"runtime": {
"loadMs": 0,
"inferenceMs": 283.9450000077486,
"totalMs": 283.9450000077486,
"pairsPerSecond": 10.56542640271226,
"grade": "excellent"
}
},
{
"precision": "q4",
"device": "webgpu",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 9.584008693695068,
"cpuScoreDifference": 0.11164829555449807,
"passed": false
},
"runtime": {
"loadMs": 0,
"inferenceMs": 271.4350000023842,
"totalMs": 271.4350000023842,
"pairsPerSecond": 11.05236981219684,
"grade": "excellent"
}
},
{
"precision": "q4",
"device": "webgpu",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 9.584008693695068,
"cpuScoreDifference": 0.11164829555449807,
"passed": false
},
"runtime": {
"loadMs": 0,
"inferenceMs": 271.294999986887,
"totalMs": 271.294999986887,
"pairsPerSecond": 11.05807331556057,
"grade": "excellent"
}
},
{
"precision": "q8",
"device": "wasm",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 9.537567138671875,
"cpuScoreDifference": 0.3185248074825542,
"passed": false
},
"runtime": {
"loadMs": 34700.92500001192,
"inferenceMs": 1942.7950000017881,
"totalMs": 36643.72000001371,
"pairsPerSecond": 1.5441670376942698,
"grade": "acceptable"
}
},
{
"precision": "q8",
"device": "wasm",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 5.734798431396484,
"cpuScoreDifference": 0.45759459786360057,
"passed": false
},
"runtime": {
"loadMs": 0,
"inferenceMs": 2038.4549999982119,
"totalMs": 2038.4549999982119,
"pairsPerSecond": 1.4717028337650975,
"grade": "acceptable"
}
},
{
"precision": "q8",
"device": "wasm",
"status": "pass",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 2.689192771911621,
"cpuScoreDifference": 0.059303751069203936,
"passed": true
},
"runtime": {
"loadMs": 0,
"inferenceMs": 1998.7549999952316,
"totalMs": 1998.7549999952316,
"pairsPerSecond": 1.500934331625015,
"grade": "acceptable"
}
},
{
"precision": "q8",
"device": "wasm",
"status": "pass",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 9.921249389648438,
"cpuScoreDifference": 0.01661645659227638,
"passed": true
},
"runtime": {
"loadMs": 0,
"inferenceMs": 1824.300000011921,
"totalMs": 1824.300000011921,
"pairsPerSecond": 1.6444663706519742,
"grade": "acceptable"
}
},
{
"precision": "q8",
"device": "wasm",
"status": "pass",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": null,
"positiveLogitMargin": null,
"cpuScoreDifference": 0.00012903131765596045,
"passed": true
},
"runtime": {
"loadMs": 0,
"inferenceMs": 1812.684999987483,
"totalMs": 1812.684999987483,
"pairsPerSecond": 1.6550034893104515,
"grade": "acceptable"
}
},
{
"precision": "q8",
"device": "wasm",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 9.537567138671875,
"cpuScoreDifference": 0.3185248074825542,
"passed": false
},
"runtime": {
"loadMs": 0,
"inferenceMs": 2084.185000002384,
"totalMs": 2084.185000002384,
"pairsPerSecond": 1.4394115685491298,
"grade": "acceptable"
}
},
{
"precision": "q8",
"device": "wasm",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 9.537567138671875,
"cpuScoreDifference": 0.3185248074825542,
"passed": false
},
"runtime": {
"loadMs": 0,
"inferenceMs": 1963.7300000041723,
"totalMs": 1963.7300000041723,
"pairsPerSecond": 1.5277049288820896,
"grade": "acceptable"
}
},
{
"precision": "q8",
"device": "wasm",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 9.537567138671875,
"cpuScoreDifference": 0.3185248074825542,
"passed": false
},
"runtime": {
"loadMs": 0,
"inferenceMs": 1901.765000000596,
"totalMs": 1901.765000000596,
"pairsPerSecond": 1.5774819706951488,
"grade": "acceptable"
}
},
{
"precision": "q8",
"device": "wasm",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 9.537567138671875,
"cpuScoreDifference": 0.3185248074825542,
"passed": false
},
"runtime": {
"loadMs": 0,
"inferenceMs": 1899.2099999934435,
"totalMs": 1899.2099999934435,
"pairsPerSecond": 1.5796041512051626,
"grade": "acceptable"
}
},
{
"precision": "q8",
"device": "wasm",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 9.537567138671875,
"cpuScoreDifference": 0.3185248074825542,
"passed": false
},
"runtime": {
"loadMs": 0,
"inferenceMs": 1914.3700000047684,
"totalMs": 1914.3700000047684,
"pairsPerSecond": 1.5670951801336876,
"grade": "acceptable"
}
},
{
"precision": "q8",
"device": "webgpu",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 6.796820163726807,
"cpuScoreDifference": 0.6276123601284331,
"passed": false
},
"runtime": {
"loadMs": 28875.92999999225,
"inferenceMs": 2579.125,
"totalMs": 31455.05499999225,
"pairsPerSecond": 1.163185188775263,
"grade": "acceptable"
}
},
{
"precision": "q8",
"device": "webgpu",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 3.573981285095215,
"cpuScoreDifference": 0.39709107674789335,
"passed": false
},
"runtime": {
"loadMs": 0,
"inferenceMs": 2474.765000000596,
"totalMs": 2474.765000000596,
"pairsPerSecond": 1.2122363133466318,
"grade": "acceptable"
}
},
{
"precision": "q8",
"device": "webgpu",
"status": "pass",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 1.6537389755249023,
"cpuScoreDifference": 0.07699993560445373,
"passed": true
},
"runtime": {
"loadMs": 0,
"inferenceMs": 2483.5349999964237,
"totalMs": 2483.5349999964237,
"pairsPerSecond": 1.2079555955540469,
"grade": "acceptable"
}
},
{
"precision": "q8",
"device": "webgpu",
"status": "pass",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 6.187666893005371,
"cpuScoreDifference": 0.0436560333540732,
"passed": true
},
"runtime": {
"loadMs": 0,
"inferenceMs": 2284.234999999404,
"totalMs": 2284.234999999404,
"pairsPerSecond": 1.3133499836929137,
"grade": "acceptable"
}
},
{
"precision": "q8",
"device": "webgpu",
"status": "pass",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": null,
"positiveLogitMargin": null,
"cpuScoreDifference": 0.0002044212635123203,
"passed": true
},
"runtime": {
"loadMs": 0,
"inferenceMs": 2293.7099999934435,
"totalMs": 2293.7099999934435,
"pairsPerSecond": 1.3079247158570941,
"grade": "acceptable"
}
},
{
"precision": "q8",
"device": "webgpu",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 6.796820163726807,
"cpuScoreDifference": 0.6276123601284331,
"passed": false
},
"runtime": {
"loadMs": 0,
"inferenceMs": 2377.7549999952316,
"totalMs": 2377.7549999952316,
"pairsPerSecond": 1.261694329317367,
"grade": "acceptable"
}
},
{
"precision": "q8",
"device": "webgpu",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 6.796820163726807,
"cpuScoreDifference": 0.6276123601284331,
"passed": false
},
"runtime": {
"loadMs": 0,
"inferenceMs": 2504.479999989271,
"totalMs": 2504.479999989271,
"pairsPerSecond": 1.1978534466287818,
"grade": "acceptable"
}
},
{
"precision": "q8",
"device": "webgpu",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 6.796820163726807,
"cpuScoreDifference": 0.6276123601284331,
"passed": false
},
"runtime": {
"loadMs": 0,
"inferenceMs": 2375.8250000029802,
"totalMs": 2375.8250000029802,
"pairsPerSecond": 1.262719265937616,
"grade": "acceptable"
}
},
{
"precision": "q8",
"device": "webgpu",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 6.796820163726807,
"cpuScoreDifference": 0.6276123601284331,
"passed": false
},
"runtime": {
"loadMs": 0,
"inferenceMs": 2424.4899999946356,
"totalMs": 2424.4899999946356,
"pairsPerSecond": 1.2373736332204455,
"grade": "acceptable"
}
},
{
"precision": "q8",
"device": "webgpu",
"status": "fail",
"quality": {
"finite": true,
"correctTop": true,
"top1Accuracy": 1,
"positiveLogitMargin": 6.796820163726807,
"cpuScoreDifference": 0.6276123601284331,
"passed": false
},
"runtime": {
"loadMs": 0,
"inferenceMs": 2447.5750000029802,
"totalMs": 2447.5750000029802,
"pairsPerSecond": 1.2257029917352265,
"grade": "acceptable"
}
}
]