Apertus v1.1 0.5B Instruct — INT4
4 configurations testedQuality criteria
Exact-match answer, non-empty/non-repetitive output, and streamed text equal to final text.
| Precision | Device | Result | |
|---|---|---|---|
Q4 | GPU · WebGPUAccelerated | Passed | Fast |
Q4F16 | GPU · WebGPUAccelerated | Passed | Slow |
Q4F16 | CPU · WASM | Passed | Very slow |
Q4 | CPU · WASM | Passed | Very slow |
Test setup & sample details
- Quality gate
- Exact-match answer, non-empty/non-repetitive output, and streamed text equal to final text.
- Tested
- 2026-09-09T16:50:14.459Z
- Environment
- Not recorded
- Runtime
- Transformers.js 4.2.0
- Artifact revision
- defefbcf863cad09633955d5e82c8a1978f1c214
- Fixture
{ "file": "apertus-v1.1-0.5b.json", "url": "https://f005.backblazeb2.com/file/hubters-webai-public/assets/benchmarks/apertus-v1.1-0.5b.json", "size": 397, "sha1": "01931912845892dd559d20d4f628d65a2e6999d8" }- Rerun when
- artifact revision change; runtime version change; fixture or quality gate change; browser/backend change
Per-case quality & failure details
[
{
"precision": "q4",
"device": "wasm",
"status": "pass",
"quality": {
"grade": "exact-match",
"output": "The capital of France is Paris.",
"nonEmpty": true,
"nonRepetitive": true
},
"runtime": {
"loadMs": 67673.6,
"inferenceMs": 5725,
"totalMs": 75436.5,
"firstChunkMs": 1929.3,
"outputTokens": 7,
"tokensPerSecond": 1.223,
"grade": "very-slow"
}
},
{
"precision": "q4",
"device": "webgpu",
"status": "pass",
"quality": {
"grade": "exact-match",
"output": "The capital of France is Paris.",
"nonEmpty": true,
"nonRepetitive": true
},
"runtime": {
"loadMs": 71906.8,
"inferenceMs": 333.3,
"totalMs": 74056.1,
"firstChunkMs": 265,
"outputTokens": 7,
"tokensPerSecond": 21.002,
"grade": "excellent"
}
},
{
"precision": "q4f16",
"device": "wasm",
"status": "pass",
"quality": {
"grade": "exact-match",
"output": "The capital of France is Paris.",
"nonEmpty": true,
"nonRepetitive": true
},
"runtime": {
"loadMs": 46637.6,
"inferenceMs": 5559.7,
"totalMs": 53980.7,
"firstChunkMs": 1970.8,
"outputTokens": 7,
"tokensPerSecond": 1.259,
"grade": "very-slow"
}
},
{
"precision": "q4f16",
"device": "webgpu",
"status": "pass",
"quality": {
"grade": "exact-match",
"output": "The capital of France is Paris.",
"nonEmpty": true,
"nonRepetitive": true
},
"runtime": {
"loadMs": 50756.2,
"inferenceMs": 854.7,
"totalMs": 53346,
"firstChunkMs": 501.1,
"outputTokens": 7,
"tokensPerSecond": 8.19,
"grade": "slow"
}
}
]