Laya Typed Decisions — Q4
2 configurations testedQuality criteria
Every agent-observability, customer-service, invoice, and security case must return finite normalized non-degenerate distributions and the checked-in expected choice/yes-no result or ordered-score interval.
| Precision | Device | Result | |
|---|---|---|---|
Q4 | GPU · WebGPUAccelerated | Passed | Moderate |
Q4 | CPU · WASM | Passed | Very slow |
Test setup & sample details
- Quality gate
- Every agent-observability, customer-service, invoice, and security case must return finite normalized non-degenerate distributions and the checked-in expected choice/yes-no result or ordered-score interval.
- Tested
- 2026-09-25T09:51:15.016Z
- Environment
- Isolated headless Google Chrome on macOS; Transformers.js 4.2.0
- Runtime
- Transformers.js 4.2.0
- Artifact revision
- 7388ca40228f08fd72fb3d2fdbd29153e9bc63d4
- Fixture
{ "file": "laya-typed-decisions.json", "url": "https://f005.backblazeb2.com/file/hubters-webai-public/assets/benchmarks/laya-typed-decisions.json", "size": 2242, "sha1": "cee490dc2efc280f3d781dd6c7ec47c6d23ba737", "source": "WebAI-authored regression cases representing the source model's four intended workflow families.", "license": "CC0-1.0", "licenseUrl": "https://creativecommons.org/publicdomain/zero/1.0/", "preprocessing": "Pinned ModernBERT tokenizer; each question/options prefix uses Laya's marker format and the shared state is truncated to the remaining context." }- Rerun when
- artifact revision change; runtime version change; fixture or quality-gate change; browser/backend behavior change
Per-case quality & failure details
[
{
"precision": "q4",
"device": "wasm",
"status": "pass",
"quality": {
"grade": "typed-decision-pass",
"cases": [
{
"domain": "agent-observability",
"type": "choice",
"pass": true
},
{
"domain": "agent-observability",
"type": "noul",
"pass": true
},
{
"domain": "customer-service",
"type": "choice",
"pass": true
},
{
"domain": "customer-service",
"type": "noul",
"pass": true
},
{
"domain": "invoice-processing",
"type": "choice",
"pass": true
},
{
"domain": "invoice-processing",
"type": "score",
"pass": true
},
{
"domain": "security-incidents",
"type": "choice",
"pass": true
},
{
"domain": "security-incidents",
"type": "noul",
"pass": true
}
]
},
"runtime": {
"loadMs": 53499.4,
"inferenceMs": 9939.8,
"totalMs": 64657.9,
"grade": "very-slow",
"questionsPerSecond": 0.8048
}
},
{
"precision": "q4",
"device": "webgpu",
"status": "pass",
"quality": {
"grade": "typed-decision-pass",
"cases": [
{
"domain": "agent-observability",
"type": "choice",
"pass": true
},
{
"domain": "agent-observability",
"type": "noul",
"pass": true
},
{
"domain": "customer-service",
"type": "choice",
"pass": true
},
{
"domain": "customer-service",
"type": "noul",
"pass": true
},
{
"domain": "invoice-processing",
"type": "choice",
"pass": true
},
{
"domain": "invoice-processing",
"type": "score",
"pass": true
},
{
"domain": "security-incidents",
"type": "choice",
"pass": true
},
{
"domain": "security-incidents",
"type": "noul",
"pass": true
}
]
},
"runtime": {
"loadMs": 53566.3,
"inferenceMs": 938.7,
"totalMs": 55818.6,
"grade": "good",
"questionsPerSecond": 8.5224
}
}
]