All MiniLM L6 v2
16 configurations testedQuality criteria
Finite normalized distinct 384d vectors; relevant passage cosine exceeds unrelated passage by >=0.1.
| Precision | Device | Result | |
|---|---|---|---|
FP16 | GPU · WebGPUAccelerated | Passed | Very fast |
Q4F16 | GPU · WebGPUAccelerated | Passed | Very fast |
Q4 | GPU · WebGPUAccelerated | Passed | Very fast |
FP32 | GPU · WebGPUAccelerated | Passed | Very fast |
INT8 | GPU · WebGPUAccelerated | Passed | Fast |
UINT8 | GPU · WebGPUAccelerated | Passed | Fast |
Q8 | GPU · WebGPUAccelerated | Passed | Fast |
BNB4 | GPU · WebGPUAccelerated | Passed | Fast |
FP32 | CPU · WASM | Passed | Very fast |
UINT8 | CPU · WASM | Passed | Very fast |
INT8 | CPU · WASM | Passed | Very fast |
Q8 | CPU · WASM | Passed | Very fast |
BNB4 | CPU · WASM | Passed | Very fast |
Q4 | CPU · WASM | Passed | Very fast |
FP16 | CPU · WASM | Failed | — |
Q4F16 | CPU · WASM | Failed | — |
Test setup & sample details
- Quality gate
- Finite normalized distinct 384d vectors; relevant passage cosine exceeds unrelated passage by >=0.1.
- Tested
- 2026-09-15T18:03:03.122Z
- Environment
- Headless Chrome on macOS; Transformers.js 4.2.0
- Runtime
- Transformers.js 4.2.0
- Artifact revision
- 751bff37182d3f1213fa05d7196b954e230abad9
- Fixture
{ "file": "all-minilm-l6-v2.json", "url": "https://f005.backblazeb2.com/file/hubters-webai-public/assets/benchmarks/all-minilm-l6-v2.json", "size": 407, "sha1": "8beb837ef1dfd6c35b4a100d7d947f414f03c0f6" }- Rerun when
- artifact revision change; runtime version change; fixture or quality-gate change; browser/backend behavior change
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "wasm",
"status": "pass",
"quality": {
"shape": true,
"finite": true,
"normalized": true,
"distinct": true,
"positive": 0.7724030499931017,
"negative": 0.04601099547020955,
"margin": 0.7263920545228921,
"minimumMargin": 0.1,
"grade": "semantic-retrieval-pass"
},
"runtime": {
"loadMs": 14700.90000000596,
"inferenceMs": 37.900000005960464,
"totalMs": 15961.40000000596,
"grade": "excellent",
"embeddingsPerSecond": 79.15567281077033
}
},
{
"precision": "fp32",
"device": "webgpu",
"status": "pass",
"quality": {
"shape": true,
"finite": true,
"normalized": true,
"distinct": true,
"positive": 0.7724028277553725,
"negative": 0.04601110846368907,
"margin": 0.7263917192916834,
"minimumMargin": 0.1,
"grade": "semantic-retrieval-pass"
},
"runtime": {
"loadMs": 14810.300000011921,
"inferenceMs": 87,
"totalMs": 16120.699999988079,
"grade": "excellent",
"embeddingsPerSecond": 34.48275862068966
}
},
{
"precision": "fp16",
"device": "wasm",
"status": "fail",
"quality": "Not embedded",
"runtime": {
"loadMs": null,
"inferenceMs": null,
"totalMs": 10966.800000011921,
"grade": "failed"
},
"failure": "Error: Can't create a session. ERROR_CODE: 1, ERROR_MESSAGE: /mnt/vss/_work/1/s/onnxruntime/core/graph/graph_utils.cc:30 int onnxruntime::graph_utils::GetIndexFromName(const Node &, const std::string &, bool) itr != node_args.end() was false. Attempting to get index by a name which does not exist:InsertedPrecisionFreeCast_/encoder/layer.5/output/LayerNorm/Constant_output_0for node: /embeddings/LayerNorm/Mul/SimplifiedLayerNormFusion/\n"
},
{
"precision": "fp16",
"device": "webgpu",
"status": "pass",
"quality": {
"shape": true,
"finite": true,
"normalized": true,
"distinct": true,
"positive": 0.7730415426686783,
"negative": 0.0456449256363155,
"margin": 0.7273966170323628,
"minimumMargin": 0.1,
"grade": "semantic-retrieval-pass"
},
"runtime": {
"loadMs": 9982.09999999404,
"inferenceMs": 82.40000000596046,
"totalMs": 11385.700000017881,
"grade": "excellent",
"embeddingsPerSecond": 36.40776698765768
}
},
{
"precision": "int8",
"device": "wasm",
"status": "pass",
"quality": {
"shape": true,
"finite": true,
"normalized": true,
"distinct": true,
"positive": 0.7722416643690213,
"negative": 0.027980527186719884,
"margin": 0.7442611371823015,
"minimumMargin": 0.1,
"grade": "semantic-retrieval-pass"
},
"runtime": {
"loadMs": 8187.200000017881,
"inferenceMs": 43.79999998211861,
"totalMs": 9548.200000017881,
"grade": "excellent",
"embeddingsPerSecond": 68.49315071289391
}
},
{
"precision": "int8",
"device": "webgpu",
"status": "pass",
"quality": {
"shape": true,
"finite": true,
"normalized": true,
"distinct": true,
"positive": 0.7722416117012072,
"negative": 0.02798057330092668,
"margin": 0.7442610384002806,
"minimumMargin": 0.1,
"grade": "semantic-retrieval-pass"
},
"runtime": {
"loadMs": 7859.9000000059605,
"inferenceMs": 143.30000001192093,
"totalMs": 9281.40000000596,
"grade": "excellent",
"embeddingsPerSecond": 20.93510118458084
}
},
{
"precision": "uint8",
"device": "wasm",
"status": "pass",
"quality": {
"shape": true,
"finite": true,
"normalized": true,
"distinct": true,
"positive": 0.7846678365487807,
"negative": 0.0552927046653054,
"margin": 0.7293751318834752,
"minimumMargin": 0.1,
"grade": "semantic-retrieval-pass"
},
"runtime": {
"loadMs": 7780.699999988079,
"inferenceMs": 43.400000005960464,
"totalMs": 9043.40000000596,
"grade": "excellent",
"embeddingsPerSecond": 69.12442395364025
}
},
{
"precision": "uint8",
"device": "webgpu",
"status": "pass",
"quality": {
"shape": true,
"finite": true,
"normalized": true,
"distinct": true,
"positive": 0.7846677477370225,
"negative": 0.0552895126171424,
"margin": 0.7293782351198801,
"minimumMargin": 0.1,
"grade": "semantic-retrieval-pass"
},
"runtime": {
"loadMs": 8371,
"inferenceMs": 163.90000000596046,
"totalMs": 9865.09999999404,
"grade": "good",
"embeddingsPerSecond": 18.303843806533866
}
},
{
"precision": "q8",
"device": "wasm",
"status": "pass",
"quality": {
"shape": true,
"finite": true,
"normalized": true,
"distinct": true,
"positive": 0.7722416643690213,
"negative": 0.027980527186719884,
"margin": 0.7442611371823015,
"minimumMargin": 0.1,
"grade": "semantic-retrieval-pass"
},
"runtime": {
"loadMs": 7439.0999999940395,
"inferenceMs": 43.80000001192093,
"totalMs": 8761.700000017881,
"grade": "excellent",
"embeddingsPerSecond": 68.4931506662899
}
},
{
"precision": "q8",
"device": "webgpu",
"status": "pass",
"quality": {
"shape": true,
"finite": true,
"normalized": true,
"distinct": true,
"positive": 0.7722416117012072,
"negative": 0.02798057330092668,
"margin": 0.7442610384002806,
"minimumMargin": 0.1,
"grade": "semantic-retrieval-pass"
},
"runtime": {
"loadMs": 8023.799999982119,
"inferenceMs": 165.59999999403954,
"totalMs": 9478.40000000596,
"grade": "good",
"embeddingsPerSecond": 18.115942029637555
}
},
{
"precision": "q4",
"device": "wasm",
"status": "pass",
"quality": {
"shape": true,
"finite": true,
"normalized": true,
"distinct": true,
"positive": 0.7961162499844112,
"negative": 0.028139081710697626,
"margin": 0.7679771682737135,
"minimumMargin": 0.1,
"grade": "semantic-retrieval-pass"
},
"runtime": {
"loadMs": 12157.700000017881,
"inferenceMs": 53.19999998807907,
"totalMs": 13494.59999999404,
"grade": "excellent",
"embeddingsPerSecond": 56.39097745624498
}
},
{
"precision": "q4",
"device": "webgpu",
"status": "pass",
"quality": {
"shape": true,
"finite": true,
"normalized": true,
"distinct": true,
"positive": 0.7961161390535308,
"negative": 0.02813908655516071,
"margin": 0.7679770524983701,
"minimumMargin": 0.1,
"grade": "semantic-retrieval-pass"
},
"runtime": {
"loadMs": 12957,
"inferenceMs": 85,
"totalMs": 14310.700000017881,
"grade": "excellent",
"embeddingsPerSecond": 35.29411764705882
}
},
{
"precision": "q4f16",
"device": "wasm",
"status": "fail",
"quality": "Not embedded",
"runtime": {
"loadMs": null,
"inferenceMs": null,
"totalMs": 10332,
"grade": "failed"
},
"failure": "Error: Can't create a session. ERROR_CODE: 1, ERROR_MESSAGE: /mnt/vss/_work/1/s/onnxruntime/core/graph/graph_utils.cc:30 int onnxruntime::graph_utils::GetIndexFromName(const Node &, const std::string &, bool) itr != node_args.end() was false. Attempting to get index by a name which does not exist:InsertedPrecisionFreeCast_/embeddings/LayerNorm/Constant_output_0for node: /embeddings/LayerNorm/Mul/SimplifiedLayerNormFusion/\n"
},
{
"precision": "q4f16",
"device": "webgpu",
"status": "pass",
"quality": {
"shape": true,
"finite": true,
"normalized": true,
"distinct": true,
"positive": 0.7963711455441236,
"negative": 0.028056996170858772,
"margin": 0.7683141493732648,
"minimumMargin": 0.1,
"grade": "semantic-retrieval-pass"
},
"runtime": {
"loadMs": 9289.5,
"inferenceMs": 84,
"totalMs": 10610.199999988079,
"grade": "excellent",
"embeddingsPerSecond": 35.714285714285715
}
},
{
"precision": "bnb4",
"device": "wasm",
"status": "pass",
"quality": {
"shape": true,
"finite": true,
"normalized": true,
"distinct": true,
"positive": 0.800362821756695,
"negative": 0.06411063220401463,
"margin": 0.7362521895526805,
"minimumMargin": 0.1,
"grade": "semantic-retrieval-pass"
},
"runtime": {
"loadMs": 11390.5,
"inferenceMs": 51.19999998807907,
"totalMs": 12718.59999999404,
"grade": "excellent",
"embeddingsPerSecond": 58.59375001364242
}
},
{
"precision": "bnb4",
"device": "webgpu",
"status": "pass",
"quality": {
"shape": true,
"finite": true,
"normalized": true,
"distinct": true,
"positive": 0.800362987038872,
"negative": 0.06411066535998207,
"margin": 0.7362523216788899,
"minimumMargin": 0.1,
"grade": "semantic-retrieval-pass"
},
"runtime": {
"loadMs": 13376.299999982119,
"inferenceMs": 174.5,
"totalMs": 14713.799999982119,
"grade": "good",
"embeddingsPerSecond": 17.191977077363898
}
}
]