A model is still running

A model is still running or downloading. Leaving will stop the current operation. Stay on this page to let it finish.

Skip to content
LocalAI.toolsBeta

Join the waitlist

Get an email when the new apps and model runs launch.

No account needed.

MiniLM 2

Compare meaning, explore text vectors, and build semantic search on your device.

Browser inferencePowered by WebAI · Private · Unlimited runs
About this model

Compare English sentences by meaning, group similar text, or prepare text for semantic search. Converts each input into a 384-number vector; it does not write answers.

License
Apache-2.0
Model released
Unknown
Source repository created
Mar 2, 2022
Model weights · FP32
90.4 MB

Best for

  • Lightweight English semantic search.
  • Clustering and similarity prototypes.
  • Small-download embedding performance baseline.

Keep in mind

  • Primarily English; multilingual retrieval is not certified.
  • Inputs beyond 256 wordpieces are truncated; chunk long documents.
  • Similarity does not verify facts and may reproduce training-data biases.
  • Close rankings and domain-specific text require application evaluation.
  • Do not use similarity alone for consequential decisions about people.

Processed on your deviceInference runs in this browser. Input/output sharing requires consent for each run. Basic run logs include device information and approximate connection location. Model and runtime files download from external hosts. Saved downloads may be cleared by your browser.

Sources & technical details
Original model
sentence-transformers/all-MiniLM-L6-v2
Release verification
Unknown
Release evidence
Exact original checkpoint public release date has not been verified. Repository creation, training, paper, artifact conversion and update dates are not release evidence.
Source updated
Jun 1, 2026
Browser model files
Xenova/all-MiniLM-L6-v2
Files updated
Jul 22, 2025
Artifact revision
751bff37182d3f1213fa05d7196b954e230abad9

Input

TEXT → VECTORS

Sample result

All MiniLM L6 v2

Output

Explore what your text means

Generate vectors from one or more texts. Add multiple texts to compare their meaning.

Benchmark evidence

Saved worker evidence is separate from benchmarks run on your device. It does not guarantee speed or quality on every browser.

All MiniLM L6 v2

16 configurations tested
Passed14
Failed2
TestedSep 15, 2026
RuntimeTransformers.js 4.2.0

Quality criteria

Finite normalized distinct 384d vectors; relevant passage cosine exceeds unrelated passage by >=0.1.

PrecisionDeviceResult
FP16GPU · WebGPUAcceleratedPassedVery fast
Q4F16GPU · WebGPUAcceleratedPassedVery fast
Q4GPU · WebGPUAcceleratedPassedVery fast
FP32GPU · WebGPUAcceleratedPassedVery fast
INT8GPU · WebGPUAcceleratedPassedFast
UINT8GPU · WebGPUAcceleratedPassedFast
Q8GPU · WebGPUAcceleratedPassedFast
BNB4GPU · WebGPUAcceleratedPassedFast
FP32CPU · WASMPassedVery fast
UINT8CPU · WASMPassedVery fast
INT8CPU · WASMPassedVery fast
Q8CPU · WASMPassedVery fast
BNB4CPU · WASMPassedVery fast
Q4CPU · WASMPassedVery fast
FP16CPU · WASMFailed—
Q4F16CPU · WASMFailed—
Test setup & sample details
Quality gate
Finite normalized distinct 384d vectors; relevant passage cosine exceeds unrelated passage by >=0.1.
Tested
2026-09-15T18:03:03.122Z
Environment
Headless Chrome on macOS; Transformers.js 4.2.0
Runtime
Transformers.js 4.2.0
Artifact revision
751bff37182d3f1213fa05d7196b954e230abad9
Fixture
{
  "file": "all-minilm-l6-v2.json",
  "url": "https://f005.backblazeb2.com/file/hubters-webai-public/assets/benchmarks/all-minilm-l6-v2.json",
  "size": 407,
  "sha1": "8beb837ef1dfd6c35b4a100d7d947f414f03c0f6"
}
Rerun when
artifact revision change; runtime version change; fixture or quality-gate change; browser/backend behavior change
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.7724030499931017,
      "negative": 0.04601099547020955,
      "margin": 0.7263920545228921,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass"
    },
    "runtime": {
      "loadMs": 14700.90000000596,
      "inferenceMs": 37.900000005960464,
      "totalMs": 15961.40000000596,
      "grade": "excellent",
      "embeddingsPerSecond": 79.15567281077033
    }
  },
  {
    "precision": "fp32",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.7724028277553725,
      "negative": 0.04601110846368907,
      "margin": 0.7263917192916834,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass"
    },
    "runtime": {
      "loadMs": 14810.300000011921,
      "inferenceMs": 87,
      "totalMs": 16120.699999988079,
      "grade": "excellent",
      "embeddingsPerSecond": 34.48275862068966
    }
  },
  {
    "precision": "fp16",
    "device": "wasm",
    "status": "fail",
    "quality": "Not embedded",
    "runtime": {
      "loadMs": null,
      "inferenceMs": null,
      "totalMs": 10966.800000011921,
      "grade": "failed"
    },
    "failure": "Error: Can't create a session. ERROR_CODE: 1, ERROR_MESSAGE: /mnt/vss/_work/1/s/onnxruntime/core/graph/graph_utils.cc:30 int onnxruntime::graph_utils::GetIndexFromName(const Node &, const std::string &, bool) itr != node_args.end() was false. Attempting to get index by a name which does not exist:InsertedPrecisionFreeCast_/encoder/layer.5/output/LayerNorm/Constant_output_0for node: /embeddings/LayerNorm/Mul/SimplifiedLayerNormFusion/\n"
  },
  {
    "precision": "fp16",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.7730415426686783,
      "negative": 0.0456449256363155,
      "margin": 0.7273966170323628,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass"
    },
    "runtime": {
      "loadMs": 9982.09999999404,
      "inferenceMs": 82.40000000596046,
      "totalMs": 11385.700000017881,
      "grade": "excellent",
      "embeddingsPerSecond": 36.40776698765768
    }
  },
  {
    "precision": "int8",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.7722416643690213,
      "negative": 0.027980527186719884,
      "margin": 0.7442611371823015,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass"
    },
    "runtime": {
      "loadMs": 8187.200000017881,
      "inferenceMs": 43.79999998211861,
      "totalMs": 9548.200000017881,
      "grade": "excellent",
      "embeddingsPerSecond": 68.49315071289391
    }
  },
  {
    "precision": "int8",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.7722416117012072,
      "negative": 0.02798057330092668,
      "margin": 0.7442610384002806,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass"
    },
    "runtime": {
      "loadMs": 7859.9000000059605,
      "inferenceMs": 143.30000001192093,
      "totalMs": 9281.40000000596,
      "grade": "excellent",
      "embeddingsPerSecond": 20.93510118458084
    }
  },
  {
    "precision": "uint8",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.7846678365487807,
      "negative": 0.0552927046653054,
      "margin": 0.7293751318834752,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass"
    },
    "runtime": {
      "loadMs": 7780.699999988079,
      "inferenceMs": 43.400000005960464,
      "totalMs": 9043.40000000596,
      "grade": "excellent",
      "embeddingsPerSecond": 69.12442395364025
    }
  },
  {
    "precision": "uint8",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.7846677477370225,
      "negative": 0.0552895126171424,
      "margin": 0.7293782351198801,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass"
    },
    "runtime": {
      "loadMs": 8371,
      "inferenceMs": 163.90000000596046,
      "totalMs": 9865.09999999404,
      "grade": "good",
      "embeddingsPerSecond": 18.303843806533866
    }
  },
  {
    "precision": "q8",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.7722416643690213,
      "negative": 0.027980527186719884,
      "margin": 0.7442611371823015,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass"
    },
    "runtime": {
      "loadMs": 7439.0999999940395,
      "inferenceMs": 43.80000001192093,
      "totalMs": 8761.700000017881,
      "grade": "excellent",
      "embeddingsPerSecond": 68.4931506662899
    }
  },
  {
    "precision": "q8",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.7722416117012072,
      "negative": 0.02798057330092668,
      "margin": 0.7442610384002806,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass"
    },
    "runtime": {
      "loadMs": 8023.799999982119,
      "inferenceMs": 165.59999999403954,
      "totalMs": 9478.40000000596,
      "grade": "good",
      "embeddingsPerSecond": 18.115942029637555
    }
  },
  {
    "precision": "q4",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.7961162499844112,
      "negative": 0.028139081710697626,
      "margin": 0.7679771682737135,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass"
    },
    "runtime": {
      "loadMs": 12157.700000017881,
      "inferenceMs": 53.19999998807907,
      "totalMs": 13494.59999999404,
      "grade": "excellent",
      "embeddingsPerSecond": 56.39097745624498
    }
  },
  {
    "precision": "q4",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.7961161390535308,
      "negative": 0.02813908655516071,
      "margin": 0.7679770524983701,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass"
    },
    "runtime": {
      "loadMs": 12957,
      "inferenceMs": 85,
      "totalMs": 14310.700000017881,
      "grade": "excellent",
      "embeddingsPerSecond": 35.29411764705882
    }
  },
  {
    "precision": "q4f16",
    "device": "wasm",
    "status": "fail",
    "quality": "Not embedded",
    "runtime": {
      "loadMs": null,
      "inferenceMs": null,
      "totalMs": 10332,
      "grade": "failed"
    },
    "failure": "Error: Can't create a session. ERROR_CODE: 1, ERROR_MESSAGE: /mnt/vss/_work/1/s/onnxruntime/core/graph/graph_utils.cc:30 int onnxruntime::graph_utils::GetIndexFromName(const Node &, const std::string &, bool) itr != node_args.end() was false. Attempting to get index by a name which does not exist:InsertedPrecisionFreeCast_/embeddings/LayerNorm/Constant_output_0for node: /embeddings/LayerNorm/Mul/SimplifiedLayerNormFusion/\n"
  },
  {
    "precision": "q4f16",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.7963711455441236,
      "negative": 0.028056996170858772,
      "margin": 0.7683141493732648,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass"
    },
    "runtime": {
      "loadMs": 9289.5,
      "inferenceMs": 84,
      "totalMs": 10610.199999988079,
      "grade": "excellent",
      "embeddingsPerSecond": 35.714285714285715
    }
  },
  {
    "precision": "bnb4",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.800362821756695,
      "negative": 0.06411063220401463,
      "margin": 0.7362521895526805,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass"
    },
    "runtime": {
      "loadMs": 11390.5,
      "inferenceMs": 51.19999998807907,
      "totalMs": 12718.59999999404,
      "grade": "excellent",
      "embeddingsPerSecond": 58.59375001364242
    }
  },
  {
    "precision": "bnb4",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.800362987038872,
      "negative": 0.06411066535998207,
      "margin": 0.7362523216788899,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass"
    },
    "runtime": {
      "loadMs": 13376.299999982119,
      "inferenceMs": 174.5,
      "totalMs": 14713.799999982119,
      "grade": "good",
      "embeddingsPerSecond": 17.191977077363898
    }
  }
]

Paraphrase Multilingual MiniLM L12 v2

16 configurations tested
Passed14
Failed2
TestedSep 10, 2026
RuntimeTransformers.js 4.2.0

Quality criteria

Finite normalized 384d vectors, distinct rows, and cross-lingual paraphrase cosine exceeds an unrelated sentence by at least 0.1.

PrecisionDeviceResult
FP16GPU · WebGPUAcceleratedPassedVery fast
Q8GPU · WebGPUAcceleratedPassedFast
FP32GPU · WebGPUAcceleratedPassedFast
INT8GPU · WebGPUAcceleratedPassedFast
UINT8GPU · WebGPUAcceleratedPassedFast
Q4F16GPU · WebGPUAcceleratedPassedFast
BNB4GPU · WebGPUAcceleratedPassedFast
Q4GPU · WebGPUAcceleratedPassedFast
UINT8CPU · WASMPassedVery fast
Q8CPU · WASMPassedVery fast
INT8CPU · WASMPassedVery fast
BNB4CPU · WASMPassedVery fast
FP32CPU · WASMPassedVery fast
Q4CPU · WASMPassedVery fast
Q4F16CPU · WASMFailed—
FP16CPU · WASMFailed—
Test setup & sample details
Quality gate
Finite normalized 384d vectors, distinct rows, and cross-lingual paraphrase cosine exceeds an unrelated sentence by at least 0.1.
Tested
2026-09-10T17:01:39.383Z
Environment
Not recorded
Runtime
Transformers.js 4.2.0
Artifact revision
2c4055b12046f11709e9df2c122e59ffbdc2f900
Fixture
{
  "file": "paraphrase-multilingual-minilm-l12-v2.json",
  "url": "https://f005.backblazeb2.com/file/hubters-webai-public/assets/benchmarks/paraphrase-multilingual-minilm-l12-v2.json",
  "size": 523,
  "sha1": "0294528b8c4ca8006eadb7a4872cad2a5ce29416"
}
Rerun when
artifact revision change; runtime version change; fixture or quality gate change; browser/backend change
Per-case quality & failure details
[
  {
    "precision": "q4f16",
    "device": "wasm",
    "status": "fail",
    "quality": "Not embedded",
    "runtime": "Not embedded",
    "failure": "ONNX session graph error"
  },
  {
    "precision": "q4f16",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "positive": 0.9634434633752343,
      "negative": 0.016400515664729432,
      "margin": 0.9470429477105049
    },
    "runtime": {
      "loadMs": 27484.400000095367,
      "inferenceMs": 216,
      "totalMs": 29123.600000023842,
      "embeddingsPerSecond": 13.88888888888889,
      "grade": "good"
    }
  },
  {
    "precision": "q4",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "positive": 0.9633238890429541,
      "negative": 0.016378795891804777,
      "margin": 0.9469450931511493
    },
    "runtime": {
      "loadMs": 59198.199999928474,
      "inferenceMs": 93.10000002384186,
      "totalMs": 60594.10000002384,
      "embeddingsPerSecond": 32.223415673810244,
      "grade": "excellent"
    }
  },
  {
    "precision": "q4",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "positive": 0.9633241034293016,
      "negative": 0.016378732683020834,
      "margin": 0.9469453707462808
    },
    "runtime": {
      "loadMs": 104547.10000002384,
      "inferenceMs": 265.39999997615814,
      "totalMs": 105967.29999995232,
      "embeddingsPerSecond": 11.303692540578377,
      "grade": "good"
    }
  },
  {
    "precision": "q8",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "positive": 0.9600488226006761,
      "negative": 0.019562819127059206,
      "margin": 0.9404860034736169
    },
    "runtime": {
      "loadMs": 19012.800000071526,
      "inferenceMs": 79,
      "totalMs": 20356.700000047684,
      "embeddingsPerSecond": 37.974683544303794,
      "grade": "excellent"
    }
  },
  {
    "precision": "q8",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "positive": 0.9597591200892055,
      "negative": 0.019554869530899063,
      "margin": 0.9402042505583065
    },
    "runtime": {
      "loadMs": 20864.5,
      "inferenceMs": 132.5,
      "totalMs": 22297.699999928474,
      "embeddingsPerSecond": 22.641509433962263,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "wasm",
    "status": "fail",
    "quality": "Not embedded",
    "runtime": "Not embedded",
    "failure": "ONNX session graph error"
  },
  {
    "precision": "fp16",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "positive": 0.9614530147869136,
      "negative": 0.010634397365533989,
      "margin": 0.9508186174213796
    },
    "runtime": {
      "loadMs": 32166.100000023842,
      "inferenceMs": 82.60000002384186,
      "totalMs": 33593.39999997616,
      "embeddingsPerSecond": 36.319612580315656,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "positive": 0.9615690552802658,
      "negative": 0.010496432821365476,
      "margin": 0.9510726224589003
    },
    "runtime": {
      "loadMs": 54524.200000047684,
      "inferenceMs": 88.39999997615814,
      "totalMs": 55772.10000002384,
      "embeddingsPerSecond": 33.936651592863264,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "positive": 0.9615689143010782,
      "negative": 0.010496399676341445,
      "margin": 0.9510725146247367
    },
    "runtime": {
      "loadMs": 59002.199999928474,
      "inferenceMs": 135.40000009536743,
      "totalMs": 60304,
      "embeddingsPerSecond": 22.15657310108556,
      "grade": "excellent"
    }
  },
  {
    "precision": "int8",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "positive": 0.9671473708784826,
      "negative": 0.01563588563693314,
      "margin": 0.9515114852415494
    },
    "runtime": {
      "loadMs": 21194.399999976158,
      "inferenceMs": 79.19999992847443,
      "totalMs": 22536.100000023842,
      "embeddingsPerSecond": 37.87878791299624,
      "grade": "excellent"
    }
  },
  {
    "precision": "int8",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "positive": 0.9673156267541712,
      "negative": 0.014866507112901387,
      "margin": 0.9524491196412698
    },
    "runtime": {
      "loadMs": 19456.100000023842,
      "inferenceMs": 143,
      "totalMs": 20852.5,
      "embeddingsPerSecond": 20.97902097902098,
      "grade": "excellent"
    }
  },
  {
    "precision": "uint8",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "positive": 0.96247490412721,
      "negative": -0.0003,
      "margin": 0.9627356916629597
    },
    "runtime": {
      "loadMs": 20047,
      "inferenceMs": 76.19999992847443,
      "totalMs": 21320,
      "embeddingsPerSecond": 39.37007877711243,
      "grade": "excellent"
    }
  },
  {
    "precision": "uint8",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "positive": 0.9614893528663216,
      "negative": -0.0023,
      "margin": 0.9638067241958327
    },
    "runtime": {
      "loadMs": 20321.099999904633,
      "inferenceMs": 148.70000004768372,
      "totalMs": 21787.400000095367,
      "embeddingsPerSecond": 20.174848682165354,
      "grade": "excellent"
    }
  },
  {
    "precision": "bnb4",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "positive": 0.9632869658872978,
      "negative": 0.02356272016744846,
      "margin": 0.9397242457198494
    },
    "runtime": {
      "loadMs": 53410.89999997616,
      "inferenceMs": 79.20000004768372,
      "totalMs": 54663.60000002384,
      "embeddingsPerSecond": 37.87878785598231,
      "grade": "excellent"
    }
  },
  {
    "precision": "bnb4",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "positive": 0.9632869214774838,
      "negative": 0.02356271629532229,
      "margin": 0.9397242051821616
    },
    "runtime": {
      "loadMs": 51658.10000002384,
      "inferenceMs": 224.29999995231628,
      "totalMs": 53029.5,
      "embeddingsPerSecond": 13.374944273908904,
      "grade": "good"
    }
  }
]

all-MiniLM-L12-v2

16 configurations tested
Passed13
Failed3
TestedNov 13, 2026
RuntimeTransformers.js 4.2.0

Quality criteria

Finite normalized distinct 384d vectors; relevant passage cosine exceeds unrelated passage by >=0.1.

PrecisionDeviceResult
FP32GPU · WebGPUAcceleratedPassedVery fast
UINT8GPU · WebGPUAcceleratedPassedFast
Q4GPU · WebGPUAcceleratedPassedFast
Q8GPU · WebGPUAcceleratedPassedFast
INT8GPU · WebGPUAcceleratedPassedFast
Q4F16GPU · WebGPUAcceleratedPassedFast
BNB4GPU · WebGPUAcceleratedPassedModerate
FP32CPU · WASMPassedVery fast
INT8CPU · WASMPassedVery fast
Q8CPU · WASMPassedVery fast
UINT8CPU · WASMPassedVery fast
BNB4CPU · WASMPassedVery fast
Q4CPU · WASMPassedVery fast
FP16CPU · WASMFailed—
FP16GPU · WebGPUFailed—
Q4F16CPU · WASMFailed—
Test setup & sample details
Quality gate
Finite normalized distinct 384d vectors; relevant passage cosine exceeds unrelated passage by >=0.1.
Tested
2026-11-13T18:16:29.146Z
Environment
Headless Chrome on macOS; Transformers.js 4.2.0
Runtime
Transformers.js 4.2.0
Artifact revision
beeb2e4b69e95f188a15cc2e90d09fd035dac229
Fixture
{
  "file": "all-minilm-l6-v2.json",
  "url": "https://f005.backblazeb2.com/file/hubters-webai-public/assets/benchmarks/all-minilm-l6-v2.json",
  "size": 407,
  "sha1": "8beb837ef1dfd6c35b4a100d7d947f414f03c0f6"
}
Rerun when
artifact revision change; runtime version change; fixture or quality-gate change; browser/backend behavior change
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.5739818342873608,
      "negative": 0.12616282341486126,
      "margin": 0.44781901087249953,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 25976.5,
      "inferenceMs": 56.19999998807907,
      "totalMs": 27569.19999998808,
      "grade": "excellent",
      "embeddingsPerSecond": 53.380782929472396
    }
  },
  {
    "precision": "bnb4",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.5914765638738706,
      "negative": 0.13942048398779072,
      "margin": 0.4520560798860799,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 15932.899999976158,
      "inferenceMs": 71.60000002384186,
      "totalMs": 17624.200000047684,
      "grade": "excellent",
      "embeddingsPerSecond": 41.899441326830164
    }
  },
  {
    "precision": "bnb4",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.5914762663923855,
      "negative": 0.13942057348804957,
      "margin": 0.4520556929043359,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 22206.19999998808,
      "inferenceMs": 511.5,
      "totalMs": 24583.400000035763,
      "grade": "slow",
      "embeddingsPerSecond": 5.865102639296188
    }
  },
  {
    "precision": "fp16",
    "device": "wasm",
    "status": "fail",
    "quality": "Not embedded",
    "runtime": {
      "loadMs": null,
      "inferenceMs": null,
      "totalMs": 36240.80000001192,
      "grade": "failed"
    },
    "failure": "Error: Can't create a session. ERROR_CODE: 1, ERROR_MESSAGE: /mnt/vss/_work/1/s/onnxruntime/core/graph/graph_utils.cc:30 int onnxruntime::graph_utils::GetIndexFromName(const Node &, const std::string &, bool) itr != node_args.end() was false. Attempting to get index by a name which does not exist:InsertedPrecisionFreeCast_/encoder/layer.11/output/LayerNorm/Constant_output_0for node: /embeddings/LayerNorm/Mul/SimplifiedLayerNormFusion/\n"
  },
  {
    "precision": "fp16",
    "device": "webgpu",
    "status": "fail",
    "quality": "Not embedded",
    "runtime": {
      "loadMs": null,
      "inferenceMs": null,
      "totalMs": null,
      "grade": "failed"
    },
    "failure": "page.waitForFunction: Timeout 45000ms exceeded."
  },
  {
    "precision": "fp32",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.5739817618146679,
      "negative": 0.12616281771727408,
      "margin": 0.44781894409739376,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 21417.19999998808,
      "inferenceMs": 66.5,
      "totalMs": 23529.600000023842,
      "grade": "excellent",
      "embeddingsPerSecond": 45.11278195488722
    }
  },
  {
    "precision": "int8",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.5986834519005806,
      "negative": 0.06870924786510275,
      "margin": 0.5299742040354778,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 11322.799999952316,
      "inferenceMs": 62,
      "totalMs": 13465.300000011921,
      "grade": "excellent",
      "embeddingsPerSecond": 48.38709677419355
    }
  },
  {
    "precision": "int8",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.596024307629536,
      "negative": 0.06604237554904815,
      "margin": 0.5299819320804878,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 12088.700000047684,
      "inferenceMs": 243,
      "totalMs": 14165.199999988079,
      "grade": "good",
      "embeddingsPerSecond": 12.345679012345679
    }
  },
  {
    "precision": "q4",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.6233366090165307,
      "negative": 0.1880638395929659,
      "margin": 0.43527276942356474,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 12021.300000011921,
      "inferenceMs": 85.5,
      "totalMs": 14037.300000011921,
      "grade": "excellent",
      "embeddingsPerSecond": 35.08771929824561
    }
  },
  {
    "precision": "q4",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.6233365821537403,
      "negative": 0.1880640041037178,
      "margin": 0.4352725780500225,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 12183.099999964237,
      "inferenceMs": 141.4000000357628,
      "totalMs": 14216.199999988079,
      "grade": "excellent",
      "embeddingsPerSecond": 21.216407349655178
    }
  },
  {
    "precision": "q4f16",
    "device": "wasm",
    "status": "fail",
    "quality": "Not embedded",
    "runtime": {
      "loadMs": null,
      "inferenceMs": null,
      "totalMs": 13679.300000011921,
      "grade": "failed"
    },
    "failure": "Error: Can't create a session. ERROR_CODE: 1, ERROR_MESSAGE: /mnt/vss/_work/1/s/onnxruntime/core/graph/graph_utils.cc:30 int onnxruntime::graph_utils::GetIndexFromName(const Node &, const std::string &, bool) itr != node_args.end() was false. Attempting to get index by a name which does not exist:InsertedPrecisionFreeCast_/embeddings/LayerNorm/Constant_output_0for node: /embeddings/LayerNorm/Mul/SimplifiedLayerNormFusion/\n"
  },
  {
    "precision": "q4f16",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.5698506240022674,
      "negative": 0.15584329337574188,
      "margin": 0.4140073306265255,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 11033.699999988079,
      "inferenceMs": 253.19999998807907,
      "totalMs": 13178.199999988079,
      "grade": "good",
      "embeddingsPerSecond": 11.848341232785321
    }
  },
  {
    "precision": "q8",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.5768044887372922,
      "negative": 0.12436029366354168,
      "margin": 0.4524441950737505,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 13451.5,
      "inferenceMs": 62.30000001192093,
      "totalMs": 15300.699999988079,
      "grade": "excellent",
      "embeddingsPerSecond": 48.15409308869917
    }
  },
  {
    "precision": "q8",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.5768716353484884,
      "negative": 0.12754861499537565,
      "margin": 0.4493230203531128,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 11609,
      "inferenceMs": 182.19999998807907,
      "totalMs": 13934.5,
      "grade": "good",
      "embeddingsPerSecond": 16.465422613591016
    }
  },
  {
    "precision": "uint8",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.587981360532366,
      "negative": 0.11671918541204714,
      "margin": 0.4712621751203189,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 8178.300000011921,
      "inferenceMs": 63.69999998807907,
      "totalMs": 10079.900000035763,
      "grade": "excellent",
      "embeddingsPerSecond": 47.09576139028925
    }
  },
  {
    "precision": "uint8",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.5924704984231938,
      "negative": 0.11981835579690409,
      "margin": 0.4726521426262897,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 10464.800000011921,
      "inferenceMs": 127.89999997615814,
      "totalMs": 12705.5,
      "grade": "excellent",
      "embeddingsPerSecond": 23.45582486754676
    }
  }
]

paraphrase-MiniLM-L12-v2

2 configurations tested
Passed2
Failed0
TestedNov 13, 2026
RuntimeTransformers.js 4.2.0

Quality criteria

Finite normalized distinct 384d vectors; relevant passage cosine exceeds unrelated passage by >=0.1.

PrecisionDeviceResult
FP32GPU · WebGPUAcceleratedPassedVery fast
FP32CPU · WASMPassedVery fast
Test setup & sample details
Quality gate
Finite normalized distinct 384d vectors; relevant passage cosine exceeds unrelated passage by >=0.1.
Tested
2026-11-13T18:13:13.677Z
Environment
Headless Chrome on macOS; Transformers.js 4.2.0
Runtime
Transformers.js 4.2.0
Artifact revision
2b9058918c3d228389532ccf813578658e4af706
Fixture
{
  "file": "all-minilm-l6-v2.json",
  "url": "https://f005.backblazeb2.com/file/hubters-webai-public/assets/benchmarks/all-minilm-l6-v2.json",
  "size": 407,
  "sha1": "8beb837ef1dfd6c35b4a100d7d947f414f03c0f6"
}
Rerun when
artifact revision change; runtime version change; fixture or quality-gate change; browser/backend behavior change
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.7479657056439987,
      "negative": 0.0019258669752217818,
      "margin": 0.7460398386687769,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 29233.69999998808,
      "inferenceMs": 55.80000001192093,
      "totalMs": 31892.600000023842,
      "grade": "excellent",
      "embeddingsPerSecond": 53.7634408487292
    }
  },
  {
    "precision": "fp32",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.74796528623038,
      "negative": 0.001925864927367944,
      "margin": 0.7460394213030122,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 24914.899999976158,
      "inferenceMs": 68.69999998807907,
      "totalMs": 27068.399999976158,
      "grade": "excellent",
      "embeddingsPerSecond": 43.668122278319714
    }
  }
]

paraphrase-MiniLM-L3-v2

16 configurations tested
Passed14
Failed2
TestedNov 13, 2026
RuntimeTransformers.js 4.2.0

Quality criteria

Finite normalized distinct 384d vectors; relevant passage cosine exceeds unrelated passage by >=0.1.

PrecisionDeviceResult
Q4GPU · WebGPUAcceleratedPassedVery fast
BNB4GPU · WebGPUAcceleratedPassedVery fast
INT8GPU · WebGPUAcceleratedPassedVery fast
UINT8GPU · WebGPUAcceleratedPassedVery fast
FP32GPU · WebGPUAcceleratedPassedFast
Q8GPU · WebGPUAcceleratedPassedFast
Q4F16GPU · WebGPUAcceleratedPassedFast
FP16GPU · WebGPUAcceleratedPassedFast
FP32CPU · WASMPassedVery fast
INT8CPU · WASMPassedVery fast
UINT8CPU · WASMPassedVery fast
BNB4CPU · WASMPassedVery fast
Q4CPU · WASMPassedVery fast
Q8CPU · WASMPassedVery fast
FP16CPU · WASMFailed—
Q4F16CPU · WASMFailed—
Test setup & sample details
Quality gate
Finite normalized distinct 384d vectors; relevant passage cosine exceeds unrelated passage by >=0.1.
Tested
2026-11-13T18:15:54.957Z
Environment
Headless Chrome on macOS; Transformers.js 4.2.0
Runtime
Transformers.js 4.2.0
Artifact revision
4b544e74dfc3256b2b56849ea5d7064fee1ac846
Fixture
{
  "file": "all-minilm-l6-v2.json",
  "url": "https://f005.backblazeb2.com/file/hubters-webai-public/assets/benchmarks/all-minilm-l6-v2.json",
  "size": 407,
  "sha1": "8beb837ef1dfd6c35b4a100d7d947f414f03c0f6"
}
Rerun when
artifact revision change; runtime version change; fixture or quality-gate change; browser/backend behavior change
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.6697117257640937,
      "negative": -0.004775593478939169,
      "margin": 0.6744873192430328,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 28253.5,
      "inferenceMs": 27,
      "totalMs": 30284.299999952316,
      "grade": "excellent",
      "embeddingsPerSecond": 111.11111111111111
    }
  },
  {
    "precision": "bnb4",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.6775639064604935,
      "negative": 0.009015702889898088,
      "margin": 0.6685482035705954,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 18098.100000023842,
      "inferenceMs": 34.59999996423721,
      "totalMs": 19792.30000001192,
      "grade": "excellent",
      "embeddingsPerSecond": 86.7052024017578
    }
  },
  {
    "precision": "bnb4",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.6775638585654236,
      "negative": 0.009015723152486275,
      "margin": 0.6685481354129373,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 24746.099999964237,
      "inferenceMs": 82.40000003576279,
      "totalMs": 27134.299999952316,
      "grade": "excellent",
      "embeddingsPerSecond": 36.40776697448977
    }
  },
  {
    "precision": "fp16",
    "device": "wasm",
    "status": "fail",
    "quality": "Not embedded",
    "runtime": {
      "loadMs": null,
      "inferenceMs": null,
      "totalMs": 15224.699999988079,
      "grade": "failed"
    },
    "failure": "Error: Can't create a session. ERROR_CODE: 1, ERROR_MESSAGE: /mnt/vss/_work/1/s/onnxruntime/core/graph/graph_utils.cc:30 int onnxruntime::graph_utils::GetIndexFromName(const Node &, const std::string &, bool) itr != node_args.end() was false. Attempting to get index by a name which does not exist:InsertedPrecisionFreeCast_/encoder/layer.2/output/LayerNorm/Constant_output_0for node: /embeddings/LayerNorm/Mul/SimplifiedLayerNormFusion/\n"
  },
  {
    "precision": "fp16",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.6693299737403484,
      "negative": -0.004375843601711549,
      "margin": 0.6737058173420599,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 29634.600000023842,
      "inferenceMs": 365.69999998807907,
      "totalMs": 32250.600000023842,
      "grade": "slow",
      "embeddingsPerSecond": 8.20344544735519
    }
  },
  {
    "precision": "fp32",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.6697116583043821,
      "negative": -0.004775597503239975,
      "margin": 0.674487255807622,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 15188,
      "inferenceMs": 110.40000003576279,
      "totalMs": 17310,
      "grade": "excellent",
      "embeddingsPerSecond": 27.17391303467559
    }
  },
  {
    "precision": "int8",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.696192796783385,
      "negative": 0.018922340672735228,
      "margin": 0.6772704561106497,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 10253,
      "inferenceMs": 31.80000001192093,
      "totalMs": 12174.300000011921,
      "grade": "excellent",
      "embeddingsPerSecond": 94.33962260614415
    }
  },
  {
    "precision": "int8",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.6961928090462685,
      "negative": 0.018922335055483776,
      "margin": 0.6772704739907847,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 13561.5,
      "inferenceMs": 91.09999996423721,
      "totalMs": 15623.600000023842,
      "grade": "excellent",
      "embeddingsPerSecond": 32.930845237954976
    }
  },
  {
    "precision": "q4",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.6786971399192798,
      "negative": 0.003737967603939488,
      "margin": 0.6749591723153403,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 12982.300000011921,
      "inferenceMs": 34.89999997615814,
      "totalMs": 14969.200000047684,
      "grade": "excellent",
      "embeddingsPerSecond": 85.9598854455428
    }
  },
  {
    "precision": "q4",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.6786971956317747,
      "negative": 0.0037379660383623694,
      "margin": 0.6749592295934123,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 13121.100000023842,
      "inferenceMs": 63,
      "totalMs": 15003.699999988079,
      "grade": "excellent",
      "embeddingsPerSecond": 47.61904761904762
    }
  },
  {
    "precision": "q4f16",
    "device": "wasm",
    "status": "fail",
    "quality": "Not embedded",
    "runtime": {
      "loadMs": null,
      "inferenceMs": null,
      "totalMs": 14932.199999988079,
      "grade": "failed"
    },
    "failure": "Error: Can't create a session. ERROR_CODE: 1, ERROR_MESSAGE: /mnt/vss/_work/1/s/onnxruntime/core/graph/graph_utils.cc:30 int onnxruntime::graph_utils::GetIndexFromName(const Node &, const std::string &, bool) itr != node_args.end() was false. Attempting to get index by a name which does not exist:InsertedPrecisionFreeCast_/embeddings/LayerNorm/Constant_output_0for node: /embeddings/LayerNorm/Mul/SimplifiedLayerNormFusion/\n"
  },
  {
    "precision": "q4f16",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.6787079227484198,
      "negative": 0.0036446366154952335,
      "margin": 0.6750632861329247,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 10031.300000011921,
      "inferenceMs": 166.89999997615814,
      "totalMs": 12255.599999964237,
      "grade": "good",
      "embeddingsPerSecond": 17.97483523324478
    }
  },
  {
    "precision": "q8",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.6573493432371056,
      "negative": -0.005512995002828147,
      "margin": 0.6628623382399337,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 8369.100000023842,
      "inferenceMs": 35.19999998807907,
      "totalMs": 10470.400000035763,
      "grade": "excellent",
      "embeddingsPerSecond": 85.22727275613603
    }
  },
  {
    "precision": "q8",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.6573491675783596,
      "negative": -0.005512968765636145,
      "margin": 0.6628621363439957,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 8343.600000023842,
      "inferenceMs": 135.5,
      "totalMs": 10970.300000011921,
      "grade": "excellent",
      "embeddingsPerSecond": 22.14022140221402
    }
  },
  {
    "precision": "uint8",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.6818060045129054,
      "negative": 0.011369705543991939,
      "margin": 0.6704362989689134,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 12033.800000011921,
      "inferenceMs": 31.899999976158142,
      "totalMs": 13830.199999988079,
      "grade": "excellent",
      "embeddingsPerSecond": 94.04388721762324
    }
  },
  {
    "precision": "uint8",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.6818059593320251,
      "negative": 0.01136971159497422,
      "margin": 0.6704362477370509,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 9898.399999976158,
      "inferenceMs": 95.69999998807907,
      "totalMs": 11916.600000023842,
      "grade": "excellent",
      "embeddingsPerSecond": 31.34796238635002
    }
  }
]

paraphrase-MiniLM-L6-v2

16 configurations tested
Passed14
Failed2
TestedNov 13, 2026
RuntimeTransformers.js 4.2.0

Quality criteria

Finite normalized distinct 384d vectors; relevant passage cosine exceeds unrelated passage by >=0.1.

PrecisionDeviceResult
Q4GPU · WebGPUAcceleratedPassedVery fast
Q4F16GPU · WebGPUAcceleratedPassedVery fast
FP32GPU · WebGPUAcceleratedPassedVery fast
Q8GPU · WebGPUAcceleratedPassedVery fast
UINT8GPU · WebGPUAcceleratedPassedVery fast
INT8GPU · WebGPUAcceleratedPassedVery fast
FP16GPU · WebGPUAcceleratedPassedFast
BNB4GPU · WebGPUAcceleratedPassedFast
FP32CPU · WASMPassedVery fast
INT8CPU · WASMPassedVery fast
UINT8CPU · WASMPassedVery fast
Q8CPU · WASMPassedVery fast
BNB4CPU · WASMPassedVery fast
Q4CPU · WASMPassedVery fast
FP16CPU · WASMFailed—
Q4F16CPU · WASMFailed—
Test setup & sample details
Quality gate
Finite normalized distinct 384d vectors; relevant passage cosine exceeds unrelated passage by >=0.1.
Tested
2026-11-13T18:15:44.600Z
Environment
Headless Chrome on macOS; Transformers.js 4.2.0
Runtime
Transformers.js 4.2.0
Artifact revision
2bb9dc00941ae0b65da96d5dd95d4a4564161858
Fixture
{
  "file": "all-minilm-l6-v2.json",
  "url": "https://f005.backblazeb2.com/file/hubters-webai-public/assets/benchmarks/all-minilm-l6-v2.json",
  "size": 407,
  "sha1": "8beb837ef1dfd6c35b4a100d7d947f414f03c0f6"
}
Rerun when
artifact revision change; runtime version change; fixture or quality-gate change; browser/backend behavior change
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.7438060336584993,
      "negative": 0.05782012873178438,
      "margin": 0.685985904926715,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 31046.30000001192,
      "inferenceMs": 35.59999996423721,
      "totalMs": 33991,
      "grade": "excellent",
      "embeddingsPerSecond": 84.26966300600331
    }
  },
  {
    "precision": "bnb4",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.757420258716462,
      "negative": 0.07118993837326786,
      "margin": 0.6862303203431942,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 15728,
      "inferenceMs": 48.80000001192093,
      "totalMs": 17420,
      "grade": "excellent",
      "embeddingsPerSecond": 61.47540982104828
    }
  },
  {
    "precision": "bnb4",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.7574202719220392,
      "negative": 0.07118995337505891,
      "margin": 0.6862303185469802,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 21278.899999976158,
      "inferenceMs": 315.10000002384186,
      "totalMs": 23240.100000023842,
      "grade": "slow",
      "embeddingsPerSecond": 9.520787051009224
    }
  },
  {
    "precision": "fp16",
    "device": "wasm",
    "status": "fail",
    "quality": "Not embedded",
    "runtime": {
      "loadMs": null,
      "inferenceMs": null,
      "totalMs": 13969.899999976158,
      "grade": "failed"
    },
    "failure": "Error: Can't create a session. ERROR_CODE: 1, ERROR_MESSAGE: /mnt/vss/_work/1/s/onnxruntime/core/graph/graph_utils.cc:30 int onnxruntime::graph_utils::GetIndexFromName(const Node &, const std::string &, bool) itr != node_args.end() was false. Attempting to get index by a name which does not exist:InsertedPrecisionFreeCast_/encoder/layer.5/output/LayerNorm/Constant_output_0for node: /embeddings/LayerNorm/Mul/SimplifiedLayerNormFusion/\n"
  },
  {
    "precision": "fp16",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.7440043875914684,
      "negative": 0.05768908006017934,
      "margin": 0.686315307531289,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 21300.30000001192,
      "inferenceMs": 116.40000003576279,
      "totalMs": 23545,
      "grade": "excellent",
      "embeddingsPerSecond": 25.77319586837009
    }
  },
  {
    "precision": "fp32",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.743806008374053,
      "negative": 0.057820069081379634,
      "margin": 0.6859859392926734,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 16620.100000023842,
      "inferenceMs": 71.5,
      "totalMs": 19521.600000023842,
      "grade": "excellent",
      "embeddingsPerSecond": 41.95804195804196
    }
  },
  {
    "precision": "int8",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.7421291908673712,
      "negative": 0.04964718156782162,
      "margin": 0.6924820092995495,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 10885.700000047684,
      "inferenceMs": 41,
      "totalMs": 13129.400000035763,
      "grade": "excellent",
      "embeddingsPerSecond": 73.17073170731707
    }
  },
  {
    "precision": "int8",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.7363855547286932,
      "negative": 0.05434707396315597,
      "margin": 0.6820384807655372,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 7819.800000011921,
      "inferenceMs": 95.5,
      "totalMs": 9804,
      "grade": "excellent",
      "embeddingsPerSecond": 31.413612565445025
    }
  },
  {
    "precision": "q4",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.7496135885623504,
      "negative": 0.07209727639382013,
      "margin": 0.6775163121685303,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 14136.5,
      "inferenceMs": 51.30000001192093,
      "totalMs": 16556.69999998808,
      "grade": "excellent",
      "embeddingsPerSecond": 58.4795321501534
    }
  },
  {
    "precision": "q4",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.7496134786006227,
      "negative": 0.07209717280402926,
      "margin": 0.6775163057965934,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 10861.099999964237,
      "inferenceMs": 60.60000002384186,
      "totalMs": 12593.899999976158,
      "grade": "excellent",
      "embeddingsPerSecond": 49.50495047557277
    }
  },
  {
    "precision": "q4f16",
    "device": "wasm",
    "status": "fail",
    "quality": "Not embedded",
    "runtime": {
      "loadMs": null,
      "inferenceMs": null,
      "totalMs": 15682.899999976158,
      "grade": "failed"
    },
    "failure": "Error: Can't create a session. ERROR_CODE: 1, ERROR_MESSAGE: /mnt/vss/_work/1/s/onnxruntime/core/graph/graph_utils.cc:30 int onnxruntime::graph_utils::GetIndexFromName(const Node &, const std::string &, bool) itr != node_args.end() was false. Attempting to get index by a name which does not exist:InsertedPrecisionFreeCast_/embeddings/LayerNorm/Constant_output_0for node: /embeddings/LayerNorm/Mul/SimplifiedLayerNormFusion/\n"
  },
  {
    "precision": "q4f16",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.7498807671365737,
      "negative": 0.07194917595177977,
      "margin": 0.677931591184794,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 8465.600000023842,
      "inferenceMs": 67.5,
      "totalMs": 10489.299999952316,
      "grade": "excellent",
      "embeddingsPerSecond": 44.44444444444444
    }
  },
  {
    "precision": "q8",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.7399365996872257,
      "negative": 0.05856839379330692,
      "margin": 0.6813682058939188,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 9550,
      "inferenceMs": 42.39999997615814,
      "totalMs": 12652.800000011921,
      "grade": "excellent",
      "embeddingsPerSecond": 70.75471702091802
    }
  },
  {
    "precision": "q8",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.7417797296650741,
      "negative": 0.05841436708095037,
      "margin": 0.6833653625841237,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 16252.199999988079,
      "inferenceMs": 92.89999997615814,
      "totalMs": 18531.5,
      "grade": "excellent",
      "embeddingsPerSecond": 32.29278795231346
    }
  },
  {
    "precision": "uint8",
    "device": "wasm",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.7370353117322448,
      "negative": 0.08594494052405473,
      "margin": 0.65109037120819,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 11457.099999964237,
      "inferenceMs": 41.5,
      "totalMs": 13417.700000047684,
      "grade": "excellent",
      "embeddingsPerSecond": 72.28915662650601
    }
  },
  {
    "precision": "uint8",
    "device": "webgpu",
    "status": "pass",
    "quality": {
      "shape": true,
      "finite": true,
      "normalized": true,
      "distinct": true,
      "positive": 0.7370354243685373,
      "negative": 0.08594499673800529,
      "margin": 0.651090427630532,
      "minimumMargin": 0.1,
      "grade": "semantic-retrieval-pass",
      "passed": true
    },
    "runtime": {
      "loadMs": 9571.899999976158,
      "inferenceMs": 95.39999997615814,
      "totalMs": 11798,
      "grade": "excellent",
      "embeddingsPerSecond": 31.446540888362097
    }
  }
]

Something not quite right?

Tell us what happened with MiniLM 2. Loading, results, or the page itself—we’re here for all of it.

Attachments Optional

Up to 3 files · 50 MB each

Browser, OS, CPU/GPU, memory, screen, storage, and network details are included automatically. Model inputs, results, and browsing history are not collected.