A model is still running

A model is still running or downloading. Leaving will stop the current operation. Stay on this page to let it finish.

Skip to content
LocalAI.toolsBeta

Join the waitlist

Get an email when the new apps and model runs launch.

No account needed.

BGE-M3 Desktop

Create 1024-dimensional dense text embeddings for multilingual search with the exact Multilingual checkpoint.

Desktop inferencePrivate · Runs on your computer
About this model

Create 1024-dimensional dense text embeddings for multilingual search with the exact Multilingual checkpoint.

License
MIT
Model released
Unknown
Source repository created
Jan 27, 2024
Model weights · BF16
2.3 GB

Best for

  • Multilingual and cross-lingual dense semantic search and retrieval.
  • Clustering, similarity, and classification prototypes.

Keep in mind

  • Retrieval quality depends on language, domain, and input quality.
  • This adapter exposes dense embeddings only; BGE-M3 lexical weights and multi-vector interaction outputs are not exposed.
  • Semantic similarity is not factual verification and can encode training-data bias.
  • Long inputs are truncated and should be chunked.
  • Quantization can change ranking margins.
  • Four diagnostic English/cross-language fixtures only; not a multilingual leaderboard or all-language validation.
  • Native adapter exposes dense CLS embeddings only, not sparse lexical weights or ColBERT vectors.
  • Windows, Intel Mac, other Apple hardware/OS and source-outage recovery remain unverified.
  • FP16/BF16 are runtime casts of original weights, not separate checkpoint variants.

Processed on your deviceYour input is processed locally in the desktop app. Model and runtime files download from external hosts.

Target platforms · verification required

  • macos · aarch64, x86_64
  • windows · x86_64

Recommended configuration

BF16 · mps (verified platform only)

Sources & technical details
Original model
BAAI/bge-m3
Release verification
Unknown
Release evidence
Exact original checkpoint public release date has not been verified. Repository creation, training, paper, artifact conversion and update dates are not release evidence.
Source updated
Jul 3, 2024
Native model files
BAAI/bge-m3
Files updated
Jul 3, 2024
Artifact revision
5617a9f61b028005a4858fdac845db406aefb181

Desktop app

Sample result

BGE-M3 Multilingual Desktop

Run BGE-M3 Desktop on your computer

Download the desktop app to load this model, provide input and view results locally.

Public desktop downloads are coming soon.

Device requirements

macOS Apple SiliconCPU / FP32Tested16.0 GiBRAM estimate
macOS Apple SiliconMPS / FP32Tested16.0 GiBRAM estimate
macOS Apple SiliconMPS / FP16Tested16.0 GiBRAM estimate
macOS Apple SiliconMPS / BF16Tested16.0 GiBRAM estimate
Windows Intel / AMDCPU / FP32Unverified16.0 GiBRAM estimate
macOS Intel / AMDCPU / FP32Unverified16.0 GiBRAM estimate
If your device falls short

MemoryAdvisory only: close other apps, reduce image/tile/token/batch settings or choose a smaller verified model/precision. CPU can need more RAM than GPU.

GPUIf the selected GPU runtime fails, existing CPU fallback may recover when enabled, but CPU has its own memory/runtime needs.

PlatformUse a compatible OS/architecture/runtime or another model. These estimates do not add hardware detection or block runs.

Benchmark evidence

Saved desktop measurements apply to the recorded hardware and runtime. Fallback cases report the device that actually ran; they do not prove GPU support on the requested device.

BGE-M3 Multilingual Desktop

48 configurations tested
Passed48
Failed0
TestedNov 13, 2026
Runtimetransformers 5.22.1

Quality criteria

{ "file": "research/bge-m3-family-expansion/quality-gates.json", "sha256": "e39823c58c7328c1f308cb1502358ee29930d02e2a3f8cb766f828bb1200b591", "description": "Finite unit1024d vectors, per-fixture positive-minus-negative cosine margins. Fresh inference<=10s, warm<=5s, load<=120s." }

PrecisionDeviceResultPlatformInference / load
BF16mpsAcceleratedPassedVery fastmacos · aarch6446 ms / 2525 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6446 ms / 2519 ms
BF16mpsAcceleratedPassedVery fastmacos · aarch6447 ms / 2568 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6447 ms / 2532 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6447 ms / 2700 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6448 ms / 2519 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6449 ms / 2687 ms
BF16mpsAcceleratedPassedVery fastmacos · aarch6450 ms / 2525 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6450 ms / 2700 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6451 ms / 2687 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6472 ms / 2532 ms
BF16mpsAcceleratedPassedVery fastmacos · aarch6474 ms / 2568 ms
BF16mpsAcceleratedPassedFastmacos · aarch64103 ms / 2525 ms
FP16mpsAcceleratedPassedFastmacos · aarch64103 ms / 2532 ms
FP32mpsAcceleratedPassedFastmacos · aarch64104 ms / 2700 ms
BF16mpsAcceleratedPassedFastmacos · aarch64105 ms / 2568 ms
FP16mpsAcceleratedPassedFastmacos · aarch64105 ms / 2519 ms
FP32mpsAcceleratedPassedFastmacos · aarch64108 ms / 2687 ms
BF16mpsAcceleratedPassedFastmacos · aarch64140 ms / 2580 ms
FP32mpsAcceleratedPassedFastmacos · aarch64140 ms / 2759 ms
FP16mpsAcceleratedPassedFastmacos · aarch64141 ms / 2552 ms
FP16mpsAcceleratedPassedFastmacos · aarch64141 ms / 2552 ms
FP32mpsAcceleratedPassedFastmacos · aarch64158 ms / 2759 ms
BF16mpsAcceleratedPassedFastmacos · aarch64173 ms / 2568 ms
FP16mpsAcceleratedPassedFastmacos · aarch64174 ms / 2532 ms
BF16mpsAcceleratedPassedFastmacos · aarch64179 ms / 2525 ms
FP16mpsAcceleratedPassedFastmacos · aarch64188 ms / 2519 ms
FP32mpsAcceleratedPassedFastmacos · aarch64189 ms / 2700 ms
FP32mpsAcceleratedPassedFastmacos · aarch64205 ms / 2687 ms
BF16mpsAcceleratedPassedFastmacos · aarch64219 ms / 2580 ms
FP32mpsAcceleratedPassedFastmacos · aarch64357 ms / 2759 ms
FP16mpsAcceleratedPassedFastmacos · aarch64373 ms / 2552 ms
FP32mpsAcceleratedPassedFastmacos · aarch64389 ms / 2759 ms
BF16mpsAcceleratedPassedFastmacos · aarch64400 ms / 2580 ms
FP16mpsAcceleratedPassedFastmacos · aarch64404 ms / 2552 ms
BF16mpsAcceleratedPassedFastmacos · aarch64411 ms / 2580 ms
FP32cpuPassedVery fastmacos · aarch6442 ms / 2338 ms
FP32cpuPassedVery fastmacos · aarch6442 ms / 2321 ms
FP32cpuPassedVery fastmacos · aarch6446 ms / 8682 ms
FP32cpuPassedVery fastmacos · aarch6453 ms / 2338 ms
FP32cpuPassedVery fastmacos · aarch6453 ms / 2321 ms
FP32cpuPassedVery fastmacos · aarch6456 ms / 8682 ms
FP32cpuPassedFastmacos · aarch64104 ms / 2321 ms
FP32cpuPassedFastmacos · aarch64109 ms / 8682 ms
FP32cpuPassedFastmacos · aarch64110 ms / 2338 ms
FP32cpuPassedFastmacos · aarch64510 ms / 2321 ms
FP32cpuPassedFastmacos · aarch64511 ms / 2338 ms
FP32cpuPassedFastmacos · aarch64525 ms / 8682 ms
Test setup & sample details
Quality gate
{ "file": "research/bge-m3-family-expansion/quality-gates.json", "sha256": "e39823c58c7328c1f308cb1502358ee29930d02e2a3f8cb766f828bb1200b591", "description": "Finite unit1024d vectors, per-fixture positive-minus-negative cosine margins. Fresh inference<=10s, warm<=5s, load<=120s." }
Tested
2026-11-13T04:37:34.622578+00:00
Environment
{ "os": "macos", "osVersion": "27.2", "architecture": "aarch64", "hardware": "Apple M3 Max;Mac15,11;96GiB RAM", "cpu": "Apple M3 Max", "gpu": "Apple M3 Max", "driver": "OS-managed Metal/MPS macOS27.2 build26B118", "runtime": "App-owned CPython3.13.18;Torch2.10.0;Transformers5.22.1", "timingScope": "Fresh model/helper per cold repetition, local original snapshot load, shared OS/driver caches may be warm. Four fixtures plus five warm calls; no network transfer included.", "concurrentUploadActivity": "none; uploads excluded" }
Runtime
transformers 5.22.1
Artifact revision
5617a9f61b028005a4858fdac845db406aefb181
Fixture
{
  "file": "model-samples/native/bge-m3/quality-gates.json",
  "url": "https://assets.localai.tools/samples/v1/files/a10b184352cd5c00018eb629107ced06f6e55a9ea2cfd4cffb2f8574feb696a1/bge-m3-quality-gates.json",
  "size": 4205,
  "sha1": "c53d7a6368c4e7cc451f1fae92f04f27ca7c3884",
  "sha256": "a10b184352cd5c00018eb629107ced06f6e55a9ea2cfd4cffb2f8574feb696a1"
}
Rerun when
artifact-revision; runtime-version; fixture-checksum; quality-gate; backend-behavior
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000077172053,
        0.999999979059761,
        0.9999999974625967
      ],
      "positiveCosine": 0.7432051905918331,
      "negativeCosine": 0.35828289135672914,
      "margin": 0.38492229923510396,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 109.16208298294805,
      "durationMs": 110.0628330023028,
      "embeddingsPerSecond": 27.48206994610595,
      "inputTokens": [
        3,
        16
      ],
      "outputDevice": "cpu",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": null,
      "loadMs": 8681.698000000324,
      "totalMs": 8790.860082983272,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000576072567,
        0.9999999135592897,
        0.9999998999542602
      ],
      "positiveCosine": 0.7633974456480599,
      "negativeCosine": 0.3020763840952494,
      "margin": 0.4613210615528105,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 55.70595900644548,
      "durationMs": 56.1079999897629,
      "embeddingsPerSecond": 53.85420255762734,
      "inputTokens": [
        3,
        12
      ],
      "outputDevice": "cpu",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": null,
      "loadMs": 8681.698000000324,
      "totalMs": 8737.40395900677,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000438697059,
        0.9999999506879391,
        1.0000000712395805
      ],
      "positiveCosine": 0.9439428010712417,
      "negativeCosine": 0.43277716354344437,
      "margin": 0.5111656375277973,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 46.14387499168515,
      "durationMs": 46.53837499790825,
      "embeddingsPerSecond": 65.01404575451416,
      "inputTokens": [
        3,
        9
      ],
      "outputDevice": "cpu",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": null,
      "loadMs": 8681.698000000324,
      "totalMs": 8727.84187499201,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000479274251,
        0.9999999177035241,
        0.9999999945266407
      ],
      "positiveCosine": 0.7258266848073294,
      "negativeCosine": 0.3863602191538423,
      "margin": 0.3394664656534871,
      "minimumMargin": 0.05,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 525.4946249770001,
      "durationMs": 526.7516250023618,
      "embeddingsPerSecond": 5.70890710848147,
      "inputTokens": [
        3,
        412
      ],
      "outputDevice": "cpu",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": null,
      "loadMs": 8681.698000000324,
      "totalMs": 9207.192624977324,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000077172053,
        0.999999979059761,
        0.9999999974625967
      ],
      "positiveCosine": 0.7432051905918331,
      "negativeCosine": 0.35828289135672914,
      "margin": 0.38492229923510396,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 109.90562499500811,
      "durationMs": 110.48520900658332,
      "embeddingsPerSecond": 27.296146126608708,
      "inputTokens": [
        3,
        16
      ],
      "outputDevice": "cpu",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": null,
      "loadMs": 2338.448916998459,
      "totalMs": 2448.354541993467,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000576072567,
        0.9999999135592897,
        0.9999998999542602
      ],
      "positiveCosine": 0.7633974456480599,
      "negativeCosine": 0.3020763840952494,
      "margin": 0.4613210615528105,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 53.28379201819189,
      "durationMs": 53.62704099388793,
      "embeddingsPerSecond": 56.30229918650975,
      "inputTokens": [
        3,
        12
      ],
      "outputDevice": "cpu",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": null,
      "loadMs": 2338.448916998459,
      "totalMs": 2391.732709016651,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000438697059,
        0.9999999506879391,
        1.0000000712395805
      ],
      "positiveCosine": 0.9439428010712417,
      "negativeCosine": 0.43277716354344437,
      "margin": 0.5111656375277973,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 42.05408302368596,
      "durationMs": 42.39675001008436,
      "embeddingsPerSecond": 71.33671178397401,
      "inputTokens": [
        3,
        9
      ],
      "outputDevice": "cpu",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": null,
      "loadMs": 2338.448916998459,
      "totalMs": 2380.503000022145,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000479274251,
        0.9999999177035241,
        0.9999999945266407
      ],
      "positiveCosine": 0.7258266848073294,
      "negativeCosine": 0.3863602191538423,
      "margin": 0.3394664656534871,
      "minimumMargin": 0.05,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 511.1463750072289,
      "durationMs": 512.3583340027835,
      "embeddingsPerSecond": 5.869160277146781,
      "inputTokens": [
        3,
        412
      ],
      "outputDevice": "cpu",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": null,
      "loadMs": 2338.448916998459,
      "totalMs": 2849.595292005688,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000077172053,
        0.999999979059761,
        0.9999999974625967
      ],
      "positiveCosine": 0.7432051905918331,
      "negativeCosine": 0.35828289135672914,
      "margin": 0.38492229923510396,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 103.72541699325666,
      "durationMs": 104.24270899966359,
      "embeddingsPerSecond": 28.92251568576518,
      "inputTokens": [
        3,
        16
      ],
      "outputDevice": "cpu",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": null,
      "loadMs": 2321.281832992099,
      "totalMs": 2425.007249985356,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000576072567,
        0.9999999135592897,
        0.9999998999542602
      ],
      "positiveCosine": 0.7633974456480599,
      "negativeCosine": 0.3020763840952494,
      "margin": 0.4613210615528105,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 53.44929202692583,
      "durationMs": 53.82724999799393,
      "embeddingsPerSecond": 56.12796514664231,
      "inputTokens": [
        3,
        12
      ],
      "outputDevice": "cpu",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": null,
      "loadMs": 2321.281832992099,
      "totalMs": 2374.731125019025,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000438697059,
        0.9999999506879391,
        1.0000000712395805
      ],
      "positiveCosine": 0.9439428010712417,
      "negativeCosine": 0.43277716354344437,
      "margin": 0.5111656375277973,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 42.35966599662788,
      "durationMs": 42.678625002736226,
      "embeddingsPerSecond": 70.82208816846716,
      "inputTokens": [
        3,
        9
      ],
      "outputDevice": "cpu",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": null,
      "loadMs": 2321.281832992099,
      "totalMs": 2363.641498988727,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000479274251,
        0.9999999177035241,
        0.9999999945266407
      ],
      "positiveCosine": 0.7258266848073294,
      "negativeCosine": 0.3863602191538423,
      "margin": 0.3394664656534871,
      "minimumMargin": 0.05,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 510.2804169873707,
      "durationMs": 511.4870000106748,
      "embeddingsPerSecond": 5.879120381910029,
      "inputTokens": [
        3,
        412
      ],
      "outputDevice": "cpu",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": null,
      "loadMs": 2321.281832992099,
      "totalMs": 2831.56224997947,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000211870428,
        0.9999999929230169,
        0.9999999767896206
      ],
      "positiveCosine": 0.7432051033610728,
      "negativeCosine": 0.3582827292675386,
      "margin": 0.3849223740935342,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 389.2305839981418,
      "durationMs": 390.8427499991376,
      "embeddingsPerSecond": 7.707513549383164,
      "inputTokens": [
        3,
        16
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 3201220608,
      "loadMs": 2759.243124979548,
      "totalMs": 3148.47370897769,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999424989322,
        0.9999999961769176,
        1.000000019394219
      ],
      "positiveCosine": 0.7633974690508457,
      "negativeCosine": 0.30207613433089925,
      "margin": 0.46132133471994646,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 157.86604100139812,
      "durationMs": 159.33712502010167,
      "embeddingsPerSecond": 19.003453693840534,
      "inputTokens": [
        3,
        12
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 3201368064,
      "loadMs": 2759.243124979548,
      "totalMs": 2917.1091659809463,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000281018404,
        1.0000000021396556,
        1.000000037993779
      ],
      "positiveCosine": 0.943942833072416,
      "negativeCosine": 0.4327768926856611,
      "margin": 0.5111659403867549,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 140.21754200803116,
      "durationMs": 141.72612500260584,
      "embeddingsPerSecond": 21.395325841813506,
      "inputTokens": [
        3,
        9
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 3201482752,
      "loadMs": 2759.243124979548,
      "totalMs": 2899.4606669875793,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000085974663,
        1.000000023292101,
        0.9999999888165177
      ],
      "positiveCosine": 0.7258271404676749,
      "negativeCosine": 0.3863600948947857,
      "margin": 0.33946704557288926,
      "minimumMargin": 0.05,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 356.92737498902716,
      "durationMs": 358.89820801094174,
      "embeddingsPerSecond": 8.405071199966738,
      "inputTokens": [
        3,
        412
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 3300392960,
      "loadMs": 2759.243124979548,
      "totalMs": 3116.1704999685753,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000211870428,
        0.9999999929230169,
        0.9999999767896206
      ],
      "positiveCosine": 0.7432051033610728,
      "negativeCosine": 0.3582827292675386,
      "margin": 0.3849223740935342,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 107.8387500019744,
      "durationMs": 109.19695897609927,
      "embeddingsPerSecond": 27.819313557928606,
      "inputTokens": [
        3,
        16
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 3201220608,
      "loadMs": 2686.592708021635,
      "totalMs": 2794.4314580236096,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999424989322,
        0.9999999961769176,
        1.000000019394219
      ],
      "positiveCosine": 0.7633974690508457,
      "negativeCosine": 0.30207613433089925,
      "margin": 0.46132133471994646,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 51.26441700849682,
      "durationMs": 52.793458016822115,
      "embeddingsPerSecond": 58.52012321729446,
      "inputTokens": [
        3,
        12
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 3201368064,
      "loadMs": 2686.592708021635,
      "totalMs": 2737.857125030132,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000281018404,
        1.0000000021396556,
        1.000000037993779
      ],
      "positiveCosine": 0.943942833072416,
      "negativeCosine": 0.4327768926856611,
      "margin": 0.5111659403867549,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 48.75154100591317,
      "durationMs": 50.24775001220405,
      "embeddingsPerSecond": 61.53651634593713,
      "inputTokens": [
        3,
        9
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 3201499136,
      "loadMs": 2686.592708021635,
      "totalMs": 2735.3442490275484,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000085974663,
        1.000000023292101,
        0.9999999888165177
      ],
      "positiveCosine": 0.7258271404676749,
      "negativeCosine": 0.3863600948947857,
      "margin": 0.33946704557288926,
      "minimumMargin": 0.05,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 205.0472919945605,
      "durationMs": 206.8556250014808,
      "embeddingsPerSecond": 14.630771130006359,
      "inputTokens": [
        3,
        412
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 3300409344,
      "loadMs": 2686.592708021635,
      "totalMs": 2891.6400000161957,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000211870428,
        0.9999999929230169,
        0.9999999767896206
      ],
      "positiveCosine": 0.7432051033610728,
      "negativeCosine": 0.3582827292675386,
      "margin": 0.3849223740935342,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 103.86162501526996,
      "durationMs": 105.55958401528187,
      "embeddingsPerSecond": 28.88458561628449,
      "inputTokens": [
        3,
        16
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 3201220608,
      "loadMs": 2700.319291994674,
      "totalMs": 2804.180917009944,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999424989322,
        0.9999999961769176,
        1.000000019394219
      ],
      "positiveCosine": 0.7633974690508457,
      "negativeCosine": 0.30207613433089925,
      "margin": 0.46132133471994646,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 49.756749998778105,
      "durationMs": 51.254042016807944,
      "embeddingsPerSecond": 60.293327037510934,
      "inputTokens": [
        3,
        12
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 3201368064,
      "loadMs": 2700.319291994674,
      "totalMs": 2750.0760419934522,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000281018404,
        1.0000000021396556,
        1.000000037993779
      ],
      "positiveCosine": 0.943942833072416,
      "negativeCosine": 0.4327768926856611,
      "margin": 0.5111659403867549,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 47.07674999372102,
      "durationMs": 48.63008399843238,
      "embeddingsPerSecond": 63.725724490329775,
      "inputTokens": [
        3,
        9
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 3201499136,
      "loadMs": 2700.319291994674,
      "totalMs": 2747.396041988395,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000085974663,
        1.000000023292101,
        0.9999999888165177
      ],
      "positiveCosine": 0.7258271404676749,
      "negativeCosine": 0.3863600948947857,
      "margin": 0.33946704557288926,
      "minimumMargin": 0.05,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 189.49908399372362,
      "durationMs": 191.4200419851113,
      "embeddingsPerSecond": 15.831211089650242,
      "inputTokens": [
        3,
        412
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 3300409344,
      "loadMs": 2700.319291994674,
      "totalMs": 2889.8183759883977,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000092764376,
        1.0000000034604957,
        0.9999999673304596
      ],
      "positiveCosine": 0.743267347077721,
      "negativeCosine": 0.3583154440084353,
      "margin": 0.38495190306928573,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 404.2042499931995,
      "durationMs": 405.73304099962115,
      "embeddingsPerSecond": 7.421990243918695,
      "inputTokens": [
        3,
        16
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169080320,
      "loadMs": 2552.4202919914387,
      "totalMs": 2956.624541984638,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999997661901,
        1.0000000106659337,
        1.0000000237122044
      ],
      "positiveCosine": 0.7634482543094263,
      "negativeCosine": 0.3020474317270439,
      "margin": 0.4614008225823824,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 141.02437501423992,
      "durationMs": 142.54337499733083,
      "embeddingsPerSecond": 21.272918243368036,
      "inputTokens": [
        3,
        12
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169227776,
      "loadMs": 2552.4202919914387,
      "totalMs": 2693.4446670056786,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000271237672,
        1.0000000247315426,
        1.0000000081035167
      ],
      "positiveCosine": 0.943929555707606,
      "negativeCosine": 0.4328989041527131,
      "margin": 0.5110306515548928,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 141.25799998873845,
      "durationMs": 142.5396249978803,
      "embeddingsPerSecond": 21.23773520961057,
      "inputTokens": [
        3,
        9
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169358848,
      "loadMs": 2552.4202919914387,
      "totalMs": 2693.678291980177,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.000000038799976,
        1.0000000304194427,
        0.9999999895591469
      ],
      "positiveCosine": 0.7258514105169573,
      "negativeCosine": 0.38639587927503916,
      "margin": 0.3394555312419182,
      "minimumMargin": 0.05,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 372.6414160046261,
      "durationMs": 375.5367499834392,
      "embeddingsPerSecond": 8.050634929861788,
      "inputTokens": [
        3,
        412
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1335361536,
      "loadMs": 2552.4202919914387,
      "totalMs": 2925.061707996065,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000092764376,
        1.0000000034604957,
        0.9999999673304596
      ],
      "positiveCosine": 0.743267347077721,
      "negativeCosine": 0.3583154440084353,
      "margin": 0.38495190306928573,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 102.8521670086775,
      "durationMs": 104.09149999031797,
      "embeddingsPerSecond": 29.16807771047638,
      "inputTokens": [
        3,
        16
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169080320,
      "loadMs": 2531.7601659917273,
      "totalMs": 2634.6123330004048,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999997661901,
        1.0000000106659337,
        1.0000000237122044
      ],
      "positiveCosine": 0.7634482543094263,
      "negativeCosine": 0.3020474317270439,
      "margin": 0.4614008225823824,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 72.45362500543706,
      "durationMs": 73.96245902054943,
      "embeddingsPerSecond": 41.40579577315661,
      "inputTokens": [
        3,
        12
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169227776,
      "loadMs": 2531.7601659917273,
      "totalMs": 2604.2137909971643,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000271237672,
        1.0000000247315426,
        1.0000000081035167
      ],
      "positiveCosine": 0.943929555707606,
      "negativeCosine": 0.4328989041527131,
      "margin": 0.5110306515548928,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 46.965040994109586,
      "durationMs": 48.260957992170006,
      "embeddingsPerSecond": 63.8772997212174,
      "inputTokens": [
        3,
        9
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169358848,
      "loadMs": 2531.7601659917273,
      "totalMs": 2578.725206985837,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.000000038799976,
        1.0000000304194427,
        0.9999999895591469
      ],
      "positiveCosine": 0.7258514105169573,
      "negativeCosine": 0.38639587927503916,
      "margin": 0.3394555312419182,
      "minimumMargin": 0.05,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 173.794041009387,
      "durationMs": 175.71400001179427,
      "embeddingsPerSecond": 17.261811639663545,
      "inputTokens": [
        3,
        412
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1335377920,
      "loadMs": 2531.7601659917273,
      "totalMs": 2705.5542070011143,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000092764376,
        1.0000000034604957,
        0.9999999673304596
      ],
      "positiveCosine": 0.743267347077721,
      "negativeCosine": 0.3583154440084353,
      "margin": 0.38495190306928573,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 105.4312500054948,
      "durationMs": 106.879166996805,
      "embeddingsPerSecond": 28.454561620427036,
      "inputTokens": [
        3,
        16
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169080320,
      "loadMs": 2518.8049169955775,
      "totalMs": 2624.2361670010723,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999997661901,
        1.0000000106659337,
        1.0000000237122044
      ],
      "positiveCosine": 0.7634482543094263,
      "negativeCosine": 0.3020474317270439,
      "margin": 0.4614008225823824,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 48.29462498310022,
      "durationMs": 50.29379201005213,
      "embeddingsPerSecond": 62.118714060825454,
      "inputTokens": [
        3,
        12
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169227776,
      "loadMs": 2518.8049169955775,
      "totalMs": 2567.0995419786777,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000271237672,
        1.0000000247315426,
        1.0000000081035167
      ],
      "positiveCosine": 0.943929555707606,
      "negativeCosine": 0.4328989041527131,
      "margin": 0.5110306515548928,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 46.454541006824,
      "durationMs": 47.99516699858941,
      "embeddingsPerSecond": 64.57926254312385,
      "inputTokens": [
        3,
        9
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169358848,
      "loadMs": 2518.8049169955775,
      "totalMs": 2565.2594580024015,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.000000038799976,
        1.0000000304194427,
        0.9999999895591469
      ],
      "positiveCosine": 0.7258514105169573,
      "negativeCosine": 0.38639587927503916,
      "margin": 0.3394555312419182,
      "minimumMargin": 0.05,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 187.76245901244693,
      "durationMs": 190.2451670030132,
      "embeddingsPerSecond": 15.977634803989906,
      "inputTokens": [
        3,
        412
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1335377920,
      "loadMs": 2518.8049169955775,
      "totalMs": 2706.5673760080244,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999553672709,
        0.9999999663604554,
        1.000000035671252
      ],
      "positiveCosine": 0.7433577291917177,
      "negativeCosine": 0.35905120972474774,
      "margin": 0.38430651946697,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 410.6048330140766,
      "durationMs": 412.3247909883503,
      "embeddingsPerSecond": 7.306294906414685,
      "inputTokens": [
        3,
        16
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169080320,
      "loadMs": 2579.764666996198,
      "totalMs": 2990.3695000102744,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "bf16",
      "precision": "bf16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.00000000698783,
        1.0000000450324653,
        1.0000000189376141
      ],
      "positiveCosine": 0.7637661497915144,
      "negativeCosine": 0.3040330550405722,
      "margin": 0.45973309475094215,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 218.5002919868566,
      "durationMs": 220.28341601253487,
      "embeddingsPerSecond": 13.72995876902745,
      "inputTokens": [
        3,
        12
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169227776,
      "loadMs": 2579.764666996198,
      "totalMs": 2798.2649589830544,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "bf16",
      "precision": "bf16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999848599367,
        0.9999999961452326,
        0.999999999672102
      ],
      "positiveCosine": 0.9443278206375563,
      "negativeCosine": 0.4328479293612124,
      "margin": 0.5114798912763439,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 140.1039590127766,
      "durationMs": 141.9616669882089,
      "embeddingsPerSecond": 21.412671141765657,
      "inputTokens": [
        3,
        9
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169358848,
      "loadMs": 2579.764666996198,
      "totalMs": 2719.8686260089744,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "bf16",
      "precision": "bf16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.99999998146247,
        0.9999999892591173,
        1.0000000013889696
      ],
      "positiveCosine": 0.7220528607715129,
      "negativeCosine": 0.3861278659731936,
      "margin": 0.3359249947983193,
      "minimumMargin": 0.05,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 399.7247919905931,
      "durationMs": 402.0047080121003,
      "embeddingsPerSecond": 7.5051637029073754,
      "inputTokens": [
        3,
        412
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1335361536,
      "loadMs": 2579.764666996198,
      "totalMs": 2979.489458986791,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "bf16",
      "precision": "bf16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999553672709,
        0.9999999663604554,
        1.000000035671252
      ],
      "positiveCosine": 0.7433577291917177,
      "negativeCosine": 0.35905120972474774,
      "margin": 0.38430651946697,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 105.01891700550914,
      "durationMs": 106.43608402460814,
      "embeddingsPerSecond": 28.566282014150122,
      "inputTokens": [
        3,
        16
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169080320,
      "loadMs": 2567.703541019,
      "totalMs": 2672.7224580245093,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "bf16",
      "precision": "bf16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.00000000698783,
        1.0000000450324653,
        1.0000000189376141
      ],
      "positiveCosine": 0.7637661497915144,
      "negativeCosine": 0.3040330550405722,
      "margin": 0.45973309475094215,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 73.63445800729096,
      "durationMs": 75.12541601317935,
      "embeddingsPerSecond": 40.741795094124996,
      "inputTokens": [
        3,
        12
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169227776,
      "loadMs": 2567.703541019,
      "totalMs": 2641.337999026291,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "bf16",
      "precision": "bf16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999848599367,
        0.9999999961452326,
        0.999999999672102
      ],
      "positiveCosine": 0.9443278206375563,
      "negativeCosine": 0.4328479293612124,
      "margin": 0.5114798912763439,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 46.8056250247173,
      "durationMs": 48.62195797613822,
      "embeddingsPerSecond": 64.09486035953474,
      "inputTokens": [
        3,
        9
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169358848,
      "loadMs": 2567.703541019,
      "totalMs": 2614.5091660437174,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "bf16",
      "precision": "bf16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.99999998146247,
        0.9999999892591173,
        1.0000000013889696
      ],
      "positiveCosine": 0.7220528607715129,
      "negativeCosine": 0.3861278659731936,
      "margin": 0.3359249947983193,
      "minimumMargin": 0.05,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 173.04504200001247,
      "durationMs": 174.95295801199973,
      "embeddingsPerSecond": 17.336526752380365,
      "inputTokens": [
        3,
        412
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1335377920,
      "loadMs": 2567.703541019,
      "totalMs": 2740.7485830190126,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "bf16",
      "precision": "bf16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999553672709,
        0.9999999663604554,
        1.000000035671252
      ],
      "positiveCosine": 0.7433577291917177,
      "negativeCosine": 0.35905120972474774,
      "margin": 0.38430651946697,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 102.74033300811425,
      "durationMs": 104.07937498530373,
      "embeddingsPerSecond": 29.199827489006342,
      "inputTokens": [
        3,
        16
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169080320,
      "loadMs": 2524.6627499873284,
      "totalMs": 2627.4030829954427,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "bf16",
      "precision": "bf16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.00000000698783,
        1.0000000450324653,
        1.0000000189376141
      ],
      "positiveCosine": 0.7637661497915144,
      "negativeCosine": 0.3040330550405722,
      "margin": 0.45973309475094215,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 49.537874991074204,
      "durationMs": 50.5452920042444,
      "embeddingsPerSecond": 60.55972325297651,
      "inputTokens": [
        3,
        12
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169227776,
      "loadMs": 2524.6627499873284,
      "totalMs": 2574.2006249784026,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "bf16",
      "precision": "bf16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999848599367,
        0.9999999961452326,
        0.999999999672102
      ],
      "positiveCosine": 0.9443278206375563,
      "negativeCosine": 0.4328479293612124,
      "margin": 0.5114798912763439,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 45.952499989653006,
      "durationMs": 47.46812497614883,
      "embeddingsPerSecond": 65.28480497634517,
      "inputTokens": [
        3,
        9
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169358848,
      "loadMs": 2524.6627499873284,
      "totalMs": 2570.6152499769814,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "bf16",
      "precision": "bf16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.99999998146247,
        0.9999999892591173,
        1.0000000013889696
      ],
      "positiveCosine": 0.7220528607715129,
      "negativeCosine": 0.3861278659731936,
      "margin": 0.3359249947983193,
      "minimumMargin": 0.05,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 178.93258298863657,
      "durationMs": 181.38854100834578,
      "embeddingsPerSecond": 16.76609117183828,
      "inputTokens": [
        3,
        412
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1335377920,
      "loadMs": 2524.6627499873284,
      "totalMs": 2703.595332975965,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "bf16",
      "precision": "bf16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  }
]

BGE-M3 Unsupervised Desktop

48 configurations tested
Passed48
Failed0
TestedNov 13, 2026
Runtimetransformers 5.22.1

Quality criteria

{ "file": "research/bge-m3-family-expansion/quality-gates.json", "sha256": "e39823c58c7328c1f308cb1502358ee29930d02e2a3f8cb766f828bb1200b591", "description": "Finite unit1024d vectors, per-fixture positive-minus-negative cosine margins. Fresh inference<=10s, warm<=5s, load<=120s." }

PrecisionDeviceResultPlatformInference / load
FP16mpsAcceleratedPassedVery fastmacos · aarch6445 ms / 2451 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6446 ms / 2435 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6446 ms / 2605 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6448 ms / 2451 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6448 ms / 2616 ms
BF16mpsAcceleratedPassedVery fastmacos · aarch6448 ms / 2412 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6448 ms / 2670 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6449 ms / 2435 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6449 ms / 2605 ms
BF16mpsAcceleratedPassedVery fastmacos · aarch6449 ms / 2411 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6451 ms / 2475 ms
BF16mpsAcceleratedPassedVery fastmacos · aarch6451 ms / 2412 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6451 ms / 2475 ms
BF16mpsAcceleratedPassedVery fastmacos · aarch6452 ms / 2396 ms
BF16mpsAcceleratedPassedVery fastmacos · aarch6457 ms / 2396 ms
BF16mpsAcceleratedPassedVery fastmacos · aarch6465 ms / 2411 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6473 ms / 2616 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6474 ms / 2670 ms
FP32mpsAcceleratedPassedFastmacos · aarch64102 ms / 2616 ms
BF16mpsAcceleratedPassedFastmacos · aarch64102 ms / 2411 ms
FP32mpsAcceleratedPassedFastmacos · aarch64106 ms / 2605 ms
FP32mpsAcceleratedPassedFastmacos · aarch64107 ms / 2670 ms
FP16mpsAcceleratedPassedFastmacos · aarch64108 ms / 2475 ms
BF16mpsAcceleratedPassedFastmacos · aarch64110 ms / 2396 ms
FP16mpsAcceleratedPassedFastmacos · aarch64119 ms / 2451 ms
FP16mpsAcceleratedPassedFastmacos · aarch64122 ms / 2435 ms
BF16mpsAcceleratedPassedFastmacos · aarch64129 ms / 2412 ms
BF16mpsAcceleratedPassedFastmacos · aarch64173 ms / 2411 ms
FP16mpsAcceleratedPassedFastmacos · aarch64173 ms / 2451 ms
FP16mpsAcceleratedPassedFastmacos · aarch64175 ms / 2435 ms
BF16mpsAcceleratedPassedFastmacos · aarch64175 ms / 2412 ms
FP32mpsAcceleratedPassedFastmacos · aarch64190 ms / 2616 ms
FP32mpsAcceleratedPassedFastmacos · aarch64191 ms / 2605 ms
FP32mpsAcceleratedPassedFastmacos · aarch64194 ms / 2670 ms
FP16mpsAcceleratedPassedFastmacos · aarch64217 ms / 2475 ms
BF16mpsAcceleratedPassedFastmacos · aarch64234 ms / 2396 ms
FP32cpuPassedVery fastmacos · aarch6443 ms / 2265 ms
FP32cpuPassedVery fastmacos · aarch6444 ms / 2242 ms
FP32cpuPassedVery fastmacos · aarch6445 ms / 2303 ms
FP32cpuPassedVery fastmacos · aarch6454 ms / 2242 ms
FP32cpuPassedVery fastmacos · aarch6454 ms / 2265 ms
FP32cpuPassedVery fastmacos · aarch6456 ms / 2303 ms
FP32cpuPassedFastmacos · aarch64105 ms / 2265 ms
FP32cpuPassedFastmacos · aarch64106 ms / 2242 ms
FP32cpuPassedFastmacos · aarch64109 ms / 2303 ms
FP32cpuPassedFastmacos · aarch64516 ms / 2265 ms
FP32cpuPassedFastmacos · aarch64516 ms / 2242 ms
FP32cpuPassedFastmacos · aarch64522 ms / 2303 ms
Test setup & sample details
Quality gate
{ "file": "research/bge-m3-family-expansion/quality-gates.json", "sha256": "e39823c58c7328c1f308cb1502358ee29930d02e2a3f8cb766f828bb1200b591", "description": "Finite unit1024d vectors, per-fixture positive-minus-negative cosine margins. Fresh inference<=10s, warm<=5s, load<=120s." }
Tested
2026-11-13T04:37:34.622578+00:00
Environment
{ "os": "macos", "osVersion": "27.2", "architecture": "aarch64", "hardware": "Apple M3 Max;Mac15,11;96GiB RAM", "cpu": "Apple M3 Max", "gpu": "Apple M3 Max", "driver": "OS-managed Metal/MPS macOS27.2 build26B118", "runtime": "App-owned CPython3.13.18;Torch2.10.0;Transformers5.22.1", "timingScope": "Fresh model/helper per cold repetition, local original snapshot load, shared OS/driver caches may be warm. Four fixtures plus five warm calls; no network transfer included.", "concurrentUploadActivity": "none; uploads excluded" }
Runtime
transformers 5.22.1
Artifact revision
46f03bc86361cf88102b0b517b36c8259f2946b1
Fixture
{
  "file": "model-samples/native/bge-m3/quality-gates.json",
  "url": "https://assets.localai.tools/samples/v1/files/a10b184352cd5c00018eb629107ced06f6e55a9ea2cfd4cffb2f8574feb696a1/bge-m3-quality-gates.json",
  "size": 4205,
  "sha1": "c53d7a6368c4e7cc451f1fae92f04f27ca7c3884",
  "sha256": "a10b184352cd5c00018eb629107ced06f6e55a9ea2cfd4cffb2f8574feb696a1"
}
Rerun when
artifact-revision; runtime-version; fixture-checksum; quality-gate; backend-behavior
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999919190489,
        0.9999998906490281,
        1.0000000328663003
      ],
      "positiveCosine": 0.7831185302683791,
      "negativeCosine": 0.26061313651513,
      "margin": 0.5225053937532491,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 108.70691601303406,
      "durationMs": 109.34275001636706,
      "embeddingsPerSecond": 27.59714018232563,
      "inputTokens": [
        3,
        16
      ],
      "outputDevice": "cpu",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": null,
      "loadMs": 2303.188458987279,
      "totalMs": 2411.895375000313,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.999999951622245,
        0.9999999394727269,
        1.0000000062896863
      ],
      "positiveCosine": 0.7908431861636811,
      "negativeCosine": 0.3053681719134734,
      "margin": 0.4854750142502077,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 56.3091250078287,
      "durationMs": 56.66520798695274,
      "embeddingsPerSecond": 53.277332929306006,
      "inputTokens": [
        3,
        12
      ],
      "outputDevice": "cpu",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": null,
      "loadMs": 2303.188458987279,
      "totalMs": 2359.4975839951076,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.000000001575941,
        0.9999999429643861,
        0.9999999350934491
      ],
      "positiveCosine": 0.9031304553219311,
      "negativeCosine": 0.32214035217271186,
      "margin": 0.5809901031492193,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 44.886208983371034,
      "durationMs": 45.20929197315127,
      "embeddingsPerSecond": 66.83567331585985,
      "inputTokens": [
        3,
        9
      ],
      "outputDevice": "cpu",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": null,
      "loadMs": 2303.188458987279,
      "totalMs": 2348.07466797065,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.999999896287797,
        0.9999999322157993,
        0.9999999758104792
      ],
      "positiveCosine": 0.6053540306618209,
      "negativeCosine": 0.29909317256996343,
      "margin": 0.3062608580918575,
      "minimumMargin": 0.05,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 521.5836249990389,
      "durationMs": 522.8406250244007,
      "embeddingsPerSecond": 5.751714310443561,
      "inputTokens": [
        3,
        412
      ],
      "outputDevice": "cpu",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": null,
      "loadMs": 2303.188458987279,
      "totalMs": 2824.7720839863177,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999919190489,
        0.9999998906490281,
        1.0000000328663003
      ],
      "positiveCosine": 0.7831185302683791,
      "negativeCosine": 0.26061313651513,
      "margin": 0.5225053937532491,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 104.54091700376011,
      "durationMs": 105.09950001141988,
      "embeddingsPerSecond": 28.696897693102276,
      "inputTokens": [
        3,
        16
      ],
      "outputDevice": "cpu",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": null,
      "loadMs": 2264.6997919946443,
      "totalMs": 2369.2407089984044,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.999999951622245,
        0.9999999394727269,
        1.0000000062896863
      ],
      "positiveCosine": 0.7908431861636811,
      "negativeCosine": 0.3053681719134734,
      "margin": 0.4854750142502077,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 54.39504102105275,
      "durationMs": 54.76291599916294,
      "embeddingsPerSecond": 55.15208636094046,
      "inputTokens": [
        3,
        12
      ],
      "outputDevice": "cpu",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": null,
      "loadMs": 2264.6997919946443,
      "totalMs": 2319.094833015697,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.000000001575941,
        0.9999999429643861,
        0.9999999350934491
      ],
      "positiveCosine": 0.9031304553219311,
      "negativeCosine": 0.32214035217271186,
      "margin": 0.5809901031492193,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 43.383332988014445,
      "durationMs": 43.72795898234472,
      "embeddingsPerSecond": 69.15098018930018,
      "inputTokens": [
        3,
        9
      ],
      "outputDevice": "cpu",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": null,
      "loadMs": 2264.6997919946443,
      "totalMs": 2308.083124982659,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.999999896287797,
        0.9999999322157993,
        0.9999999758104792
      ],
      "positiveCosine": 0.6053540306618209,
      "negativeCosine": 0.29909317256996343,
      "margin": 0.3062608580918575,
      "minimumMargin": 0.05,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 515.8579999988433,
      "durationMs": 517.0528340095188,
      "embeddingsPerSecond": 5.815553892750964,
      "inputTokens": [
        3,
        412
      ],
      "outputDevice": "cpu",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": null,
      "loadMs": 2264.6997919946443,
      "totalMs": 2780.5577919934876,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999919190489,
        0.9999998906490281,
        1.0000000328663003
      ],
      "positiveCosine": 0.7831185302683791,
      "negativeCosine": 0.26061313651513,
      "margin": 0.5225053937532491,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 105.73974999715574,
      "durationMs": 106.27895899233408,
      "embeddingsPerSecond": 28.37154428756164,
      "inputTokens": [
        3,
        16
      ],
      "outputDevice": "cpu",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": null,
      "loadMs": 2241.5913750010077,
      "totalMs": 2347.3311249981634,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.999999951622245,
        0.9999999394727269,
        1.0000000062896863
      ],
      "positiveCosine": 0.7908431861636811,
      "negativeCosine": 0.3053681719134734,
      "margin": 0.4854750142502077,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 54.32512500556186,
      "durationMs": 54.681291978340596,
      "embeddingsPerSecond": 55.22306666929634,
      "inputTokens": [
        3,
        12
      ],
      "outputDevice": "cpu",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": null,
      "loadMs": 2241.5913750010077,
      "totalMs": 2295.9165000065695,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.000000001575941,
        0.9999999429643861,
        0.9999999350934491
      ],
      "positiveCosine": 0.9031304553219311,
      "negativeCosine": 0.32214035217271186,
      "margin": 0.5809901031492193,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 43.5617080074735,
      "durationMs": 43.90608301036991,
      "embeddingsPerSecond": 68.86782307721535,
      "inputTokens": [
        3,
        9
      ],
      "outputDevice": "cpu",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": null,
      "loadMs": 2241.5913750010077,
      "totalMs": 2285.153083008481,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.999999896287797,
        0.9999999322157993,
        0.9999999758104792
      ],
      "positiveCosine": 0.6053540306618209,
      "negativeCosine": 0.29909317256996343,
      "margin": 0.3062608580918575,
      "minimumMargin": 0.05,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 516.3936669996474,
      "durationMs": 517.554874997586,
      "embeddingsPerSecond": 5.809521285244671,
      "inputTokens": [
        3,
        412
      ],
      "outputDevice": "cpu",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": null,
      "loadMs": 2241.5913750010077,
      "totalMs": 2757.985042000655,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000391316581,
        1.0000000126105106,
        0.9999999853970182
      ],
      "positiveCosine": 0.7831185799856215,
      "negativeCosine": 0.26061306939861517,
      "margin": 0.5225055105870063,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 106.79116600658745,
      "durationMs": 108.02129202056676,
      "embeddingsPerSecond": 28.092211295969406,
      "inputTokens": [
        3,
        16
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 3201220608,
      "loadMs": 2670.103458018275,
      "totalMs": 2776.8946240248624,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999641638627,
        1.0000000055064022,
        0.9999999914255607
      ],
      "positiveCosine": 0.7908433501065775,
      "negativeCosine": 0.3053681673377815,
      "margin": 0.485475182768796,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 48.4540420002304,
      "durationMs": 49.598832993069664,
      "embeddingsPerSecond": 61.91433936483018,
      "inputTokens": [
        3,
        12
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 3201368064,
      "loadMs": 2670.103458018275,
      "totalMs": 2718.5575000185054,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999921459959,
        0.9999999645904396,
        0.9999999996279445
      ],
      "positiveCosine": 0.9031305300960925,
      "negativeCosine": 0.32214022430992617,
      "margin": 0.5809903057861663,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 74.23920801375061,
      "durationMs": 75.60870799352415,
      "embeddingsPerSecond": 40.4099138482773,
      "inputTokens": [
        3,
        9
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 3201499136,
      "loadMs": 2670.103458018275,
      "totalMs": 2744.3426660320256,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.000000001672196,
        0.9999999964118276,
        1.0000000068642085
      ],
      "positiveCosine": 0.6053540969128707,
      "negativeCosine": 0.29909310374507025,
      "margin": 0.3062609931678004,
      "minimumMargin": 0.05,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 193.9882499864325,
      "durationMs": 195.97079200320877,
      "embeddingsPerSecond": 15.464854186837705,
      "inputTokens": [
        3,
        412
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 3300409344,
      "loadMs": 2670.103458018275,
      "totalMs": 2864.0917080047075,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000391316581,
        1.0000000126105106,
        0.9999999853970182
      ],
      "positiveCosine": 0.7831185799856215,
      "negativeCosine": 0.26061306939861517,
      "margin": 0.5225055105870063,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 106.22333400533535,
      "durationMs": 108.12370802159421,
      "embeddingsPerSecond": 28.242382223187583,
      "inputTokens": [
        3,
        16
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 3201220608,
      "loadMs": 2604.8076250008307,
      "totalMs": 2711.030959006166,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999641638627,
        1.0000000055064022,
        0.9999999914255607
      ],
      "positiveCosine": 0.7908433501065775,
      "negativeCosine": 0.3053681673377815,
      "margin": 0.485475182768796,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 48.634083999786526,
      "durationMs": 50.17108301399276,
      "embeddingsPerSecond": 61.68513423658125,
      "inputTokens": [
        3,
        12
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 3201368064,
      "loadMs": 2604.8076250008307,
      "totalMs": 2653.4417090006173,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999921459959,
        0.9999999645904396,
        0.9999999996279445
      ],
      "positiveCosine": 0.9031305300960925,
      "negativeCosine": 0.32214022430992617,
      "margin": 0.5809903057861663,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 45.65954199642874,
      "durationMs": 46.76174998166971,
      "embeddingsPerSecond": 65.70368139554805,
      "inputTokens": [
        3,
        9
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 3201499136,
      "loadMs": 2604.8076250008307,
      "totalMs": 2650.4671669972595,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.000000001672196,
        0.9999999964118276,
        1.0000000068642085
      ],
      "positiveCosine": 0.6053540969128707,
      "negativeCosine": 0.29909310374507025,
      "margin": 0.3062609931678004,
      "minimumMargin": 0.05,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 191.27533299615607,
      "durationMs": 193.4274999948684,
      "embeddingsPerSecond": 15.684196979337054,
      "inputTokens": [
        3,
        412
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 3300409344,
      "loadMs": 2604.8076250008307,
      "totalMs": 2796.082957996987,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000391316581,
        1.0000000126105106,
        0.9999999853970182
      ],
      "positiveCosine": 0.7831185799856215,
      "negativeCosine": 0.26061306939861517,
      "margin": 0.5225055105870063,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 102.22508298465982,
      "durationMs": 103.91816700575873,
      "embeddingsPerSecond": 29.347004789912358,
      "inputTokens": [
        3,
        16
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 3201220608,
      "loadMs": 2615.6165410066023,
      "totalMs": 2717.841623991262,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999641638627,
        1.0000000055064022,
        0.9999999914255607
      ],
      "positiveCosine": 0.7908433501065775,
      "negativeCosine": 0.3053681673377815,
      "margin": 0.485475182768796,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 73.29283299623057,
      "durationMs": 74.40583399147727,
      "embeddingsPerSecond": 40.93169655693742,
      "inputTokens": [
        3,
        12
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 3201368064,
      "loadMs": 2615.6165410066023,
      "totalMs": 2688.909374002833,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999921459959,
        0.9999999645904396,
        0.9999999996279445
      ],
      "positiveCosine": 0.9031305300960925,
      "negativeCosine": 0.32214022430992617,
      "margin": 0.5809903057861663,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 47.678416973212734,
      "durationMs": 49.208082986297086,
      "embeddingsPerSecond": 62.92155214980179,
      "inputTokens": [
        3,
        9
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 3201499136,
      "loadMs": 2615.6165410066023,
      "totalMs": 2663.294957979815,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.000000001672196,
        0.9999999964118276,
        1.0000000068642085
      ],
      "positiveCosine": 0.6053540969128707,
      "negativeCosine": 0.29909310374507025,
      "margin": 0.3062609931678004,
      "minimumMargin": 0.05,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 190.07966699427925,
      "durationMs": 192.3997919948306,
      "embeddingsPerSecond": 15.782855933192948,
      "inputTokens": [
        3,
        412
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 3300409344,
      "loadMs": 2615.6165410066023,
      "totalMs": 2805.6962080008816,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.000000047842044,
        0.9999999824377083,
        1.0000000347496425
      ],
      "positiveCosine": 0.783096706976447,
      "negativeCosine": 0.2605986425781911,
      "margin": 0.5224980643982559,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 107.95000000507571,
      "durationMs": 109.85766598605551,
      "embeddingsPerSecond": 27.79064381527506,
      "inputTokens": [
        3,
        16
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169080320,
      "loadMs": 2474.754249997204,
      "totalMs": 2582.70425000228,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000249173837,
        0.9999999989486017,
        1.0000000119134322
      ],
      "positiveCosine": 0.7909186035281347,
      "negativeCosine": 0.30540580734029427,
      "margin": 0.48551279618784043,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 51.12533297506161,
      "durationMs": 52.177875011693686,
      "embeddingsPerSecond": 58.67932442539529,
      "inputTokens": [
        3,
        12
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169227776,
      "loadMs": 2474.754249997204,
      "totalMs": 2525.879582972266,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999770355187,
        0.9999999901279779,
        1.000000031815608
      ],
      "positiveCosine": 0.9031291504711986,
      "negativeCosine": 0.3221932846111982,
      "margin": 0.5809358658600003,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 51.38158399495296,
      "durationMs": 52.92908300179988,
      "embeddingsPerSecond": 58.38667800305027,
      "inputTokens": [
        3,
        9
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169358848,
      "loadMs": 2474.754249997204,
      "totalMs": 2526.135833992157,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999856587625,
        1.0000000463918306,
        1.0000000508813236
      ],
      "positiveCosine": 0.6052298903529097,
      "negativeCosine": 0.2991912456410045,
      "margin": 0.30603864471190523,
      "minimumMargin": 0.05,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 217.45079098036513,
      "durationMs": 219.83733400702477,
      "embeddingsPerSecond": 13.796224821600612,
      "inputTokens": [
        3,
        412
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1335377920,
      "loadMs": 2474.754249997204,
      "totalMs": 2692.2050409775693,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.000000047842044,
        0.9999999824377083,
        1.0000000347496425
      ],
      "positiveCosine": 0.783096706976447,
      "negativeCosine": 0.2605986425781911,
      "margin": 0.5224980643982559,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 118.97816698183306,
      "durationMs": 120.40824998985045,
      "embeddingsPerSecond": 25.2147101951745,
      "inputTokens": [
        3,
        16
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169080320,
      "loadMs": 2450.545666011749,
      "totalMs": 2569.523832993582,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000249173837,
        0.9999999989486017,
        1.0000000119134322
      ],
      "positiveCosine": 0.7909186035281347,
      "negativeCosine": 0.30540580734029427,
      "margin": 0.48551279618784043,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 47.552208008710295,
      "durationMs": 48.778958996990696,
      "embeddingsPerSecond": 63.088553100425536,
      "inputTokens": [
        3,
        12
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169227776,
      "loadMs": 2450.545666011749,
      "totalMs": 2498.097874020459,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999770355187,
        0.9999999901279779,
        1.000000031815608
      ],
      "positiveCosine": 0.9031291504711986,
      "negativeCosine": 0.3221932846111982,
      "margin": 0.5809358658600003,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 45.469332981156185,
      "durationMs": 46.52912501478568,
      "embeddingsPerSecond": 65.9785354943141,
      "inputTokens": [
        3,
        9
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169358848,
      "loadMs": 2450.545666011749,
      "totalMs": 2496.014998992905,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999856587625,
        1.0000000463918306,
        1.0000000508813236
      ],
      "positiveCosine": 0.6052298903529097,
      "negativeCosine": 0.2991912456410045,
      "margin": 0.30603864471190523,
      "minimumMargin": 0.05,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 173.43391699250787,
      "durationMs": 175.36533399834298,
      "embeddingsPerSecond": 17.29765464577264,
      "inputTokens": [
        3,
        412
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1335377920,
      "loadMs": 2450.545666011749,
      "totalMs": 2623.9795830042567,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.000000047842044,
        0.9999999824377083,
        1.0000000347496425
      ],
      "positiveCosine": 0.783096706976447,
      "negativeCosine": 0.2605986425781911,
      "margin": 0.5224980643982559,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 122.07816599402577,
      "durationMs": 124.14224998792633,
      "embeddingsPerSecond": 24.57441898452843,
      "inputTokens": [
        3,
        16
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169080320,
      "loadMs": 2435.1607080025133,
      "totalMs": 2557.238873996539,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000249173837,
        0.9999999989486017,
        1.0000000119134322
      ],
      "positiveCosine": 0.7909186035281347,
      "negativeCosine": 0.30540580734029427,
      "margin": 0.48551279618784043,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 48.502457997528836,
      "durationMs": 49.7327079938259,
      "embeddingsPerSecond": 61.852535394244306,
      "inputTokens": [
        3,
        12
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169227776,
      "loadMs": 2435.1607080025133,
      "totalMs": 2483.663166000042,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999770355187,
        0.9999999901279779,
        1.000000031815608
      ],
      "positiveCosine": 0.9031291504711986,
      "negativeCosine": 0.3221932846111982,
      "margin": 0.5809358658600003,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 45.51262498716824,
      "durationMs": 47.56970901507884,
      "embeddingsPerSecond": 65.91577613565062,
      "inputTokens": [
        3,
        9
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169358848,
      "loadMs": 2435.1607080025133,
      "totalMs": 2480.6733329896815,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999856587625,
        1.0000000463918306,
        1.0000000508813236
      ],
      "positiveCosine": 0.6052298903529097,
      "negativeCosine": 0.2991912456410045,
      "margin": 0.30603864471190523,
      "minimumMargin": 0.05,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 174.631749978289,
      "durationMs": 176.83991699595936,
      "embeddingsPerSecond": 17.179006683337786,
      "inputTokens": [
        3,
        412
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1335377920,
      "loadMs": 2435.1607080025133,
      "totalMs": 2609.7924579808023,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000610214153,
        0.9999999944458311,
        1.0000000364364932
      ],
      "positiveCosine": 0.7833779773270386,
      "negativeCosine": 0.2620974510507668,
      "margin": 0.5212805262762719,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 110.14529201202095,
      "durationMs": 112.05020800116472,
      "embeddingsPerSecond": 27.236751977311826,
      "inputTokens": [
        3,
        16
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169080320,
      "loadMs": 2395.604416989954,
      "totalMs": 2505.749709001975,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "bf16",
      "precision": "bf16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999879816657,
        1.000000023902974,
        1.000000022931096
      ],
      "positiveCosine": 0.7906741263036614,
      "negativeCosine": 0.3070256342610816,
      "margin": 0.4836484920425798,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 57.11699998937547,
      "durationMs": 59.18620800366625,
      "embeddingsPerSecond": 52.52376701433969,
      "inputTokens": [
        3,
        12
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169227776,
      "loadMs": 2395.604416989954,
      "totalMs": 2452.7214169793297,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "bf16",
      "precision": "bf16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999691077012,
        1.0000000308523842,
        1.000000019194924
      ],
      "positiveCosine": 0.9036814311043841,
      "negativeCosine": 0.32307195044404424,
      "margin": 0.5806094806603399,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 52.239707991248,
      "durationMs": 53.896832978352904,
      "embeddingsPerSecond": 57.42757981156032,
      "inputTokens": [
        3,
        9
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169358848,
      "loadMs": 2395.604416989954,
      "totalMs": 2447.844124981202,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "bf16",
      "precision": "bf16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999946080774,
        0.9999999952509451,
        0.9999999779599524
      ],
      "positiveCosine": 0.606599438618612,
      "negativeCosine": 0.29973293045521876,
      "margin": 0.3068665081633932,
      "minimumMargin": 0.05,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 233.88874999363907,
      "durationMs": 236.69350001728162,
      "embeddingsPerSecond": 12.826610942516856,
      "inputTokens": [
        3,
        412
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1335377920,
      "loadMs": 2395.604416989954,
      "totalMs": 2629.4931669835933,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "bf16",
      "precision": "bf16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000610214153,
        0.9999999944458311,
        1.0000000364364932
      ],
      "positiveCosine": 0.7833779773270386,
      "negativeCosine": 0.2620974510507668,
      "margin": 0.5212805262762719,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 128.65258299279958,
      "durationMs": 130.38779198541306,
      "embeddingsPerSecond": 23.318614599194667,
      "inputTokens": [
        3,
        16
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169080320,
      "loadMs": 2411.522624985082,
      "totalMs": 2540.1752079778817,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "bf16",
      "precision": "bf16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999879816657,
        1.000000023902974,
        1.000000022931096
      ],
      "positiveCosine": 0.7906741263036614,
      "negativeCosine": 0.3070256342610816,
      "margin": 0.4836484920425798,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 51.21158401016146,
      "durationMs": 52.237500000046566,
      "embeddingsPerSecond": 58.580496151119576,
      "inputTokens": [
        3,
        12
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169227776,
      "loadMs": 2411.522624985082,
      "totalMs": 2462.7342089952435,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "bf16",
      "precision": "bf16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999691077012,
        1.0000000308523842,
        1.000000019194924
      ],
      "positiveCosine": 0.9036814311043841,
      "negativeCosine": 0.32307195044404424,
      "margin": 0.5806094806603399,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 48.01475000567734,
      "durationMs": 49.492624995764345,
      "embeddingsPerSecond": 62.48080016339302,
      "inputTokens": [
        3,
        9
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169358848,
      "loadMs": 2411.522624985082,
      "totalMs": 2459.5373749907594,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "bf16",
      "precision": "bf16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999946080774,
        0.9999999952509451,
        0.9999999779599524
      ],
      "positiveCosine": 0.606599438618612,
      "negativeCosine": 0.29973293045521876,
      "margin": 0.3068665081633932,
      "minimumMargin": 0.05,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 175.24341700482182,
      "durationMs": 177.02770797768608,
      "embeddingsPerSecond": 17.119045332912307,
      "inputTokens": [
        3,
        412
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1335377920,
      "loadMs": 2411.522624985082,
      "totalMs": 2586.766041989904,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "bf16",
      "precision": "bf16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        1.0000000610214153,
        0.9999999944458311,
        1.0000000364364932
      ],
      "positiveCosine": 0.7833779773270386,
      "negativeCosine": 0.2620974510507668,
      "margin": 0.5212805262762719,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 102.38758299965411,
      "durationMs": 104.04912501689978,
      "embeddingsPerSecond": 29.30042796312649,
      "inputTokens": [
        3,
        16
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169080320,
      "loadMs": 2410.865083016688,
      "totalMs": 2513.252666016342,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "bf16",
      "precision": "bf16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999879816657,
        1.000000023902974,
        1.000000022931096
      ],
      "positiveCosine": 0.7906741263036614,
      "negativeCosine": 0.3070256342610816,
      "margin": 0.4836484920425798,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 49.33604202233255,
      "durationMs": 50.47779198503122,
      "embeddingsPerSecond": 60.8074721243754,
      "inputTokens": [
        3,
        12
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169227776,
      "loadMs": 2410.865083016688,
      "totalMs": 2460.2011250390206,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "bf16",
      "precision": "bf16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999691077012,
        1.0000000308523842,
        1.000000019194924
      ],
      "positiveCosine": 0.9036814311043841,
      "negativeCosine": 0.32307195044404424,
      "margin": 0.5806094806603399,
      "minimumMargin": 0.1,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 65.32241599052213,
      "durationMs": 66.90779200289398,
      "embeddingsPerSecond": 45.92604168889407,
      "inputTokens": [
        3,
        9
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1169358848,
      "loadMs": 2410.865083016688,
      "totalMs": 2476.18749900721,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "bf16",
      "precision": "bf16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "finite": true,
      "norms": [
        0.9999999946080774,
        0.9999999952509451,
        0.9999999779599524
      ],
      "positiveCosine": 0.606599438618612,
      "negativeCosine": 0.29973293045521876,
      "margin": 0.3068665081633932,
      "minimumMargin": 0.05,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "inferenceMs": 173.06604099576361,
      "durationMs": 175.21312498138286,
      "embeddingsPerSecond": 17.33442322213539,
      "inputTokens": [
        3,
        412
      ],
      "outputDevice": "mps",
      "rssScope": "model process",
      "driverAllocatedMemoryBytes": 1335377920,
      "loadMs": 2410.865083016688,
      "totalMs": 2583.9311240124516,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "bf16",
      "precision": "bf16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  }
]
Release validation still required
  • Sample publication, complete dependency backup and forced model/runtime-source recovery pending under user upload exclusion.

Something not quite right?

Tell us what happened with BGE-M3 Desktop. Loading, results, or the page itself—we’re here for all of it.

Attachments Optional

Up to 3 files · 50 MB each

Browser, OS, CPU/GPU, memory, screen, storage, and network details are included automatically. Model inputs, results, and browsing history are not collected.