BGE-M3 Desktop
Create 1024-dimensional dense text embeddings for multilingual search with the exact Multilingual checkpoint.
Desktop app
Run BGE-M3 Desktop on your computer
Download the desktop app to load this model, provide input and view results locally.
Public desktop downloads are coming soon.Device requirements
If your device falls short
MemoryAdvisory only: close other apps, reduce image/tile/token/batch settings or choose a smaller verified model/precision. CPU can need more RAM than GPU.
GPUIf the selected GPU runtime fails, existing CPU fallback may recover when enabled, but CPU has its own memory/runtime needs.
PlatformUse a compatible OS/architecture/runtime or another model. These estimates do not add hardware detection or block runs.
Benchmark evidence
Saved desktop measurements apply to the recorded hardware and runtime. Fallback cases report the device that actually ran; they do not prove GPU support on the requested device.
BGE-M3 Multilingual Desktop
48 configurations testedQuality criteria
{ "file": "research/bge-m3-family-expansion/quality-gates.json", "sha256": "e39823c58c7328c1f308cb1502358ee29930d02e2a3f8cb766f828bb1200b591", "description": "Finite unit1024d vectors, per-fixture positive-minus-negative cosine margins. Fresh inference<=10s, warm<=5s, load<=120s." }
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
BF16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 46 ms / 2525 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 46 ms / 2519 ms |
BF16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 47 ms / 2568 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 47 ms / 2532 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 47 ms / 2700 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 48 ms / 2519 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 49 ms / 2687 ms |
BF16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 50 ms / 2525 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 50 ms / 2700 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 51 ms / 2687 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 72 ms / 2532 ms |
BF16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 74 ms / 2568 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 103 ms / 2525 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 103 ms / 2532 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 104 ms / 2700 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 105 ms / 2568 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 105 ms / 2519 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 108 ms / 2687 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 140 ms / 2580 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 140 ms / 2759 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 141 ms / 2552 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 141 ms / 2552 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 158 ms / 2759 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 173 ms / 2568 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 174 ms / 2532 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 179 ms / 2525 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 188 ms / 2519 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 189 ms / 2700 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 205 ms / 2687 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 219 ms / 2580 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 357 ms / 2759 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 373 ms / 2552 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 389 ms / 2759 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 400 ms / 2580 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 404 ms / 2552 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 411 ms / 2580 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 42 ms / 2338 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 42 ms / 2321 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 46 ms / 8682 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 53 ms / 2338 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 53 ms / 2321 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 56 ms / 8682 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 104 ms / 2321 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 109 ms / 8682 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 110 ms / 2338 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 510 ms / 2321 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 511 ms / 2338 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 525 ms / 8682 ms |
Test setup & sample details
- Quality gate
- { "file": "research/bge-m3-family-expansion/quality-gates.json", "sha256": "e39823c58c7328c1f308cb1502358ee29930d02e2a3f8cb766f828bb1200b591", "description": "Finite unit1024d vectors, per-fixture positive-minus-negative cosine margins. Fresh inference<=10s, warm<=5s, load<=120s." }
- Tested
- 2026-11-13T04:37:34.622578+00:00
- Environment
- { "os": "macos", "osVersion": "27.2", "architecture": "aarch64", "hardware": "Apple M3 Max;Mac15,11;96GiB RAM", "cpu": "Apple M3 Max", "gpu": "Apple M3 Max", "driver": "OS-managed Metal/MPS macOS27.2 build26B118", "runtime": "App-owned CPython3.13.18;Torch2.10.0;Transformers5.22.1", "timingScope": "Fresh model/helper per cold repetition, local original snapshot load, shared OS/driver caches may be warm. Four fixtures plus five warm calls; no network transfer included.", "concurrentUploadActivity": "none; uploads excluded" }
- Runtime
- transformers 5.22.1
- Artifact revision
- 5617a9f61b028005a4858fdac845db406aefb181
- Fixture
{ "file": "model-samples/native/bge-m3/quality-gates.json", "url": "https://assets.localai.tools/samples/v1/files/a10b184352cd5c00018eb629107ced06f6e55a9ea2cfd4cffb2f8574feb696a1/bge-m3-quality-gates.json", "size": 4205, "sha1": "c53d7a6368c4e7cc451f1fae92f04f27ca7c3884", "sha256": "a10b184352cd5c00018eb629107ced06f6e55a9ea2cfd4cffb2f8574feb696a1" }- Rerun when
- artifact-revision; runtime-version; fixture-checksum; quality-gate; backend-behavior
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000077172053,
0.999999979059761,
0.9999999974625967
],
"positiveCosine": 0.7432051905918331,
"negativeCosine": 0.35828289135672914,
"margin": 0.38492229923510396,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 109.16208298294805,
"durationMs": 110.0628330023028,
"embeddingsPerSecond": 27.48206994610595,
"inputTokens": [
3,
16
],
"outputDevice": "cpu",
"rssScope": "model process",
"driverAllocatedMemoryBytes": null,
"loadMs": 8681.698000000324,
"totalMs": 8790.860082983272,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000576072567,
0.9999999135592897,
0.9999998999542602
],
"positiveCosine": 0.7633974456480599,
"negativeCosine": 0.3020763840952494,
"margin": 0.4613210615528105,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 55.70595900644548,
"durationMs": 56.1079999897629,
"embeddingsPerSecond": 53.85420255762734,
"inputTokens": [
3,
12
],
"outputDevice": "cpu",
"rssScope": "model process",
"driverAllocatedMemoryBytes": null,
"loadMs": 8681.698000000324,
"totalMs": 8737.40395900677,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000438697059,
0.9999999506879391,
1.0000000712395805
],
"positiveCosine": 0.9439428010712417,
"negativeCosine": 0.43277716354344437,
"margin": 0.5111656375277973,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 46.14387499168515,
"durationMs": 46.53837499790825,
"embeddingsPerSecond": 65.01404575451416,
"inputTokens": [
3,
9
],
"outputDevice": "cpu",
"rssScope": "model process",
"driverAllocatedMemoryBytes": null,
"loadMs": 8681.698000000324,
"totalMs": 8727.84187499201,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000479274251,
0.9999999177035241,
0.9999999945266407
],
"positiveCosine": 0.7258266848073294,
"negativeCosine": 0.3863602191538423,
"margin": 0.3394664656534871,
"minimumMargin": 0.05,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 525.4946249770001,
"durationMs": 526.7516250023618,
"embeddingsPerSecond": 5.70890710848147,
"inputTokens": [
3,
412
],
"outputDevice": "cpu",
"rssScope": "model process",
"driverAllocatedMemoryBytes": null,
"loadMs": 8681.698000000324,
"totalMs": 9207.192624977324,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000077172053,
0.999999979059761,
0.9999999974625967
],
"positiveCosine": 0.7432051905918331,
"negativeCosine": 0.35828289135672914,
"margin": 0.38492229923510396,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 109.90562499500811,
"durationMs": 110.48520900658332,
"embeddingsPerSecond": 27.296146126608708,
"inputTokens": [
3,
16
],
"outputDevice": "cpu",
"rssScope": "model process",
"driverAllocatedMemoryBytes": null,
"loadMs": 2338.448916998459,
"totalMs": 2448.354541993467,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000576072567,
0.9999999135592897,
0.9999998999542602
],
"positiveCosine": 0.7633974456480599,
"negativeCosine": 0.3020763840952494,
"margin": 0.4613210615528105,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 53.28379201819189,
"durationMs": 53.62704099388793,
"embeddingsPerSecond": 56.30229918650975,
"inputTokens": [
3,
12
],
"outputDevice": "cpu",
"rssScope": "model process",
"driverAllocatedMemoryBytes": null,
"loadMs": 2338.448916998459,
"totalMs": 2391.732709016651,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000438697059,
0.9999999506879391,
1.0000000712395805
],
"positiveCosine": 0.9439428010712417,
"negativeCosine": 0.43277716354344437,
"margin": 0.5111656375277973,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 42.05408302368596,
"durationMs": 42.39675001008436,
"embeddingsPerSecond": 71.33671178397401,
"inputTokens": [
3,
9
],
"outputDevice": "cpu",
"rssScope": "model process",
"driverAllocatedMemoryBytes": null,
"loadMs": 2338.448916998459,
"totalMs": 2380.503000022145,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000479274251,
0.9999999177035241,
0.9999999945266407
],
"positiveCosine": 0.7258266848073294,
"negativeCosine": 0.3863602191538423,
"margin": 0.3394664656534871,
"minimumMargin": 0.05,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 511.1463750072289,
"durationMs": 512.3583340027835,
"embeddingsPerSecond": 5.869160277146781,
"inputTokens": [
3,
412
],
"outputDevice": "cpu",
"rssScope": "model process",
"driverAllocatedMemoryBytes": null,
"loadMs": 2338.448916998459,
"totalMs": 2849.595292005688,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000077172053,
0.999999979059761,
0.9999999974625967
],
"positiveCosine": 0.7432051905918331,
"negativeCosine": 0.35828289135672914,
"margin": 0.38492229923510396,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 103.72541699325666,
"durationMs": 104.24270899966359,
"embeddingsPerSecond": 28.92251568576518,
"inputTokens": [
3,
16
],
"outputDevice": "cpu",
"rssScope": "model process",
"driverAllocatedMemoryBytes": null,
"loadMs": 2321.281832992099,
"totalMs": 2425.007249985356,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000576072567,
0.9999999135592897,
0.9999998999542602
],
"positiveCosine": 0.7633974456480599,
"negativeCosine": 0.3020763840952494,
"margin": 0.4613210615528105,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 53.44929202692583,
"durationMs": 53.82724999799393,
"embeddingsPerSecond": 56.12796514664231,
"inputTokens": [
3,
12
],
"outputDevice": "cpu",
"rssScope": "model process",
"driverAllocatedMemoryBytes": null,
"loadMs": 2321.281832992099,
"totalMs": 2374.731125019025,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000438697059,
0.9999999506879391,
1.0000000712395805
],
"positiveCosine": 0.9439428010712417,
"negativeCosine": 0.43277716354344437,
"margin": 0.5111656375277973,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 42.35966599662788,
"durationMs": 42.678625002736226,
"embeddingsPerSecond": 70.82208816846716,
"inputTokens": [
3,
9
],
"outputDevice": "cpu",
"rssScope": "model process",
"driverAllocatedMemoryBytes": null,
"loadMs": 2321.281832992099,
"totalMs": 2363.641498988727,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000479274251,
0.9999999177035241,
0.9999999945266407
],
"positiveCosine": 0.7258266848073294,
"negativeCosine": 0.3863602191538423,
"margin": 0.3394664656534871,
"minimumMargin": 0.05,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 510.2804169873707,
"durationMs": 511.4870000106748,
"embeddingsPerSecond": 5.879120381910029,
"inputTokens": [
3,
412
],
"outputDevice": "cpu",
"rssScope": "model process",
"driverAllocatedMemoryBytes": null,
"loadMs": 2321.281832992099,
"totalMs": 2831.56224997947,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000211870428,
0.9999999929230169,
0.9999999767896206
],
"positiveCosine": 0.7432051033610728,
"negativeCosine": 0.3582827292675386,
"margin": 0.3849223740935342,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 389.2305839981418,
"durationMs": 390.8427499991376,
"embeddingsPerSecond": 7.707513549383164,
"inputTokens": [
3,
16
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 3201220608,
"loadMs": 2759.243124979548,
"totalMs": 3148.47370897769,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999424989322,
0.9999999961769176,
1.000000019394219
],
"positiveCosine": 0.7633974690508457,
"negativeCosine": 0.30207613433089925,
"margin": 0.46132133471994646,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 157.86604100139812,
"durationMs": 159.33712502010167,
"embeddingsPerSecond": 19.003453693840534,
"inputTokens": [
3,
12
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 3201368064,
"loadMs": 2759.243124979548,
"totalMs": 2917.1091659809463,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000281018404,
1.0000000021396556,
1.000000037993779
],
"positiveCosine": 0.943942833072416,
"negativeCosine": 0.4327768926856611,
"margin": 0.5111659403867549,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 140.21754200803116,
"durationMs": 141.72612500260584,
"embeddingsPerSecond": 21.395325841813506,
"inputTokens": [
3,
9
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 3201482752,
"loadMs": 2759.243124979548,
"totalMs": 2899.4606669875793,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000085974663,
1.000000023292101,
0.9999999888165177
],
"positiveCosine": 0.7258271404676749,
"negativeCosine": 0.3863600948947857,
"margin": 0.33946704557288926,
"minimumMargin": 0.05,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 356.92737498902716,
"durationMs": 358.89820801094174,
"embeddingsPerSecond": 8.405071199966738,
"inputTokens": [
3,
412
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 3300392960,
"loadMs": 2759.243124979548,
"totalMs": 3116.1704999685753,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000211870428,
0.9999999929230169,
0.9999999767896206
],
"positiveCosine": 0.7432051033610728,
"negativeCosine": 0.3582827292675386,
"margin": 0.3849223740935342,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 107.8387500019744,
"durationMs": 109.19695897609927,
"embeddingsPerSecond": 27.819313557928606,
"inputTokens": [
3,
16
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 3201220608,
"loadMs": 2686.592708021635,
"totalMs": 2794.4314580236096,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999424989322,
0.9999999961769176,
1.000000019394219
],
"positiveCosine": 0.7633974690508457,
"negativeCosine": 0.30207613433089925,
"margin": 0.46132133471994646,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 51.26441700849682,
"durationMs": 52.793458016822115,
"embeddingsPerSecond": 58.52012321729446,
"inputTokens": [
3,
12
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 3201368064,
"loadMs": 2686.592708021635,
"totalMs": 2737.857125030132,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000281018404,
1.0000000021396556,
1.000000037993779
],
"positiveCosine": 0.943942833072416,
"negativeCosine": 0.4327768926856611,
"margin": 0.5111659403867549,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 48.75154100591317,
"durationMs": 50.24775001220405,
"embeddingsPerSecond": 61.53651634593713,
"inputTokens": [
3,
9
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 3201499136,
"loadMs": 2686.592708021635,
"totalMs": 2735.3442490275484,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000085974663,
1.000000023292101,
0.9999999888165177
],
"positiveCosine": 0.7258271404676749,
"negativeCosine": 0.3863600948947857,
"margin": 0.33946704557288926,
"minimumMargin": 0.05,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 205.0472919945605,
"durationMs": 206.8556250014808,
"embeddingsPerSecond": 14.630771130006359,
"inputTokens": [
3,
412
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 3300409344,
"loadMs": 2686.592708021635,
"totalMs": 2891.6400000161957,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000211870428,
0.9999999929230169,
0.9999999767896206
],
"positiveCosine": 0.7432051033610728,
"negativeCosine": 0.3582827292675386,
"margin": 0.3849223740935342,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 103.86162501526996,
"durationMs": 105.55958401528187,
"embeddingsPerSecond": 28.88458561628449,
"inputTokens": [
3,
16
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 3201220608,
"loadMs": 2700.319291994674,
"totalMs": 2804.180917009944,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999424989322,
0.9999999961769176,
1.000000019394219
],
"positiveCosine": 0.7633974690508457,
"negativeCosine": 0.30207613433089925,
"margin": 0.46132133471994646,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 49.756749998778105,
"durationMs": 51.254042016807944,
"embeddingsPerSecond": 60.293327037510934,
"inputTokens": [
3,
12
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 3201368064,
"loadMs": 2700.319291994674,
"totalMs": 2750.0760419934522,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000281018404,
1.0000000021396556,
1.000000037993779
],
"positiveCosine": 0.943942833072416,
"negativeCosine": 0.4327768926856611,
"margin": 0.5111659403867549,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 47.07674999372102,
"durationMs": 48.63008399843238,
"embeddingsPerSecond": 63.725724490329775,
"inputTokens": [
3,
9
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 3201499136,
"loadMs": 2700.319291994674,
"totalMs": 2747.396041988395,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000085974663,
1.000000023292101,
0.9999999888165177
],
"positiveCosine": 0.7258271404676749,
"negativeCosine": 0.3863600948947857,
"margin": 0.33946704557288926,
"minimumMargin": 0.05,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 189.49908399372362,
"durationMs": 191.4200419851113,
"embeddingsPerSecond": 15.831211089650242,
"inputTokens": [
3,
412
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 3300409344,
"loadMs": 2700.319291994674,
"totalMs": 2889.8183759883977,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000092764376,
1.0000000034604957,
0.9999999673304596
],
"positiveCosine": 0.743267347077721,
"negativeCosine": 0.3583154440084353,
"margin": 0.38495190306928573,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 404.2042499931995,
"durationMs": 405.73304099962115,
"embeddingsPerSecond": 7.421990243918695,
"inputTokens": [
3,
16
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169080320,
"loadMs": 2552.4202919914387,
"totalMs": 2956.624541984638,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999997661901,
1.0000000106659337,
1.0000000237122044
],
"positiveCosine": 0.7634482543094263,
"negativeCosine": 0.3020474317270439,
"margin": 0.4614008225823824,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 141.02437501423992,
"durationMs": 142.54337499733083,
"embeddingsPerSecond": 21.272918243368036,
"inputTokens": [
3,
12
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169227776,
"loadMs": 2552.4202919914387,
"totalMs": 2693.4446670056786,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000271237672,
1.0000000247315426,
1.0000000081035167
],
"positiveCosine": 0.943929555707606,
"negativeCosine": 0.4328989041527131,
"margin": 0.5110306515548928,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 141.25799998873845,
"durationMs": 142.5396249978803,
"embeddingsPerSecond": 21.23773520961057,
"inputTokens": [
3,
9
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169358848,
"loadMs": 2552.4202919914387,
"totalMs": 2693.678291980177,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.000000038799976,
1.0000000304194427,
0.9999999895591469
],
"positiveCosine": 0.7258514105169573,
"negativeCosine": 0.38639587927503916,
"margin": 0.3394555312419182,
"minimumMargin": 0.05,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 372.6414160046261,
"durationMs": 375.5367499834392,
"embeddingsPerSecond": 8.050634929861788,
"inputTokens": [
3,
412
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1335361536,
"loadMs": 2552.4202919914387,
"totalMs": 2925.061707996065,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000092764376,
1.0000000034604957,
0.9999999673304596
],
"positiveCosine": 0.743267347077721,
"negativeCosine": 0.3583154440084353,
"margin": 0.38495190306928573,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 102.8521670086775,
"durationMs": 104.09149999031797,
"embeddingsPerSecond": 29.16807771047638,
"inputTokens": [
3,
16
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169080320,
"loadMs": 2531.7601659917273,
"totalMs": 2634.6123330004048,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999997661901,
1.0000000106659337,
1.0000000237122044
],
"positiveCosine": 0.7634482543094263,
"negativeCosine": 0.3020474317270439,
"margin": 0.4614008225823824,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 72.45362500543706,
"durationMs": 73.96245902054943,
"embeddingsPerSecond": 41.40579577315661,
"inputTokens": [
3,
12
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169227776,
"loadMs": 2531.7601659917273,
"totalMs": 2604.2137909971643,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000271237672,
1.0000000247315426,
1.0000000081035167
],
"positiveCosine": 0.943929555707606,
"negativeCosine": 0.4328989041527131,
"margin": 0.5110306515548928,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 46.965040994109586,
"durationMs": 48.260957992170006,
"embeddingsPerSecond": 63.8772997212174,
"inputTokens": [
3,
9
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169358848,
"loadMs": 2531.7601659917273,
"totalMs": 2578.725206985837,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.000000038799976,
1.0000000304194427,
0.9999999895591469
],
"positiveCosine": 0.7258514105169573,
"negativeCosine": 0.38639587927503916,
"margin": 0.3394555312419182,
"minimumMargin": 0.05,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 173.794041009387,
"durationMs": 175.71400001179427,
"embeddingsPerSecond": 17.261811639663545,
"inputTokens": [
3,
412
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1335377920,
"loadMs": 2531.7601659917273,
"totalMs": 2705.5542070011143,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000092764376,
1.0000000034604957,
0.9999999673304596
],
"positiveCosine": 0.743267347077721,
"negativeCosine": 0.3583154440084353,
"margin": 0.38495190306928573,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 105.4312500054948,
"durationMs": 106.879166996805,
"embeddingsPerSecond": 28.454561620427036,
"inputTokens": [
3,
16
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169080320,
"loadMs": 2518.8049169955775,
"totalMs": 2624.2361670010723,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999997661901,
1.0000000106659337,
1.0000000237122044
],
"positiveCosine": 0.7634482543094263,
"negativeCosine": 0.3020474317270439,
"margin": 0.4614008225823824,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 48.29462498310022,
"durationMs": 50.29379201005213,
"embeddingsPerSecond": 62.118714060825454,
"inputTokens": [
3,
12
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169227776,
"loadMs": 2518.8049169955775,
"totalMs": 2567.0995419786777,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000271237672,
1.0000000247315426,
1.0000000081035167
],
"positiveCosine": 0.943929555707606,
"negativeCosine": 0.4328989041527131,
"margin": 0.5110306515548928,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 46.454541006824,
"durationMs": 47.99516699858941,
"embeddingsPerSecond": 64.57926254312385,
"inputTokens": [
3,
9
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169358848,
"loadMs": 2518.8049169955775,
"totalMs": 2565.2594580024015,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.000000038799976,
1.0000000304194427,
0.9999999895591469
],
"positiveCosine": 0.7258514105169573,
"negativeCosine": 0.38639587927503916,
"margin": 0.3394555312419182,
"minimumMargin": 0.05,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 187.76245901244693,
"durationMs": 190.2451670030132,
"embeddingsPerSecond": 15.977634803989906,
"inputTokens": [
3,
412
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1335377920,
"loadMs": 2518.8049169955775,
"totalMs": 2706.5673760080244,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999553672709,
0.9999999663604554,
1.000000035671252
],
"positiveCosine": 0.7433577291917177,
"negativeCosine": 0.35905120972474774,
"margin": 0.38430651946697,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 410.6048330140766,
"durationMs": 412.3247909883503,
"embeddingsPerSecond": 7.306294906414685,
"inputTokens": [
3,
16
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169080320,
"loadMs": 2579.764666996198,
"totalMs": 2990.3695000102744,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "bf16",
"precision": "bf16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.00000000698783,
1.0000000450324653,
1.0000000189376141
],
"positiveCosine": 0.7637661497915144,
"negativeCosine": 0.3040330550405722,
"margin": 0.45973309475094215,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 218.5002919868566,
"durationMs": 220.28341601253487,
"embeddingsPerSecond": 13.72995876902745,
"inputTokens": [
3,
12
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169227776,
"loadMs": 2579.764666996198,
"totalMs": 2798.2649589830544,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "bf16",
"precision": "bf16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999848599367,
0.9999999961452326,
0.999999999672102
],
"positiveCosine": 0.9443278206375563,
"negativeCosine": 0.4328479293612124,
"margin": 0.5114798912763439,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 140.1039590127766,
"durationMs": 141.9616669882089,
"embeddingsPerSecond": 21.412671141765657,
"inputTokens": [
3,
9
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169358848,
"loadMs": 2579.764666996198,
"totalMs": 2719.8686260089744,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "bf16",
"precision": "bf16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.99999998146247,
0.9999999892591173,
1.0000000013889696
],
"positiveCosine": 0.7220528607715129,
"negativeCosine": 0.3861278659731936,
"margin": 0.3359249947983193,
"minimumMargin": 0.05,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 399.7247919905931,
"durationMs": 402.0047080121003,
"embeddingsPerSecond": 7.5051637029073754,
"inputTokens": [
3,
412
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1335361536,
"loadMs": 2579.764666996198,
"totalMs": 2979.489458986791,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "bf16",
"precision": "bf16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999553672709,
0.9999999663604554,
1.000000035671252
],
"positiveCosine": 0.7433577291917177,
"negativeCosine": 0.35905120972474774,
"margin": 0.38430651946697,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 105.01891700550914,
"durationMs": 106.43608402460814,
"embeddingsPerSecond": 28.566282014150122,
"inputTokens": [
3,
16
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169080320,
"loadMs": 2567.703541019,
"totalMs": 2672.7224580245093,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "bf16",
"precision": "bf16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.00000000698783,
1.0000000450324653,
1.0000000189376141
],
"positiveCosine": 0.7637661497915144,
"negativeCosine": 0.3040330550405722,
"margin": 0.45973309475094215,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 73.63445800729096,
"durationMs": 75.12541601317935,
"embeddingsPerSecond": 40.741795094124996,
"inputTokens": [
3,
12
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169227776,
"loadMs": 2567.703541019,
"totalMs": 2641.337999026291,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "bf16",
"precision": "bf16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999848599367,
0.9999999961452326,
0.999999999672102
],
"positiveCosine": 0.9443278206375563,
"negativeCosine": 0.4328479293612124,
"margin": 0.5114798912763439,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 46.8056250247173,
"durationMs": 48.62195797613822,
"embeddingsPerSecond": 64.09486035953474,
"inputTokens": [
3,
9
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169358848,
"loadMs": 2567.703541019,
"totalMs": 2614.5091660437174,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "bf16",
"precision": "bf16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.99999998146247,
0.9999999892591173,
1.0000000013889696
],
"positiveCosine": 0.7220528607715129,
"negativeCosine": 0.3861278659731936,
"margin": 0.3359249947983193,
"minimumMargin": 0.05,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 173.04504200001247,
"durationMs": 174.95295801199973,
"embeddingsPerSecond": 17.336526752380365,
"inputTokens": [
3,
412
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1335377920,
"loadMs": 2567.703541019,
"totalMs": 2740.7485830190126,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "bf16",
"precision": "bf16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999553672709,
0.9999999663604554,
1.000000035671252
],
"positiveCosine": 0.7433577291917177,
"negativeCosine": 0.35905120972474774,
"margin": 0.38430651946697,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 102.74033300811425,
"durationMs": 104.07937498530373,
"embeddingsPerSecond": 29.199827489006342,
"inputTokens": [
3,
16
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169080320,
"loadMs": 2524.6627499873284,
"totalMs": 2627.4030829954427,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "bf16",
"precision": "bf16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.00000000698783,
1.0000000450324653,
1.0000000189376141
],
"positiveCosine": 0.7637661497915144,
"negativeCosine": 0.3040330550405722,
"margin": 0.45973309475094215,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 49.537874991074204,
"durationMs": 50.5452920042444,
"embeddingsPerSecond": 60.55972325297651,
"inputTokens": [
3,
12
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169227776,
"loadMs": 2524.6627499873284,
"totalMs": 2574.2006249784026,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "bf16",
"precision": "bf16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999848599367,
0.9999999961452326,
0.999999999672102
],
"positiveCosine": 0.9443278206375563,
"negativeCosine": 0.4328479293612124,
"margin": 0.5114798912763439,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 45.952499989653006,
"durationMs": 47.46812497614883,
"embeddingsPerSecond": 65.28480497634517,
"inputTokens": [
3,
9
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169358848,
"loadMs": 2524.6627499873284,
"totalMs": 2570.6152499769814,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "bf16",
"precision": "bf16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.99999998146247,
0.9999999892591173,
1.0000000013889696
],
"positiveCosine": 0.7220528607715129,
"negativeCosine": 0.3861278659731936,
"margin": 0.3359249947983193,
"minimumMargin": 0.05,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 178.93258298863657,
"durationMs": 181.38854100834578,
"embeddingsPerSecond": 16.76609117183828,
"inputTokens": [
3,
412
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1335377920,
"loadMs": 2524.6627499873284,
"totalMs": 2703.595332975965,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "5617a9f61b028005a4858fdac845db406aefb181",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "bf16",
"precision": "bf16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
}
]BGE-M3 Unsupervised Desktop
48 configurations testedQuality criteria
{ "file": "research/bge-m3-family-expansion/quality-gates.json", "sha256": "e39823c58c7328c1f308cb1502358ee29930d02e2a3f8cb766f828bb1200b591", "description": "Finite unit1024d vectors, per-fixture positive-minus-negative cosine margins. Fresh inference<=10s, warm<=5s, load<=120s." }
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 45 ms / 2451 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 46 ms / 2435 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 46 ms / 2605 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 48 ms / 2451 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 48 ms / 2616 ms |
BF16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 48 ms / 2412 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 48 ms / 2670 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 49 ms / 2435 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 49 ms / 2605 ms |
BF16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 49 ms / 2411 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 51 ms / 2475 ms |
BF16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 51 ms / 2412 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 51 ms / 2475 ms |
BF16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 52 ms / 2396 ms |
BF16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 57 ms / 2396 ms |
BF16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 65 ms / 2411 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 73 ms / 2616 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 74 ms / 2670 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 102 ms / 2616 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 102 ms / 2411 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 106 ms / 2605 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 107 ms / 2670 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 108 ms / 2475 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 110 ms / 2396 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 119 ms / 2451 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 122 ms / 2435 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 129 ms / 2412 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 173 ms / 2411 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 173 ms / 2451 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 175 ms / 2435 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 175 ms / 2412 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 190 ms / 2616 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 191 ms / 2605 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 194 ms / 2670 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 217 ms / 2475 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 234 ms / 2396 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 43 ms / 2265 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 44 ms / 2242 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 45 ms / 2303 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 54 ms / 2242 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 54 ms / 2265 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 56 ms / 2303 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 105 ms / 2265 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 106 ms / 2242 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 109 ms / 2303 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 516 ms / 2265 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 516 ms / 2242 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 522 ms / 2303 ms |
Test setup & sample details
- Quality gate
- { "file": "research/bge-m3-family-expansion/quality-gates.json", "sha256": "e39823c58c7328c1f308cb1502358ee29930d02e2a3f8cb766f828bb1200b591", "description": "Finite unit1024d vectors, per-fixture positive-minus-negative cosine margins. Fresh inference<=10s, warm<=5s, load<=120s." }
- Tested
- 2026-11-13T04:37:34.622578+00:00
- Environment
- { "os": "macos", "osVersion": "27.2", "architecture": "aarch64", "hardware": "Apple M3 Max;Mac15,11;96GiB RAM", "cpu": "Apple M3 Max", "gpu": "Apple M3 Max", "driver": "OS-managed Metal/MPS macOS27.2 build26B118", "runtime": "App-owned CPython3.13.18;Torch2.10.0;Transformers5.22.1", "timingScope": "Fresh model/helper per cold repetition, local original snapshot load, shared OS/driver caches may be warm. Four fixtures plus five warm calls; no network transfer included.", "concurrentUploadActivity": "none; uploads excluded" }
- Runtime
- transformers 5.22.1
- Artifact revision
- 46f03bc86361cf88102b0b517b36c8259f2946b1
- Fixture
{ "file": "model-samples/native/bge-m3/quality-gates.json", "url": "https://assets.localai.tools/samples/v1/files/a10b184352cd5c00018eb629107ced06f6e55a9ea2cfd4cffb2f8574feb696a1/bge-m3-quality-gates.json", "size": 4205, "sha1": "c53d7a6368c4e7cc451f1fae92f04f27ca7c3884", "sha256": "a10b184352cd5c00018eb629107ced06f6e55a9ea2cfd4cffb2f8574feb696a1" }- Rerun when
- artifact-revision; runtime-version; fixture-checksum; quality-gate; backend-behavior
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999919190489,
0.9999998906490281,
1.0000000328663003
],
"positiveCosine": 0.7831185302683791,
"negativeCosine": 0.26061313651513,
"margin": 0.5225053937532491,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 108.70691601303406,
"durationMs": 109.34275001636706,
"embeddingsPerSecond": 27.59714018232563,
"inputTokens": [
3,
16
],
"outputDevice": "cpu",
"rssScope": "model process",
"driverAllocatedMemoryBytes": null,
"loadMs": 2303.188458987279,
"totalMs": 2411.895375000313,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.999999951622245,
0.9999999394727269,
1.0000000062896863
],
"positiveCosine": 0.7908431861636811,
"negativeCosine": 0.3053681719134734,
"margin": 0.4854750142502077,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 56.3091250078287,
"durationMs": 56.66520798695274,
"embeddingsPerSecond": 53.277332929306006,
"inputTokens": [
3,
12
],
"outputDevice": "cpu",
"rssScope": "model process",
"driverAllocatedMemoryBytes": null,
"loadMs": 2303.188458987279,
"totalMs": 2359.4975839951076,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.000000001575941,
0.9999999429643861,
0.9999999350934491
],
"positiveCosine": 0.9031304553219311,
"negativeCosine": 0.32214035217271186,
"margin": 0.5809901031492193,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 44.886208983371034,
"durationMs": 45.20929197315127,
"embeddingsPerSecond": 66.83567331585985,
"inputTokens": [
3,
9
],
"outputDevice": "cpu",
"rssScope": "model process",
"driverAllocatedMemoryBytes": null,
"loadMs": 2303.188458987279,
"totalMs": 2348.07466797065,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.999999896287797,
0.9999999322157993,
0.9999999758104792
],
"positiveCosine": 0.6053540306618209,
"negativeCosine": 0.29909317256996343,
"margin": 0.3062608580918575,
"minimumMargin": 0.05,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 521.5836249990389,
"durationMs": 522.8406250244007,
"embeddingsPerSecond": 5.751714310443561,
"inputTokens": [
3,
412
],
"outputDevice": "cpu",
"rssScope": "model process",
"driverAllocatedMemoryBytes": null,
"loadMs": 2303.188458987279,
"totalMs": 2824.7720839863177,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999919190489,
0.9999998906490281,
1.0000000328663003
],
"positiveCosine": 0.7831185302683791,
"negativeCosine": 0.26061313651513,
"margin": 0.5225053937532491,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 104.54091700376011,
"durationMs": 105.09950001141988,
"embeddingsPerSecond": 28.696897693102276,
"inputTokens": [
3,
16
],
"outputDevice": "cpu",
"rssScope": "model process",
"driverAllocatedMemoryBytes": null,
"loadMs": 2264.6997919946443,
"totalMs": 2369.2407089984044,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.999999951622245,
0.9999999394727269,
1.0000000062896863
],
"positiveCosine": 0.7908431861636811,
"negativeCosine": 0.3053681719134734,
"margin": 0.4854750142502077,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 54.39504102105275,
"durationMs": 54.76291599916294,
"embeddingsPerSecond": 55.15208636094046,
"inputTokens": [
3,
12
],
"outputDevice": "cpu",
"rssScope": "model process",
"driverAllocatedMemoryBytes": null,
"loadMs": 2264.6997919946443,
"totalMs": 2319.094833015697,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.000000001575941,
0.9999999429643861,
0.9999999350934491
],
"positiveCosine": 0.9031304553219311,
"negativeCosine": 0.32214035217271186,
"margin": 0.5809901031492193,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 43.383332988014445,
"durationMs": 43.72795898234472,
"embeddingsPerSecond": 69.15098018930018,
"inputTokens": [
3,
9
],
"outputDevice": "cpu",
"rssScope": "model process",
"driverAllocatedMemoryBytes": null,
"loadMs": 2264.6997919946443,
"totalMs": 2308.083124982659,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.999999896287797,
0.9999999322157993,
0.9999999758104792
],
"positiveCosine": 0.6053540306618209,
"negativeCosine": 0.29909317256996343,
"margin": 0.3062608580918575,
"minimumMargin": 0.05,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 515.8579999988433,
"durationMs": 517.0528340095188,
"embeddingsPerSecond": 5.815553892750964,
"inputTokens": [
3,
412
],
"outputDevice": "cpu",
"rssScope": "model process",
"driverAllocatedMemoryBytes": null,
"loadMs": 2264.6997919946443,
"totalMs": 2780.5577919934876,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999919190489,
0.9999998906490281,
1.0000000328663003
],
"positiveCosine": 0.7831185302683791,
"negativeCosine": 0.26061313651513,
"margin": 0.5225053937532491,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 105.73974999715574,
"durationMs": 106.27895899233408,
"embeddingsPerSecond": 28.37154428756164,
"inputTokens": [
3,
16
],
"outputDevice": "cpu",
"rssScope": "model process",
"driverAllocatedMemoryBytes": null,
"loadMs": 2241.5913750010077,
"totalMs": 2347.3311249981634,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.999999951622245,
0.9999999394727269,
1.0000000062896863
],
"positiveCosine": 0.7908431861636811,
"negativeCosine": 0.3053681719134734,
"margin": 0.4854750142502077,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 54.32512500556186,
"durationMs": 54.681291978340596,
"embeddingsPerSecond": 55.22306666929634,
"inputTokens": [
3,
12
],
"outputDevice": "cpu",
"rssScope": "model process",
"driverAllocatedMemoryBytes": null,
"loadMs": 2241.5913750010077,
"totalMs": 2295.9165000065695,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.000000001575941,
0.9999999429643861,
0.9999999350934491
],
"positiveCosine": 0.9031304553219311,
"negativeCosine": 0.32214035217271186,
"margin": 0.5809901031492193,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 43.5617080074735,
"durationMs": 43.90608301036991,
"embeddingsPerSecond": 68.86782307721535,
"inputTokens": [
3,
9
],
"outputDevice": "cpu",
"rssScope": "model process",
"driverAllocatedMemoryBytes": null,
"loadMs": 2241.5913750010077,
"totalMs": 2285.153083008481,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.999999896287797,
0.9999999322157993,
0.9999999758104792
],
"positiveCosine": 0.6053540306618209,
"negativeCosine": 0.29909317256996343,
"margin": 0.3062608580918575,
"minimumMargin": 0.05,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 516.3936669996474,
"durationMs": 517.554874997586,
"embeddingsPerSecond": 5.809521285244671,
"inputTokens": [
3,
412
],
"outputDevice": "cpu",
"rssScope": "model process",
"driverAllocatedMemoryBytes": null,
"loadMs": 2241.5913750010077,
"totalMs": 2757.985042000655,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000391316581,
1.0000000126105106,
0.9999999853970182
],
"positiveCosine": 0.7831185799856215,
"negativeCosine": 0.26061306939861517,
"margin": 0.5225055105870063,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 106.79116600658745,
"durationMs": 108.02129202056676,
"embeddingsPerSecond": 28.092211295969406,
"inputTokens": [
3,
16
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 3201220608,
"loadMs": 2670.103458018275,
"totalMs": 2776.8946240248624,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999641638627,
1.0000000055064022,
0.9999999914255607
],
"positiveCosine": 0.7908433501065775,
"negativeCosine": 0.3053681673377815,
"margin": 0.485475182768796,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 48.4540420002304,
"durationMs": 49.598832993069664,
"embeddingsPerSecond": 61.91433936483018,
"inputTokens": [
3,
12
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 3201368064,
"loadMs": 2670.103458018275,
"totalMs": 2718.5575000185054,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999921459959,
0.9999999645904396,
0.9999999996279445
],
"positiveCosine": 0.9031305300960925,
"negativeCosine": 0.32214022430992617,
"margin": 0.5809903057861663,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 74.23920801375061,
"durationMs": 75.60870799352415,
"embeddingsPerSecond": 40.4099138482773,
"inputTokens": [
3,
9
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 3201499136,
"loadMs": 2670.103458018275,
"totalMs": 2744.3426660320256,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.000000001672196,
0.9999999964118276,
1.0000000068642085
],
"positiveCosine": 0.6053540969128707,
"negativeCosine": 0.29909310374507025,
"margin": 0.3062609931678004,
"minimumMargin": 0.05,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 193.9882499864325,
"durationMs": 195.97079200320877,
"embeddingsPerSecond": 15.464854186837705,
"inputTokens": [
3,
412
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 3300409344,
"loadMs": 2670.103458018275,
"totalMs": 2864.0917080047075,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000391316581,
1.0000000126105106,
0.9999999853970182
],
"positiveCosine": 0.7831185799856215,
"negativeCosine": 0.26061306939861517,
"margin": 0.5225055105870063,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 106.22333400533535,
"durationMs": 108.12370802159421,
"embeddingsPerSecond": 28.242382223187583,
"inputTokens": [
3,
16
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 3201220608,
"loadMs": 2604.8076250008307,
"totalMs": 2711.030959006166,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999641638627,
1.0000000055064022,
0.9999999914255607
],
"positiveCosine": 0.7908433501065775,
"negativeCosine": 0.3053681673377815,
"margin": 0.485475182768796,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 48.634083999786526,
"durationMs": 50.17108301399276,
"embeddingsPerSecond": 61.68513423658125,
"inputTokens": [
3,
12
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 3201368064,
"loadMs": 2604.8076250008307,
"totalMs": 2653.4417090006173,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999921459959,
0.9999999645904396,
0.9999999996279445
],
"positiveCosine": 0.9031305300960925,
"negativeCosine": 0.32214022430992617,
"margin": 0.5809903057861663,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 45.65954199642874,
"durationMs": 46.76174998166971,
"embeddingsPerSecond": 65.70368139554805,
"inputTokens": [
3,
9
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 3201499136,
"loadMs": 2604.8076250008307,
"totalMs": 2650.4671669972595,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.000000001672196,
0.9999999964118276,
1.0000000068642085
],
"positiveCosine": 0.6053540969128707,
"negativeCosine": 0.29909310374507025,
"margin": 0.3062609931678004,
"minimumMargin": 0.05,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 191.27533299615607,
"durationMs": 193.4274999948684,
"embeddingsPerSecond": 15.684196979337054,
"inputTokens": [
3,
412
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 3300409344,
"loadMs": 2604.8076250008307,
"totalMs": 2796.082957996987,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000391316581,
1.0000000126105106,
0.9999999853970182
],
"positiveCosine": 0.7831185799856215,
"negativeCosine": 0.26061306939861517,
"margin": 0.5225055105870063,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 102.22508298465982,
"durationMs": 103.91816700575873,
"embeddingsPerSecond": 29.347004789912358,
"inputTokens": [
3,
16
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 3201220608,
"loadMs": 2615.6165410066023,
"totalMs": 2717.841623991262,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999641638627,
1.0000000055064022,
0.9999999914255607
],
"positiveCosine": 0.7908433501065775,
"negativeCosine": 0.3053681673377815,
"margin": 0.485475182768796,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 73.29283299623057,
"durationMs": 74.40583399147727,
"embeddingsPerSecond": 40.93169655693742,
"inputTokens": [
3,
12
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 3201368064,
"loadMs": 2615.6165410066023,
"totalMs": 2688.909374002833,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999921459959,
0.9999999645904396,
0.9999999996279445
],
"positiveCosine": 0.9031305300960925,
"negativeCosine": 0.32214022430992617,
"margin": 0.5809903057861663,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 47.678416973212734,
"durationMs": 49.208082986297086,
"embeddingsPerSecond": 62.92155214980179,
"inputTokens": [
3,
9
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 3201499136,
"loadMs": 2615.6165410066023,
"totalMs": 2663.294957979815,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.000000001672196,
0.9999999964118276,
1.0000000068642085
],
"positiveCosine": 0.6053540969128707,
"negativeCosine": 0.29909310374507025,
"margin": 0.3062609931678004,
"minimumMargin": 0.05,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 190.07966699427925,
"durationMs": 192.3997919948306,
"embeddingsPerSecond": 15.782855933192948,
"inputTokens": [
3,
412
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 3300409344,
"loadMs": 2615.6165410066023,
"totalMs": 2805.6962080008816,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.000000047842044,
0.9999999824377083,
1.0000000347496425
],
"positiveCosine": 0.783096706976447,
"negativeCosine": 0.2605986425781911,
"margin": 0.5224980643982559,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 107.95000000507571,
"durationMs": 109.85766598605551,
"embeddingsPerSecond": 27.79064381527506,
"inputTokens": [
3,
16
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169080320,
"loadMs": 2474.754249997204,
"totalMs": 2582.70425000228,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000249173837,
0.9999999989486017,
1.0000000119134322
],
"positiveCosine": 0.7909186035281347,
"negativeCosine": 0.30540580734029427,
"margin": 0.48551279618784043,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 51.12533297506161,
"durationMs": 52.177875011693686,
"embeddingsPerSecond": 58.67932442539529,
"inputTokens": [
3,
12
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169227776,
"loadMs": 2474.754249997204,
"totalMs": 2525.879582972266,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999770355187,
0.9999999901279779,
1.000000031815608
],
"positiveCosine": 0.9031291504711986,
"negativeCosine": 0.3221932846111982,
"margin": 0.5809358658600003,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 51.38158399495296,
"durationMs": 52.92908300179988,
"embeddingsPerSecond": 58.38667800305027,
"inputTokens": [
3,
9
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169358848,
"loadMs": 2474.754249997204,
"totalMs": 2526.135833992157,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999856587625,
1.0000000463918306,
1.0000000508813236
],
"positiveCosine": 0.6052298903529097,
"negativeCosine": 0.2991912456410045,
"margin": 0.30603864471190523,
"minimumMargin": 0.05,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 217.45079098036513,
"durationMs": 219.83733400702477,
"embeddingsPerSecond": 13.796224821600612,
"inputTokens": [
3,
412
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1335377920,
"loadMs": 2474.754249997204,
"totalMs": 2692.2050409775693,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.000000047842044,
0.9999999824377083,
1.0000000347496425
],
"positiveCosine": 0.783096706976447,
"negativeCosine": 0.2605986425781911,
"margin": 0.5224980643982559,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 118.97816698183306,
"durationMs": 120.40824998985045,
"embeddingsPerSecond": 25.2147101951745,
"inputTokens": [
3,
16
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169080320,
"loadMs": 2450.545666011749,
"totalMs": 2569.523832993582,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000249173837,
0.9999999989486017,
1.0000000119134322
],
"positiveCosine": 0.7909186035281347,
"negativeCosine": 0.30540580734029427,
"margin": 0.48551279618784043,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 47.552208008710295,
"durationMs": 48.778958996990696,
"embeddingsPerSecond": 63.088553100425536,
"inputTokens": [
3,
12
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169227776,
"loadMs": 2450.545666011749,
"totalMs": 2498.097874020459,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999770355187,
0.9999999901279779,
1.000000031815608
],
"positiveCosine": 0.9031291504711986,
"negativeCosine": 0.3221932846111982,
"margin": 0.5809358658600003,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 45.469332981156185,
"durationMs": 46.52912501478568,
"embeddingsPerSecond": 65.9785354943141,
"inputTokens": [
3,
9
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169358848,
"loadMs": 2450.545666011749,
"totalMs": 2496.014998992905,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999856587625,
1.0000000463918306,
1.0000000508813236
],
"positiveCosine": 0.6052298903529097,
"negativeCosine": 0.2991912456410045,
"margin": 0.30603864471190523,
"minimumMargin": 0.05,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 173.43391699250787,
"durationMs": 175.36533399834298,
"embeddingsPerSecond": 17.29765464577264,
"inputTokens": [
3,
412
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1335377920,
"loadMs": 2450.545666011749,
"totalMs": 2623.9795830042567,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.000000047842044,
0.9999999824377083,
1.0000000347496425
],
"positiveCosine": 0.783096706976447,
"negativeCosine": 0.2605986425781911,
"margin": 0.5224980643982559,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 122.07816599402577,
"durationMs": 124.14224998792633,
"embeddingsPerSecond": 24.57441898452843,
"inputTokens": [
3,
16
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169080320,
"loadMs": 2435.1607080025133,
"totalMs": 2557.238873996539,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000249173837,
0.9999999989486017,
1.0000000119134322
],
"positiveCosine": 0.7909186035281347,
"negativeCosine": 0.30540580734029427,
"margin": 0.48551279618784043,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 48.502457997528836,
"durationMs": 49.7327079938259,
"embeddingsPerSecond": 61.852535394244306,
"inputTokens": [
3,
12
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169227776,
"loadMs": 2435.1607080025133,
"totalMs": 2483.663166000042,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999770355187,
0.9999999901279779,
1.000000031815608
],
"positiveCosine": 0.9031291504711986,
"negativeCosine": 0.3221932846111982,
"margin": 0.5809358658600003,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 45.51262498716824,
"durationMs": 47.56970901507884,
"embeddingsPerSecond": 65.91577613565062,
"inputTokens": [
3,
9
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169358848,
"loadMs": 2435.1607080025133,
"totalMs": 2480.6733329896815,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999856587625,
1.0000000463918306,
1.0000000508813236
],
"positiveCosine": 0.6052298903529097,
"negativeCosine": 0.2991912456410045,
"margin": 0.30603864471190523,
"minimumMargin": 0.05,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 174.631749978289,
"durationMs": 176.83991699595936,
"embeddingsPerSecond": 17.179006683337786,
"inputTokens": [
3,
412
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1335377920,
"loadMs": 2435.1607080025133,
"totalMs": 2609.7924579808023,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000610214153,
0.9999999944458311,
1.0000000364364932
],
"positiveCosine": 0.7833779773270386,
"negativeCosine": 0.2620974510507668,
"margin": 0.5212805262762719,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 110.14529201202095,
"durationMs": 112.05020800116472,
"embeddingsPerSecond": 27.236751977311826,
"inputTokens": [
3,
16
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169080320,
"loadMs": 2395.604416989954,
"totalMs": 2505.749709001975,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "bf16",
"precision": "bf16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999879816657,
1.000000023902974,
1.000000022931096
],
"positiveCosine": 0.7906741263036614,
"negativeCosine": 0.3070256342610816,
"margin": 0.4836484920425798,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 57.11699998937547,
"durationMs": 59.18620800366625,
"embeddingsPerSecond": 52.52376701433969,
"inputTokens": [
3,
12
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169227776,
"loadMs": 2395.604416989954,
"totalMs": 2452.7214169793297,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "bf16",
"precision": "bf16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999691077012,
1.0000000308523842,
1.000000019194924
],
"positiveCosine": 0.9036814311043841,
"negativeCosine": 0.32307195044404424,
"margin": 0.5806094806603399,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 52.239707991248,
"durationMs": 53.896832978352904,
"embeddingsPerSecond": 57.42757981156032,
"inputTokens": [
3,
9
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169358848,
"loadMs": 2395.604416989954,
"totalMs": 2447.844124981202,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "bf16",
"precision": "bf16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999946080774,
0.9999999952509451,
0.9999999779599524
],
"positiveCosine": 0.606599438618612,
"negativeCosine": 0.29973293045521876,
"margin": 0.3068665081633932,
"minimumMargin": 0.05,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 233.88874999363907,
"durationMs": 236.69350001728162,
"embeddingsPerSecond": 12.826610942516856,
"inputTokens": [
3,
412
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1335377920,
"loadMs": 2395.604416989954,
"totalMs": 2629.4931669835933,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "bf16",
"precision": "bf16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000610214153,
0.9999999944458311,
1.0000000364364932
],
"positiveCosine": 0.7833779773270386,
"negativeCosine": 0.2620974510507668,
"margin": 0.5212805262762719,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 128.65258299279958,
"durationMs": 130.38779198541306,
"embeddingsPerSecond": 23.318614599194667,
"inputTokens": [
3,
16
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169080320,
"loadMs": 2411.522624985082,
"totalMs": 2540.1752079778817,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "bf16",
"precision": "bf16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999879816657,
1.000000023902974,
1.000000022931096
],
"positiveCosine": 0.7906741263036614,
"negativeCosine": 0.3070256342610816,
"margin": 0.4836484920425798,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 51.21158401016146,
"durationMs": 52.237500000046566,
"embeddingsPerSecond": 58.580496151119576,
"inputTokens": [
3,
12
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169227776,
"loadMs": 2411.522624985082,
"totalMs": 2462.7342089952435,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "bf16",
"precision": "bf16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999691077012,
1.0000000308523842,
1.000000019194924
],
"positiveCosine": 0.9036814311043841,
"negativeCosine": 0.32307195044404424,
"margin": 0.5806094806603399,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 48.01475000567734,
"durationMs": 49.492624995764345,
"embeddingsPerSecond": 62.48080016339302,
"inputTokens": [
3,
9
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169358848,
"loadMs": 2411.522624985082,
"totalMs": 2459.5373749907594,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "bf16",
"precision": "bf16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999946080774,
0.9999999952509451,
0.9999999779599524
],
"positiveCosine": 0.606599438618612,
"negativeCosine": 0.29973293045521876,
"margin": 0.3068665081633932,
"minimumMargin": 0.05,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 175.24341700482182,
"durationMs": 177.02770797768608,
"embeddingsPerSecond": 17.119045332912307,
"inputTokens": [
3,
412
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1335377920,
"loadMs": 2411.522624985082,
"totalMs": 2586.766041989904,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "bf16",
"precision": "bf16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
1.0000000610214153,
0.9999999944458311,
1.0000000364364932
],
"positiveCosine": 0.7833779773270386,
"negativeCosine": 0.2620974510507668,
"margin": 0.5212805262762719,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 102.38758299965411,
"durationMs": 104.04912501689978,
"embeddingsPerSecond": 29.30042796312649,
"inputTokens": [
3,
16
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169080320,
"loadMs": 2410.865083016688,
"totalMs": 2513.252666016342,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "bf16",
"precision": "bf16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999879816657,
1.000000023902974,
1.000000022931096
],
"positiveCosine": 0.7906741263036614,
"negativeCosine": 0.3070256342610816,
"margin": 0.4836484920425798,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 49.33604202233255,
"durationMs": 50.47779198503122,
"embeddingsPerSecond": 60.8074721243754,
"inputTokens": [
3,
12
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169227776,
"loadMs": 2410.865083016688,
"totalMs": 2460.2011250390206,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "bf16",
"precision": "bf16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999691077012,
1.0000000308523842,
1.000000019194924
],
"positiveCosine": 0.9036814311043841,
"negativeCosine": 0.32307195044404424,
"margin": 0.5806094806603399,
"minimumMargin": 0.1,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 65.32241599052213,
"durationMs": 66.90779200289398,
"embeddingsPerSecond": 45.92604168889407,
"inputTokens": [
3,
9
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1169358848,
"loadMs": 2410.865083016688,
"totalMs": 2476.18749900721,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "bf16",
"precision": "bf16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"finite": true,
"norms": [
0.9999999946080774,
0.9999999952509451,
0.9999999779599524
],
"positiveCosine": 0.606599438618612,
"negativeCosine": 0.29973293045521876,
"margin": 0.3068665081633932,
"minimumMargin": 0.05,
"passed": true,
"speedPassed": true
},
"runtime": {
"inferenceMs": 173.06604099576361,
"durationMs": 175.21312498138286,
"embeddingsPerSecond": 17.33442322213539,
"inputTokens": [
3,
412
],
"outputDevice": "mps",
"rssScope": "model process",
"driverAllocatedMemoryBytes": 1335377920,
"loadMs": 2410.865083016688,
"totalMs": 2583.9311240124516,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "46f03bc86361cf88102b0b517b36c8259f2946b1",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "bf16",
"precision": "bf16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
}
]Release validation still required
- Sample publication, complete dependency backup and forced model/runtime-source recovery pending under user upload exclusion.