BERT NER Desktop
Extract English people, organizations, locations and miscellaneous named entity spans with the original Base · Cased checkpoint.
Desktop app
Run BERT NER Desktop on your computer
Download the desktop app to load this model, provide input and view results locally.
Public desktop downloads are coming soon.Device requirements
If your device falls short
MemoryAdvisory only: close other apps, reduce image/tile/token/batch settings or choose a smaller verified model/precision. CPU can need more RAM than GPU.
GPUIf the selected GPU runtime fails, existing CPU fallback may recover when enabled, but CPU has its own memory/runtime needs.
PlatformUse a compatible OS/architecture/runtime or another model. These estimates do not add hardware detection or block runs.
Benchmark evidence
Saved desktop measurements apply to the recorded hardware and runtime. Fallback cases report the device that actually ran; they do not prove GPU support on the requested device.
BERT Base Cased NER Desktop
72 configurations testedQuality criteria
{ "file": "research/bert-ner-family-expansion/quality-gates.json", "sha256": "4362bea8f9d1c5d1975a56c1ded21d9244114c3e347968ebe9ac4e72e2b705e3", "description": "Six frozen NER cases: selected-span recall for non-exhaustive NASA reference; exact-span recall/F1 for five complete references, including valid UTF-16 offsets and no-entity negative; <=1000ms per text." }
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 8 ms / 1931 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 8 ms / 1875 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 8 ms / 2559 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 8 ms / 1931 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 8 ms / 1875 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 9 ms / 1931 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 9 ms / 1905 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 9 ms / 2559 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 10 ms / 1912 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 10 ms / 1875 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 10 ms / 2559 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 10 ms / 1905 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 10 ms / 1912 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 11 ms / 1905 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 12 ms / 1912 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 14 ms / 1909 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 15 ms / 1909 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 17 ms / 1909 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 18 ms / 1931 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 19 ms / 2559 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 20 ms / 1875 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 22 ms / 1909 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 22 ms / 1905 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 24 ms / 1912 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 24 ms / 1875 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 25 ms / 2559 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 26 ms / 1905 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 27 ms / 1909 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 31 ms / 1912 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 31 ms / 1931 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 63 ms / 1909 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 63 ms / 1875 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 65 ms / 1905 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 67 ms / 1912 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 72 ms / 1931 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 73 ms / 2559 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 16 ms / 1663 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 16 ms / 1639 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 17 ms / 1724 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 17 ms / 1663 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 17 ms / 1639 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 17 ms / 1624 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 17 ms / 1724 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 17 ms / 1663 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 17 ms / 1639 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 17 ms / 1724 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 17 ms / 1642 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 18 ms / 1624 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 18 ms / 1624 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 18 ms / 1663 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 18 ms / 1724 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 18 ms / 1624 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 18 ms / 1639 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 18 ms / 1642 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 18 ms / 1639 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 18 ms / 1639 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 18 ms / 1663 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 18 ms / 1642 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 19 ms / 1642 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 19 ms / 1624 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 19 ms / 1639 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 19 ms / 1639 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 19 ms / 1639 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 19 ms / 1724 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 19 ms / 1642 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 20 ms / 1639 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 336 ms / 1642 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 336 ms / 1639 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 338 ms / 1624 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 340 ms / 1724 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 342 ms / 1639 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 357 ms / 1663 ms |
Test setup & sample details
- Quality gate
- { "file": "research/bert-ner-family-expansion/quality-gates.json", "sha256": "4362bea8f9d1c5d1975a56c1ded21d9244114c3e347968ebe9ac4e72e2b705e3", "description": "Six frozen NER cases: selected-span recall for non-exhaustive NASA reference; exact-span recall/F1 for five complete references, including valid UTF-16 offsets and no-entity negative; <=1000ms per text." }
- Tested
- 2026-11-12T22:40:54.760961+00:00
- Environment
- { "os": "macos", "osVersion": "27.2", "architecture": "aarch64", "hardware": "Apple M3 Max;Mac15,11;96GiB RAM", "cpu": "Apple M3 Max", "gpu": "Apple M3 Max", "driver": "OS-managed Metal/MPS macOS27.2 build26B118", "runtime": "App-owned CPython3.13.18;Torch2.10.0;Transformers5.22.1", "timingScope": "Fresh helper/model process per cold repetition; runtime/tokenizer/model startup in load. OS files/driver cache may be warm. Five warm texts follow six fixtures. Snapshot loads, not Internet transfer.", "concurrentUploadActivity": "none; uploads excluded" }
- Runtime
- transformers 5.22.1
- Artifact revision
- d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc
- Fixture
{ "file": "model-samples/native/bert-ner/quality-gates.json", "url": "https://assets.localai.tools/samples/v1/files/4362bea8f9d1c5d1975a56c1ded21d9244114c3e347968ebe9ac4e72e2b705e3/bert-ner-quality-gates.json", "size": 4105, "sha1": "50eaec89241b58910bbfe13b9c7c1d4a2dac1816", "sha256": "4362bea8f9d1c5d1975a56c1ded21d9244114c3e347968ebe9ac4e72e2b705e3" }- Rerun when
- artifact-revision; runtime-version; fixture-checksum; quality-gate; backend-behavior
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.5,
"recall": 1,
"f1": 0.6666666666666666,
"expectedSpans": 4,
"actualSpans": 8,
"speedPassed": true
},
"runtime": {
"durationMs": 357.0229170145467,
"inferenceMs": 357.0229170145467,
"charactersPerSecond": 548.9843667150759,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1663.4898750053253,
"totalMs": 2020.512792019872,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 18.224625004222617,
"inferenceMs": 18.224625004222617,
"charactersPerSecond": 2249.703354143109,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1663.4898750053253,
"totalMs": 1681.714500009548,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 17.17658300185576,
"inferenceMs": 17.17658300185576,
"charactersPerSecond": 1746.5639118536437,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1663.4898750053253,
"totalMs": 1680.666458007181,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 17.598167003598064,
"inferenceMs": 17.598167003598064,
"charactersPerSecond": 2500.260395926684,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1663.4898750053253,
"totalMs": 1681.0880420089234,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 16.262125020148233,
"inferenceMs": 16.262125020148233,
"charactersPerSecond": 1721.792199070474,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1663.4898750053253,
"totalMs": 1679.7520000254735,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 16.71024999814108,
"inferenceMs": 16.71024999814108,
"charactersPerSecond": 2573.2708968916386,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1663.4898750053253,
"totalMs": 1680.2001250034664,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.5,
"recall": 1,
"f1": 0.6666666666666666,
"expectedSpans": 4,
"actualSpans": 8,
"speedPassed": true
},
"runtime": {
"durationMs": 335.8510420075618,
"inferenceMs": 335.8510420075618,
"charactersPerSecond": 583.5920556577788,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1642.0322500052862,
"totalMs": 1977.883292012848,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 19.20745801180601,
"inferenceMs": 19.20745801180601,
"charactersPerSecond": 2134.5875115176113,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1642.0322500052862,
"totalMs": 1661.2397080170922,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 18.023457989329472,
"inferenceMs": 18.023457989329472,
"charactersPerSecond": 1664.4974575778447,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1642.0322500052862,
"totalMs": 1660.0557079946157,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 18.450000003213063,
"inferenceMs": 18.450000003213063,
"charactersPerSecond": 2384.823847823166,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1642.0322500052862,
"totalMs": 1660.4822500084992,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 17.45891702012159,
"inferenceMs": 17.45891702012159,
"charactersPerSecond": 1603.7649968626174,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1642.0322500052862,
"totalMs": 1659.4911670254078,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 18.568041996331885,
"inferenceMs": 18.568041996331885,
"charactersPerSecond": 2315.806912139398,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1642.0322500052862,
"totalMs": 1660.600292001618,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.5,
"recall": 1,
"f1": 0.6666666666666666,
"expectedSpans": 4,
"actualSpans": 8,
"speedPassed": true
},
"runtime": {
"durationMs": 341.87833400210366,
"inferenceMs": 341.87833400210366,
"charactersPerSecond": 573.3033670358121,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1638.9899170026183,
"totalMs": 1980.868251004722,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 18.17079199827276,
"inferenceMs": 18.17079199827276,
"charactersPerSecond": 2256.368352237882,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1638.9899170026183,
"totalMs": 1657.160709000891,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 17.408000014256686,
"inferenceMs": 17.408000014256686,
"charactersPerSecond": 1723.3455868239203,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1638.9899170026183,
"totalMs": 1656.397917016875,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 17.741582996677607,
"inferenceMs": 17.741582996677607,
"charactersPerSecond": 2480.049272279689,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1638.9899170026183,
"totalMs": 1656.731499999296,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 16.496916010510176,
"inferenceMs": 16.496916010510176,
"charactersPerSecond": 1697.2869342464503,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1638.9899170026183,
"totalMs": 1655.4868330131285,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 16.76833297824487,
"inferenceMs": 16.76833297824487,
"charactersPerSecond": 2564.3574740427644,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1638.9899170026183,
"totalMs": 1655.7582499808632,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.5,
"recall": 1,
"f1": 0.6666666666666666,
"expectedSpans": 4,
"actualSpans": 8,
"speedPassed": true
},
"runtime": {
"durationMs": 336.3414999912493,
"inferenceMs": 336.3414999912493,
"charactersPerSecond": 582.7410533790787,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1639.0415420173667,
"totalMs": 1975.383042008616,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 20.12829200248234,
"inferenceMs": 20.12829200248234,
"charactersPerSecond": 2036.9338836570753,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1639.0415420173667,
"totalMs": 1659.169834019849,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 18.808625027304515,
"inferenceMs": 18.808625027304515,
"charactersPerSecond": 1595.0129239351068,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1639.0415420173667,
"totalMs": 1657.8501670446713,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 18.77199998125434,
"inferenceMs": 18.77199998125434,
"charactersPerSecond": 2343.9164736809216,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1639.0415420173667,
"totalMs": 1657.813541998621,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 18.188459012890235,
"inferenceMs": 18.188459012890235,
"charactersPerSecond": 1539.4377269760064,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1639.0415420173667,
"totalMs": 1657.230001030257,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 19.07145799486898,
"inferenceMs": 19.07145799486898,
"charactersPerSecond": 2254.678169417818,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1639.0415420173667,
"totalMs": 1658.1130000122357,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.5,
"recall": 1,
"f1": 0.6666666666666666,
"expectedSpans": 4,
"actualSpans": 8,
"speedPassed": true
},
"runtime": {
"durationMs": 340.4717080120463,
"inferenceMs": 340.4717080120463,
"charactersPerSecond": 575.6719145458785,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1723.603374994127,
"totalMs": 2064.0750830061734,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 19.071541988523677,
"inferenceMs": 19.071541988523677,
"charactersPerSecond": 2149.7999492999465,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1723.603374994127,
"totalMs": 1742.6749169826508,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 17.673541005933657,
"inferenceMs": 17.673541005933657,
"charactersPerSecond": 1697.4527057100722,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1723.603374994127,
"totalMs": 1741.2769160000607,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 17.447874997742474,
"inferenceMs": 17.447874997742474,
"charactersPerSecond": 2521.797067304358,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1723.603374994127,
"totalMs": 1741.0512499918696,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 16.666999988956377,
"inferenceMs": 16.666999988956377,
"charactersPerSecond": 1679.9664017851392,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1723.603374994127,
"totalMs": 1740.2703749830835,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 17.041459010215476,
"inferenceMs": 17.041459010215476,
"charactersPerSecond": 2523.258130317581,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1723.603374994127,
"totalMs": 1740.6448340043426,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.5,
"recall": 1,
"f1": 0.6666666666666666,
"expectedSpans": 4,
"actualSpans": 8,
"speedPassed": true
},
"runtime": {
"durationMs": 337.64375001192093,
"inferenceMs": 337.64375001192093,
"charactersPerSecond": 580.4934934915276,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1624.2325420025736,
"totalMs": 1961.8762920144945,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 18.60416601994075,
"inferenceMs": 18.60416601994075,
"charactersPerSecond": 2203.8074674271575,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1624.2325420025736,
"totalMs": 1642.8367080225144,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 17.508832999737933,
"inferenceMs": 17.508832999737933,
"charactersPerSecond": 1713.4208773622452,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1624.2325420025736,
"totalMs": 1641.7413750023115,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 17.713541019475088,
"inferenceMs": 17.713541019475088,
"charactersPerSecond": 2483.9753921378206,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1624.2325420025736,
"totalMs": 1641.9460830220487,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 16.828499996336177,
"inferenceMs": 16.828499996336177,
"charactersPerSecond": 1663.8440744033057,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1624.2325420025736,
"totalMs": 1641.0610419989098,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 17.58424998843111,
"inferenceMs": 17.58424998843111,
"charactersPerSecond": 2445.3701481888743,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1624.2325420025736,
"totalMs": 1641.8167919910047,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.5,
"recall": 1,
"f1": 0.6666666666666666,
"expectedSpans": 4,
"actualSpans": 8,
"speedPassed": true
},
"runtime": {
"durationMs": 64.68320899875835,
"inferenceMs": 64.68320899875835,
"charactersPerSecond": 3030.1526939358005,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093599232,
"loadMs": 1904.5657920069061,
"totalMs": 1969.2490010056645,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 25.65645900904201,
"inferenceMs": 25.65645900904201,
"charactersPerSecond": 1598.0381386827592,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093648384,
"loadMs": 1904.5657920069061,
"totalMs": 1930.2222510159481,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 22.217916004592553,
"inferenceMs": 22.217916004592553,
"charactersPerSecond": 1350.2616534241488,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093664768,
"loadMs": 1904.5657920069061,
"totalMs": 1926.7837080114987,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 9.770208998816088,
"inferenceMs": 9.770208998816088,
"charactersPerSecond": 4503.486056985243,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093664768,
"loadMs": 1904.5657920069061,
"totalMs": 1914.3360010057222,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 8.942707994719967,
"inferenceMs": 8.942707994719967,
"charactersPerSecond": 3131.042634572437,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093664768,
"loadMs": 1904.5657920069061,
"totalMs": 1913.508500001626,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 11.300375015707687,
"inferenceMs": 11.300375015707687,
"charactersPerSecond": 3805.1834510119684,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093664768,
"loadMs": 1904.5657920069061,
"totalMs": 1915.8661670226138,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.5,
"recall": 1,
"f1": 0.6666666666666666,
"expectedSpans": 4,
"actualSpans": 8,
"speedPassed": true
},
"runtime": {
"durationMs": 62.68800000543706,
"inferenceMs": 62.68800000543706,
"charactersPerSecond": 3126.5952013623105,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093599232,
"loadMs": 1909.015665994957,
"totalMs": 1971.7036660003942,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 26.972624997142702,
"inferenceMs": 26.972624997142702,
"charactersPerSecond": 1520.059690309833,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093648384,
"loadMs": 1909.015665994957,
"totalMs": 1935.9882909920998,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 21.57616597833112,
"inferenceMs": 21.57616597833112,
"charactersPerSecond": 1390.4231192014797,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093664768,
"loadMs": 1909.015665994957,
"totalMs": 1930.5918319732882,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 13.741000002482906,
"inferenceMs": 13.741000002482906,
"charactersPerSecond": 3202.0959167491083,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093664768,
"loadMs": 1909.015665994957,
"totalMs": 1922.75666599744,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 15.185666998149827,
"inferenceMs": 15.185666998149827,
"charactersPerSecond": 1843.8439354301283,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093664768,
"loadMs": 1909.015665994957,
"totalMs": 1924.201332993107,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 17.308875016169623,
"inferenceMs": 17.308875016169623,
"charactersPerSecond": 2484.2746833534943,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093664768,
"loadMs": 1909.015665994957,
"totalMs": 1926.3245410111267,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.5,
"recall": 1,
"f1": 0.6666666666666666,
"expectedSpans": 4,
"actualSpans": 8,
"speedPassed": true
},
"runtime": {
"durationMs": 66.6626250022091,
"inferenceMs": 66.6626250022091,
"charactersPerSecond": 2940.1782482088706,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093599232,
"loadMs": 1911.9291660026647,
"totalMs": 1978.5917910048738,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 31.12012500059791,
"inferenceMs": 31.12012500059791,
"charactersPerSecond": 1317.4754278529495,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093648384,
"loadMs": 1911.9291660026647,
"totalMs": 1943.0492910032626,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 24.17487499769777,
"inferenceMs": 24.17487499769777,
"charactersPerSecond": 1240.9578127232082,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093664768,
"loadMs": 1911.9291660026647,
"totalMs": 1936.1040410003625,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 11.622208985500038,
"inferenceMs": 11.622208985500038,
"charactersPerSecond": 3785.8551721875556,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093664768,
"loadMs": 1911.9291660026647,
"totalMs": 1923.5513749881648,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 9.638457995606586,
"inferenceMs": 9.638457995606586,
"charactersPerSecond": 2905.029000776161,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093664768,
"loadMs": 1911.9291660026647,
"totalMs": 1921.5676239982713,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 10.433083982206881,
"inferenceMs": 10.433083982206881,
"charactersPerSecond": 4121.504252561795,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093664768,
"loadMs": 1911.9291660026647,
"totalMs": 1922.3622499848716,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.5,
"recall": 1,
"f1": 0.6666666666666666,
"expectedSpans": 4,
"actualSpans": 8,
"speedPassed": true
},
"runtime": {
"durationMs": 71.84720900841057,
"inferenceMs": 71.84720900841057,
"charactersPerSecond": 2728.011327163117,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093599232,
"loadMs": 1931.1824580072425,
"totalMs": 2003.029667015653,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 31.280416005756706,
"inferenceMs": 31.280416005756706,
"charactersPerSecond": 1310.7242561113812,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093648384,
"loadMs": 1931.1824580072425,
"totalMs": 1962.4628740129992,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 18.477291014278308,
"inferenceMs": 18.477291014278308,
"charactersPerSecond": 1623.6146292666783,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093664768,
"loadMs": 1931.1824580072425,
"totalMs": 1949.6597490215208,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 8.602542016888037,
"inferenceMs": 8.602542016888037,
"charactersPerSecond": 5114.767229688809,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093664768,
"loadMs": 1931.1824580072425,
"totalMs": 1939.7850000241306,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 7.511000003432855,
"inferenceMs": 7.511000003432855,
"charactersPerSecond": 3727.865795127517,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093664768,
"loadMs": 1931.1824580072425,
"totalMs": 1938.6934580106754,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 8.330540993483737,
"inferenceMs": 8.330540993483737,
"charactersPerSecond": 5161.72959638938,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093664768,
"loadMs": 1931.1824580072425,
"totalMs": 1939.5129990007263,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.5,
"recall": 1,
"f1": 0.6666666666666666,
"expectedSpans": 4,
"actualSpans": 8,
"speedPassed": true
},
"runtime": {
"durationMs": 73.15591699443758,
"inferenceMs": 73.15591699443758,
"charactersPerSecond": 2679.209120089396,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093599232,
"loadMs": 2559.489957988262,
"totalMs": 2632.6458749826998,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 24.987250013509765,
"inferenceMs": 24.987250013509765,
"charactersPerSecond": 1640.8368258945134,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093648384,
"loadMs": 2559.489957988262,
"totalMs": 2584.477208001772,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 19.188541977200657,
"inferenceMs": 19.188541977200657,
"charactersPerSecond": 1563.433013078599,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093664768,
"loadMs": 2559.489957988262,
"totalMs": 2578.678499965463,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 9.67349999700673,
"inferenceMs": 9.67349999700673,
"charactersPerSecond": 4548.50881414327,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093664768,
"loadMs": 2559.489957988262,
"totalMs": 2569.163457985269,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 7.777249993523583,
"inferenceMs": 7.777249993523583,
"charactersPerSecond": 3600.244305290004,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093664768,
"loadMs": 2559.489957988262,
"totalMs": 2567.2672079817858,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 9.300082980189472,
"inferenceMs": 9.300082980189472,
"charactersPerSecond": 4623.614659309626,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093664768,
"loadMs": 2559.489957988262,
"totalMs": 2568.7900409684516,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.5,
"recall": 1,
"f1": 0.6666666666666666,
"expectedSpans": 4,
"actualSpans": 8,
"speedPassed": true
},
"runtime": {
"durationMs": 63.124499982222915,
"inferenceMs": 63.124499982222915,
"charactersPerSecond": 3104.9750897860167,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093599232,
"loadMs": 1875.1310000079684,
"totalMs": 1938.2554999901913,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 24.292082991451025,
"inferenceMs": 24.292082991451025,
"charactersPerSecond": 1687.7926859721704,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093648384,
"loadMs": 1875.1310000079684,
"totalMs": 1899.4230829994194,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 20.357333996798843,
"inferenceMs": 20.357333996798843,
"charactersPerSecond": 1473.6703737688565,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093664768,
"loadMs": 1875.1310000079684,
"totalMs": 1895.4883340047672,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 9.647041995776817,
"inferenceMs": 9.647041995776817,
"charactersPerSecond": 4560.983565663119,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093664768,
"loadMs": 1875.1310000079684,
"totalMs": 1884.7780420037452,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 7.6264580129645765,
"inferenceMs": 7.6264580129645765,
"charactersPerSecond": 3671.429115901704,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093664768,
"loadMs": 1875.1310000079684,
"totalMs": 1882.757458020933,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 8.402166015002877,
"inferenceMs": 8.402166015002877,
"charactersPerSecond": 5117.727967195526,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093664768,
"loadMs": 1875.1310000079684,
"totalMs": 1883.5331660229713,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
}
]BERT Base Uncased NER Desktop
72 configurations testedQuality criteria
{ "file": "research/bert-ner-family-expansion/quality-gates.json", "sha256": "4362bea8f9d1c5d1975a56c1ded21d9244114c3e347968ebe9ac4e72e2b705e3", "description": "Six frozen NER cases: selected-span recall for non-exhaustive NASA reference; exact-span recall/F1 for five complete references, including valid UTF-16 offsets and no-entity negative; <=1000ms per text." }
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 8 ms / 1863 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 8 ms / 1863 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 8 ms / 1916 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 8 ms / 1973 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 9 ms / 1851 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 9 ms / 1914 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 9 ms / 1863 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 9 ms / 1881 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 9 ms / 1916 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 10 ms / 1914 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 10 ms / 1914 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 10 ms / 1851 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 10 ms / 1851 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 11 ms / 1881 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 11 ms / 1973 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 11 ms / 1973 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 12 ms / 1881 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 14 ms / 1916 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 20 ms / 1863 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 20 ms / 1881 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 21 ms / 1973 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 22 ms / 1916 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 22 ms / 1851 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 22 ms / 1914 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 24 ms / 1914 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 25 ms / 1851 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 25 ms / 1881 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 25 ms / 1863 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 28 ms / 1973 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 29 ms / 1916 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 63 ms / 1863 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 63 ms / 1973 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 63 ms / 1881 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 64 ms / 1914 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 70 ms / 1916 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 72 ms / 1851 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 17 ms / 1632 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 17 ms / 1636 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 17 ms / 1632 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 17 ms / 1636 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 17 ms / 1635 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 17 ms / 1631 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 18 ms / 1632 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 18 ms / 1632 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 18 ms / 1636 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 18 ms / 1636 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 18 ms / 1631 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 18 ms / 1631 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 18 ms / 1636 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 18 ms / 1631 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 18 ms / 1635 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 18 ms / 1734 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 19 ms / 1636 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 19 ms / 1631 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 19 ms / 1632 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 19 ms / 1636 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 19 ms / 1734 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 19 ms / 1636 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 19 ms / 1635 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 19 ms / 1734 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 20 ms / 1636 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 20 ms / 1636 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 20 ms / 1635 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 20 ms / 1734 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 21 ms / 1734 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 22 ms / 1635 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 341 ms / 1636 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 342 ms / 1631 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 342 ms / 1636 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 342 ms / 1632 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 342 ms / 1635 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 347 ms / 1734 ms |
Test setup & sample details
- Quality gate
- { "file": "research/bert-ner-family-expansion/quality-gates.json", "sha256": "4362bea8f9d1c5d1975a56c1ded21d9244114c3e347968ebe9ac4e72e2b705e3", "description": "Six frozen NER cases: selected-span recall for non-exhaustive NASA reference; exact-span recall/F1 for five complete references, including valid UTF-16 offsets and no-entity negative; <=1000ms per text." }
- Tested
- 2026-11-12T22:40:54.760961+00:00
- Environment
- { "os": "macos", "osVersion": "27.2", "architecture": "aarch64", "hardware": "Apple M3 Max;Mac15,11;96GiB RAM", "cpu": "Apple M3 Max", "gpu": "Apple M3 Max", "driver": "OS-managed Metal/MPS macOS27.2 build26B118", "runtime": "App-owned CPython3.13.18;Torch2.10.0;Transformers5.22.1", "timingScope": "Fresh helper/model process per cold repetition; runtime/tokenizer/model startup in load. OS files/driver cache may be warm. Five warm texts follow six fixtures. Snapshot loads, not Internet transfer.", "concurrentUploadActivity": "none; uploads excluded" }
- Runtime
- transformers 5.22.1
- Artifact revision
- 301540b48433e31000058882c971ddd7bc726547
- Fixture
{ "file": "model-samples/native/bert-ner/quality-gates.json", "url": "https://assets.localai.tools/samples/v1/files/4362bea8f9d1c5d1975a56c1ded21d9244114c3e347968ebe9ac4e72e2b705e3/bert-ner-quality-gates.json", "size": 4105, "sha1": "50eaec89241b58910bbfe13b9c7c1d4a2dac1816", "sha256": "4362bea8f9d1c5d1975a56c1ded21d9244114c3e347968ebe9ac4e72e2b705e3" }- Rerun when
- artifact-revision; runtime-version; fixture-checksum; quality-gate; backend-behavior
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.6666666666666666,
"recall": 1,
"f1": 0.8,
"expectedSpans": 4,
"actualSpans": 6,
"speedPassed": true
},
"runtime": {
"durationMs": 342.4395000038203,
"inferenceMs": 342.4395000038203,
"charactersPerSecond": 572.3638774084573,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1634.8025419865735,
"totalMs": 1977.2420419903938,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 20.126791001530364,
"inferenceMs": 20.126791001530364,
"charactersPerSecond": 2037.0857926076003,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1634.8025419865735,
"totalMs": 1654.9293329881039,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 19.463665987132117,
"inferenceMs": 19.463665987132117,
"charactersPerSecond": 1541.3334784841509,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1634.8025419865735,
"totalMs": 1654.2662079737056,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 22.291708999546245,
"inferenceMs": 22.291708999546245,
"charactersPerSecond": 1973.8280273125597,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1634.8025419865735,
"totalMs": 1657.0942509861197,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 18.42937502078712,
"inferenceMs": 18.42937502078712,
"charactersPerSecond": 1519.313594108202,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1634.8025419865735,
"totalMs": 1653.2319170073606,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 17.104041995480657,
"inferenceMs": 17.104041995480657,
"charactersPerSecond": 2514.025632734166,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1634.8025419865735,
"totalMs": 1651.9065839820541,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.6666666666666666,
"recall": 1,
"f1": 0.8,
"expectedSpans": 4,
"actualSpans": 6,
"speedPassed": true
},
"runtime": {
"durationMs": 347.41437502088957,
"inferenceMs": 347.41437502088957,
"charactersPerSecond": 564.167789511343,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1734.2115829815157,
"totalMs": 2081.6259580024052,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 19.4819160096813,
"inferenceMs": 19.4819160096813,
"charactersPerSecond": 2104.5157970923165,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1734.2115829815157,
"totalMs": 1753.693498991197,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 19.00374999968335,
"inferenceMs": 19.00374999968335,
"charactersPerSecond": 1578.6357955929684,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1734.2115829815157,
"totalMs": 1753.215332981199,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 20.399958011694252,
"inferenceMs": 20.399958011694252,
"charactersPerSecond": 2156.8671844705295,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1734.2115829815157,
"totalMs": 1754.61154099321,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 20.83541700267233,
"inferenceMs": 20.83541700267233,
"charactersPerSecond": 1343.8655917665938,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1734.2115829815157,
"totalMs": 1755.046999984188,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 18.49795802263543,
"inferenceMs": 18.49795802263543,
"charactersPerSecond": 2324.580904950812,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1734.2115829815157,
"totalMs": 1752.709541004151,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.6666666666666666,
"recall": 1,
"f1": 0.8,
"expectedSpans": 4,
"actualSpans": 6,
"speedPassed": true
},
"runtime": {
"durationMs": 341.68783397763036,
"inferenceMs": 341.68783397763036,
"charactersPerSecond": 573.6229988593382,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1631.2773750105407,
"totalMs": 1972.965208988171,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 18.383625021670014,
"inferenceMs": 18.383625021670014,
"charactersPerSecond": 2230.2456643709033,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1631.2773750105407,
"totalMs": 1649.6610000322107,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 17.317374993581325,
"inferenceMs": 17.317374993581325,
"charactersPerSecond": 1732.364172463753,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1631.2773750105407,
"totalMs": 1648.594750004122,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 18.115832994226366,
"inferenceMs": 18.115832994226366,
"charactersPerSecond": 2428.8146183519734,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1631.2773750105407,
"totalMs": 1649.393208004767,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 17.934500006958842,
"inferenceMs": 17.934500006958842,
"charactersPerSecond": 1561.2367219122696,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1631.2773750105407,
"totalMs": 1649.2118750174996,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 18.787792010698467,
"inferenceMs": 18.787792010698467,
"charactersPerSecond": 2288.7202485270327,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1631.2773750105407,
"totalMs": 1650.0651670212392,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.6666666666666666,
"recall": 1,
"f1": 0.8,
"expectedSpans": 4,
"actualSpans": 6,
"speedPassed": true
},
"runtime": {
"durationMs": 342.10483299102634,
"inferenceMs": 342.10483299102634,
"charactersPerSecond": 572.9237973236736,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1632.2316250007134,
"totalMs": 1974.3364579917397,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 18.79999999073334,
"inferenceMs": 18.79999999073334,
"charactersPerSecond": 2180.851064904745,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1632.2316250007134,
"totalMs": 1651.0316249914467,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 17.70170798408799,
"inferenceMs": 17.70170798408799,
"charactersPerSecond": 1694.7517170075853,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1632.2316250007134,
"totalMs": 1649.9333329848014,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 17.61199999600649,
"inferenceMs": 17.61199999600649,
"charactersPerSecond": 2498.296616510162,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1632.2316250007134,
"totalMs": 1649.8436249967199,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 16.930207988480106,
"inferenceMs": 16.930207988480106,
"charactersPerSecond": 1653.8485539605988,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1632.2316250007134,
"totalMs": 1649.1618329891935,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 17.045582993887365,
"inferenceMs": 17.045582993887365,
"charactersPerSecond": 2522.647656898566,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1632.2316250007134,
"totalMs": 1649.2772079946008,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.6666666666666666,
"recall": 1,
"f1": 0.8,
"expectedSpans": 4,
"actualSpans": 6,
"speedPassed": true
},
"runtime": {
"durationMs": 341.0425829933956,
"inferenceMs": 341.0425829933956,
"charactersPerSecond": 574.7082909109787,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1635.539750015596,
"totalMs": 1976.5823330089916,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 18.88708400656469,
"inferenceMs": 18.88708400656469,
"charactersPerSecond": 2170.795660449723,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1635.539750015596,
"totalMs": 1654.4268340221606,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 17.79458299279213,
"inferenceMs": 17.79458299279213,
"charactersPerSecond": 1685.9063239724019,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1635.539750015596,
"totalMs": 1653.334333008388,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 17.78020799974911,
"inferenceMs": 17.78020799974911,
"charactersPerSecond": 2474.66171377865,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1635.539750015596,
"totalMs": 1653.319958015345,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 16.998292005155236,
"inferenceMs": 16.998292005155236,
"charactersPerSecond": 1647.2243206263413,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1635.539750015596,
"totalMs": 1652.5380420207512,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 17.062166996765882,
"inferenceMs": 17.062166996765882,
"charactersPerSecond": 2520.1957059821657,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1635.539750015596,
"totalMs": 1652.6019170123618,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.6666666666666666,
"recall": 1,
"f1": 0.8,
"expectedSpans": 4,
"actualSpans": 6,
"speedPassed": true
},
"runtime": {
"durationMs": 341.8487499875482,
"inferenceMs": 341.8487499875482,
"charactersPerSecond": 573.3529814198217,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1636.2530410115141,
"totalMs": 1978.1017909990624,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 19.668332999572158,
"inferenceMs": 19.668332999572158,
"charactersPerSecond": 2084.5691396872253,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1636.2530410115141,
"totalMs": 1655.9213740110863,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 19.535291008651257,
"inferenceMs": 19.535291008651257,
"charactersPerSecond": 1535.6822678870983,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1636.2530410115141,
"totalMs": 1655.7883320201654,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 19.092291011475027,
"inferenceMs": 19.092291011475027,
"charactersPerSecond": 2304.5950836154084,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1636.2530410115141,
"totalMs": 1655.3453320229892,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 18.786541011650115,
"inferenceMs": 18.786541011650115,
"charactersPerSecond": 1490.4287054565464,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1636.2530410115141,
"totalMs": 1655.0395820231643,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 18.16916701500304,
"inferenceMs": 18.16916701500304,
"charactersPerSecond": 2366.6467463529343,
"placement": {
"cpu/torch.float32": 199
},
"loadMs": 1636.2530410115141,
"totalMs": 1654.4222080265172,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.6666666666666666,
"recall": 1,
"f1": 0.8,
"expectedSpans": 4,
"actualSpans": 6,
"speedPassed": true
},
"runtime": {
"durationMs": 62.7345830143895,
"inferenceMs": 62.7345830143895,
"charactersPerSecond": 3124.273575789023,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093566464,
"loadMs": 1973.4319580020383,
"totalMs": 2036.1665410164278,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 28.451833000872284,
"inferenceMs": 28.451833000872284,
"charactersPerSecond": 1441.0319362813289,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093615616,
"loadMs": 1973.4319580020383,
"totalMs": 2001.8837910029106,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 21.106875006807968,
"inferenceMs": 21.106875006807968,
"charactersPerSecond": 1421.3378337780262,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093632000,
"loadMs": 1973.4319580020383,
"totalMs": 1994.5388330088463,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 10.601582995150238,
"inferenceMs": 10.601582995150238,
"charactersPerSecond": 4150.323590366465,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093632000,
"loadMs": 1973.4319580020383,
"totalMs": 1984.0335409971885,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 8.488166989991441,
"inferenceMs": 8.488166989991441,
"charactersPerSecond": 3298.7098431281256,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093632000,
"loadMs": 1973.4319580020383,
"totalMs": 1981.9201249920297,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 10.567499994067475,
"inferenceMs": 10.567499994067475,
"charactersPerSecond": 4069.0797278580476,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093632000,
"loadMs": 1973.4319580020383,
"totalMs": 1983.9994579961058,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.6666666666666666,
"recall": 1,
"f1": 0.8,
"expectedSpans": 4,
"actualSpans": 6,
"speedPassed": true
},
"runtime": {
"durationMs": 63.15129200811498,
"inferenceMs": 63.15129200811498,
"charactersPerSecond": 3103.6577996664564,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093566464,
"loadMs": 1881.06220800546,
"totalMs": 1944.213500013575,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 25.280874979216605,
"inferenceMs": 25.280874979216605,
"charactersPerSecond": 1621.7793107915006,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093615616,
"loadMs": 1881.06220800546,
"totalMs": 1906.3430829846766,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 20.40816700900905,
"inferenceMs": 20.40816700900905,
"charactersPerSecond": 1469.9997303411278,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093632000,
"loadMs": 1881.06220800546,
"totalMs": 1901.470375014469,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 10.54954098071903,
"inferenceMs": 10.54954098071903,
"charactersPerSecond": 4170.797580711524,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093632000,
"loadMs": 1881.06220800546,
"totalMs": 1891.611748986179,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 9.151874983217567,
"inferenceMs": 9.151874983217567,
"charactersPerSecond": 3059.482352124079,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093632000,
"loadMs": 1881.06220800546,
"totalMs": 1890.2140829886775,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 11.531708005350083,
"inferenceMs": 11.531708005350083,
"charactersPerSecond": 3728.8491852248035,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093632000,
"loadMs": 1881.06220800546,
"totalMs": 1892.59391601081,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.6666666666666666,
"recall": 1,
"f1": 0.8,
"expectedSpans": 4,
"actualSpans": 6,
"speedPassed": true
},
"runtime": {
"durationMs": 71.88345899339765,
"inferenceMs": 71.88345899339765,
"charactersPerSecond": 2726.6356230576243,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093566464,
"loadMs": 1850.8375409874134,
"totalMs": 1922.720999980811,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 25.170542008709162,
"inferenceMs": 25.170542008709162,
"charactersPerSecond": 1628.8882450689282,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093615616,
"loadMs": 1850.8375409874134,
"totalMs": 1876.0080829961225,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 21.807750017615035,
"inferenceMs": 21.807750017615035,
"charactersPerSecond": 1375.6577352440183,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093632000,
"loadMs": 1850.8375409874134,
"totalMs": 1872.6452910050284,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 10.022375005064532,
"inferenceMs": 10.022375005064532,
"charactersPerSecond": 4390.176976791011,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093632000,
"loadMs": 1850.8375409874134,
"totalMs": 1860.859915992478,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 8.659958984935656,
"inferenceMs": 8.659958984935656,
"charactersPerSecond": 3233.271664300849,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093632000,
"loadMs": 1850.8375409874134,
"totalMs": 1859.497499972349,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 10.162374994251877,
"inferenceMs": 10.162374994251877,
"charactersPerSecond": 4231.294360257519,
"placement": {
"mps:0/torch.float32": 199
},
"driverAllocatedMemoryBytes": 1093632000,
"loadMs": 1850.8375409874134,
"totalMs": 1860.9999159816653,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.6666666666666666,
"recall": 1,
"f1": 0.8,
"expectedSpans": 4,
"actualSpans": 6,
"speedPassed": true
},
"runtime": {
"durationMs": 62.66983397654258,
"inferenceMs": 62.66983397654258,
"charactersPerSecond": 3127.501503727665,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093566464,
"loadMs": 1862.922250002157,
"totalMs": 1925.5920839786995,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 25.403125007869676,
"inferenceMs": 25.403125007869676,
"charactersPerSecond": 1613.974658129602,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093615616,
"loadMs": 1862.922250002157,
"totalMs": 1888.3253750100266,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 20.405666989972815,
"inferenceMs": 20.405666989972815,
"charactersPerSecond": 1470.1798287084546,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093632000,
"loadMs": 1862.922250002157,
"totalMs": 1883.3279169921298,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 9.107624995522201,
"inferenceMs": 9.107624995522201,
"charactersPerSecond": 4831.116786388632,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093632000,
"loadMs": 1862.922250002157,
"totalMs": 1872.0298749976791,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 7.836624979972839,
"inferenceMs": 7.836624979972839,
"charactersPerSecond": 3572.9666880265904,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093632000,
"loadMs": 1862.922250002157,
"totalMs": 1870.7588749821298,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 8.301541995024309,
"inferenceMs": 8.301541995024309,
"charactersPerSecond": 5179.760582524656,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093632000,
"loadMs": 1862.922250002157,
"totalMs": 1871.2237919971813,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.6666666666666666,
"recall": 1,
"f1": 0.8,
"expectedSpans": 4,
"actualSpans": 6,
"speedPassed": true
},
"runtime": {
"durationMs": 70.09320799261332,
"inferenceMs": 70.09320799261332,
"charactersPerSecond": 2796.2766381110023,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093566464,
"loadMs": 1915.7919579884037,
"totalMs": 1985.885165981017,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 29.141042003175244,
"inferenceMs": 29.141042003175244,
"charactersPerSecond": 1406.9503758833534,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093615616,
"loadMs": 1915.7919579884037,
"totalMs": 1944.932999991579,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 21.509875019546598,
"inferenceMs": 21.509875019546598,
"charactersPerSecond": 1394.7082432017014,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093632000,
"loadMs": 1915.7919579884037,
"totalMs": 1937.3018330079503,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 14.300207985797897,
"inferenceMs": 14.300207985797897,
"charactersPerSecond": 3076.8783253850675,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093632000,
"loadMs": 1915.7919579884037,
"totalMs": 1930.0921659742016,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 8.392415998969227,
"inferenceMs": 8.392415998969227,
"charactersPerSecond": 3336.345577178135,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093632000,
"loadMs": 1915.7919579884037,
"totalMs": 1924.184373987373,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 9.18154200189747,
"inferenceMs": 9.18154200189747,
"charactersPerSecond": 4683.30918609462,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093632000,
"loadMs": 1915.7919579884037,
"totalMs": 1924.9734999903012,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.6666666666666666,
"recall": 1,
"f1": 0.8,
"expectedSpans": 4,
"actualSpans": 6,
"speedPassed": true
},
"runtime": {
"durationMs": 64.02608400094323,
"inferenceMs": 64.02608400094323,
"charactersPerSecond": 3061.2523482946817,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093566464,
"loadMs": 1913.961709011346,
"totalMs": 1977.9877930122893,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 23.988291999557987,
"inferenceMs": 23.988291999557987,
"charactersPerSecond": 1709.1671220591893,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093615616,
"loadMs": 1913.961709011346,
"totalMs": 1937.950001010904,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 21.961000020382926,
"inferenceMs": 21.961000020382926,
"charactersPerSecond": 1366.0580106623443,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093632000,
"loadMs": 1913.961709011346,
"totalMs": 1935.922709031729,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 9.934500005329028,
"inferenceMs": 9.934500005329028,
"charactersPerSecond": 4429.0100132264015,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093632000,
"loadMs": 1913.961709011346,
"totalMs": 1923.8962090166751,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 8.689207985298708,
"inferenceMs": 8.689207985298708,
"charactersPerSecond": 3222.3880527860842,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093632000,
"loadMs": 1913.961709011346,
"totalMs": 1922.6509169966448,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 9.785290982108563,
"inferenceMs": 9.785290982108563,
"charactersPerSecond": 4394.350671699109,
"placement": {
"mps:0/torch.float16": 199
},
"driverAllocatedMemoryBytes": 1093632000,
"loadMs": 1913.961709011346,
"totalMs": 1923.7469999934547,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
}
]BERT Large Cased NER Desktop
48 configurations testedQuality criteria
{ "file": "research/bert-ner-family-expansion/quality-gates.json", "sha256": "4362bea8f9d1c5d1975a56c1ded21d9244114c3e347968ebe9ac4e72e2b705e3", "description": "Six frozen NER cases: selected-span recall for non-exhaustive NASA reference; exact-span recall/F1 for five complete references, including valid UTF-16 offsets and no-entity negative; <=1000ms per text." }
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 13 ms / 2332 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 14 ms / 2354 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 14 ms / 2397 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 15 ms / 2332 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 16 ms / 2354 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 17 ms / 2397 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 17 ms / 2374 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 17 ms / 2397 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 17 ms / 2354 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 18 ms / 2332 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 19 ms / 2397 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 22 ms / 2397 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 22 ms / 2374 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 23 ms / 2374 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 27 ms / 2397 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 28 ms / 2475 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 29 ms / 2374 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 32 ms / 2397 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 33 ms / 2475 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 33 ms / 2475 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 37 ms / 2397 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 37 ms / 2332 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 38 ms / 2374 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 42 ms / 2397 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 43 ms / 2397 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 43 ms / 2332 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 72 ms / 2374 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 73 ms / 2397 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 74 ms / 2332 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 75 ms / 2397 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 169 ms / 2475 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 171 ms / 2354 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 171 ms / 2475 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 192 ms / 2354 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 216 ms / 2475 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 230 ms / 2354 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 1184 ms / 1650 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 55 ms / 1650 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 54 ms / 1650 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 55 ms / 1650 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 55 ms / 1650 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 57 ms / 1650 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 1153 ms / 1665 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 57 ms / 1665 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 54 ms / 1665 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 55 ms / 1665 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 54 ms / 1665 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 55 ms / 1665 ms |
Test setup & sample details
- Quality gate
- { "file": "research/bert-ner-family-expansion/quality-gates.json", "sha256": "4362bea8f9d1c5d1975a56c1ded21d9244114c3e347968ebe9ac4e72e2b705e3", "description": "Six frozen NER cases: selected-span recall for non-exhaustive NASA reference; exact-span recall/F1 for five complete references, including valid UTF-16 offsets and no-entity negative; <=1000ms per text." }
- Tested
- 2026-11-12T22:40:54.760961+00:00
- Environment
- { "os": "macos", "osVersion": "27.2", "architecture": "aarch64", "hardware": "Apple M3 Max;Mac15,11;96GiB RAM", "cpu": "Apple M3 Max", "gpu": "Apple M3 Max", "driver": "OS-managed Metal/MPS macOS27.2 build26B118", "runtime": "App-owned CPython3.13.18;Torch2.10.0;Transformers5.22.1", "timingScope": "Fresh helper/model process per cold repetition; runtime/tokenizer/model startup in load. OS files/driver cache may be warm. Five warm texts follow six fixtures. Snapshot loads, not Internet transfer.", "concurrentUploadActivity": "none; uploads excluded" }
- Runtime
- transformers 5.22.1
- Artifact revision
- 6fe43d9ec0bba0f67e367ecd74399216fc409c7f
- Fixture
{ "file": "model-samples/native/bert-ner/quality-gates.json", "url": "https://assets.localai.tools/samples/v1/files/4362bea8f9d1c5d1975a56c1ded21d9244114c3e347968ebe9ac4e72e2b705e3/bert-ner-quality-gates.json", "size": 4105, "sha1": "50eaec89241b58910bbfe13b9c7c1d4a2dac1816", "sha256": "4362bea8f9d1c5d1975a56c1ded21d9244114c3e347968ebe9ac4e72e2b705e3" }- Rerun when
- artifact-revision; runtime-version; fixture-checksum; quality-gate; backend-behavior
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": false,
"validOffsetsAndScores": true,
"precision": 0.5,
"recall": 1,
"f1": 0.6666666666666666,
"expectedSpans": 4,
"actualSpans": 8,
"speedPassed": false
},
"runtime": {
"durationMs": 1183.6334579857066,
"inferenceMs": 1183.6334579857066,
"charactersPerSecond": 165.59180435263335,
"placement": {
"cpu/torch.float32": 391
},
"loadMs": 1650.0002079992555,
"totalMs": 2833.633665984962,
"grade": "failed",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 55.331667012069374,
"inferenceMs": 55.331667012069374,
"charactersPerSecond": 740.9861696568216,
"placement": {
"cpu/torch.float32": 391
},
"loadMs": 1650.0002079992555,
"totalMs": 1705.3318750113249,
"grade": "failed",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 54.47579099563882,
"inferenceMs": 54.47579099563882,
"charactersPerSecond": 550.7033390740801,
"placement": {
"cpu/torch.float32": 391
},
"loadMs": 1650.0002079992555,
"totalMs": 1704.4759989948943,
"grade": "failed",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 54.65445900335908,
"inferenceMs": 54.65445900335908,
"charactersPerSecond": 805.0578269797849,
"placement": {
"cpu/torch.float32": 391
},
"loadMs": 1650.0002079992555,
"totalMs": 1704.6546670026146,
"grade": "failed",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 54.53295799088664,
"inferenceMs": 54.53295799088664,
"charactersPerSecond": 513.4509667471048,
"placement": {
"cpu/torch.float32": 391
},
"loadMs": 1650.0002079992555,
"totalMs": 1704.5331659901422,
"grade": "failed",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 57.404541003052145,
"inferenceMs": 57.404541003052145,
"charactersPerSecond": 749.0696597977106,
"placement": {
"cpu/torch.float32": 391
},
"loadMs": 1650.0002079992555,
"totalMs": 1707.4047490023077,
"grade": "failed",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": false,
"validOffsetsAndScores": true,
"precision": 0.5,
"recall": 1,
"f1": 0.6666666666666666,
"expectedSpans": 4,
"actualSpans": 8,
"speedPassed": false
},
"runtime": {
"durationMs": 1152.9059169988614,
"inferenceMs": 1152.9059169988614,
"charactersPerSecond": 170.00519913212796,
"placement": {
"cpu/torch.float32": 391
},
"loadMs": 1664.7429169970565,
"totalMs": 2817.648833995918,
"grade": "failed",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 56.73645800561644,
"inferenceMs": 56.73645800561644,
"charactersPerSecond": 722.6394005057795,
"placement": {
"cpu/torch.float32": 391
},
"loadMs": 1664.7429169970565,
"totalMs": 1721.479375002673,
"grade": "failed",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 53.88679099269211,
"inferenceMs": 53.88679099269211,
"charactersPerSecond": 556.7227041608113,
"placement": {
"cpu/torch.float32": 391
},
"loadMs": 1664.7429169970565,
"totalMs": 1718.6297079897486,
"grade": "failed",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 55.30670899315737,
"inferenceMs": 55.30670899315737,
"charactersPerSecond": 795.563518441203,
"placement": {
"cpu/torch.float32": 391
},
"loadMs": 1664.7429169970565,
"totalMs": 1720.0496259902138,
"grade": "failed",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 53.76837501535192,
"inferenceMs": 53.76837501535192,
"charactersPerSecond": 520.7522078174289,
"placement": {
"cpu/torch.float32": 391
},
"loadMs": 1664.7429169970565,
"totalMs": 1718.5112920124084,
"grade": "failed",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 54.95524997240864,
"inferenceMs": 54.95524997240864,
"charactersPerSecond": 782.4548159018291,
"placement": {
"cpu/torch.float32": 391
},
"loadMs": 1664.7429169970565,
"totalMs": 1719.698166969465,
"grade": "failed",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.5,
"recall": 1,
"f1": 0.6666666666666666,
"expectedSpans": 4,
"actualSpans": 8,
"speedPassed": true
},
"runtime": {
"durationMs": 171.35033299564384,
"inferenceMs": 171.35033299564384,
"charactersPerSecond": 1143.8553784717933,
"placement": {
"mps:0/torch.float32": 391
},
"driverAllocatedMemoryBytes": 2167406592,
"loadMs": 2475.1819579978473,
"totalMs": 2646.532290993491,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 168.81716600619256,
"inferenceMs": 168.81716600619256,
"charactersPerSecond": 242.8662971305657,
"placement": {
"mps:0/torch.float32": 391
},
"driverAllocatedMemoryBytes": 2167455744,
"loadMs": 2475.1819579978473,
"totalMs": 2643.99912400404,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 216.1658749973867,
"inferenceMs": 216.1658749973867,
"charactersPerSecond": 138.78231242726298,
"placement": {
"mps:0/torch.float32": 391
},
"driverAllocatedMemoryBytes": 2167472128,
"loadMs": 2475.1819579978473,
"totalMs": 2691.347832995234,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 33.432124997489154,
"inferenceMs": 33.432124997489154,
"charactersPerSecond": 1316.0994104713511,
"placement": {
"mps:0/torch.float32": 391
},
"driverAllocatedMemoryBytes": 2167472128,
"loadMs": 2475.1819579978473,
"totalMs": 2508.6140829953365,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 28.224124980624765,
"inferenceMs": 28.224124980624765,
"charactersPerSecond": 992.0590990587442,
"placement": {
"mps:0/torch.float32": 391
},
"driverAllocatedMemoryBytes": 2167472128,
"loadMs": 2475.1819579978473,
"totalMs": 2503.406082978472,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 33.00024999771267,
"inferenceMs": 33.00024999771267,
"charactersPerSecond": 1303.020431753712,
"placement": {
"mps:0/torch.float32": 391
},
"driverAllocatedMemoryBytes": 2167472128,
"loadMs": 2475.1819579978473,
"totalMs": 2508.18220799556,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.5,
"recall": 1,
"f1": 0.6666666666666666,
"expectedSpans": 4,
"actualSpans": 8,
"speedPassed": true
},
"runtime": {
"durationMs": 71.61295900004916,
"inferenceMs": 71.61295900004916,
"charactersPerSecond": 2736.9348053313292,
"placement": {
"mps:0/torch.float32": 391
},
"driverAllocatedMemoryBytes": 2167406592,
"loadMs": 2373.788291006349,
"totalMs": 2445.401250006398,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 38.14087502541952,
"inferenceMs": 38.14087502541952,
"charactersPerSecond": 1074.9622281259929,
"placement": {
"mps:0/torch.float32": 391
},
"driverAllocatedMemoryBytes": 2167472128,
"loadMs": 2373.788291006349,
"totalMs": 2411.9291660317685,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 28.916584007674828,
"inferenceMs": 28.916584007674828,
"charactersPerSecond": 1037.4669425696209,
"placement": {
"mps:0/torch.float32": 391
},
"driverAllocatedMemoryBytes": 2167488512,
"loadMs": 2373.788291006349,
"totalMs": 2402.704875014024,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 22.447166993515566,
"inferenceMs": 22.447166993515566,
"charactersPerSecond": 1960.1582690907283,
"placement": {
"mps:0/torch.float32": 391
},
"driverAllocatedMemoryBytes": 2167488512,
"loadMs": 2373.788291006349,
"totalMs": 2396.2354579998646,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 16.8270829890389,
"inferenceMs": 16.8270829890389,
"charactersPerSecond": 1663.9841865782143,
"placement": {
"mps:0/torch.float32": 391
},
"driverAllocatedMemoryBytes": 2167488512,
"loadMs": 2373.788291006349,
"totalMs": 2390.615373995388,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 22.81712502008304,
"inferenceMs": 22.81712502008304,
"charactersPerSecond": 1884.5494321546873,
"placement": {
"mps:0/torch.float32": 391
},
"driverAllocatedMemoryBytes": 2167488512,
"loadMs": 2373.788291006349,
"totalMs": 2396.605416026432,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.5,
"recall": 1,
"f1": 0.6666666666666666,
"expectedSpans": 4,
"actualSpans": 8,
"speedPassed": true
},
"runtime": {
"durationMs": 74.9443750246428,
"inferenceMs": 74.9443750246428,
"charactersPerSecond": 2615.272993277379,
"placement": {
"mps:0/torch.float32": 391
},
"driverAllocatedMemoryBytes": 2167406592,
"loadMs": 2396.73199999379,
"totalMs": 2471.6763750184327,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 41.54837501118891,
"inferenceMs": 41.54837501118891,
"charactersPerSecond": 986.8015292766268,
"placement": {
"mps:0/torch.float32": 391
},
"driverAllocatedMemoryBytes": 2167472128,
"loadMs": 2396.73199999379,
"totalMs": 2438.280375004979,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 43.094499997096136,
"inferenceMs": 43.094499997096136,
"charactersPerSecond": 696.1445196491782,
"placement": {
"mps:0/torch.float32": 391
},
"driverAllocatedMemoryBytes": 2167488512,
"loadMs": 2396.73199999379,
"totalMs": 2439.826499990886,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 37.0672499993816,
"inferenceMs": 37.0672499993816,
"charactersPerSecond": 1187.0316789277342,
"placement": {
"mps:0/torch.float32": 391
},
"driverAllocatedMemoryBytes": 2167488512,
"loadMs": 2396.73199999379,
"totalMs": 2433.7992499931715,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 19.356625009095296,
"inferenceMs": 19.356625009095296,
"charactersPerSecond": 1446.533163030402,
"placement": {
"mps:0/torch.float32": 391
},
"driverAllocatedMemoryBytes": 2167488512,
"loadMs": 2396.73199999379,
"totalMs": 2416.0886250028852,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 22.314417001325637,
"inferenceMs": 22.314417001325637,
"charactersPerSecond": 1927.0053077096072,
"placement": {
"mps:0/torch.float32": 391
},
"driverAllocatedMemoryBytes": 2167488512,
"loadMs": 2396.73199999379,
"totalMs": 2419.0464169951156,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.5,
"recall": 1,
"f1": 0.6666666666666666,
"expectedSpans": 4,
"actualSpans": 8,
"speedPassed": true
},
"runtime": {
"durationMs": 171.07170898816548,
"inferenceMs": 171.07170898816548,
"charactersPerSecond": 1145.718372484132,
"placement": {
"mps:0/torch.float16": 391
},
"driverAllocatedMemoryBytes": 1093664768,
"loadMs": 2354.464125004597,
"totalMs": 2525.5358339927625,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 192.4099580210168,
"inferenceMs": 192.4099580210168,
"charactersPerSecond": 213.08668439874407,
"placement": {
"mps:0/torch.float16": 391
},
"driverAllocatedMemoryBytes": 1093730304,
"loadMs": 2354.464125004597,
"totalMs": 2546.874083025614,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 230.0700829946436,
"inferenceMs": 230.0700829946436,
"charactersPerSecond": 130.39505010609506,
"placement": {
"mps:0/torch.float16": 391
},
"driverAllocatedMemoryBytes": 1093746688,
"loadMs": 2354.464125004597,
"totalMs": 2584.5342079992406,
"grade": "good",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 16.20320900110528,
"inferenceMs": 16.20320900110528,
"charactersPerSecond": 2715.511476584583,
"placement": {
"mps:0/torch.float16": 391
},
"driverAllocatedMemoryBytes": 1093746688,
"loadMs": 2354.464125004597,
"totalMs": 2370.6673340057023,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 14.021832990692928,
"inferenceMs": 14.021832990692928,
"charactersPerSecond": 1996.885857832222,
"placement": {
"mps:0/torch.float16": 391
},
"driverAllocatedMemoryBytes": 1093746688,
"loadMs": 2354.464125004597,
"totalMs": 2368.48595799529,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 17.096291994675994,
"inferenceMs": 17.096291994675994,
"charactersPerSecond": 2515.1652775578914,
"placement": {
"mps:0/torch.float16": 391
},
"driverAllocatedMemoryBytes": 1093746688,
"loadMs": 2354.464125004597,
"totalMs": 2371.560416999273,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.5,
"recall": 1,
"f1": 0.6666666666666666,
"expectedSpans": 4,
"actualSpans": 8,
"speedPassed": true
},
"runtime": {
"durationMs": 72.65924999956042,
"inferenceMs": 72.65924999956042,
"charactersPerSecond": 2697.5230270225165,
"placement": {
"mps:0/torch.float16": 391
},
"driverAllocatedMemoryBytes": 1093664768,
"loadMs": 2397.322583012283,
"totalMs": 2469.9818330118433,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 31.737207988044247,
"inferenceMs": 31.737207988044247,
"charactersPerSecond": 1291.859070131347,
"placement": {
"mps:0/torch.float16": 391
},
"driverAllocatedMemoryBytes": 1093730304,
"loadMs": 2397.322583012283,
"totalMs": 2429.059791000327,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 26.725125004304573,
"inferenceMs": 26.725125004304573,
"charactersPerSecond": 1122.539183452573,
"placement": {
"mps:0/torch.float16": 391
},
"driverAllocatedMemoryBytes": 1093746688,
"loadMs": 2397.322583012283,
"totalMs": 2424.0477080165874,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 16.88370801275596,
"inferenceMs": 16.88370801275596,
"charactersPerSecond": 2606.062599919234,
"placement": {
"mps:0/torch.float16": 391
},
"driverAllocatedMemoryBytes": 1093746688,
"loadMs": 2397.322583012283,
"totalMs": 2414.206291025039,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 14.18720799847506,
"inferenceMs": 14.18720799847506,
"charactersPerSecond": 1973.6089019777278,
"placement": {
"mps:0/torch.float16": 391
},
"driverAllocatedMemoryBytes": 1093746688,
"loadMs": 2397.322583012283,
"totalMs": 2411.509791010758,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 16.671250021317974,
"inferenceMs": 16.671250021317974,
"charactersPerSecond": 2579.2906917606506,
"placement": {
"mps:0/torch.float16": 391
},
"driverAllocatedMemoryBytes": 1093746688,
"loadMs": 2397.322583012283,
"totalMs": 2413.993833033601,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 0.5,
"recall": 1,
"f1": 0.6666666666666666,
"expectedSpans": 4,
"actualSpans": 8,
"speedPassed": true
},
"runtime": {
"durationMs": 74.18345799669623,
"inferenceMs": 74.18345799669623,
"charactersPerSecond": 2642.0984582402302,
"placement": {
"mps:0/torch.float16": 391
},
"driverAllocatedMemoryBytes": 1093664768,
"loadMs": 2331.688500009477,
"totalMs": 2405.8719580061734,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 43.247791996691376,
"inferenceMs": 43.247791996691376,
"charactersPerSecond": 948.0252772936167,
"placement": {
"mps:0/torch.float16": 391
},
"driverAllocatedMemoryBytes": 1093730304,
"loadMs": 2331.688500009477,
"totalMs": 2374.9362920061685,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 37.195708020590246,
"inferenceMs": 37.195708020590246,
"charactersPerSecond": 806.5446686320112,
"placement": {
"mps:0/torch.float16": 391
},
"driverAllocatedMemoryBytes": 1093746688,
"loadMs": 2331.688500009477,
"totalMs": 2368.8842080300674,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 0,
"actualSpans": 0,
"speedPassed": true
},
"runtime": {
"durationMs": 18.45416601281613,
"inferenceMs": 18.45416601281613,
"charactersPerSecond": 2384.285476214026,
"placement": {
"mps:0/torch.float16": 391
},
"driverAllocatedMemoryBytes": 1093746688,
"loadMs": 2331.688500009477,
"totalMs": 2350.1426660222933,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 2,
"actualSpans": 2,
"speedPassed": true
},
"runtime": {
"durationMs": 12.643750000279397,
"inferenceMs": 12.643750000279397,
"charactersPerSecond": 2214.5328719233826,
"placement": {
"mps:0/torch.float16": 391
},
"driverAllocatedMemoryBytes": 1093746688,
"loadMs": 2331.688500009477,
"totalMs": 2344.3322500097565,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"validOffsetsAndScores": true,
"precision": 1,
"recall": 1,
"f1": 1,
"expectedSpans": 4,
"actualSpans": 4,
"speedPassed": true
},
"runtime": {
"durationMs": 15.244000009261072,
"inferenceMs": 15.244000009261072,
"charactersPerSecond": 2820.7819452818508,
"placement": {
"mps:0/torch.float16": 391
},
"driverAllocatedMemoryBytes": 1093746688,
"loadMs": 2331.688500009477,
"totalMs": 2346.932500018738,
"grade": "excellent",
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false
}
}
]Release validation still required
- Sample publication, complete dependency backup and forced model/runtime-source recovery pending under user upload exclusion.