A model is still running

A model is still running or downloading. Leaving will stop the current operation. Stay on this page to let it finish.

Skip to content
LocalAI.toolsBeta

Join the waitlist

Get an email when the new apps and model runs launch.

No account needed.

BERT NER Desktop

Extract English people, organizations, locations and miscellaneous named entity spans with the original Base · Cased checkpoint.

Desktop inferencePrivate · Runs on your computer
About this model

Extract English people, organizations, locations and miscellaneous named entity spans with the original Base · Cased checkpoint.

Created by
dslim
License
MIT
Model released
Unknown
Source repository created
Mar 2, 2022
Model weights · FP16
433.3 MB

Best for

  • Highlight named entities in English news-like text.
  • Prototype local document indexing, search annotation, and information-extraction workflows.
  • Provide candidate spans for human-reviewed downstream processing.

Keep in mind

  • The model recognizes only PER, ORG, LOC, and MISC; dates, quantities, products, events, and other entity types are not represented consistently.
  • It was trained on English Reuters news from a limited historical period and may generalize poorly to other languages, domains, emerging names, informal spelling, or unusual capitalization.
  • Subword tokenization can split, truncate, merge, or omit entity spans; character offsets and labels require application-level review.
  • Confidence scores are not calibrated guarantees and must not be used alone for consequential decisions about people.
  • Inputs longer than 512 tokens are truncated, so entities beyond the retained prefix are omitted.
  • Six short English NER sanity fixtures only, not a full CoNLL-2003, privacy, factuality or multilingual validation.
  • Default simple BIO aggregation can return partial-word spans. First whole-word aggregation is the measured native default. Large CPU exceeds the frozen1second NASA case budget and is unavailable; GPU-only support has no CPU recovery.
  • Tokenizer/preprocessing, BIO aggregation and serialization run on CPU; model forward/parameters use MPS.
  • Windows, Intel Mac and other Apple hardware/OS remain unverified.

Processed on your deviceYour input is processed locally in the desktop app. Model and runtime files download from external hosts.

Target platforms · verification required

  • macos · aarch64, x86_64
  • windows · x86_64

Recommended configuration

FP16 · mps (verified platform only)

Sources & technical details
Original model
dslim/bert-base-NER
Release verification
Unknown
Release evidence
Exact original checkpoint public release date has not been verified. Repository creation, training, paper, artifact conversion and update dates are not release evidence.
Source updated
Oct 8, 2024
Native model files
dslim/bert-base-NER
Files updated
Oct 8, 2024
Artifact revision
d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc

Desktop app

Sample result

BERT Base Cased NER Desktop

Run BERT NER Desktop on your computer

Download the desktop app to load this model, provide input and view results locally.

Public desktop downloads are coming soon.

Device requirements

macOS Apple SiliconCPU / FP32Tested8.0 GiBRAM estimate
macOS Apple SiliconMPS / FP16Tested8.0 GiBRAM estimate
macOS Apple SiliconMPS / FP32Tested8.0 GiBRAM estimate
Windows Intel / AMDCPU / FP32Unverified8.0 GiBRAM estimate
macOS Intel / AMDCPU / FP32Unverified8.0 GiBRAM estimate
If your device falls short

MemoryAdvisory only: close other apps, reduce image/tile/token/batch settings or choose a smaller verified model/precision. CPU can need more RAM than GPU.

GPUIf the selected GPU runtime fails, existing CPU fallback may recover when enabled, but CPU has its own memory/runtime needs.

PlatformUse a compatible OS/architecture/runtime or another model. These estimates do not add hardware detection or block runs.

Benchmark evidence

Saved desktop measurements apply to the recorded hardware and runtime. Fallback cases report the device that actually ran; they do not prove GPU support on the requested device.

BERT Base Cased NER Desktop

72 configurations tested
Passed72
Failed0
TestedNov 12, 2026
Runtimetransformers 5.22.1

Quality criteria

{ "file": "research/bert-ner-family-expansion/quality-gates.json", "sha256": "4362bea8f9d1c5d1975a56c1ded21d9244114c3e347968ebe9ac4e72e2b705e3", "description": "Six frozen NER cases: selected-span recall for non-exhaustive NASA reference; exact-span recall/F1 for five complete references, including valid UTF-16 offsets and no-entity negative; <=1000ms per text." }

PrecisionDeviceResultPlatformInference / load
FP16mpsAcceleratedPassedVery fastmacos · aarch648 ms / 1931 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch648 ms / 1875 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch648 ms / 2559 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch648 ms / 1931 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch648 ms / 1875 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch649 ms / 1931 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch649 ms / 1905 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch649 ms / 2559 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6410 ms / 1912 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6410 ms / 1875 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6410 ms / 2559 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6410 ms / 1905 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6410 ms / 1912 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6411 ms / 1905 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6412 ms / 1912 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6414 ms / 1909 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6415 ms / 1909 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6417 ms / 1909 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6418 ms / 1931 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6419 ms / 2559 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6420 ms / 1875 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6422 ms / 1909 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6422 ms / 1905 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6424 ms / 1912 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6424 ms / 1875 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6425 ms / 2559 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6426 ms / 1905 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6427 ms / 1909 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6431 ms / 1912 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6431 ms / 1931 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6463 ms / 1909 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6463 ms / 1875 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6465 ms / 1905 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6467 ms / 1912 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6472 ms / 1931 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6473 ms / 2559 ms
FP32cpuPassedVery fastmacos · aarch6416 ms / 1663 ms
FP32cpuPassedVery fastmacos · aarch6416 ms / 1639 ms
FP32cpuPassedVery fastmacos · aarch6417 ms / 1724 ms
FP32cpuPassedVery fastmacos · aarch6417 ms / 1663 ms
FP32cpuPassedVery fastmacos · aarch6417 ms / 1639 ms
FP32cpuPassedVery fastmacos · aarch6417 ms / 1624 ms
FP32cpuPassedVery fastmacos · aarch6417 ms / 1724 ms
FP32cpuPassedVery fastmacos · aarch6417 ms / 1663 ms
FP32cpuPassedVery fastmacos · aarch6417 ms / 1639 ms
FP32cpuPassedVery fastmacos · aarch6417 ms / 1724 ms
FP32cpuPassedVery fastmacos · aarch6417 ms / 1642 ms
FP32cpuPassedVery fastmacos · aarch6418 ms / 1624 ms
FP32cpuPassedVery fastmacos · aarch6418 ms / 1624 ms
FP32cpuPassedVery fastmacos · aarch6418 ms / 1663 ms
FP32cpuPassedVery fastmacos · aarch6418 ms / 1724 ms
FP32cpuPassedVery fastmacos · aarch6418 ms / 1624 ms
FP32cpuPassedVery fastmacos · aarch6418 ms / 1639 ms
FP32cpuPassedVery fastmacos · aarch6418 ms / 1642 ms
FP32cpuPassedVery fastmacos · aarch6418 ms / 1639 ms
FP32cpuPassedVery fastmacos · aarch6418 ms / 1639 ms
FP32cpuPassedVery fastmacos · aarch6418 ms / 1663 ms
FP32cpuPassedVery fastmacos · aarch6418 ms / 1642 ms
FP32cpuPassedVery fastmacos · aarch6419 ms / 1642 ms
FP32cpuPassedVery fastmacos · aarch6419 ms / 1624 ms
FP32cpuPassedVery fastmacos · aarch6419 ms / 1639 ms
FP32cpuPassedVery fastmacos · aarch6419 ms / 1639 ms
FP32cpuPassedVery fastmacos · aarch6419 ms / 1639 ms
FP32cpuPassedVery fastmacos · aarch6419 ms / 1724 ms
FP32cpuPassedVery fastmacos · aarch6419 ms / 1642 ms
FP32cpuPassedVery fastmacos · aarch6420 ms / 1639 ms
FP32cpuPassedFastmacos · aarch64336 ms / 1642 ms
FP32cpuPassedFastmacos · aarch64336 ms / 1639 ms
FP32cpuPassedFastmacos · aarch64338 ms / 1624 ms
FP32cpuPassedFastmacos · aarch64340 ms / 1724 ms
FP32cpuPassedFastmacos · aarch64342 ms / 1639 ms
FP32cpuPassedFastmacos · aarch64357 ms / 1663 ms
Test setup & sample details
Quality gate
{ "file": "research/bert-ner-family-expansion/quality-gates.json", "sha256": "4362bea8f9d1c5d1975a56c1ded21d9244114c3e347968ebe9ac4e72e2b705e3", "description": "Six frozen NER cases: selected-span recall for non-exhaustive NASA reference; exact-span recall/F1 for five complete references, including valid UTF-16 offsets and no-entity negative; <=1000ms per text." }
Tested
2026-11-12T22:40:54.760961+00:00
Environment
{ "os": "macos", "osVersion": "27.2", "architecture": "aarch64", "hardware": "Apple M3 Max;Mac15,11;96GiB RAM", "cpu": "Apple M3 Max", "gpu": "Apple M3 Max", "driver": "OS-managed Metal/MPS macOS27.2 build26B118", "runtime": "App-owned CPython3.13.18;Torch2.10.0;Transformers5.22.1", "timingScope": "Fresh helper/model process per cold repetition; runtime/tokenizer/model startup in load. OS files/driver cache may be warm. Five warm texts follow six fixtures. Snapshot loads, not Internet transfer.", "concurrentUploadActivity": "none; uploads excluded" }
Runtime
transformers 5.22.1
Artifact revision
d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc
Fixture
{
  "file": "model-samples/native/bert-ner/quality-gates.json",
  "url": "https://assets.localai.tools/samples/v1/files/4362bea8f9d1c5d1975a56c1ded21d9244114c3e347968ebe9ac4e72e2b705e3/bert-ner-quality-gates.json",
  "size": 4105,
  "sha1": "50eaec89241b58910bbfe13b9c7c1d4a2dac1816",
  "sha256": "4362bea8f9d1c5d1975a56c1ded21d9244114c3e347968ebe9ac4e72e2b705e3"
}
Rerun when
artifact-revision; runtime-version; fixture-checksum; quality-gate; backend-behavior
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.5,
      "recall": 1,
      "f1": 0.6666666666666666,
      "expectedSpans": 4,
      "actualSpans": 8,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 357.0229170145467,
      "inferenceMs": 357.0229170145467,
      "charactersPerSecond": 548.9843667150759,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1663.4898750053253,
      "totalMs": 2020.512792019872,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 18.224625004222617,
      "inferenceMs": 18.224625004222617,
      "charactersPerSecond": 2249.703354143109,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1663.4898750053253,
      "totalMs": 1681.714500009548,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 17.17658300185576,
      "inferenceMs": 17.17658300185576,
      "charactersPerSecond": 1746.5639118536437,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1663.4898750053253,
      "totalMs": 1680.666458007181,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 17.598167003598064,
      "inferenceMs": 17.598167003598064,
      "charactersPerSecond": 2500.260395926684,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1663.4898750053253,
      "totalMs": 1681.0880420089234,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 16.262125020148233,
      "inferenceMs": 16.262125020148233,
      "charactersPerSecond": 1721.792199070474,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1663.4898750053253,
      "totalMs": 1679.7520000254735,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 16.71024999814108,
      "inferenceMs": 16.71024999814108,
      "charactersPerSecond": 2573.2708968916386,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1663.4898750053253,
      "totalMs": 1680.2001250034664,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.5,
      "recall": 1,
      "f1": 0.6666666666666666,
      "expectedSpans": 4,
      "actualSpans": 8,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 335.8510420075618,
      "inferenceMs": 335.8510420075618,
      "charactersPerSecond": 583.5920556577788,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1642.0322500052862,
      "totalMs": 1977.883292012848,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 19.20745801180601,
      "inferenceMs": 19.20745801180601,
      "charactersPerSecond": 2134.5875115176113,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1642.0322500052862,
      "totalMs": 1661.2397080170922,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 18.023457989329472,
      "inferenceMs": 18.023457989329472,
      "charactersPerSecond": 1664.4974575778447,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1642.0322500052862,
      "totalMs": 1660.0557079946157,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 18.450000003213063,
      "inferenceMs": 18.450000003213063,
      "charactersPerSecond": 2384.823847823166,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1642.0322500052862,
      "totalMs": 1660.4822500084992,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 17.45891702012159,
      "inferenceMs": 17.45891702012159,
      "charactersPerSecond": 1603.7649968626174,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1642.0322500052862,
      "totalMs": 1659.4911670254078,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 18.568041996331885,
      "inferenceMs": 18.568041996331885,
      "charactersPerSecond": 2315.806912139398,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1642.0322500052862,
      "totalMs": 1660.600292001618,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.5,
      "recall": 1,
      "f1": 0.6666666666666666,
      "expectedSpans": 4,
      "actualSpans": 8,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 341.87833400210366,
      "inferenceMs": 341.87833400210366,
      "charactersPerSecond": 573.3033670358121,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1638.9899170026183,
      "totalMs": 1980.868251004722,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 18.17079199827276,
      "inferenceMs": 18.17079199827276,
      "charactersPerSecond": 2256.368352237882,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1638.9899170026183,
      "totalMs": 1657.160709000891,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 17.408000014256686,
      "inferenceMs": 17.408000014256686,
      "charactersPerSecond": 1723.3455868239203,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1638.9899170026183,
      "totalMs": 1656.397917016875,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 17.741582996677607,
      "inferenceMs": 17.741582996677607,
      "charactersPerSecond": 2480.049272279689,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1638.9899170026183,
      "totalMs": 1656.731499999296,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 16.496916010510176,
      "inferenceMs": 16.496916010510176,
      "charactersPerSecond": 1697.2869342464503,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1638.9899170026183,
      "totalMs": 1655.4868330131285,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 16.76833297824487,
      "inferenceMs": 16.76833297824487,
      "charactersPerSecond": 2564.3574740427644,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1638.9899170026183,
      "totalMs": 1655.7582499808632,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.5,
      "recall": 1,
      "f1": 0.6666666666666666,
      "expectedSpans": 4,
      "actualSpans": 8,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 336.3414999912493,
      "inferenceMs": 336.3414999912493,
      "charactersPerSecond": 582.7410533790787,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1639.0415420173667,
      "totalMs": 1975.383042008616,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 20.12829200248234,
      "inferenceMs": 20.12829200248234,
      "charactersPerSecond": 2036.9338836570753,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1639.0415420173667,
      "totalMs": 1659.169834019849,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 18.808625027304515,
      "inferenceMs": 18.808625027304515,
      "charactersPerSecond": 1595.0129239351068,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1639.0415420173667,
      "totalMs": 1657.8501670446713,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 18.77199998125434,
      "inferenceMs": 18.77199998125434,
      "charactersPerSecond": 2343.9164736809216,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1639.0415420173667,
      "totalMs": 1657.813541998621,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 18.188459012890235,
      "inferenceMs": 18.188459012890235,
      "charactersPerSecond": 1539.4377269760064,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1639.0415420173667,
      "totalMs": 1657.230001030257,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 19.07145799486898,
      "inferenceMs": 19.07145799486898,
      "charactersPerSecond": 2254.678169417818,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1639.0415420173667,
      "totalMs": 1658.1130000122357,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.5,
      "recall": 1,
      "f1": 0.6666666666666666,
      "expectedSpans": 4,
      "actualSpans": 8,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 340.4717080120463,
      "inferenceMs": 340.4717080120463,
      "charactersPerSecond": 575.6719145458785,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1723.603374994127,
      "totalMs": 2064.0750830061734,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 19.071541988523677,
      "inferenceMs": 19.071541988523677,
      "charactersPerSecond": 2149.7999492999465,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1723.603374994127,
      "totalMs": 1742.6749169826508,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 17.673541005933657,
      "inferenceMs": 17.673541005933657,
      "charactersPerSecond": 1697.4527057100722,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1723.603374994127,
      "totalMs": 1741.2769160000607,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 17.447874997742474,
      "inferenceMs": 17.447874997742474,
      "charactersPerSecond": 2521.797067304358,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1723.603374994127,
      "totalMs": 1741.0512499918696,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 16.666999988956377,
      "inferenceMs": 16.666999988956377,
      "charactersPerSecond": 1679.9664017851392,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1723.603374994127,
      "totalMs": 1740.2703749830835,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 17.041459010215476,
      "inferenceMs": 17.041459010215476,
      "charactersPerSecond": 2523.258130317581,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1723.603374994127,
      "totalMs": 1740.6448340043426,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.5,
      "recall": 1,
      "f1": 0.6666666666666666,
      "expectedSpans": 4,
      "actualSpans": 8,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 337.64375001192093,
      "inferenceMs": 337.64375001192093,
      "charactersPerSecond": 580.4934934915276,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1624.2325420025736,
      "totalMs": 1961.8762920144945,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 18.60416601994075,
      "inferenceMs": 18.60416601994075,
      "charactersPerSecond": 2203.8074674271575,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1624.2325420025736,
      "totalMs": 1642.8367080225144,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 17.508832999737933,
      "inferenceMs": 17.508832999737933,
      "charactersPerSecond": 1713.4208773622452,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1624.2325420025736,
      "totalMs": 1641.7413750023115,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 17.713541019475088,
      "inferenceMs": 17.713541019475088,
      "charactersPerSecond": 2483.9753921378206,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1624.2325420025736,
      "totalMs": 1641.9460830220487,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 16.828499996336177,
      "inferenceMs": 16.828499996336177,
      "charactersPerSecond": 1663.8440744033057,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1624.2325420025736,
      "totalMs": 1641.0610419989098,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 17.58424998843111,
      "inferenceMs": 17.58424998843111,
      "charactersPerSecond": 2445.3701481888743,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1624.2325420025736,
      "totalMs": 1641.8167919910047,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.5,
      "recall": 1,
      "f1": 0.6666666666666666,
      "expectedSpans": 4,
      "actualSpans": 8,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 64.68320899875835,
      "inferenceMs": 64.68320899875835,
      "charactersPerSecond": 3030.1526939358005,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093599232,
      "loadMs": 1904.5657920069061,
      "totalMs": 1969.2490010056645,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 25.65645900904201,
      "inferenceMs": 25.65645900904201,
      "charactersPerSecond": 1598.0381386827592,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093648384,
      "loadMs": 1904.5657920069061,
      "totalMs": 1930.2222510159481,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 22.217916004592553,
      "inferenceMs": 22.217916004592553,
      "charactersPerSecond": 1350.2616534241488,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093664768,
      "loadMs": 1904.5657920069061,
      "totalMs": 1926.7837080114987,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.770208998816088,
      "inferenceMs": 9.770208998816088,
      "charactersPerSecond": 4503.486056985243,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093664768,
      "loadMs": 1904.5657920069061,
      "totalMs": 1914.3360010057222,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 8.942707994719967,
      "inferenceMs": 8.942707994719967,
      "charactersPerSecond": 3131.042634572437,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093664768,
      "loadMs": 1904.5657920069061,
      "totalMs": 1913.508500001626,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 11.300375015707687,
      "inferenceMs": 11.300375015707687,
      "charactersPerSecond": 3805.1834510119684,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093664768,
      "loadMs": 1904.5657920069061,
      "totalMs": 1915.8661670226138,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.5,
      "recall": 1,
      "f1": 0.6666666666666666,
      "expectedSpans": 4,
      "actualSpans": 8,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 62.68800000543706,
      "inferenceMs": 62.68800000543706,
      "charactersPerSecond": 3126.5952013623105,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093599232,
      "loadMs": 1909.015665994957,
      "totalMs": 1971.7036660003942,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 26.972624997142702,
      "inferenceMs": 26.972624997142702,
      "charactersPerSecond": 1520.059690309833,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093648384,
      "loadMs": 1909.015665994957,
      "totalMs": 1935.9882909920998,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 21.57616597833112,
      "inferenceMs": 21.57616597833112,
      "charactersPerSecond": 1390.4231192014797,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093664768,
      "loadMs": 1909.015665994957,
      "totalMs": 1930.5918319732882,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 13.741000002482906,
      "inferenceMs": 13.741000002482906,
      "charactersPerSecond": 3202.0959167491083,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093664768,
      "loadMs": 1909.015665994957,
      "totalMs": 1922.75666599744,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 15.185666998149827,
      "inferenceMs": 15.185666998149827,
      "charactersPerSecond": 1843.8439354301283,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093664768,
      "loadMs": 1909.015665994957,
      "totalMs": 1924.201332993107,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 17.308875016169623,
      "inferenceMs": 17.308875016169623,
      "charactersPerSecond": 2484.2746833534943,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093664768,
      "loadMs": 1909.015665994957,
      "totalMs": 1926.3245410111267,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.5,
      "recall": 1,
      "f1": 0.6666666666666666,
      "expectedSpans": 4,
      "actualSpans": 8,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 66.6626250022091,
      "inferenceMs": 66.6626250022091,
      "charactersPerSecond": 2940.1782482088706,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093599232,
      "loadMs": 1911.9291660026647,
      "totalMs": 1978.5917910048738,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 31.12012500059791,
      "inferenceMs": 31.12012500059791,
      "charactersPerSecond": 1317.4754278529495,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093648384,
      "loadMs": 1911.9291660026647,
      "totalMs": 1943.0492910032626,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 24.17487499769777,
      "inferenceMs": 24.17487499769777,
      "charactersPerSecond": 1240.9578127232082,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093664768,
      "loadMs": 1911.9291660026647,
      "totalMs": 1936.1040410003625,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 11.622208985500038,
      "inferenceMs": 11.622208985500038,
      "charactersPerSecond": 3785.8551721875556,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093664768,
      "loadMs": 1911.9291660026647,
      "totalMs": 1923.5513749881648,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.638457995606586,
      "inferenceMs": 9.638457995606586,
      "charactersPerSecond": 2905.029000776161,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093664768,
      "loadMs": 1911.9291660026647,
      "totalMs": 1921.5676239982713,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 10.433083982206881,
      "inferenceMs": 10.433083982206881,
      "charactersPerSecond": 4121.504252561795,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093664768,
      "loadMs": 1911.9291660026647,
      "totalMs": 1922.3622499848716,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.5,
      "recall": 1,
      "f1": 0.6666666666666666,
      "expectedSpans": 4,
      "actualSpans": 8,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 71.84720900841057,
      "inferenceMs": 71.84720900841057,
      "charactersPerSecond": 2728.011327163117,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093599232,
      "loadMs": 1931.1824580072425,
      "totalMs": 2003.029667015653,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 31.280416005756706,
      "inferenceMs": 31.280416005756706,
      "charactersPerSecond": 1310.7242561113812,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093648384,
      "loadMs": 1931.1824580072425,
      "totalMs": 1962.4628740129992,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 18.477291014278308,
      "inferenceMs": 18.477291014278308,
      "charactersPerSecond": 1623.6146292666783,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093664768,
      "loadMs": 1931.1824580072425,
      "totalMs": 1949.6597490215208,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 8.602542016888037,
      "inferenceMs": 8.602542016888037,
      "charactersPerSecond": 5114.767229688809,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093664768,
      "loadMs": 1931.1824580072425,
      "totalMs": 1939.7850000241306,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 7.511000003432855,
      "inferenceMs": 7.511000003432855,
      "charactersPerSecond": 3727.865795127517,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093664768,
      "loadMs": 1931.1824580072425,
      "totalMs": 1938.6934580106754,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 8.330540993483737,
      "inferenceMs": 8.330540993483737,
      "charactersPerSecond": 5161.72959638938,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093664768,
      "loadMs": 1931.1824580072425,
      "totalMs": 1939.5129990007263,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.5,
      "recall": 1,
      "f1": 0.6666666666666666,
      "expectedSpans": 4,
      "actualSpans": 8,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 73.15591699443758,
      "inferenceMs": 73.15591699443758,
      "charactersPerSecond": 2679.209120089396,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093599232,
      "loadMs": 2559.489957988262,
      "totalMs": 2632.6458749826998,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 24.987250013509765,
      "inferenceMs": 24.987250013509765,
      "charactersPerSecond": 1640.8368258945134,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093648384,
      "loadMs": 2559.489957988262,
      "totalMs": 2584.477208001772,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 19.188541977200657,
      "inferenceMs": 19.188541977200657,
      "charactersPerSecond": 1563.433013078599,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093664768,
      "loadMs": 2559.489957988262,
      "totalMs": 2578.678499965463,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.67349999700673,
      "inferenceMs": 9.67349999700673,
      "charactersPerSecond": 4548.50881414327,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093664768,
      "loadMs": 2559.489957988262,
      "totalMs": 2569.163457985269,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 7.777249993523583,
      "inferenceMs": 7.777249993523583,
      "charactersPerSecond": 3600.244305290004,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093664768,
      "loadMs": 2559.489957988262,
      "totalMs": 2567.2672079817858,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.300082980189472,
      "inferenceMs": 9.300082980189472,
      "charactersPerSecond": 4623.614659309626,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093664768,
      "loadMs": 2559.489957988262,
      "totalMs": 2568.7900409684516,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.5,
      "recall": 1,
      "f1": 0.6666666666666666,
      "expectedSpans": 4,
      "actualSpans": 8,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 63.124499982222915,
      "inferenceMs": 63.124499982222915,
      "charactersPerSecond": 3104.9750897860167,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093599232,
      "loadMs": 1875.1310000079684,
      "totalMs": 1938.2554999901913,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 24.292082991451025,
      "inferenceMs": 24.292082991451025,
      "charactersPerSecond": 1687.7926859721704,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093648384,
      "loadMs": 1875.1310000079684,
      "totalMs": 1899.4230829994194,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 20.357333996798843,
      "inferenceMs": 20.357333996798843,
      "charactersPerSecond": 1473.6703737688565,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093664768,
      "loadMs": 1875.1310000079684,
      "totalMs": 1895.4883340047672,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.647041995776817,
      "inferenceMs": 9.647041995776817,
      "charactersPerSecond": 4560.983565663119,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093664768,
      "loadMs": 1875.1310000079684,
      "totalMs": 1884.7780420037452,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 7.6264580129645765,
      "inferenceMs": 7.6264580129645765,
      "charactersPerSecond": 3671.429115901704,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093664768,
      "loadMs": 1875.1310000079684,
      "totalMs": 1882.757458020933,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 8.402166015002877,
      "inferenceMs": 8.402166015002877,
      "charactersPerSecond": 5117.727967195526,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093664768,
      "loadMs": 1875.1310000079684,
      "totalMs": 1883.5331660229713,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "d1a3e8f13f8c3566299d95fcfc9a8d2382a9affc",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  }
]

BERT Base Uncased NER Desktop

72 configurations tested
Passed72
Failed0
TestedNov 12, 2026
Runtimetransformers 5.22.1

Quality criteria

{ "file": "research/bert-ner-family-expansion/quality-gates.json", "sha256": "4362bea8f9d1c5d1975a56c1ded21d9244114c3e347968ebe9ac4e72e2b705e3", "description": "Six frozen NER cases: selected-span recall for non-exhaustive NASA reference; exact-span recall/F1 for five complete references, including valid UTF-16 offsets and no-entity negative; <=1000ms per text." }

PrecisionDeviceResultPlatformInference / load
FP16mpsAcceleratedPassedVery fastmacos · aarch648 ms / 1863 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch648 ms / 1863 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch648 ms / 1916 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch648 ms / 1973 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch649 ms / 1851 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch649 ms / 1914 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch649 ms / 1863 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch649 ms / 1881 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch649 ms / 1916 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6410 ms / 1914 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6410 ms / 1914 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6410 ms / 1851 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6410 ms / 1851 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6411 ms / 1881 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6411 ms / 1973 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6411 ms / 1973 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6412 ms / 1881 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6414 ms / 1916 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6420 ms / 1863 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6420 ms / 1881 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6421 ms / 1973 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6422 ms / 1916 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6422 ms / 1851 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6422 ms / 1914 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6424 ms / 1914 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6425 ms / 1851 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6425 ms / 1881 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6425 ms / 1863 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6428 ms / 1973 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6429 ms / 1916 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6463 ms / 1863 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6463 ms / 1973 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6463 ms / 1881 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6464 ms / 1914 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6470 ms / 1916 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6472 ms / 1851 ms
FP32cpuPassedVery fastmacos · aarch6417 ms / 1632 ms
FP32cpuPassedVery fastmacos · aarch6417 ms / 1636 ms
FP32cpuPassedVery fastmacos · aarch6417 ms / 1632 ms
FP32cpuPassedVery fastmacos · aarch6417 ms / 1636 ms
FP32cpuPassedVery fastmacos · aarch6417 ms / 1635 ms
FP32cpuPassedVery fastmacos · aarch6417 ms / 1631 ms
FP32cpuPassedVery fastmacos · aarch6418 ms / 1632 ms
FP32cpuPassedVery fastmacos · aarch6418 ms / 1632 ms
FP32cpuPassedVery fastmacos · aarch6418 ms / 1636 ms
FP32cpuPassedVery fastmacos · aarch6418 ms / 1636 ms
FP32cpuPassedVery fastmacos · aarch6418 ms / 1631 ms
FP32cpuPassedVery fastmacos · aarch6418 ms / 1631 ms
FP32cpuPassedVery fastmacos · aarch6418 ms / 1636 ms
FP32cpuPassedVery fastmacos · aarch6418 ms / 1631 ms
FP32cpuPassedVery fastmacos · aarch6418 ms / 1635 ms
FP32cpuPassedVery fastmacos · aarch6418 ms / 1734 ms
FP32cpuPassedVery fastmacos · aarch6419 ms / 1636 ms
FP32cpuPassedVery fastmacos · aarch6419 ms / 1631 ms
FP32cpuPassedVery fastmacos · aarch6419 ms / 1632 ms
FP32cpuPassedVery fastmacos · aarch6419 ms / 1636 ms
FP32cpuPassedVery fastmacos · aarch6419 ms / 1734 ms
FP32cpuPassedVery fastmacos · aarch6419 ms / 1636 ms
FP32cpuPassedVery fastmacos · aarch6419 ms / 1635 ms
FP32cpuPassedVery fastmacos · aarch6419 ms / 1734 ms
FP32cpuPassedVery fastmacos · aarch6420 ms / 1636 ms
FP32cpuPassedVery fastmacos · aarch6420 ms / 1636 ms
FP32cpuPassedVery fastmacos · aarch6420 ms / 1635 ms
FP32cpuPassedVery fastmacos · aarch6420 ms / 1734 ms
FP32cpuPassedVery fastmacos · aarch6421 ms / 1734 ms
FP32cpuPassedVery fastmacos · aarch6422 ms / 1635 ms
FP32cpuPassedFastmacos · aarch64341 ms / 1636 ms
FP32cpuPassedFastmacos · aarch64342 ms / 1631 ms
FP32cpuPassedFastmacos · aarch64342 ms / 1636 ms
FP32cpuPassedFastmacos · aarch64342 ms / 1632 ms
FP32cpuPassedFastmacos · aarch64342 ms / 1635 ms
FP32cpuPassedFastmacos · aarch64347 ms / 1734 ms
Test setup & sample details
Quality gate
{ "file": "research/bert-ner-family-expansion/quality-gates.json", "sha256": "4362bea8f9d1c5d1975a56c1ded21d9244114c3e347968ebe9ac4e72e2b705e3", "description": "Six frozen NER cases: selected-span recall for non-exhaustive NASA reference; exact-span recall/F1 for five complete references, including valid UTF-16 offsets and no-entity negative; <=1000ms per text." }
Tested
2026-11-12T22:40:54.760961+00:00
Environment
{ "os": "macos", "osVersion": "27.2", "architecture": "aarch64", "hardware": "Apple M3 Max;Mac15,11;96GiB RAM", "cpu": "Apple M3 Max", "gpu": "Apple M3 Max", "driver": "OS-managed Metal/MPS macOS27.2 build26B118", "runtime": "App-owned CPython3.13.18;Torch2.10.0;Transformers5.22.1", "timingScope": "Fresh helper/model process per cold repetition; runtime/tokenizer/model startup in load. OS files/driver cache may be warm. Five warm texts follow six fixtures. Snapshot loads, not Internet transfer.", "concurrentUploadActivity": "none; uploads excluded" }
Runtime
transformers 5.22.1
Artifact revision
301540b48433e31000058882c971ddd7bc726547
Fixture
{
  "file": "model-samples/native/bert-ner/quality-gates.json",
  "url": "https://assets.localai.tools/samples/v1/files/4362bea8f9d1c5d1975a56c1ded21d9244114c3e347968ebe9ac4e72e2b705e3/bert-ner-quality-gates.json",
  "size": 4105,
  "sha1": "50eaec89241b58910bbfe13b9c7c1d4a2dac1816",
  "sha256": "4362bea8f9d1c5d1975a56c1ded21d9244114c3e347968ebe9ac4e72e2b705e3"
}
Rerun when
artifact-revision; runtime-version; fixture-checksum; quality-gate; backend-behavior
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.6666666666666666,
      "recall": 1,
      "f1": 0.8,
      "expectedSpans": 4,
      "actualSpans": 6,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 342.4395000038203,
      "inferenceMs": 342.4395000038203,
      "charactersPerSecond": 572.3638774084573,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1634.8025419865735,
      "totalMs": 1977.2420419903938,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 20.126791001530364,
      "inferenceMs": 20.126791001530364,
      "charactersPerSecond": 2037.0857926076003,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1634.8025419865735,
      "totalMs": 1654.9293329881039,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 19.463665987132117,
      "inferenceMs": 19.463665987132117,
      "charactersPerSecond": 1541.3334784841509,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1634.8025419865735,
      "totalMs": 1654.2662079737056,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 22.291708999546245,
      "inferenceMs": 22.291708999546245,
      "charactersPerSecond": 1973.8280273125597,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1634.8025419865735,
      "totalMs": 1657.0942509861197,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 18.42937502078712,
      "inferenceMs": 18.42937502078712,
      "charactersPerSecond": 1519.313594108202,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1634.8025419865735,
      "totalMs": 1653.2319170073606,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 17.104041995480657,
      "inferenceMs": 17.104041995480657,
      "charactersPerSecond": 2514.025632734166,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1634.8025419865735,
      "totalMs": 1651.9065839820541,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.6666666666666666,
      "recall": 1,
      "f1": 0.8,
      "expectedSpans": 4,
      "actualSpans": 6,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 347.41437502088957,
      "inferenceMs": 347.41437502088957,
      "charactersPerSecond": 564.167789511343,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1734.2115829815157,
      "totalMs": 2081.6259580024052,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 19.4819160096813,
      "inferenceMs": 19.4819160096813,
      "charactersPerSecond": 2104.5157970923165,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1734.2115829815157,
      "totalMs": 1753.693498991197,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 19.00374999968335,
      "inferenceMs": 19.00374999968335,
      "charactersPerSecond": 1578.6357955929684,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1734.2115829815157,
      "totalMs": 1753.215332981199,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 20.399958011694252,
      "inferenceMs": 20.399958011694252,
      "charactersPerSecond": 2156.8671844705295,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1734.2115829815157,
      "totalMs": 1754.61154099321,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 20.83541700267233,
      "inferenceMs": 20.83541700267233,
      "charactersPerSecond": 1343.8655917665938,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1734.2115829815157,
      "totalMs": 1755.046999984188,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 18.49795802263543,
      "inferenceMs": 18.49795802263543,
      "charactersPerSecond": 2324.580904950812,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1734.2115829815157,
      "totalMs": 1752.709541004151,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.6666666666666666,
      "recall": 1,
      "f1": 0.8,
      "expectedSpans": 4,
      "actualSpans": 6,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 341.68783397763036,
      "inferenceMs": 341.68783397763036,
      "charactersPerSecond": 573.6229988593382,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1631.2773750105407,
      "totalMs": 1972.965208988171,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 18.383625021670014,
      "inferenceMs": 18.383625021670014,
      "charactersPerSecond": 2230.2456643709033,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1631.2773750105407,
      "totalMs": 1649.6610000322107,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 17.317374993581325,
      "inferenceMs": 17.317374993581325,
      "charactersPerSecond": 1732.364172463753,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1631.2773750105407,
      "totalMs": 1648.594750004122,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 18.115832994226366,
      "inferenceMs": 18.115832994226366,
      "charactersPerSecond": 2428.8146183519734,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1631.2773750105407,
      "totalMs": 1649.393208004767,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 17.934500006958842,
      "inferenceMs": 17.934500006958842,
      "charactersPerSecond": 1561.2367219122696,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1631.2773750105407,
      "totalMs": 1649.2118750174996,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 18.787792010698467,
      "inferenceMs": 18.787792010698467,
      "charactersPerSecond": 2288.7202485270327,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1631.2773750105407,
      "totalMs": 1650.0651670212392,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.6666666666666666,
      "recall": 1,
      "f1": 0.8,
      "expectedSpans": 4,
      "actualSpans": 6,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 342.10483299102634,
      "inferenceMs": 342.10483299102634,
      "charactersPerSecond": 572.9237973236736,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1632.2316250007134,
      "totalMs": 1974.3364579917397,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 18.79999999073334,
      "inferenceMs": 18.79999999073334,
      "charactersPerSecond": 2180.851064904745,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1632.2316250007134,
      "totalMs": 1651.0316249914467,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 17.70170798408799,
      "inferenceMs": 17.70170798408799,
      "charactersPerSecond": 1694.7517170075853,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1632.2316250007134,
      "totalMs": 1649.9333329848014,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 17.61199999600649,
      "inferenceMs": 17.61199999600649,
      "charactersPerSecond": 2498.296616510162,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1632.2316250007134,
      "totalMs": 1649.8436249967199,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 16.930207988480106,
      "inferenceMs": 16.930207988480106,
      "charactersPerSecond": 1653.8485539605988,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1632.2316250007134,
      "totalMs": 1649.1618329891935,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 17.045582993887365,
      "inferenceMs": 17.045582993887365,
      "charactersPerSecond": 2522.647656898566,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1632.2316250007134,
      "totalMs": 1649.2772079946008,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.6666666666666666,
      "recall": 1,
      "f1": 0.8,
      "expectedSpans": 4,
      "actualSpans": 6,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 341.0425829933956,
      "inferenceMs": 341.0425829933956,
      "charactersPerSecond": 574.7082909109787,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1635.539750015596,
      "totalMs": 1976.5823330089916,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 18.88708400656469,
      "inferenceMs": 18.88708400656469,
      "charactersPerSecond": 2170.795660449723,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1635.539750015596,
      "totalMs": 1654.4268340221606,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 17.79458299279213,
      "inferenceMs": 17.79458299279213,
      "charactersPerSecond": 1685.9063239724019,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1635.539750015596,
      "totalMs": 1653.334333008388,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 17.78020799974911,
      "inferenceMs": 17.78020799974911,
      "charactersPerSecond": 2474.66171377865,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1635.539750015596,
      "totalMs": 1653.319958015345,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 16.998292005155236,
      "inferenceMs": 16.998292005155236,
      "charactersPerSecond": 1647.2243206263413,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1635.539750015596,
      "totalMs": 1652.5380420207512,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 17.062166996765882,
      "inferenceMs": 17.062166996765882,
      "charactersPerSecond": 2520.1957059821657,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1635.539750015596,
      "totalMs": 1652.6019170123618,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.6666666666666666,
      "recall": 1,
      "f1": 0.8,
      "expectedSpans": 4,
      "actualSpans": 6,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 341.8487499875482,
      "inferenceMs": 341.8487499875482,
      "charactersPerSecond": 573.3529814198217,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1636.2530410115141,
      "totalMs": 1978.1017909990624,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 19.668332999572158,
      "inferenceMs": 19.668332999572158,
      "charactersPerSecond": 2084.5691396872253,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1636.2530410115141,
      "totalMs": 1655.9213740110863,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 19.535291008651257,
      "inferenceMs": 19.535291008651257,
      "charactersPerSecond": 1535.6822678870983,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1636.2530410115141,
      "totalMs": 1655.7883320201654,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 19.092291011475027,
      "inferenceMs": 19.092291011475027,
      "charactersPerSecond": 2304.5950836154084,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1636.2530410115141,
      "totalMs": 1655.3453320229892,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 18.786541011650115,
      "inferenceMs": 18.786541011650115,
      "charactersPerSecond": 1490.4287054565464,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1636.2530410115141,
      "totalMs": 1655.0395820231643,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 18.16916701500304,
      "inferenceMs": 18.16916701500304,
      "charactersPerSecond": 2366.6467463529343,
      "placement": {
        "cpu/torch.float32": 199
      },
      "loadMs": 1636.2530410115141,
      "totalMs": 1654.4222080265172,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.6666666666666666,
      "recall": 1,
      "f1": 0.8,
      "expectedSpans": 4,
      "actualSpans": 6,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 62.7345830143895,
      "inferenceMs": 62.7345830143895,
      "charactersPerSecond": 3124.273575789023,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093566464,
      "loadMs": 1973.4319580020383,
      "totalMs": 2036.1665410164278,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 28.451833000872284,
      "inferenceMs": 28.451833000872284,
      "charactersPerSecond": 1441.0319362813289,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093615616,
      "loadMs": 1973.4319580020383,
      "totalMs": 2001.8837910029106,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 21.106875006807968,
      "inferenceMs": 21.106875006807968,
      "charactersPerSecond": 1421.3378337780262,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093632000,
      "loadMs": 1973.4319580020383,
      "totalMs": 1994.5388330088463,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 10.601582995150238,
      "inferenceMs": 10.601582995150238,
      "charactersPerSecond": 4150.323590366465,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093632000,
      "loadMs": 1973.4319580020383,
      "totalMs": 1984.0335409971885,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 8.488166989991441,
      "inferenceMs": 8.488166989991441,
      "charactersPerSecond": 3298.7098431281256,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093632000,
      "loadMs": 1973.4319580020383,
      "totalMs": 1981.9201249920297,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 10.567499994067475,
      "inferenceMs": 10.567499994067475,
      "charactersPerSecond": 4069.0797278580476,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093632000,
      "loadMs": 1973.4319580020383,
      "totalMs": 1983.9994579961058,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.6666666666666666,
      "recall": 1,
      "f1": 0.8,
      "expectedSpans": 4,
      "actualSpans": 6,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 63.15129200811498,
      "inferenceMs": 63.15129200811498,
      "charactersPerSecond": 3103.6577996664564,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093566464,
      "loadMs": 1881.06220800546,
      "totalMs": 1944.213500013575,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 25.280874979216605,
      "inferenceMs": 25.280874979216605,
      "charactersPerSecond": 1621.7793107915006,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093615616,
      "loadMs": 1881.06220800546,
      "totalMs": 1906.3430829846766,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 20.40816700900905,
      "inferenceMs": 20.40816700900905,
      "charactersPerSecond": 1469.9997303411278,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093632000,
      "loadMs": 1881.06220800546,
      "totalMs": 1901.470375014469,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 10.54954098071903,
      "inferenceMs": 10.54954098071903,
      "charactersPerSecond": 4170.797580711524,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093632000,
      "loadMs": 1881.06220800546,
      "totalMs": 1891.611748986179,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.151874983217567,
      "inferenceMs": 9.151874983217567,
      "charactersPerSecond": 3059.482352124079,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093632000,
      "loadMs": 1881.06220800546,
      "totalMs": 1890.2140829886775,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 11.531708005350083,
      "inferenceMs": 11.531708005350083,
      "charactersPerSecond": 3728.8491852248035,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093632000,
      "loadMs": 1881.06220800546,
      "totalMs": 1892.59391601081,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.6666666666666666,
      "recall": 1,
      "f1": 0.8,
      "expectedSpans": 4,
      "actualSpans": 6,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 71.88345899339765,
      "inferenceMs": 71.88345899339765,
      "charactersPerSecond": 2726.6356230576243,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093566464,
      "loadMs": 1850.8375409874134,
      "totalMs": 1922.720999980811,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 25.170542008709162,
      "inferenceMs": 25.170542008709162,
      "charactersPerSecond": 1628.8882450689282,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093615616,
      "loadMs": 1850.8375409874134,
      "totalMs": 1876.0080829961225,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 21.807750017615035,
      "inferenceMs": 21.807750017615035,
      "charactersPerSecond": 1375.6577352440183,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093632000,
      "loadMs": 1850.8375409874134,
      "totalMs": 1872.6452910050284,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 10.022375005064532,
      "inferenceMs": 10.022375005064532,
      "charactersPerSecond": 4390.176976791011,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093632000,
      "loadMs": 1850.8375409874134,
      "totalMs": 1860.859915992478,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 8.659958984935656,
      "inferenceMs": 8.659958984935656,
      "charactersPerSecond": 3233.271664300849,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093632000,
      "loadMs": 1850.8375409874134,
      "totalMs": 1859.497499972349,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 10.162374994251877,
      "inferenceMs": 10.162374994251877,
      "charactersPerSecond": 4231.294360257519,
      "placement": {
        "mps:0/torch.float32": 199
      },
      "driverAllocatedMemoryBytes": 1093632000,
      "loadMs": 1850.8375409874134,
      "totalMs": 1860.9999159816653,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.6666666666666666,
      "recall": 1,
      "f1": 0.8,
      "expectedSpans": 4,
      "actualSpans": 6,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 62.66983397654258,
      "inferenceMs": 62.66983397654258,
      "charactersPerSecond": 3127.501503727665,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093566464,
      "loadMs": 1862.922250002157,
      "totalMs": 1925.5920839786995,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 25.403125007869676,
      "inferenceMs": 25.403125007869676,
      "charactersPerSecond": 1613.974658129602,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093615616,
      "loadMs": 1862.922250002157,
      "totalMs": 1888.3253750100266,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 20.405666989972815,
      "inferenceMs": 20.405666989972815,
      "charactersPerSecond": 1470.1798287084546,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093632000,
      "loadMs": 1862.922250002157,
      "totalMs": 1883.3279169921298,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.107624995522201,
      "inferenceMs": 9.107624995522201,
      "charactersPerSecond": 4831.116786388632,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093632000,
      "loadMs": 1862.922250002157,
      "totalMs": 1872.0298749976791,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 7.836624979972839,
      "inferenceMs": 7.836624979972839,
      "charactersPerSecond": 3572.9666880265904,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093632000,
      "loadMs": 1862.922250002157,
      "totalMs": 1870.7588749821298,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 8.301541995024309,
      "inferenceMs": 8.301541995024309,
      "charactersPerSecond": 5179.760582524656,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093632000,
      "loadMs": 1862.922250002157,
      "totalMs": 1871.2237919971813,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.6666666666666666,
      "recall": 1,
      "f1": 0.8,
      "expectedSpans": 4,
      "actualSpans": 6,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 70.09320799261332,
      "inferenceMs": 70.09320799261332,
      "charactersPerSecond": 2796.2766381110023,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093566464,
      "loadMs": 1915.7919579884037,
      "totalMs": 1985.885165981017,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 29.141042003175244,
      "inferenceMs": 29.141042003175244,
      "charactersPerSecond": 1406.9503758833534,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093615616,
      "loadMs": 1915.7919579884037,
      "totalMs": 1944.932999991579,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 21.509875019546598,
      "inferenceMs": 21.509875019546598,
      "charactersPerSecond": 1394.7082432017014,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093632000,
      "loadMs": 1915.7919579884037,
      "totalMs": 1937.3018330079503,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 14.300207985797897,
      "inferenceMs": 14.300207985797897,
      "charactersPerSecond": 3076.8783253850675,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093632000,
      "loadMs": 1915.7919579884037,
      "totalMs": 1930.0921659742016,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 8.392415998969227,
      "inferenceMs": 8.392415998969227,
      "charactersPerSecond": 3336.345577178135,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093632000,
      "loadMs": 1915.7919579884037,
      "totalMs": 1924.184373987373,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.18154200189747,
      "inferenceMs": 9.18154200189747,
      "charactersPerSecond": 4683.30918609462,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093632000,
      "loadMs": 1915.7919579884037,
      "totalMs": 1924.9734999903012,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.6666666666666666,
      "recall": 1,
      "f1": 0.8,
      "expectedSpans": 4,
      "actualSpans": 6,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 64.02608400094323,
      "inferenceMs": 64.02608400094323,
      "charactersPerSecond": 3061.2523482946817,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093566464,
      "loadMs": 1913.961709011346,
      "totalMs": 1977.9877930122893,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 23.988291999557987,
      "inferenceMs": 23.988291999557987,
      "charactersPerSecond": 1709.1671220591893,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093615616,
      "loadMs": 1913.961709011346,
      "totalMs": 1937.950001010904,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 21.961000020382926,
      "inferenceMs": 21.961000020382926,
      "charactersPerSecond": 1366.0580106623443,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093632000,
      "loadMs": 1913.961709011346,
      "totalMs": 1935.922709031729,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.934500005329028,
      "inferenceMs": 9.934500005329028,
      "charactersPerSecond": 4429.0100132264015,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093632000,
      "loadMs": 1913.961709011346,
      "totalMs": 1923.8962090166751,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 8.689207985298708,
      "inferenceMs": 8.689207985298708,
      "charactersPerSecond": 3222.3880527860842,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093632000,
      "loadMs": 1913.961709011346,
      "totalMs": 1922.6509169966448,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.785290982108563,
      "inferenceMs": 9.785290982108563,
      "charactersPerSecond": 4394.350671699109,
      "placement": {
        "mps:0/torch.float16": 199
      },
      "driverAllocatedMemoryBytes": 1093632000,
      "loadMs": 1913.961709011346,
      "totalMs": 1923.7469999934547,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "301540b48433e31000058882c971ddd7bc726547",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  }
]

BERT Large Cased NER Desktop

48 configurations tested
Passed36
Failed12
TestedNov 12, 2026
Runtimetransformers 5.22.1

Quality criteria

{ "file": "research/bert-ner-family-expansion/quality-gates.json", "sha256": "4362bea8f9d1c5d1975a56c1ded21d9244114c3e347968ebe9ac4e72e2b705e3", "description": "Six frozen NER cases: selected-span recall for non-exhaustive NASA reference; exact-span recall/F1 for five complete references, including valid UTF-16 offsets and no-entity negative; <=1000ms per text." }

PrecisionDeviceResultPlatformInference / load
FP16mpsAcceleratedPassedVery fastmacos · aarch6413 ms / 2332 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6414 ms / 2354 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6414 ms / 2397 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6415 ms / 2332 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6416 ms / 2354 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6417 ms / 2397 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6417 ms / 2374 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6417 ms / 2397 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6417 ms / 2354 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6418 ms / 2332 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6419 ms / 2397 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6422 ms / 2397 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6422 ms / 2374 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6423 ms / 2374 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6427 ms / 2397 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6428 ms / 2475 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6429 ms / 2374 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6432 ms / 2397 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6433 ms / 2475 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6433 ms / 2475 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6437 ms / 2397 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6437 ms / 2332 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6438 ms / 2374 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6442 ms / 2397 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6443 ms / 2397 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6443 ms / 2332 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6472 ms / 2374 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6473 ms / 2397 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6474 ms / 2332 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6475 ms / 2397 ms
FP32mpsAcceleratedPassedFastmacos · aarch64169 ms / 2475 ms
FP16mpsAcceleratedPassedFastmacos · aarch64171 ms / 2354 ms
FP32mpsAcceleratedPassedFastmacos · aarch64171 ms / 2475 ms
FP16mpsAcceleratedPassedFastmacos · aarch64192 ms / 2354 ms
FP32mpsAcceleratedPassedFastmacos · aarch64216 ms / 2475 ms
FP16mpsAcceleratedPassedFastmacos · aarch64230 ms / 2354 ms
FP32cpuFailed—macos · aarch641184 ms / 1650 ms
FP32cpuFailed—macos · aarch6455 ms / 1650 ms
FP32cpuFailed—macos · aarch6454 ms / 1650 ms
FP32cpuFailed—macos · aarch6455 ms / 1650 ms
FP32cpuFailed—macos · aarch6455 ms / 1650 ms
FP32cpuFailed—macos · aarch6457 ms / 1650 ms
FP32cpuFailed—macos · aarch641153 ms / 1665 ms
FP32cpuFailed—macos · aarch6457 ms / 1665 ms
FP32cpuFailed—macos · aarch6454 ms / 1665 ms
FP32cpuFailed—macos · aarch6455 ms / 1665 ms
FP32cpuFailed—macos · aarch6454 ms / 1665 ms
FP32cpuFailed—macos · aarch6455 ms / 1665 ms
Test setup & sample details
Quality gate
{ "file": "research/bert-ner-family-expansion/quality-gates.json", "sha256": "4362bea8f9d1c5d1975a56c1ded21d9244114c3e347968ebe9ac4e72e2b705e3", "description": "Six frozen NER cases: selected-span recall for non-exhaustive NASA reference; exact-span recall/F1 for five complete references, including valid UTF-16 offsets and no-entity negative; <=1000ms per text." }
Tested
2026-11-12T22:40:54.760961+00:00
Environment
{ "os": "macos", "osVersion": "27.2", "architecture": "aarch64", "hardware": "Apple M3 Max;Mac15,11;96GiB RAM", "cpu": "Apple M3 Max", "gpu": "Apple M3 Max", "driver": "OS-managed Metal/MPS macOS27.2 build26B118", "runtime": "App-owned CPython3.13.18;Torch2.10.0;Transformers5.22.1", "timingScope": "Fresh helper/model process per cold repetition; runtime/tokenizer/model startup in load. OS files/driver cache may be warm. Five warm texts follow six fixtures. Snapshot loads, not Internet transfer.", "concurrentUploadActivity": "none; uploads excluded" }
Runtime
transformers 5.22.1
Artifact revision
6fe43d9ec0bba0f67e367ecd74399216fc409c7f
Fixture
{
  "file": "model-samples/native/bert-ner/quality-gates.json",
  "url": "https://assets.localai.tools/samples/v1/files/4362bea8f9d1c5d1975a56c1ded21d9244114c3e347968ebe9ac4e72e2b705e3/bert-ner-quality-gates.json",
  "size": 4105,
  "sha1": "50eaec89241b58910bbfe13b9c7c1d4a2dac1816",
  "sha256": "4362bea8f9d1c5d1975a56c1ded21d9244114c3e347968ebe9ac4e72e2b705e3"
}
Rerun when
artifact-revision; runtime-version; fixture-checksum; quality-gate; backend-behavior
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": false,
      "validOffsetsAndScores": true,
      "precision": 0.5,
      "recall": 1,
      "f1": 0.6666666666666666,
      "expectedSpans": 4,
      "actualSpans": 8,
      "speedPassed": false
    },
    "runtime": {
      "durationMs": 1183.6334579857066,
      "inferenceMs": 1183.6334579857066,
      "charactersPerSecond": 165.59180435263335,
      "placement": {
        "cpu/torch.float32": 391
      },
      "loadMs": 1650.0002079992555,
      "totalMs": 2833.633665984962,
      "grade": "failed",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 55.331667012069374,
      "inferenceMs": 55.331667012069374,
      "charactersPerSecond": 740.9861696568216,
      "placement": {
        "cpu/torch.float32": 391
      },
      "loadMs": 1650.0002079992555,
      "totalMs": 1705.3318750113249,
      "grade": "failed",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 54.47579099563882,
      "inferenceMs": 54.47579099563882,
      "charactersPerSecond": 550.7033390740801,
      "placement": {
        "cpu/torch.float32": 391
      },
      "loadMs": 1650.0002079992555,
      "totalMs": 1704.4759989948943,
      "grade": "failed",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 54.65445900335908,
      "inferenceMs": 54.65445900335908,
      "charactersPerSecond": 805.0578269797849,
      "placement": {
        "cpu/torch.float32": 391
      },
      "loadMs": 1650.0002079992555,
      "totalMs": 1704.6546670026146,
      "grade": "failed",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 54.53295799088664,
      "inferenceMs": 54.53295799088664,
      "charactersPerSecond": 513.4509667471048,
      "placement": {
        "cpu/torch.float32": 391
      },
      "loadMs": 1650.0002079992555,
      "totalMs": 1704.5331659901422,
      "grade": "failed",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 57.404541003052145,
      "inferenceMs": 57.404541003052145,
      "charactersPerSecond": 749.0696597977106,
      "placement": {
        "cpu/torch.float32": 391
      },
      "loadMs": 1650.0002079992555,
      "totalMs": 1707.4047490023077,
      "grade": "failed",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": false,
      "validOffsetsAndScores": true,
      "precision": 0.5,
      "recall": 1,
      "f1": 0.6666666666666666,
      "expectedSpans": 4,
      "actualSpans": 8,
      "speedPassed": false
    },
    "runtime": {
      "durationMs": 1152.9059169988614,
      "inferenceMs": 1152.9059169988614,
      "charactersPerSecond": 170.00519913212796,
      "placement": {
        "cpu/torch.float32": 391
      },
      "loadMs": 1664.7429169970565,
      "totalMs": 2817.648833995918,
      "grade": "failed",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 56.73645800561644,
      "inferenceMs": 56.73645800561644,
      "charactersPerSecond": 722.6394005057795,
      "placement": {
        "cpu/torch.float32": 391
      },
      "loadMs": 1664.7429169970565,
      "totalMs": 1721.479375002673,
      "grade": "failed",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 53.88679099269211,
      "inferenceMs": 53.88679099269211,
      "charactersPerSecond": 556.7227041608113,
      "placement": {
        "cpu/torch.float32": 391
      },
      "loadMs": 1664.7429169970565,
      "totalMs": 1718.6297079897486,
      "grade": "failed",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 55.30670899315737,
      "inferenceMs": 55.30670899315737,
      "charactersPerSecond": 795.563518441203,
      "placement": {
        "cpu/torch.float32": 391
      },
      "loadMs": 1664.7429169970565,
      "totalMs": 1720.0496259902138,
      "grade": "failed",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 53.76837501535192,
      "inferenceMs": 53.76837501535192,
      "charactersPerSecond": 520.7522078174289,
      "placement": {
        "cpu/torch.float32": 391
      },
      "loadMs": 1664.7429169970565,
      "totalMs": 1718.5112920124084,
      "grade": "failed",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 54.95524997240864,
      "inferenceMs": 54.95524997240864,
      "charactersPerSecond": 782.4548159018291,
      "placement": {
        "cpu/torch.float32": 391
      },
      "loadMs": 1664.7429169970565,
      "totalMs": 1719.698166969465,
      "grade": "failed",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.5,
      "recall": 1,
      "f1": 0.6666666666666666,
      "expectedSpans": 4,
      "actualSpans": 8,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 171.35033299564384,
      "inferenceMs": 171.35033299564384,
      "charactersPerSecond": 1143.8553784717933,
      "placement": {
        "mps:0/torch.float32": 391
      },
      "driverAllocatedMemoryBytes": 2167406592,
      "loadMs": 2475.1819579978473,
      "totalMs": 2646.532290993491,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 168.81716600619256,
      "inferenceMs": 168.81716600619256,
      "charactersPerSecond": 242.8662971305657,
      "placement": {
        "mps:0/torch.float32": 391
      },
      "driverAllocatedMemoryBytes": 2167455744,
      "loadMs": 2475.1819579978473,
      "totalMs": 2643.99912400404,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 216.1658749973867,
      "inferenceMs": 216.1658749973867,
      "charactersPerSecond": 138.78231242726298,
      "placement": {
        "mps:0/torch.float32": 391
      },
      "driverAllocatedMemoryBytes": 2167472128,
      "loadMs": 2475.1819579978473,
      "totalMs": 2691.347832995234,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 33.432124997489154,
      "inferenceMs": 33.432124997489154,
      "charactersPerSecond": 1316.0994104713511,
      "placement": {
        "mps:0/torch.float32": 391
      },
      "driverAllocatedMemoryBytes": 2167472128,
      "loadMs": 2475.1819579978473,
      "totalMs": 2508.6140829953365,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 28.224124980624765,
      "inferenceMs": 28.224124980624765,
      "charactersPerSecond": 992.0590990587442,
      "placement": {
        "mps:0/torch.float32": 391
      },
      "driverAllocatedMemoryBytes": 2167472128,
      "loadMs": 2475.1819579978473,
      "totalMs": 2503.406082978472,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 33.00024999771267,
      "inferenceMs": 33.00024999771267,
      "charactersPerSecond": 1303.020431753712,
      "placement": {
        "mps:0/torch.float32": 391
      },
      "driverAllocatedMemoryBytes": 2167472128,
      "loadMs": 2475.1819579978473,
      "totalMs": 2508.18220799556,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.5,
      "recall": 1,
      "f1": 0.6666666666666666,
      "expectedSpans": 4,
      "actualSpans": 8,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 71.61295900004916,
      "inferenceMs": 71.61295900004916,
      "charactersPerSecond": 2736.9348053313292,
      "placement": {
        "mps:0/torch.float32": 391
      },
      "driverAllocatedMemoryBytes": 2167406592,
      "loadMs": 2373.788291006349,
      "totalMs": 2445.401250006398,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 38.14087502541952,
      "inferenceMs": 38.14087502541952,
      "charactersPerSecond": 1074.9622281259929,
      "placement": {
        "mps:0/torch.float32": 391
      },
      "driverAllocatedMemoryBytes": 2167472128,
      "loadMs": 2373.788291006349,
      "totalMs": 2411.9291660317685,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 28.916584007674828,
      "inferenceMs": 28.916584007674828,
      "charactersPerSecond": 1037.4669425696209,
      "placement": {
        "mps:0/torch.float32": 391
      },
      "driverAllocatedMemoryBytes": 2167488512,
      "loadMs": 2373.788291006349,
      "totalMs": 2402.704875014024,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 22.447166993515566,
      "inferenceMs": 22.447166993515566,
      "charactersPerSecond": 1960.1582690907283,
      "placement": {
        "mps:0/torch.float32": 391
      },
      "driverAllocatedMemoryBytes": 2167488512,
      "loadMs": 2373.788291006349,
      "totalMs": 2396.2354579998646,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 16.8270829890389,
      "inferenceMs": 16.8270829890389,
      "charactersPerSecond": 1663.9841865782143,
      "placement": {
        "mps:0/torch.float32": 391
      },
      "driverAllocatedMemoryBytes": 2167488512,
      "loadMs": 2373.788291006349,
      "totalMs": 2390.615373995388,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 22.81712502008304,
      "inferenceMs": 22.81712502008304,
      "charactersPerSecond": 1884.5494321546873,
      "placement": {
        "mps:0/torch.float32": 391
      },
      "driverAllocatedMemoryBytes": 2167488512,
      "loadMs": 2373.788291006349,
      "totalMs": 2396.605416026432,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.5,
      "recall": 1,
      "f1": 0.6666666666666666,
      "expectedSpans": 4,
      "actualSpans": 8,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 74.9443750246428,
      "inferenceMs": 74.9443750246428,
      "charactersPerSecond": 2615.272993277379,
      "placement": {
        "mps:0/torch.float32": 391
      },
      "driverAllocatedMemoryBytes": 2167406592,
      "loadMs": 2396.73199999379,
      "totalMs": 2471.6763750184327,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 41.54837501118891,
      "inferenceMs": 41.54837501118891,
      "charactersPerSecond": 986.8015292766268,
      "placement": {
        "mps:0/torch.float32": 391
      },
      "driverAllocatedMemoryBytes": 2167472128,
      "loadMs": 2396.73199999379,
      "totalMs": 2438.280375004979,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 43.094499997096136,
      "inferenceMs": 43.094499997096136,
      "charactersPerSecond": 696.1445196491782,
      "placement": {
        "mps:0/torch.float32": 391
      },
      "driverAllocatedMemoryBytes": 2167488512,
      "loadMs": 2396.73199999379,
      "totalMs": 2439.826499990886,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 37.0672499993816,
      "inferenceMs": 37.0672499993816,
      "charactersPerSecond": 1187.0316789277342,
      "placement": {
        "mps:0/torch.float32": 391
      },
      "driverAllocatedMemoryBytes": 2167488512,
      "loadMs": 2396.73199999379,
      "totalMs": 2433.7992499931715,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 19.356625009095296,
      "inferenceMs": 19.356625009095296,
      "charactersPerSecond": 1446.533163030402,
      "placement": {
        "mps:0/torch.float32": 391
      },
      "driverAllocatedMemoryBytes": 2167488512,
      "loadMs": 2396.73199999379,
      "totalMs": 2416.0886250028852,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 22.314417001325637,
      "inferenceMs": 22.314417001325637,
      "charactersPerSecond": 1927.0053077096072,
      "placement": {
        "mps:0/torch.float32": 391
      },
      "driverAllocatedMemoryBytes": 2167488512,
      "loadMs": 2396.73199999379,
      "totalMs": 2419.0464169951156,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.5,
      "recall": 1,
      "f1": 0.6666666666666666,
      "expectedSpans": 4,
      "actualSpans": 8,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 171.07170898816548,
      "inferenceMs": 171.07170898816548,
      "charactersPerSecond": 1145.718372484132,
      "placement": {
        "mps:0/torch.float16": 391
      },
      "driverAllocatedMemoryBytes": 1093664768,
      "loadMs": 2354.464125004597,
      "totalMs": 2525.5358339927625,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 192.4099580210168,
      "inferenceMs": 192.4099580210168,
      "charactersPerSecond": 213.08668439874407,
      "placement": {
        "mps:0/torch.float16": 391
      },
      "driverAllocatedMemoryBytes": 1093730304,
      "loadMs": 2354.464125004597,
      "totalMs": 2546.874083025614,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 230.0700829946436,
      "inferenceMs": 230.0700829946436,
      "charactersPerSecond": 130.39505010609506,
      "placement": {
        "mps:0/torch.float16": 391
      },
      "driverAllocatedMemoryBytes": 1093746688,
      "loadMs": 2354.464125004597,
      "totalMs": 2584.5342079992406,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 16.20320900110528,
      "inferenceMs": 16.20320900110528,
      "charactersPerSecond": 2715.511476584583,
      "placement": {
        "mps:0/torch.float16": 391
      },
      "driverAllocatedMemoryBytes": 1093746688,
      "loadMs": 2354.464125004597,
      "totalMs": 2370.6673340057023,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 14.021832990692928,
      "inferenceMs": 14.021832990692928,
      "charactersPerSecond": 1996.885857832222,
      "placement": {
        "mps:0/torch.float16": 391
      },
      "driverAllocatedMemoryBytes": 1093746688,
      "loadMs": 2354.464125004597,
      "totalMs": 2368.48595799529,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 17.096291994675994,
      "inferenceMs": 17.096291994675994,
      "charactersPerSecond": 2515.1652775578914,
      "placement": {
        "mps:0/torch.float16": 391
      },
      "driverAllocatedMemoryBytes": 1093746688,
      "loadMs": 2354.464125004597,
      "totalMs": 2371.560416999273,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.5,
      "recall": 1,
      "f1": 0.6666666666666666,
      "expectedSpans": 4,
      "actualSpans": 8,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 72.65924999956042,
      "inferenceMs": 72.65924999956042,
      "charactersPerSecond": 2697.5230270225165,
      "placement": {
        "mps:0/torch.float16": 391
      },
      "driverAllocatedMemoryBytes": 1093664768,
      "loadMs": 2397.322583012283,
      "totalMs": 2469.9818330118433,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 31.737207988044247,
      "inferenceMs": 31.737207988044247,
      "charactersPerSecond": 1291.859070131347,
      "placement": {
        "mps:0/torch.float16": 391
      },
      "driverAllocatedMemoryBytes": 1093730304,
      "loadMs": 2397.322583012283,
      "totalMs": 2429.059791000327,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 26.725125004304573,
      "inferenceMs": 26.725125004304573,
      "charactersPerSecond": 1122.539183452573,
      "placement": {
        "mps:0/torch.float16": 391
      },
      "driverAllocatedMemoryBytes": 1093746688,
      "loadMs": 2397.322583012283,
      "totalMs": 2424.0477080165874,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 16.88370801275596,
      "inferenceMs": 16.88370801275596,
      "charactersPerSecond": 2606.062599919234,
      "placement": {
        "mps:0/torch.float16": 391
      },
      "driverAllocatedMemoryBytes": 1093746688,
      "loadMs": 2397.322583012283,
      "totalMs": 2414.206291025039,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 14.18720799847506,
      "inferenceMs": 14.18720799847506,
      "charactersPerSecond": 1973.6089019777278,
      "placement": {
        "mps:0/torch.float16": 391
      },
      "driverAllocatedMemoryBytes": 1093746688,
      "loadMs": 2397.322583012283,
      "totalMs": 2411.509791010758,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 16.671250021317974,
      "inferenceMs": 16.671250021317974,
      "charactersPerSecond": 2579.2906917606506,
      "placement": {
        "mps:0/torch.float16": 391
      },
      "driverAllocatedMemoryBytes": 1093746688,
      "loadMs": 2397.322583012283,
      "totalMs": 2413.993833033601,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 0.5,
      "recall": 1,
      "f1": 0.6666666666666666,
      "expectedSpans": 4,
      "actualSpans": 8,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 74.18345799669623,
      "inferenceMs": 74.18345799669623,
      "charactersPerSecond": 2642.0984582402302,
      "placement": {
        "mps:0/torch.float16": 391
      },
      "driverAllocatedMemoryBytes": 1093664768,
      "loadMs": 2331.688500009477,
      "totalMs": 2405.8719580061734,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 43.247791996691376,
      "inferenceMs": 43.247791996691376,
      "charactersPerSecond": 948.0252772936167,
      "placement": {
        "mps:0/torch.float16": 391
      },
      "driverAllocatedMemoryBytes": 1093730304,
      "loadMs": 2331.688500009477,
      "totalMs": 2374.9362920061685,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 37.195708020590246,
      "inferenceMs": 37.195708020590246,
      "charactersPerSecond": 806.5446686320112,
      "placement": {
        "mps:0/torch.float16": 391
      },
      "driverAllocatedMemoryBytes": 1093746688,
      "loadMs": 2331.688500009477,
      "totalMs": 2368.8842080300674,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 0,
      "actualSpans": 0,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 18.45416601281613,
      "inferenceMs": 18.45416601281613,
      "charactersPerSecond": 2384.285476214026,
      "placement": {
        "mps:0/torch.float16": 391
      },
      "driverAllocatedMemoryBytes": 1093746688,
      "loadMs": 2331.688500009477,
      "totalMs": 2350.1426660222933,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 2,
      "actualSpans": 2,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 12.643750000279397,
      "inferenceMs": 12.643750000279397,
      "charactersPerSecond": 2214.5328719233826,
      "placement": {
        "mps:0/torch.float16": 391
      },
      "driverAllocatedMemoryBytes": 1093746688,
      "loadMs": 2331.688500009477,
      "totalMs": 2344.3322500097565,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "validOffsetsAndScores": true,
      "precision": 1,
      "recall": 1,
      "f1": 1,
      "expectedSpans": 4,
      "actualSpans": 4,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 15.244000009261072,
      "inferenceMs": 15.244000009261072,
      "charactersPerSecond": 2820.7819452818508,
      "placement": {
        "mps:0/torch.float16": 391
      },
      "driverAllocatedMemoryBytes": 1093746688,
      "loadMs": 2331.688500009477,
      "totalMs": 2346.932500018738,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "6fe43d9ec0bba0f67e367ecd74399216fc409c7f",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  }
]
Release validation still required
  • Sample publication, complete dependency backup and forced model/runtime-source recovery pending under user upload exclusion.

Something not quite right?

Tell us what happened with BERT NER Desktop. Loading, results, or the page itself—we’re here for all of it.

Attachments Optional

Up to 3 files · 50 MB each

Browser, OS, CPU/GPU, memory, screen, storage, and network details are included automatically. Model inputs, results, and browsing history are not collected.