A model is still running

A model is still running or downloading. Leaving will stop the current operation. Stay on this page to let it finish.

Skip to content
LocalAI.toolsBeta

Join the waitlist

Get an email when the new apps and model runs launch.

No account needed.

HY-MT 1.5 Desktop

Translate short multilingual text locally with the original HY-MT1.5 1.8B checkpoint and official source-aware translation prompt.

Desktop inferencePrivate · Runs on your computer
About this model

Translate short multilingual text locally with the original HY-MT1.5 1.8B checkpoint and official source-aware translation prompt.

Created by
Tencent Hunyuan
Model released
Dec 30, 2025
Source repository created
Dec 25, 2025
Model weights · BF16
4.1 GB

Best for

  • Draft multilingual text translations for human review.
  • Local short-form text translation.

Keep in mind

  • Translations can omit, invent or mistranslate facts, names, numbers, negation and idioms. Review important outputs.
  • Six short authored fixtures are a compatibility suite, not broad validation across all languages or domains.
  • The adapter limits text to16384 characters and input-plus-output to4096 tokens as a resource policy, below the underlying model context.
  • Language choices are accepted selections, not evidence of measured quality in every pair.
  • Source weights are BF16. CPU FP32 and GPU dtype casts have independent performance/memory needs.
  • App-owned pinned CPython and isolated model dependencies install on demand. Untested hardware/OS/provider combinations are unverified.
  • No dedicated content safety filter; review generated translations for consequential uses.
  • Text-only translation with genuine incremental text output. The official prompt specifies the target language; sourceLanguage is validated metadata while the checkpoint infers the source language.
  • Plain text translation only; terminology/context/format instruction modes are not exposed as dedicated controls.
  • Tencent HY Community license has use and redistribution conditions; it is not Apache-2.0.
  • Publication, dependency closure backup and forced model/runtime-source recovery remain excluded and pending.

Processed on your deviceYour input is processed locally in the desktop app. Model and runtime files download from external hosts.

Target platforms · verification required

  • macos · aarch64, x86_64
  • windows · x86_64

Recommended configuration

BF16 · mps (verified platform only)

Sources & technical details
Original model
https://huggingface.co/tencent/HY-MT1.5-1.8B
Release verification
Verified
Release evidence
Official original provider release notes explicitly name HY-MT1.5-1.8B and7B released2025-12-30. Repository creation and later updates are distinct; not the September WMT25 predecessor or Hy-MT2.
Source updated
Jan 1, 2026
Native model files
https://huggingface.co/tencent/HY-MT1.5-1.8B
Files updated
Jan 1, 2026
Artifact revision
dbad03788f49709801014c95d481a514c272ca52

Desktop app

Sample result

HY-MT1.5 1.8B Desktop

Run HY-MT 1.5 Desktop on your computer

Download the desktop app to load this model, provide input and view results locally.

Public desktop downloads are coming soon.

Device requirements

macOS Apple SiliconCPU / FP32Tested24.0 GiBRAM estimate
macOS Apple SiliconMPS / BF16Tested16.0 GiBRAM estimate
macOS Apple SiliconMPS / FP16Tested16.0 GiBRAM estimate
macOS Apple SiliconMPS / FP32Tested16.0 GiBRAM estimate
If your device falls short

MemoryAdvisory only: close other apps, reduce image/tile/token/batch settings or choose a smaller verified model/precision. CPU can need more RAM than GPU.

GPUIf the selected GPU runtime fails, existing CPU fallback may recover when enabled, but CPU has its own memory/runtime needs.

PlatformUse a compatible OS/architecture/runtime or another model. These estimates do not add hardware detection or block runs.

Benchmark evidence

Saved desktop measurements apply to the recorded hardware and runtime. Fallback cases report the device that actually ran; they do not prove GPU support on the requested device.

HY-MT1.5 1.8B Desktop

165 configurations tested
Passed165
Failed0
TestedNov 13, 2026
Runtimetransformers 5.22.1

Quality criteria

{ "minimumCharacterF2": 0.55, "requireSemanticAnchors": true, "rejectRepetitionAndTruncation": true, "frozenPolicy": "research/hy-mt15-family-expansion/quality-gates.json" }

PrecisionDeviceResultPlatformInference / load
BF16mpsAcceleratedPassedFastmacos · aarch64224 ms / 5024 ms
BF16mpsAcceleratedPassedFastmacos · aarch64224 ms / 7881 ms
BF16mpsAcceleratedPassedFastmacos · aarch64225 ms / 5071 ms
FP16mpsAcceleratedPassedFastmacos · aarch64225 ms / 5074 ms
FP16mpsAcceleratedPassedFastmacos · aarch64237 ms / 5506 ms
FP16mpsAcceleratedPassedFastmacos · aarch64241 ms / 5063 ms
BF16mpsAcceleratedPassedFastmacos · aarch64248 ms / 11266 ms
FP16mpsAcceleratedPassedFastmacos · aarch64252 ms / 5157 ms
BF16mpsAcceleratedPassedFastmacos · aarch64293 ms / 5071 ms
BF16mpsAcceleratedPassedFastmacos · aarch64293 ms / 5071 ms
BF16mpsAcceleratedPassedFastmacos · aarch64294 ms / 7881 ms
BF16mpsAcceleratedPassedFastmacos · aarch64294 ms / 7881 ms
BF16mpsAcceleratedPassedFastmacos · aarch64295 ms / 5024 ms
BF16mpsAcceleratedPassedFastmacos · aarch64295 ms / 5071 ms
BF16mpsAcceleratedPassedFastmacos · aarch64295 ms / 5024 ms
FP16mpsAcceleratedPassedFastmacos · aarch64295 ms / 5157 ms
FP16mpsAcceleratedPassedFastmacos · aarch64296 ms / 5157 ms
BF16mpsAcceleratedPassedFastmacos · aarch64299 ms / 7881 ms
FP16mpsAcceleratedPassedFastmacos · aarch64299 ms / 5157 ms
BF16mpsAcceleratedPassedFastmacos · aarch64300 ms / 5024 ms
FP16mpsAcceleratedPassedFastmacos · aarch64301 ms / 5074 ms
FP16mpsAcceleratedPassedFastmacos · aarch64301 ms / 5063 ms
FP16mpsAcceleratedPassedFastmacos · aarch64316 ms / 5074 ms
FP16mpsAcceleratedPassedFastmacos · aarch64324 ms / 5063 ms
FP16mpsAcceleratedPassedFastmacos · aarch64326 ms / 5074 ms
BF16mpsAcceleratedPassedFastmacos · aarch64334 ms / 5071 ms
FP16mpsAcceleratedPassedFastmacos · aarch64335 ms / 5074 ms
FP32mpsAcceleratedPassedFastmacos · aarch64336 ms / 5533 ms
BF16mpsAcceleratedPassedFastmacos · aarch64340 ms / 5024 ms
FP16mpsAcceleratedPassedFastmacos · aarch64340 ms / 5063 ms
FP32mpsAcceleratedPassedFastmacos · aarch64342 ms / 5676 ms
FP16mpsAcceleratedPassedFastmacos · aarch64346 ms / 5063 ms
BF16mpsAcceleratedPassedFastmacos · aarch64346 ms / 7881 ms
FP32mpsAcceleratedPassedFastmacos · aarch64357 ms / 6149 ms
FP16mpsAcceleratedPassedFastmacos · aarch64364 ms / 5063 ms
BF16mpsAcceleratedPassedFastmacos · aarch64365 ms / 5071 ms
BF16mpsAcceleratedPassedFastmacos · aarch64367 ms / 5071 ms
FP16mpsAcceleratedPassedFastmacos · aarch64369 ms / 5506 ms
FP16mpsAcceleratedPassedFastmacos · aarch64371 ms / 5157 ms
BF16mpsAcceleratedPassedFastmacos · aarch64372 ms / 7881 ms
FP16mpsAcceleratedPassedFastmacos · aarch64373 ms / 5157 ms
FP16mpsAcceleratedPassedFastmacos · aarch64376 ms / 5063 ms
FP32mpsAcceleratedPassedFastmacos · aarch64378 ms / 5208 ms
BF16mpsAcceleratedPassedFastmacos · aarch64379 ms / 5024 ms
BF16mpsAcceleratedPassedFastmacos · aarch64379 ms / 11266 ms
FP16mpsAcceleratedPassedFastmacos · aarch64380 ms / 5157 ms
BF16mpsAcceleratedPassedFastmacos · aarch64384 ms / 5024 ms
BF16mpsAcceleratedPassedFastmacos · aarch64385 ms / 7881 ms
FP16mpsAcceleratedPassedFastmacos · aarch64388 ms / 5074 ms
FP16mpsAcceleratedPassedFastmacos · aarch64398 ms / 5506 ms
FP16mpsAcceleratedPassedFastmacos · aarch64400 ms / 5506 ms
FP16mpsAcceleratedPassedFastmacos · aarch64403 ms / 5074 ms
BF16mpsAcceleratedPassedFastmacos · aarch64414 ms / 11266 ms
BF16mpsAcceleratedPassedFastmacos · aarch64430 ms / 11266 ms
FP32mpsAcceleratedPassedFastmacos · aarch64479 ms / 5533 ms
FP32mpsAcceleratedPassedFastmacos · aarch64479 ms / 5676 ms
FP32mpsAcceleratedPassedFastmacos · aarch64479 ms / 5533 ms
FP32mpsAcceleratedPassedFastmacos · aarch64483 ms / 5533 ms
FP32mpsAcceleratedPassedFastmacos · aarch64487 ms / 5208 ms
FP32mpsAcceleratedPassedFastmacos · aarch64502 ms / 5208 ms
FP32mpsAcceleratedPassedFastmacos · aarch64508 ms / 5676 ms
FP32mpsAcceleratedPassedFastmacos · aarch64512 ms / 5676 ms
FP32mpsAcceleratedPassedFastmacos · aarch64515 ms / 5208 ms
FP32mpsAcceleratedPassedFastmacos · aarch64525 ms / 5208 ms
FP32mpsAcceleratedPassedFastmacos · aarch64526 ms / 5533 ms
FP32mpsAcceleratedPassedFastmacos · aarch64528 ms / 5676 ms
BF16mpsAcceleratedPassedFastmacos · aarch64534 ms / 5071 ms
BF16mpsAcceleratedPassedFastmacos · aarch64538 ms / 5024 ms
BF16mpsAcceleratedPassedFastmacos · aarch64549 ms / 5024 ms
BF16mpsAcceleratedPassedFastmacos · aarch64551 ms / 5071 ms
FP32mpsAcceleratedPassedFastmacos · aarch64555 ms / 6149 ms
BF16mpsAcceleratedPassedFastmacos · aarch64562 ms / 7881 ms
FP16mpsAcceleratedPassedFastmacos · aarch64571 ms / 5074 ms
FP32mpsAcceleratedPassedFastmacos · aarch64582 ms / 5208 ms
FP16mpsAcceleratedPassedFastmacos · aarch64584 ms / 5063 ms
FP32mpsAcceleratedPassedFastmacos · aarch64585 ms / 5533 ms
FP16mpsAcceleratedPassedFastmacos · aarch64595 ms / 5063 ms
FP32mpsAcceleratedPassedFastmacos · aarch64599 ms / 5533 ms
FP16mpsAcceleratedPassedFastmacos · aarch64600 ms / 5074 ms
FP32mpsAcceleratedPassedFastmacos · aarch64601 ms / 5533 ms
FP16mpsAcceleratedPassedFastmacos · aarch64603 ms / 5157 ms
FP32mpsAcceleratedPassedFastmacos · aarch64606 ms / 6149 ms
FP32mpsAcceleratedPassedFastmacos · aarch64607 ms / 5676 ms
FP32mpsAcceleratedPassedFastmacos · aarch64615 ms / 5208 ms
FP32mpsAcceleratedPassedFastmacos · aarch64623 ms / 6149 ms
FP16mpsAcceleratedPassedFastmacos · aarch64625 ms / 5506 ms
FP32mpsAcceleratedPassedFastmacos · aarch64626 ms / 5676 ms
FP32mpsAcceleratedPassedFastmacos · aarch64631 ms / 5208 ms
BF16mpsAcceleratedPassedFastmacos · aarch64633 ms / 7881 ms
FP16mpsAcceleratedPassedFastmacos · aarch64642 ms / 5157 ms
FP32mpsAcceleratedPassedFastmacos · aarch64655 ms / 6149 ms
BF16mpsAcceleratedPassedFastmacos · aarch64656 ms / 11266 ms
FP32mpsAcceleratedPassedFastmacos · aarch64688 ms / 5676 ms
FP16mpsAcceleratedPassedFastmacos · aarch64707 ms / 5506 ms
FP32mpsAcceleratedPassedFastmacos · aarch64715 ms / 5533 ms
FP32mpsAcceleratedPassedFastmacos · aarch64745 ms / 5676 ms
FP32mpsAcceleratedPassedFastmacos · aarch64761 ms / 6149 ms
FP32mpsAcceleratedPassedFastmacos · aarch64769 ms / 5208 ms
BF16mpsAcceleratedPassedFastmacos · aarch64832 ms / 11266 ms
FP32cpuPassedFastmacos · aarch64684 ms / 4764 ms
FP32cpuPassedFastmacos · aarch64705 ms / 5158 ms
FP32cpuPassedFastmacos · aarch64714 ms / 8264 ms
FP32cpuPassedFastmacos · aarch64719 ms / 7025 ms
FP32cpuPassedFastmacos · aarch64741 ms / 5279 ms
FP32cpuPassedFastmacos · aarch64779 ms / 4773 ms
FP32cpuPassedFastmacos · aarch64798 ms / 5507 ms
FP32cpuPassedFastmacos · aarch64816 ms / 4703 ms
FP32cpuPassedFastmacos · aarch641121 ms / 4764 ms
FP32cpuPassedFastmacos · aarch641123 ms / 4773 ms
FP32cpuPassedFastmacos · aarch641143 ms / 7025 ms
FP32cpuPassedFastmacos · aarch641144 ms / 5158 ms
FP32cpuPassedFastmacos · aarch641145 ms / 5158 ms
FP32cpuPassedFastmacos · aarch641148 ms / 7025 ms
FP32cpuPassedFastmacos · aarch641148 ms / 5158 ms
FP32cpuPassedFastmacos · aarch641150 ms / 7025 ms
FP32cpuPassedFastmacos · aarch641151 ms / 7025 ms
FP32cpuPassedFastmacos · aarch641152 ms / 7025 ms
FP32cpuPassedFastmacos · aarch641153 ms / 4703 ms
FP32cpuPassedFastmacos · aarch641163 ms / 4773 ms
FP32cpuPassedFastmacos · aarch641172 ms / 4773 ms
FP32cpuPassedFastmacos · aarch641177 ms / 4773 ms
FP32cpuPassedFastmacos · aarch641189 ms / 8264 ms
FP32cpuPassedFastmacos · aarch641191 ms / 5158 ms
FP32cpuPassedFastmacos · aarch641205 ms / 5158 ms
FP32cpuPassedFastmacos · aarch641211 ms / 8264 ms
FP32cpuPassedFastmacos · aarch641221 ms / 4764 ms
FP32cpuPassedFastmacos · aarch641230 ms / 4764 ms
FP32cpuPassedFastmacos · aarch641232 ms / 4773 ms
FP32cpuPassedFastmacos · aarch641274 ms / 4764 ms
FP32cpuPassedFastmacos · aarch641280 ms / 4773 ms
FP32cpuPassedFastmacos · aarch641293 ms / 5279 ms
FP32cpuPassedFastmacos · aarch641294 ms / 4703 ms
FP32cpuPassedFastmacos · aarch641296 ms / 5158 ms
FP32cpuPassedFastmacos · aarch641301 ms / 7025 ms
FP32cpuPassedFastmacos · aarch641327 ms / 8264 ms
FP32cpuPassedFastmacos · aarch641328 ms / 8264 ms
FP32cpuPassedFastmacos · aarch641337 ms / 4703 ms
FP32cpuPassedFastmacos · aarch641343 ms / 5279 ms
FP32cpuPassedFastmacos · aarch641358 ms / 8264 ms
FP32cpuPassedFastmacos · aarch641363 ms / 5507 ms
FP32cpuPassedFastmacos · aarch641365 ms / 4764 ms
FP32cpuPassedFastmacos · aarch641366 ms / 4764 ms
FP32cpuPassedFastmacos · aarch641370 ms / 4703 ms
FP32cpuPassedFastmacos · aarch641388 ms / 8264 ms
FP32cpuPassedFastmacos · aarch641418 ms / 4773 ms
FP32cpuPassedFastmacos · aarch641421 ms / 5507 ms
FP32cpuPassedFastmacos · aarch641431 ms / 5158 ms
FP32cpuPassedFastmacos · aarch641460 ms / 4703 ms
FP32cpuPassedFastmacos · aarch641464 ms / 5507 ms
FP32cpuPassedFastmacos · aarch641480 ms / 8264 ms
FP32cpuPassedFastmacos · aarch641541 ms / 7025 ms
FP32cpuPassedFastmacos · aarch641558 ms / 4764 ms
FP32cpuPassedFastmacos · aarch641626 ms / 4703 ms
FP32cpuPassedFastmacos · aarch641643 ms / 5279 ms
FP32cpuPassedFastmacos · aarch641662 ms / 7025 ms
FP32cpuPassedFastmacos · aarch641666 ms / 5507 ms
FP32cpuPassedFastmacos · aarch641667 ms / 4764 ms
FP32cpuPassedFastmacos · aarch641678 ms / 4773 ms
FP32cpuPassedFastmacos · aarch641682 ms / 5279 ms
FP32cpuPassedFastmacos · aarch641683 ms / 5158 ms
FP32cpuPassedFastmacos · aarch641685 ms / 4703 ms
FP32cpuPassedFastmacos · aarch641735 ms / 8264 ms
FP32cpuPassedFastmacos · aarch641937 ms / 5507 ms
FP32cpuPassedFastmacos · aarch642006 ms / 4703 ms
FP32cpuPassedFastmacos · aarch642038 ms / 5279 ms
Test setup & sample details
Quality gate
{ "minimumCharacterF2": 0.55, "requireSemanticAnchors": true, "rejectRepetitionAndTruncation": true, "frozenPolicy": "research/hy-mt15-family-expansion/quality-gates.json" }
Tested
2026-11-13T09:50:29.956212+00:00
Environment
{ "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2", "runtime": "App-owned CPython3.13.18; Transformers5.22.1/Torch2.10.0; MPS op fallback disabled", "concurrentUploadActivity": "none; uploads excluded", "transport": "Verified exact local original snapshots, not Internet transfer timing" }
Runtime
transformers 5.22.1
Artifact revision
dbad03788f49709801014c95d481a514c272ca52
Fixture
{
  "file": "model-samples/hunyuan-translation.json",
  "url": "https://assets.localai.tools/samples/v1/files/b63cba0b5f61f8423d4234d4d28e9cdde538b8345b4ccdc8b2044b1bc88e2fa8/hunyuan-translation.json",
  "size": 3099,
  "sha1": "dd6e22e1c2966dd5a249b11eb2fb48b076505088",
  "sha256": "b63cba0b5f61f8423d4234d4d28e9cdde538b8345b4ccdc8b2044b1bc88e2fa8"
}
Rerun when
model artifact revision change; runtime version change; fixture or quality-gate change; backend/platform behavior change
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 8263.978708011564,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1387.271250016056,
      "inferenceMs": 1388.024875021074,
      "tokensPerSecond": 10.117697432306798,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 9652.003583032638,
      "residentMemoryBytes": 7643840512,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 8263.978708011564,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1734.4194170436822,
      "inferenceMs": 1735.2955000242218,
      "tokensPerSecond": 12.110392280955145,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 9999.274208035786,
      "residentMemoryBytes": 7646429184,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 8263.978708011564,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 712.7875829464756,
      "inferenceMs": 713.5959169827402,
      "tokensPerSecond": 11.230421991009774,
      "inputTokens": 26,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 8977.574624994304,
      "residentMemoryBytes": 7646609408,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 8263.978708011564,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1210.2975830202922,
      "inferenceMs": 1211.079917033203,
      "tokensPerSecond": 11.570203307373081,
      "inputTokens": 22,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 9475.058625044767,
      "residentMemoryBytes": 7646642176,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7626300625142021,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 8263.978708011564,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1479.0292080142535,
      "inferenceMs": 1479.877415986266,
      "tokensPerSecond": 12.172845618601855,
      "inputTokens": 29,
      "outputTokens": 18,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 9743.85612399783,
      "residentMemoryBytes": 7646674944,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 8263.978708011564,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1326.3028329820372,
      "inferenceMs": 1327.0972919999622,
      "tokensPerSecond": 12.066958418006541,
      "inputTokens": 21,
      "outputTokens": 16,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 9591.076000011526,
      "residentMemoryBytes": 7646691328,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 8263.978708011564,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1187.947541999165,
      "inferenceMs": 1188.7292079627514,
      "tokensPerSecond": 11.787773356206403,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 9452.707915974315,
      "residentMemoryBytes": 7648116736,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 8263.978708011564,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1356.8150830105878,
      "inferenceMs": 1357.6546250260435,
      "tokensPerSecond": 10.320433173374267,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 9621.633333037607,
      "residentMemoryBytes": 7649722368,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 8263.978708011564,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1326.8528750049882,
      "inferenceMs": 1327.6774999685585,
      "tokensPerSecond": 10.553857986467804,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 9591.656207980122,
      "residentMemoryBytes": 7649722368,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4764.446957968175,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1220.6698340014555,
      "inferenceMs": 1221.3670830242336,
      "tokensPerSecond": 11.49945068223331,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 5985.814040992409,
      "residentMemoryBytes": 7645315072,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4764.446957968175,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1666.2329169921577,
      "inferenceMs": 1667.0011659734882,
      "tokensPerSecond": 12.605575446357383,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6431.448123941664,
      "residentMemoryBytes": 7646412800,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4764.446957968175,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 683.5750839672983,
      "inferenceMs": 684.324000030756,
      "tokensPerSecond": 11.707565490660604,
      "inputTokens": 26,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 5448.770957998931,
      "residentMemoryBytes": 7646461952,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4764.446957968175,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1228.9830839727074,
      "inferenceMs": 1229.8393750097603,
      "tokensPerSecond": 11.39406871920328,
      "inputTokens": 22,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 5994.286332977936,
      "residentMemoryBytes": 7646445568,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7626300625142021,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4764.446957968175,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1556.8414999870583,
      "inferenceMs": 1557.7023340156302,
      "tokensPerSecond": 11.564255016329199,
      "inputTokens": 29,
      "outputTokens": 18,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6322.149291983806,
      "residentMemoryBytes": 7650033664,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4764.446957968175,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1364.224709046539,
      "inferenceMs": 1365.3255419922061,
      "tokensPerSecond": 11.730892954797708,
      "inputTokens": 21,
      "outputTokens": 16,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6129.7724999603815,
      "residentMemoryBytes": 7650672640,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4764.446957968175,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1365.5403750017285,
      "inferenceMs": 1366.4232079754584,
      "tokensPerSecond": 10.25508423272915,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6130.870165943634,
      "residentMemoryBytes": 7650787328,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4764.446957968175,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1273.0131249991246,
      "inferenceMs": 1273.801208997611,
      "tokensPerSecond": 11.000298020063758,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6038.248166965786,
      "residentMemoryBytes": 7650787328,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4764.446957968175,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1119.8608329868875,
      "inferenceMs": 1120.5989169538952,
      "tokensPerSecond": 12.504482634121628,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 5885.045874922071,
      "residentMemoryBytes": 7650787328,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4773.289375007153,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1122.5874170195311,
      "inferenceMs": 1123.2837090501562,
      "tokensPerSecond": 12.50581557289394,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 5896.573084057309,
      "residentMemoryBytes": 7641219072,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4773.289375007153,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1676.7585829948075,
      "inferenceMs": 1677.5338339502923,
      "tokensPerSecond": 12.526364701740365,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6450.823208957445,
      "residentMemoryBytes": 7644545024,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4773.289375007153,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 778.050999972038,
      "inferenceMs": 778.8369169575162,
      "tokensPerSecond": 10.285611608682006,
      "inputTokens": 26,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 5552.126291964669,
      "residentMemoryBytes": 7647625216,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4773.289375007153,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1162.605417019222,
      "inferenceMs": 1163.382124970667,
      "tokensPerSecond": 12.044827837910708,
      "inputTokens": 22,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 5936.67149997782,
      "residentMemoryBytes": 7647625216,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7626300625142021,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4773.289375007153,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1417.7510830340907,
      "inferenceMs": 1418.4957920224406,
      "tokensPerSecond": 12.698674358658712,
      "inputTokens": 29,
      "outputTokens": 18,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6191.785167029593,
      "residentMemoryBytes": 7647690752,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4773.289375007153,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1279.326832969673,
      "inferenceMs": 1280.0621669739485,
      "tokensPerSecond": 12.509123675777886,
      "inputTokens": 21,
      "outputTokens": 16,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6053.351541981101,
      "residentMemoryBytes": 7647690752,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4773.289375007153,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1176.3042910024524,
      "inferenceMs": 1177.1112089627422,
      "tokensPerSecond": 11.904139781843815,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 5950.400583969895,
      "residentMemoryBytes": 7647723520,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4773.289375007153,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1231.083624996245,
      "inferenceMs": 1231.8806250113994,
      "tokensPerSecond": 11.374900081368686,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6005.170000018552,
      "residentMemoryBytes": 7647789056,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4773.289375007153,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1171.1564169963822,
      "inferenceMs": 1171.9409169745632,
      "tokensPerSecond": 11.956896249935811,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 5945.230291981716,
      "residentMemoryBytes": 7647789056,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4703.418584016617,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1152.4710000376217,
      "inferenceMs": 1153.209749958478,
      "tokensPerSecond": 12.181029677660005,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 5856.628333975095,
      "residentMemoryBytes": 7644004352,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4703.418584016617,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 2005.2745830034837,
      "inferenceMs": 2006.3368330011144,
      "tokensPerSecond": 10.474389897003107,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6709.755417017732,
      "residentMemoryBytes": 7649804288,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4703.418584016617,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 814.7995830513537,
      "inferenceMs": 815.6343330047093,
      "tokensPerSecond": 9.822601255010614,
      "inputTokens": 26,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 5519.052917021327,
      "residentMemoryBytes": 7649968128,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4703.418584016617,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1336.2069579889067,
      "inferenceMs": 1337.1341250021942,
      "tokensPerSecond": 10.479879938276742,
      "inputTokens": 22,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6040.552709018812,
      "residentMemoryBytes": 7649951744,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7626300625142021,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4703.418584016617,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1624.6733749867417,
      "inferenceMs": 1625.6098339799792,
      "tokensPerSecond": 11.082657359953087,
      "inputTokens": 29,
      "outputTokens": 18,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6329.0284179965965,
      "residentMemoryBytes": 7649968128,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4703.418584016617,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1683.774542005267,
      "inferenceMs": 1685.0952500244603,
      "tokensPerSecond": 9.504622536810956,
      "inputTokens": 21,
      "outputTokens": 16,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6388.513834041078,
      "residentMemoryBytes": 7649984512,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4703.418584016617,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1459.1619999846444,
      "inferenceMs": 1460.051374975592,
      "tokensPerSecond": 9.596808924024284,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6163.469958992209,
      "residentMemoryBytes": 7650017280,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4703.418584016617,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1293.5474169789813,
      "inferenceMs": 1294.3749169935472,
      "tokensPerSecond": 10.825791365203733,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 5997.793501010165,
      "residentMemoryBytes": 7650017280,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4703.418584016617,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1369.1406659781933,
      "inferenceMs": 1369.9909169808961,
      "tokensPerSecond": 10.227605311559993,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6073.4095009975135,
      "residentMemoryBytes": 7650017280,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7024.655959045049,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1151.3842080021277,
      "inferenceMs": 1152.069333998952,
      "tokensPerSecond": 12.193826527036636,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 8176.725293044001,
      "residentMemoryBytes": 7646937088,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7024.655959045049,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1660.8542089816183,
      "inferenceMs": 1661.6177090327255,
      "tokensPerSecond": 12.646347919099275,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 8686.273668077774,
      "residentMemoryBytes": 7649427456,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7024.655959045049,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 718.4776659705676,
      "inferenceMs": 719.2265000194311,
      "tokensPerSecond": 11.13861794331888,
      "inputTokens": 26,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 7743.88245906448,
      "residentMemoryBytes": 7649460224,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7024.655959045049,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1146.771208033897,
      "inferenceMs": 1147.5826670066454,
      "tokensPerSecond": 12.211149968049083,
      "inputTokens": 22,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 8172.238626051694,
      "residentMemoryBytes": 7649525760,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7626300625142021,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7024.655959045049,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1540.0422499515116,
      "inferenceMs": 1540.8637499785982,
      "tokensPerSecond": 11.690325544116302,
      "inputTokens": 29,
      "outputTokens": 18,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 8565.519709023647,
      "residentMemoryBytes": 7649525760,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7024.655959045049,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1300.3304169978946,
      "inferenceMs": 1301.1392499902286,
      "tokensPerSecond": 12.30733058558182,
      "inputTokens": 21,
      "outputTokens": 16,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 8325.795209035277,
      "residentMemoryBytes": 7649525760,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7024.655959045049,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1141.7172910296358,
      "inferenceMs": 1142.518165986985,
      "tokensPerSecond": 12.264918740441033,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 8167.174125032034,
      "residentMemoryBytes": 7649525760,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7024.655959045049,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1150.2280000131577,
      "inferenceMs": 1150.9865829721093,
      "tokensPerSecond": 12.174488240265118,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 8175.642542017158,
      "residentMemoryBytes": 7649591296,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7024.655959045049,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1148.7836670130491,
      "inferenceMs": 1149.5374589576386,
      "tokensPerSecond": 12.189531978372779,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 8174.193418002687,
      "residentMemoryBytes": 7649591296,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5157.901541970205,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1190.6079579493962,
      "inferenceMs": 1191.3031250005588,
      "tokensPerSecond": 11.79565543180523,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6349.204666970763,
      "residentMemoryBytes": 7645626368,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5157.901541970205,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1682.2435419890098,
      "inferenceMs": 1683.0069580464624,
      "tokensPerSecond": 12.48554377687322,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6840.908500016667,
      "residentMemoryBytes": 7649492992,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5157.901541970205,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 704.7104589873925,
      "inferenceMs": 705.466957995668,
      "tokensPerSecond": 11.356451214522684,
      "inputTokens": 26,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 5863.368499965873,
      "residentMemoryBytes": 7649492992,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5157.901541970205,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1143.7126249657013,
      "inferenceMs": 1144.4560409872793,
      "tokensPerSecond": 12.243675826296593,
      "inputTokens": 22,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6302.357582957484,
      "residentMemoryBytes": 7649542144,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7626300625142021,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5157.901541970205,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1430.274999991525,
      "inferenceMs": 1431.0327499988489,
      "tokensPerSecond": 12.587652096132437,
      "inputTokens": 29,
      "outputTokens": 18,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6588.934291969053,
      "residentMemoryBytes": 7649542144,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5157.901541970205,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1295.199457963463,
      "inferenceMs": 1295.9629999822937,
      "tokensPerSecond": 12.355795558179524,
      "inputTokens": 21,
      "outputTokens": 16,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6453.864541952498,
      "residentMemoryBytes": 7649542144,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5157.901541970205,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1147.1890829852782,
      "inferenceMs": 1147.9585419874638,
      "tokensPerSecond": 12.206668928285149,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6305.860083957668,
      "residentMemoryBytes": 7649542144,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5157.901541970205,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1143.9718329929747,
      "inferenceMs": 1144.7157500078902,
      "tokensPerSecond": 12.241006617950555,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6302.617291978095,
      "residentMemoryBytes": 7649542144,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5157.901541970205,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1203.9907090365887,
      "inferenceMs": 1204.7665840364061,
      "tokensPerSecond": 11.63032713766157,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6362.668126006611,
      "residentMemoryBytes": 7649542144,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7880.543457984459,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 632.3576249997132,
      "inferenceMs": 633.113416028209,
      "tokensPerSecond": 22.309835567945015,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4365647872,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 8513.656874012668,
      "residentMemoryBytes": 618708992,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7880.543457984459,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 561.3809159840457,
      "inferenceMs": 562.1828750008717,
      "tokensPerSecond": 37.512735906480316,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366237696,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 8442.72633298533,
      "residentMemoryBytes": 678658048,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7880.543457984459,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 223.46566698979586,
      "inferenceMs": 224.1855000029318,
      "tokensPerSecond": 36.02621127090479,
      "inputTokens": 26,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366385152,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 8104.7289579873905,
      "residentMemoryBytes": 691814400,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7880.543457984459,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 345.6239169463515,
      "inferenceMs": 346.4380000368692,
      "tokensPerSecond": 40.69255330941452,
      "inputTokens": 22,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366532608,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 8226.981458021328,
      "residentMemoryBytes": 708771840,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7626300625142021,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7880.543457984459,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 371.51108297985047,
      "inferenceMs": 372.3467909730971,
      "tokensPerSecond": 48.66557742610281,
      "inputTokens": 29,
      "outputTokens": 18,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366532608,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 8252.890248957556,
      "residentMemoryBytes": 710148096,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7880.543457984459,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 384.365624981001,
      "inferenceMs": 385.1803339784965,
      "tokensPerSecond": 41.80268421553306,
      "inputTokens": 21,
      "outputTokens": 16,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366680064,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 8265.723791962955,
      "residentMemoryBytes": 727351296,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7880.543457984459,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 298.001250019297,
      "inferenceMs": 298.8052080036141,
      "tokensPerSecond": 47.22712146324092,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366680064,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 8179.348665988073,
      "residentMemoryBytes": 728399872,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7880.543457984459,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 293.61837496981025,
      "inferenceMs": 294.35687500517815,
      "tokensPerSecond": 47.90236813120964,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366680064,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 8174.900332989637,
      "residentMemoryBytes": 729448448,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7880.543457984459,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 293.51229197345674,
      "inferenceMs": 294.2489159759134,
      "tokensPerSecond": 47.93178622113254,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366680064,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 8174.792373960372,
      "residentMemoryBytes": 730480640,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5024.133917002473,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 537.315750028938,
      "inferenceMs": 538.0639170180075,
      "tokensPerSecond": 26.274511229104654,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4365647872,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5562.1978340204805,
      "residentMemoryBytes": 620658688,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5024.133917002473,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 548.2407920062542,
      "inferenceMs": 549.0343749988824,
      "tokensPerSecond": 38.41000271798246,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366237696,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5573.168292001355,
      "residentMemoryBytes": 680722432,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5024.133917002473,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 223.20000000763685,
      "inferenceMs": 223.9007079624571,
      "tokensPerSecond": 36.07069867416962,
      "inputTokens": 26,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366385152,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5248.03462496493,
      "residentMemoryBytes": 694075392,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5024.133917002473,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 339.01808300288394,
      "inferenceMs": 339.8197910282761,
      "tokensPerSecond": 41.50159690628121,
      "inputTokens": 22,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366532608,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5363.953708030749,
      "residentMemoryBytes": 711049216,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7626300625142021,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5024.133917002473,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 382.68066599266604,
      "inferenceMs": 383.5970830405131,
      "tokensPerSecond": 47.23279258632707,
      "inputTokens": 29,
      "outputTokens": 18,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366532608,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5407.731000042986,
      "residentMemoryBytes": 712605696,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5024.133917002473,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 377.7784169651568,
      "inferenceMs": 378.5880830255337,
      "tokensPerSecond": 42.472999298514324,
      "inputTokens": 21,
      "outputTokens": 16,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366680064,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5402.722000028007,
      "residentMemoryBytes": 729792512,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5024.133917002473,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 294.24187500262633,
      "inferenceMs": 294.98245799914,
      "tokensPerSecond": 47.81271073594829,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366680064,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5319.116375001613,
      "residentMemoryBytes": 730890240,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5024.133917002473,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 294.21795799862593,
      "inferenceMs": 294.9019999941811,
      "tokensPerSecond": 47.80346623780305,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366680064,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5319.035916996654,
      "residentMemoryBytes": 732004352,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5024.133917002473,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 298.8083749660291,
      "inferenceMs": 299.5647080242634,
      "tokensPerSecond": 47.084805960124534,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366680064,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5323.698625026736,
      "residentMemoryBytes": 733184000,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5071.107458032202,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 533.1033749971539,
      "inferenceMs": 533.830999978818,
      "tokensPerSecond": 26.479776190249822,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4365647872,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5604.93845801102,
      "residentMemoryBytes": 627965952,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5071.107458032202,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 550.4545000148937,
      "inferenceMs": 551.2365000322461,
      "tokensPerSecond": 38.25766978320208,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366237696,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5622.343958064448,
      "residentMemoryBytes": 686997504,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5071.107458032202,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 224.57174997543916,
      "inferenceMs": 225.32304102787748,
      "tokensPerSecond": 35.839731328308034,
      "inputTokens": 26,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366385152,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5296.4304990600795,
      "residentMemoryBytes": 700153856,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5071.107458032202,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 333.23200000450015,
      "inferenceMs": 333.99387501413,
      "tokensPerSecond": 42.208446496642274,
      "inputTokens": 22,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366532608,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5405.101333046332,
      "residentMemoryBytes": 716832768,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7626300625142021,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5071.107458032202,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 363.8015409815125,
      "inferenceMs": 364.53045799862593,
      "tokensPerSecond": 49.68461201773095,
      "inputTokens": 29,
      "outputTokens": 18,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366532608,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5435.637916030828,
      "residentMemoryBytes": 718258176,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5071.107458032202,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 365.82524998812005,
      "inferenceMs": 366.5543330134824,
      "tokensPerSecond": 43.913372104114494,
      "inputTokens": 21,
      "outputTokens": 16,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366680064,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5437.661791045684,
      "residentMemoryBytes": 735182848,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5071.107458032202,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 292.32804203638807,
      "inferenceMs": 293.03983302088454,
      "tokensPerSecond": 48.14217076224888,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366680064,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5364.1472910530865,
      "residentMemoryBytes": 736329728,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5071.107458032202,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 292.50758298439905,
      "inferenceMs": 293.21437497856095,
      "tokensPerSecond": 48.08478721944285,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366680064,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5364.321833010763,
      "residentMemoryBytes": 737476608,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5071.107458032202,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 294.1706670098938,
      "inferenceMs": 294.9345000088215,
      "tokensPerSecond": 47.834874011318,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366680064,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5366.0419580410235,
      "residentMemoryBytes": 738590720,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5074.395167001057,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 599.6171670267358,
      "inferenceMs": 600.3997080260888,
      "tokensPerSecond": 23.643999047687334,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4365664256,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5674.794875027146,
      "residentMemoryBytes": 755974144,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5074.395167001057,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 569.933167018462,
      "inferenceMs": 570.7815419882536,
      "tokensPerSecond": 36.936739459283764,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366237696,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5645.176708989311,
      "residentMemoryBytes": 816054272,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5074.395167001057,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 224.59795797476545,
      "inferenceMs": 225.3797500161454,
      "tokensPerSecond": 35.877027112724605,
      "inputTokens": 26,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366385152,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5299.774917017203,
      "residentMemoryBytes": 829341696,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5074.395167001057,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 334.7275420092046,
      "inferenceMs": 335.47591598471627,
      "tokensPerSecond": 42.00825466533489,
      "inputTokens": 22,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366532608,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5409.871082985774,
      "residentMemoryBytes": 846381056,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7626300625142021,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5074.395167001057,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 387.09595799446106,
      "inferenceMs": 388.00475001335144,
      "tokensPerSecond": 46.69679106461968,
      "inputTokens": 29,
      "outputTokens": 18,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366532608,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5462.399917014409,
      "residentMemoryBytes": 847888384,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5074.395167001057,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 402.3788750055246,
      "inferenceMs": 403.41575001366436,
      "tokensPerSecond": 39.92469204903762,
      "inputTokens": 21,
      "outputTokens": 16,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366680064,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5477.810917014722,
      "residentMemoryBytes": 865173504,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5074.395167001057,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 325.04354196134955,
      "inferenceMs": 325.9589999797754,
      "tokensPerSecond": 43.28680855815733,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366680064,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5400.354166980833,
      "residentMemoryBytes": 866287616,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5074.395167001057,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 315.57270901976153,
      "inferenceMs": 316.42429099883884,
      "tokensPerSecond": 44.51595841122479,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366680064,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5390.819457999896,
      "residentMemoryBytes": 867434496,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5074.395167001057,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 299.8407499981113,
      "inferenceMs": 300.70900003192946,
      "tokensPerSecond": 46.93805342183249,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366680064,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5375.104167032987,
      "residentMemoryBytes": 868564992,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5156.663457979448,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 602.1203750278801,
      "inferenceMs": 602.9193340218626,
      "tokensPerSecond": 23.454675472213218,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4365664256,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5759.582792001311,
      "residentMemoryBytes": 766853120,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5156.663457979448,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 640.718333015684,
      "inferenceMs": 641.6353330132551,
      "tokensPerSecond": 32.86330543876525,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366237696,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5798.298790992703,
      "residentMemoryBytes": 826753024,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5156.663457979448,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 251.0462910286151,
      "inferenceMs": 252.02825001906604,
      "tokensPerSecond": 32.12467044515411,
      "inputTokens": 26,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366385152,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5408.691707998514,
      "residentMemoryBytes": 839958528,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5156.663457979448,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 378.85370798176154,
      "inferenceMs": 379.838542023208,
      "tokensPerSecond": 37.11137839410482,
      "inputTokens": 22,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366532608,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5536.502000002656,
      "residentMemoryBytes": 856915968,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7626300625142021,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5156.663457979448,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 371.87054101377726,
      "inferenceMs": 372.64445796608925,
      "tokensPerSecond": 48.548293553688936,
      "inputTokens": 29,
      "outputTokens": 18,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366532608,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5529.307915945537,
      "residentMemoryBytes": 858259456,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5156.663457979448,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 370.26295805117115,
      "inferenceMs": 371.05224997503683,
      "tokensPerSecond": 43.39801970530571,
      "inputTokens": 21,
      "outputTokens": 16,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366680064,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5527.715707954485,
      "residentMemoryBytes": 875266048,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5156.663457979448,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 295.02229200443253,
      "inferenceMs": 295.76854198239744,
      "tokensPerSecond": 47.68399809056053,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366680064,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5452.431999961846,
      "residentMemoryBytes": 876314624,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5156.663457979448,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 298.36954199709,
      "inferenceMs": 299.0900410222821,
      "tokensPerSecond": 47.15320918895512,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366680064,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5455.75349900173,
      "residentMemoryBytes": 877363200,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5156.663457979448,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 294.27825001766905,
      "inferenceMs": 295.00270797871053,
      "tokensPerSecond": 47.82928368131091,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366680064,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5451.666165958159,
      "residentMemoryBytes": 878428160,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5063.365708047058,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 583.6201659985818,
      "inferenceMs": 584.4836250180379,
      "tokensPerSecond": 24.214476294526765,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4365664256,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5647.849333065096,
      "residentMemoryBytes": 756203520,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5063.365708047058,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 593.7089169747196,
      "inferenceMs": 594.560542027466,
      "tokensPerSecond": 35.46730640999211,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366237696,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5657.926250074524,
      "residentMemoryBytes": 815972352,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5063.365708047058,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 240.21170800551772,
      "inferenceMs": 241.02458398556337,
      "tokensPerSecond": 33.527771078237286,
      "inputTokens": 26,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366385152,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5304.390292032622,
      "residentMemoryBytes": 829079552,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5063.365708047058,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 345.4891250003129,
      "inferenceMs": 346.4089159970172,
      "tokensPerSecond": 40.71464875806509,
      "inputTokens": 22,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366532608,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5409.774624044076,
      "residentMemoryBytes": 846004224,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7626300625142021,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5063.365708047058,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 363.4190419688821,
      "inferenceMs": 364.1935000196099,
      "tokensPerSecond": 49.74820718794639,
      "inputTokens": 29,
      "outputTokens": 18,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366532608,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5427.559208066668,
      "residentMemoryBytes": 847331328,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5063.365708047058,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 374.83474996406585,
      "inferenceMs": 375.646417029202,
      "tokensPerSecond": 42.854019754646544,
      "inputTokens": 21,
      "outputTokens": 16,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366680064,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5439.01212507626,
      "residentMemoryBytes": 864223232,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5063.365708047058,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 300.4352909629233,
      "inferenceMs": 301.31387495202944,
      "tokensPerSecond": 46.76462561318818,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366680064,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5364.679582999088,
      "residentMemoryBytes": 865304576,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5063.365708047058,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 339.2296249512583,
      "inferenceMs": 340.32745799049735,
      "tokensPerSecond": 41.47975568015052,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366680064,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5403.693166037556,
      "residentMemoryBytes": 866451456,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5063.365708047058,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 323.45166604500264,
      "inferenceMs": 324.40583303105086,
      "tokensPerSecond": 43.4530706378336,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366680064,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5387.771541078109,
      "residentMemoryBytes": 867467264,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5676.194750005379,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 687.2474999981932,
      "inferenceMs": 688.2867499953136,
      "tokensPerSecond": 20.532396506806585,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7477821440,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 6364.481500000693,
      "residentMemoryBytes": 627490816,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5676.194750005379,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 744.0054580220021,
      "inferenceMs": 745.0323750381358,
      "tokensPerSecond": 28.322089832768786,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7478411264,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 6421.227125043515,
      "residentMemoryBytes": 685293568,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5676.194750005379,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 340.79741704044864,
      "inferenceMs": 341.67854103725404,
      "tokensPerSecond": 23.593777116698973,
      "inputTokens": 26,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7478558720,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 6017.873291042633,
      "residentMemoryBytes": 697794560,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5676.194750005379,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 527.1212079678662,
      "inferenceMs": 528.017416014336,
      "tokensPerSecond": 26.65537408102582,
      "inputTokens": 22,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7478706176,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 6204.212166019715,
      "residentMemoryBytes": 713981952,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7626300625142021,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5676.194750005379,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 624.746999994386,
      "inferenceMs": 625.8827499696054,
      "tokensPerSecond": 28.892788240430985,
      "inputTokens": 29,
      "outputTokens": 18,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7478706176,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 6302.077499974985,
      "residentMemoryBytes": 714915840,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5676.194750005379,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 605.4836249677464,
      "inferenceMs": 606.5831669839099,
      "tokensPerSecond": 26.51308041385848,
      "inputTokens": 21,
      "outputTokens": 16,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7478853632,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 6282.777916989289,
      "residentMemoryBytes": 731054080,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5676.194750005379,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 506.9645419716835,
      "inferenceMs": 508.17041704431176,
      "tokensPerSecond": 27.73375594321377,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7478853632,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 6184.365167049691,
      "residentMemoryBytes": 731824128,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5676.194750005379,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 511.3586249644868,
      "inferenceMs": 512.4484999687411,
      "tokensPerSecond": 27.44454303146364,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7478853632,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 6188.64324997412,
      "residentMemoryBytes": 732594176,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5676.194750005379,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 478.20354101713747,
      "inferenceMs": 479.0973329800181,
      "tokensPerSecond": 29.40663195001654,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7478853632,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 6155.292082985397,
      "residentMemoryBytes": 733282304,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5208.352167042904,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 581.2358330003917,
      "inferenceMs": 582.1381250279956,
      "tokensPerSecond": 24.28690645932201,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7477821440,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5790.490292070899,
      "residentMemoryBytes": 628883456,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5208.352167042904,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 768.2614589575678,
      "inferenceMs": 769.2001659888774,
      "tokensPerSecond": 27.41981051827274,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7478411264,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5977.552333031781,
      "residentMemoryBytes": 687456256,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5208.352167042904,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 377.204124990385,
      "inferenceMs": 378.0499999993481,
      "tokensPerSecond": 21.277336149968782,
      "inputTokens": 26,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7478558720,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5586.402167042252,
      "residentMemoryBytes": 700055552,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5208.352167042904,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 524.3797079892829,
      "inferenceMs": 525.2519160276279,
      "tokensPerSecond": 26.77990311541379,
      "inputTokens": 22,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7478706176,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5733.6040830705315,
      "residentMemoryBytes": 716455936,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7626300625142021,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5208.352167042904,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 629.9519159947522,
      "inferenceMs": 630.7920420076698,
      "tokensPerSecond": 28.64953835476948,
      "inputTokens": 29,
      "outputTokens": 18,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7478706176,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5839.144209050573,
      "residentMemoryBytes": 717406208,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5208.352167042904,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 613.5167910251766,
      "inferenceMs": 614.641375024803,
      "tokensPerSecond": 26.151350643262944,
      "inputTokens": 21,
      "outputTokens": 16,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7478853632,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5822.993542067707,
      "residentMemoryBytes": 733708288,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5208.352167042904,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 513.7178750010207,
      "inferenceMs": 514.589625003282,
      "tokensPerSecond": 27.353822886532225,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7478853632,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5722.941792046186,
      "residentMemoryBytes": 734511104,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5208.352167042904,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 501.5064590261318,
      "inferenceMs": 502.36112502170727,
      "tokensPerSecond": 28.007466342005383,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7478853632,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5710.713292064611,
      "residentMemoryBytes": 735232000,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5208.352167042904,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 486.3355839625001,
      "inferenceMs": 487.34508303459734,
      "tokensPerSecond": 28.848734560929262,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7478853632,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5695.697250077501,
      "residentMemoryBytes": 735936512,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5533.444667002186,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 598.3142919722013,
      "inferenceMs": 599.1157499956898,
      "tokensPerSecond": 23.604496557481085,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7477821440,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 6132.560416997876,
      "residentMemoryBytes": 628473856,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5533.444667002186,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 713.729584007524,
      "inferenceMs": 714.5576669718139,
      "tokensPerSecond": 29.49284251272467,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7478411264,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 6248.002333974,
      "residentMemoryBytes": 686440448,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5533.444667002186,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 334.75841698236763,
      "inferenceMs": 335.6024580425583,
      "tokensPerSecond": 24.024126178893333,
      "inputTokens": 26,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7478558720,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 5869.0471250447445,
      "residentMemoryBytes": 699154432,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5533.444667002186,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 524.8386249877512,
      "inferenceMs": 525.7356250076555,
      "tokensPerSecond": 26.755961503859982,
      "inputTokens": 22,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7478706176,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 6059.180292009842,
      "residentMemoryBytes": 715522048,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7626300625142021,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5533.444667002186,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 599.5375000056811,
      "inferenceMs": 600.5027079954743,
      "tokensPerSecond": 30.105419143414366,
      "inputTokens": 29,
      "outputTokens": 18,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7478706176,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 6133.9473749976605,
      "residentMemoryBytes": 716488704,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5533.444667002186,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 583.6932079982944,
      "inferenceMs": 584.7257919958793,
      "tokensPerSecond": 27.51343859529672,
      "inputTokens": 21,
      "outputTokens": 16,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7478853632,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 6118.170458998065,
      "residentMemoryBytes": 732610560,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5533.444667002186,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 482.1305410005152,
      "inferenceMs": 483.16487501142547,
      "tokensPerSecond": 29.158966879588377,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7478853632,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 6016.609542013612,
      "residentMemoryBytes": 733396992,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5533.444667002186,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 477.67274995567277,
      "inferenceMs": 478.7277089781128,
      "tokensPerSecond": 29.376681914457688,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7478853632,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 6012.172375980299,
      "residentMemoryBytes": 734216192,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5533.444667002186,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 478.1302909832448,
      "inferenceMs": 479.18066603597254,
      "tokensPerSecond": 29.41205567723355,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7478853632,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 6012.625333038159,
      "residentMemoryBytes": 734904320,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 11265.841416025069,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 830.5907909525558,
      "inferenceMs": 831.5384170273319,
      "tokensPerSecond": 16.972629449260808,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4365647872,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 12097.3798330524,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 11265.841416025069,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 654.7289160080254,
      "inferenceMs": 655.7486249948852,
      "tokensPerSecond": 32.14018015937715,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366237696,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 11921.590041019954,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 11265.841416025069,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 246.59404199337587,
      "inferenceMs": 247.56270903162658,
      "tokensPerSecond": 32.64910215293826,
      "inputTokens": 26,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366385152,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 11513.404125056695,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 11265.841416025069,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 377.81195796560496,
      "inferenceMs": 378.651249979157,
      "tokensPerSecond": 37.20381445521205,
      "inputTokens": 22,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366532608,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 11644.492666004226,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7626300625142021,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 11265.841416025069,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 413.0434169783257,
      "inferenceMs": 413.8617910211906,
      "tokensPerSecond": 43.74527611619332,
      "inputTokens": 29,
      "outputTokens": 18,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366532608,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 11679.70320704626,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 11265.841416025069,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 429.31795900221914,
      "inferenceMs": 430.1434999797493,
      "tokensPerSecond": 37.365626114010205,
      "inputTokens": 21,
      "outputTokens": 16,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366680064,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 11695.984916004818,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5506.533750041854,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1420.5479579977691,
      "inferenceMs": 1421.3865419733338,
      "tokensPerSecond": 9.883411919568951,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6927.920292015187,
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5506.533750041854,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1936.2250409903936,
      "inferenceMs": 1936.9679170195013,
      "tokensPerSecond": 10.8478206823294,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 7443.501667061355,
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5506.533750041854,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 796.8812920153141,
      "inferenceMs": 797.682625008747,
      "tokensPerSecond": 10.042700836310381,
      "inputTokens": 26,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6304.216375050601,
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5506.533750041854,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1362.7162500051782,
      "inferenceMs": 1363.4600000223145,
      "tokensPerSecond": 10.275803483254277,
      "inputTokens": 22,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6869.993750064168,
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7626300625142021,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5506.533750041854,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1665.011958044488,
      "inferenceMs": 1665.759916009847,
      "tokensPerSecond": 10.812746603950236,
      "inputTokens": 29,
      "outputTokens": 18,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 7172.293666051701,
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5506.533750041854,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1462.9188750404865,
      "inferenceMs": 1463.6539579951204,
      "tokensPerSecond": 10.93912222041694,
      "inputTokens": 21,
      "outputTokens": 16,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6970.187708036974,
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5279.279334004968,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1341.8798329657875,
      "inferenceMs": 1342.626957979519,
      "tokensPerSecond": 10.460440903105097,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6621.906291984487,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5279.279334004968,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 2037.523582985159,
      "inferenceMs": 2038.2564999745227,
      "tokensPerSecond": 10.308815713863662,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 7317.535833979491,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5279.279334004968,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 740.0127500295639,
      "inferenceMs": 740.7512920326553,
      "tokensPerSecond": 10.81606110201872,
      "inputTokens": 26,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6020.030626037624,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5279.279334004968,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1292.0497499872,
      "inferenceMs": 1292.7978750085458,
      "tokensPerSecond": 10.838469832861005,
      "inputTokens": 22,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6572.077209013514,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7626300625142021,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5279.279334004968,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1642.42866699351,
      "inferenceMs": 1643.2602079585195,
      "tokensPerSecond": 10.962221975500949,
      "inputTokens": 29,
      "outputTokens": 18,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6922.539541963488,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5279.279334004968,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1681.7243340192363,
      "inferenceMs": 1682.458332972601,
      "tokensPerSecond": 9.515857931454187,
      "inputTokens": 21,
      "outputTokens": 16,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 6961.7376669775695,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5505.996083957143,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 706.1722920043394,
      "inferenceMs": 707.081708998885,
      "tokensPerSecond": 19.96414058253828,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4365647872,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 6213.077792956028,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5505.996083957143,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 624.4930000393651,
      "inferenceMs": 625.2789160353132,
      "tokensPerSecond": 33.684305080729054,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366237696,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 6131.274999992456,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5505.996083957143,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 236.51220800820738,
      "inferenceMs": 237.28250002022833,
      "tokensPerSecond": 33.981260399245144,
      "inputTokens": 26,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366385152,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 5743.278583977371,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5505.996083957143,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 368.4866669937037,
      "inferenceMs": 369.3971249740571,
      "tokensPerSecond": 38.104146379886785,
      "inputTokens": 22,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366532608,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 5875.3932089312,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7626300625142021,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5505.996083957143,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 396.7707910342142,
      "inferenceMs": 397.59266696637496,
      "tokensPerSecond": 45.49961420237002,
      "inputTokens": 29,
      "outputTokens": 18,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366532608,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 5903.588750923518,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5505.996083957143,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 399.5512499823235,
      "inferenceMs": 400.3573330119252,
      "tokensPerSecond": 40.188198013315045,
      "inputTokens": 21,
      "outputTokens": 16,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 4366680064,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 5906.353416969068,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6148.680582991801,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 654.540125047788,
      "inferenceMs": 655.3363329730928,
      "tokensPerSecond": 21.561073035180677,
      "inputTokens": 23,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7477821440,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 6804.016915964894,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6148.680582991801,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 759.6767920185812,
      "inferenceMs": 760.6320839840919,
      "tokensPerSecond": 27.702062762299857,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7478411264,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 6909.312666975893,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6148.680582991801,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 356.5952089847997,
      "inferenceMs": 357.3718749685213,
      "tokensPerSecond": 22.527458837035663,
      "inputTokens": 26,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7478558720,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 6506.0524579603225,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6148.680582991801,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 554.4056249782443,
      "inferenceMs": 555.145583988633,
      "tokensPerSecond": 25.297274962376353,
      "inputTokens": 22,
      "outputTokens": 14,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7478706176,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 6703.826166980434,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7626300625142021,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6148.680582991801,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 622.5916250259615,
      "inferenceMs": 623.3744579949416,
      "tokensPerSecond": 28.958631491242773,
      "inputTokens": 29,
      "outputTokens": 18,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7478706176,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 6772.055040986743,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6148.680582991801,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 605.1942909834906,
      "inferenceMs": 606.1259579728357,
      "tokensPerSecond": 26.50773986670098,
      "inputTokens": 21,
      "outputTokens": 16,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 7478853632,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 6754.806540964637,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "dbad03788f49709801014c95d481a514c272ca52",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "excellent"
    }
  }
]

HY-MT1.5 7B Desktop

138 configurations tested
Passed137
Failed1
TestedNov 13, 2026
Runtimetransformers 5.22.1

Quality criteria

{ "minimumCharacterF2": 0.55, "requireSemanticAnchors": true, "rejectRepetitionAndTruncation": true, "frozenPolicy": "research/hy-mt15-family-expansion/quality-gates.json" }

PrecisionDeviceResultPlatformInference / load
BF16mpsAcceleratedPassedFastmacos · aarch64619 ms / 14371 ms
FP16mpsAcceleratedPassedFastmacos · aarch64624 ms / 7061 ms
FP16mpsAcceleratedPassedFastmacos · aarch64624 ms / 13975 ms
FP16mpsAcceleratedPassedFastmacos · aarch64625 ms / 6786 ms
BF16mpsAcceleratedPassedFastmacos · aarch64625 ms / 6494 ms
BF16mpsAcceleratedPassedFastmacos · aarch64634 ms / 14750 ms
BF16mpsAcceleratedPassedFastmacos · aarch64644 ms / 6628 ms
FP16mpsAcceleratedPassedFastmacos · aarch64648 ms / 7149 ms
FP16mpsAcceleratedPassedFastmacos · aarch64732 ms / 13975 ms
FP16mpsAcceleratedPassedFastmacos · aarch64740 ms / 6786 ms
FP16mpsAcceleratedPassedFastmacos · aarch64752 ms / 7149 ms
FP16mpsAcceleratedPassedFastmacos · aarch64753 ms / 7061 ms
BF16mpsAcceleratedPassedFastmacos · aarch64802 ms / 6494 ms
BF16mpsAcceleratedPassedFastmacos · aarch64807 ms / 14750 ms
FP16mpsAcceleratedPassedFastmacos · aarch64809 ms / 6786 ms
FP16mpsAcceleratedPassedFastmacos · aarch64812 ms / 7149 ms
BF16mpsAcceleratedPassedFastmacos · aarch64812 ms / 6494 ms
FP16mpsAcceleratedPassedFastmacos · aarch64812 ms / 13975 ms
FP16mpsAcceleratedPassedFastmacos · aarch64812 ms / 6786 ms
FP16mpsAcceleratedPassedFastmacos · aarch64812 ms / 13975 ms
FP16mpsAcceleratedPassedFastmacos · aarch64813 ms / 13975 ms
FP16mpsAcceleratedPassedFastmacos · aarch64815 ms / 6786 ms
BF16mpsAcceleratedPassedFastmacos · aarch64816 ms / 14371 ms
BF16mpsAcceleratedPassedFastmacos · aarch64816 ms / 14750 ms
BF16mpsAcceleratedPassedFastmacos · aarch64817 ms / 6628 ms
FP16mpsAcceleratedPassedFastmacos · aarch64817 ms / 7149 ms
BF16mpsAcceleratedPassedFastmacos · aarch64818 ms / 14750 ms
BF16mpsAcceleratedPassedFastmacos · aarch64818 ms / 14750 ms
BF16mpsAcceleratedPassedFastmacos · aarch64821 ms / 6494 ms
BF16mpsAcceleratedPassedFastmacos · aarch64822 ms / 6494 ms
BF16mpsAcceleratedPassedFastmacos · aarch64823 ms / 6628 ms
FP16mpsAcceleratedPassedFastmacos · aarch64824 ms / 7149 ms
BF16mpsAcceleratedPassedFastmacos · aarch64825 ms / 6628 ms
BF16mpsAcceleratedPassedFastmacos · aarch64862 ms / 6628 ms
FP16mpsAcceleratedPassedFastmacos · aarch64930 ms / 13975 ms
FP16mpsAcceleratedPassedFastmacos · aarch64954 ms / 7149 ms
BF16mpsAcceleratedPassedFastmacos · aarch64997 ms / 6628 ms
FP16mpsAcceleratedPassedFastmacos · aarch641001 ms / 6786 ms
BF16mpsAcceleratedPassedFastmacos · aarch641003 ms / 6494 ms
FP16mpsAcceleratedPassedFastmacos · aarch641050 ms / 7061 ms
FP32mpsAcceleratedPassedFastmacos · aarch641059 ms / 21440 ms
BF16mpsAcceleratedPassedFastmacos · aarch641072 ms / 14371 ms
FP16mpsAcceleratedPassedFastmacos · aarch641097 ms / 13975 ms
FP16mpsAcceleratedPassedFastmacos · aarch641101 ms / 6786 ms
FP16mpsAcceleratedPassedFastmacos · aarch641102 ms / 7061 ms
BF16mpsAcceleratedPassedFastmacos · aarch641102 ms / 14750 ms
FP16mpsAcceleratedPassedFastmacos · aarch641105 ms / 7149 ms
BF16mpsAcceleratedPassedFastmacos · aarch641107 ms / 14750 ms
BF16mpsAcceleratedPassedFastmacos · aarch641109 ms / 14371 ms
BF16mpsAcceleratedPassedFastmacos · aarch641111 ms / 6494 ms
BF16mpsAcceleratedPassedFastmacos · aarch641131 ms / 6628 ms
FP32mpsAcceleratedPassedFastmacos · aarch641285 ms / 21440 ms
BF16mpsAcceleratedPassedFastmacos · aarch641366 ms / 6494 ms
FP16mpsAcceleratedPassedFastmacos · aarch641367 ms / 7061 ms
FP16mpsAcceleratedPassedFastmacos · aarch641369 ms / 13975 ms
BF16mpsAcceleratedPassedFastmacos · aarch641372 ms / 14750 ms
FP16mpsAcceleratedPassedFastmacos · aarch641381 ms / 7149 ms
FP16mpsAcceleratedPassedFastmacos · aarch641388 ms / 6786 ms
BF16mpsAcceleratedPassedFastmacos · aarch641401 ms / 6628 ms
BF16mpsAcceleratedPassedFastmacos · aarch641416 ms / 14371 ms
FP16mpsAcceleratedPassedFastmacos · aarch641548 ms / 7061 ms
FP16mpsAcceleratedPassedFastmacos · aarch641548 ms / 13975 ms
BF16mpsAcceleratedPassedFastmacos · aarch641553 ms / 14371 ms
FP16mpsAcceleratedPassedFastmacos · aarch641554 ms / 6786 ms
FP16mpsAcceleratedPassedFastmacos · aarch641555 ms / 7149 ms
BF16mpsAcceleratedPassedFastmacos · aarch641568 ms / 6494 ms
BF16mpsAcceleratedPassedFastmacos · aarch641571 ms / 14750 ms
BF16mpsAcceleratedPassedFastmacos · aarch641682 ms / 6628 ms
FP32mpsAcceleratedPassedFastmacos · aarch641991 ms / 21440 ms
FP32mpsAcceleratedPassedFastmacos · aarch642447 ms / 21440 ms
FP32mpsAcceleratedPassedFastmacos · aarch642801 ms / 21440 ms
FP32cpuPassedModeratemacos · aarch642611 ms / 13188 ms
FP32cpuPassedModeratemacos · aarch642655 ms / 13543 ms
FP32cpuPassedModeratemacos · aarch642667 ms / 16399 ms
FP32cpuPassedModeratemacos · aarch642695 ms / 7213 ms
FP32cpuPassedModeratemacos · aarch642707 ms / 5703 ms
FP32cpuPassedModeratemacos · aarch642731 ms / 16329 ms
FP32cpuPassedModeratemacos · aarch642838 ms / 14972 ms
FP32cpuPassedModeratemacos · aarch643091 ms / 14643 ms
FP32cpuPassedModeratemacos · aarch643176 ms / 5703 ms
FP32cpuPassedModeratemacos · aarch643254 ms / 13543 ms
FP32cpuPassedModeratemacos · aarch643268 ms / 16399 ms
FP32cpuPassedModeratemacos · aarch643314 ms / 7213 ms
FP32cpuPassedModeratemacos · aarch643320 ms / 13188 ms
FP32cpuPassedModeratemacos · aarch643443 ms / 16329 ms
FP32cpuPassedModeratemacos · aarch643591 ms / 14972 ms
FP32cpuPassedModeratemacos · aarch643748 ms / 13543 ms
FP32cpuPassedModeratemacos · aarch643756 ms / 5703 ms
FP32cpuPassedModeratemacos · aarch643759 ms / 13188 ms
FP32cpuPassedModeratemacos · aarch643767 ms / 13188 ms
FP32cpuPassedModeratemacos · aarch643778 ms / 16399 ms
FP32cpuPassedModeratemacos · aarch643784 ms / 16399 ms
FP32cpuPassedModeratemacos · aarch643792 ms / 13543 ms
FP32cpuPassedModeratemacos · aarch643797 ms / 5703 ms
FP32cpuPassedModeratemacos · aarch643824 ms / 13543 ms
FP32cpuPassedModeratemacos · aarch643863 ms / 14643 ms
FP32cpuPassedModeratemacos · aarch643879 ms / 16399 ms
FP32cpuPassedModeratemacos · aarch643887 ms / 14643 ms
FP32cpuPassedModeratemacos · aarch643902 ms / 13188 ms
FP32cpuPassedModeratemacos · aarch643964 ms / 5703 ms
FP32cpuPassedModeratemacos · aarch643978 ms / 16329 ms
FP32cpuPassedModeratemacos · aarch643987 ms / 14643 ms
FP32cpuPassedModeratemacos · aarch644073 ms / 5703 ms
FP32cpuPassedModeratemacos · aarch644096 ms / 14643 ms
FP32cpuPassedModeratemacos · aarch644121 ms / 16329 ms
FP32cpuPassedModeratemacos · aarch644144 ms / 16329 ms
FP32cpuPassedModeratemacos · aarch644209 ms / 13543 ms
FP32cpuPassedModeratemacos · aarch644258 ms / 13188 ms
FP32cpuPassedModeratemacos · aarch644564 ms / 14972 ms
FP32cpuPassedModeratemacos · aarch644737 ms / 14643 ms
FP32cpuPassedModeratemacos · aarch644817 ms / 16399 ms
FP32cpuPassedModeratemacos · aarch645162 ms / 13188 ms
FP32cpuPassedModeratemacos · aarch645254 ms / 13543 ms
FP32cpuPassedModeratemacos · aarch645260 ms / 7213 ms
FP32cpuPassedModeratemacos · aarch645328 ms / 16329 ms
FP32cpuPassedModeratemacos · aarch645346 ms / 16329 ms
FP32cpuPassedModeratemacos · aarch645374 ms / 16399 ms
FP32cpuPassedModeratemacos · aarch645496 ms / 7213 ms
FP32cpuPassedModeratemacos · aarch645609 ms / 14972 ms
FP32cpuPassedModeratemacos · aarch645642 ms / 5703 ms
FP32cpuPassedModeratemacos · aarch645800 ms / 14643 ms
FP32cpuPassedModeratemacos · aarch646363 ms / 13188 ms
FP32cpuPassedModeratemacos · aarch646543 ms / 13543 ms
FP32cpuPassedModeratemacos · aarch646612 ms / 16329 ms
FP32cpuPassedModeratemacos · aarch646804 ms / 5703 ms
FP32cpuPassedModeratemacos · aarch646937 ms / 16399 ms
FP32cpuPassedModeratemacos · aarch646965 ms / 7213 ms
FP32cpuPassedModeratemacos · aarch646971 ms / 14972 ms
FP32cpuPassedModeratemacos · aarch647080 ms / 14643 ms
FP32cpuPassedModeratemacos · aarch647263 ms / 13188 ms
FP32cpuPassedModeratemacos · aarch647421 ms / 14643 ms
FP32cpuPassedModeratemacos · aarch647438 ms / 16399 ms
FP32cpuPassedModeratemacos · aarch647468 ms / 13543 ms
FP32cpuPassedModeratemacos · aarch647527 ms / 7213 ms
FP32cpuPassedModeratemacos · aarch647592 ms / 16329 ms
FP32cpuPassedModeratemacos · aarch647612 ms / 5703 ms
FP32cpuPassedModeratemacos · aarch647639 ms / 14972 ms
FP32mpsFailed—macos · aarch642511 ms / 21440 ms
Test setup & sample details
Quality gate
{ "minimumCharacterF2": 0.55, "requireSemanticAnchors": true, "rejectRepetitionAndTruncation": true, "frozenPolicy": "research/hy-mt15-family-expansion/quality-gates.json" }
Tested
2026-11-13T09:50:29.956212+00:00
Environment
{ "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2", "runtime": "App-owned CPython3.13.18; Transformers5.22.1/Torch2.10.0; built-in HunYuanDenseV1ForCausalLM, SDPA; MPS CPU op fallback disabled", "concurrentUploadActivity": "none; uploads excluded", "transport": "Verified exact local original snapshots, not Internet transfer timing" }
Runtime
transformers 5.22.1
Artifact revision
397085d46158ad0b437ffd74228499e8c2a35c76
Fixture
{
  "file": "model-samples/hunyuan-translation.json",
  "url": "https://assets.localai.tools/samples/v1/files/b63cba0b5f61f8423d4234d4d28e9cdde538b8345b4ccdc8b2044b1bc88e2fa8/hunyuan-translation.json",
  "size": 3099,
  "sha1": "dd6e22e1c2966dd5a249b11eb2fb48b076505088",
  "sha256": "b63cba0b5f61f8423d4234d4d28e9cdde538b8345b4ccdc8b2044b1bc88e2fa8"
}
Rerun when
model artifact revision change; runtime version change; fixture or quality-gate change; backend/platform behavior change
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14643.35041603772,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 4736.64562497288,
      "inferenceMs": 4737.382166960742,
      "tokensPerSecond": 2.5353305350718878,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 19380.732582998462,
      "residentMemoryBytes": 30583832576,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14643.35041603772,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 7079.10550001543,
      "inferenceMs": 7079.876542033162,
      "tokensPerSecond": 2.9666150387476358,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 21723.22695807088,
      "residentMemoryBytes": 30589140992,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14643.35041603772,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3090.144124988001,
      "inferenceMs": 3091.0674589686096,
      "tokensPerSecond": 2.58922398449713,
      "inputTokens": 25,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 17734.41787500633,
      "residentMemoryBytes": 30589599744,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7380931729310785,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14643.35041603772,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 4094.9658330064267,
      "inferenceMs": 4095.905166992452,
      "tokensPerSecond": 2.44233355325552,
      "inputTokens": 21,
      "outputTokens": 10,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 18739.25558303017,
      "residentMemoryBytes": 30589648896,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7613222236973526,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14643.35041603772,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 5798.689000017475,
      "inferenceMs": 5799.518915999215,
      "tokensPerSecond": 2.931896212061948,
      "inputTokens": 29,
      "outputTokens": 17,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 20442.869332036935,
      "residentMemoryBytes": 30590025728,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14643.35041603772,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 7419.755999988411,
      "inferenceMs": 7420.618749980349,
      "tokensPerSecond": 3.2347895870971493,
      "inputTokens": 20,
      "outputTokens": 24,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 22063.96916601807,
      "residentMemoryBytes": 30590074880,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14643.35041603772,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3985.92175001977,
      "inferenceMs": 3986.69987503672,
      "tokensPerSecond": 3.010862470703743,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 18630.05029107444,
      "residentMemoryBytes": 30590074880,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14643.35041603772,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3862.4612080166116,
      "inferenceMs": 3863.2483340334147,
      "tokensPerSecond": 3.107049674281603,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 18506.598750071134,
      "residentMemoryBytes": 30590074880,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14643.35041603772,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3886.4384579937905,
      "inferenceMs": 3887.221042008605,
      "tokensPerSecond": 3.0878979809913436,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 18530.571458046325,
      "residentMemoryBytes": 30590074880,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 16328.98649998242,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 5345.375541015528,
      "inferenceMs": 5346.141750051174,
      "tokensPerSecond": 2.2468486541984154,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 21675.128250033595,
      "residentMemoryBytes": 30594039808,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 16328.98649998242,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 6611.507290974259,
      "inferenceMs": 6612.341791973449,
      "tokensPerSecond": 3.176453124564577,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 22941.32829195587,
      "residentMemoryBytes": 30601330688,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 16328.98649998242,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 2729.684541991446,
      "inferenceMs": 2730.5405419901945,
      "tokensPerSecond": 2.931110252408365,
      "inputTokens": 25,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 19059.527041972615,
      "residentMemoryBytes": 30601510912,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7380931729310785,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 16328.98649998242,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3441.9345410424285,
      "inferenceMs": 3442.737208039034,
      "tokensPerSecond": 2.9056117823504453,
      "inputTokens": 21,
      "outputTokens": 10,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 19771.723708021455,
      "residentMemoryBytes": 30602493952,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7613222236973526,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 16328.98649998242,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 5327.264249965083,
      "inferenceMs": 5328.064290981274,
      "tokensPerSecond": 3.1913661659900407,
      "inputTokens": 29,
      "outputTokens": 17,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 21657.050790963694,
      "residentMemoryBytes": 30602838016,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 16328.98649998242,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 7591.606832982507,
      "inferenceMs": 7592.434207967017,
      "tokensPerSecond": 3.1615230525309155,
      "inputTokens": 20,
      "outputTokens": 24,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 23921.420707949437,
      "residentMemoryBytes": 30602821632,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 16328.98649998242,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 4120.128333044704,
      "inferenceMs": 4121.082666970324,
      "tokensPerSecond": 2.9128469811524864,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 20450.069166952744,
      "residentMemoryBytes": 30606082048,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 16328.98649998242,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 4142.697083996609,
      "inferenceMs": 4143.56300001964,
      "tokensPerSecond": 2.896872952389996,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 20472.54950000206,
      "residentMemoryBytes": 30606098432,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 16328.98649998242,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3977.4257910321467,
      "inferenceMs": 3978.2299159560353,
      "tokensPerSecond": 3.0172594785409443,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 20307.216415938456,
      "residentMemoryBytes": 30606114816,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 16398.717332980596,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 4816.455000021961,
      "inferenceMs": 4817.4360420089215,
      "tokensPerSecond": 2.4940373459185805,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 21216.153374989517,
      "residentMemoryBytes": 30561910784,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 16398.717332980596,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 6936.230708961375,
      "inferenceMs": 6937.038542004302,
      "tokensPerSecond": 3.027736480994778,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 23335.755874984898,
      "residentMemoryBytes": 30572199936,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 16398.717332980596,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 2666.007207997609,
      "inferenceMs": 2666.7731249472126,
      "tokensPerSecond": 3.001066317136659,
      "inputTokens": 25,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 19065.490457927808,
      "residentMemoryBytes": 30572314624,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7380931729310785,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 16398.717332980596,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3267.5524589722045,
      "inferenceMs": 3268.3158749714494,
      "tokensPerSecond": 3.060631804692108,
      "inputTokens": 21,
      "outputTokens": 10,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 19667.033207952045,
      "residentMemoryBytes": 30572298240,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7613222236973526,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 16398.717332980596,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 5373.106334009208,
      "inferenceMs": 5373.982458957471,
      "tokensPerSecond": 3.1641011017791683,
      "inputTokens": 29,
      "outputTokens": 17,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 21772.699791938066,
      "residentMemoryBytes": 30572314624,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 16398.717332980596,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 7436.94783304818,
      "inferenceMs": 7437.779209052678,
      "tokensPerSecond": 3.2272976847853783,
      "inputTokens": 20,
      "outputTokens": 24,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 23836.496542033274,
      "residentMemoryBytes": 30572347392,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 16398.717332980596,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3783.2388340029866,
      "inferenceMs": 3784.0559579781257,
      "tokensPerSecond": 3.1721187678391494,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 20182.77329095872,
      "residentMemoryBytes": 30572363776,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 16398.717332980596,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3777.0158330095,
      "inferenceMs": 3777.7542080148123,
      "tokensPerSecond": 3.177331294302294,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 20176.471540995408,
      "residentMemoryBytes": 30572380160,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 16398.717332980596,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3877.921375038568,
      "inferenceMs": 3878.6919999984093,
      "tokensPerSecond": 3.0946441463492493,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 20277.409332979005,
      "residentMemoryBytes": 30572396544,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 13543.445665971376,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 4208.0137499724515,
      "inferenceMs": 4208.732833038084,
      "tokensPerSecond": 2.8543154707815748,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 17752.17849900946,
      "residentMemoryBytes": 30575886336,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 13543.445665971376,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 6542.252958985046,
      "inferenceMs": 6543.048000021372,
      "tokensPerSecond": 3.2100408443683577,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 20086.493665992748,
      "residentMemoryBytes": 30584406016,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 13543.445665971376,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 2654.4713340117596,
      "inferenceMs": 2655.2497089724056,
      "tokensPerSecond": 3.014114722475746,
      "inputTokens": 25,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 16198.695374943782,
      "residentMemoryBytes": 30584487936,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7380931729310785,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 13543.445665971376,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3252.9463329701684,
      "inferenceMs": 3253.725249960553,
      "tokensPerSecond": 3.074417308568935,
      "inputTokens": 21,
      "outputTokens": 10,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 16797.17091593193,
      "residentMemoryBytes": 30584619008,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7613222236973526,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 13543.445665971376,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 5252.987666986883,
      "inferenceMs": 5253.785000008065,
      "tokensPerSecond": 3.2364442150910633,
      "inputTokens": 29,
      "outputTokens": 17,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 18797.23066597944,
      "residentMemoryBytes": 30584635392,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 13543.445665971376,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 7466.854624974076,
      "inferenceMs": 7467.611791973468,
      "tokensPerSecond": 3.2143165659826036,
      "inputTokens": 20,
      "outputTokens": 24,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 21011.057457944844,
      "residentMemoryBytes": 30586437632,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 13543.445665971376,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3790.886834030971,
      "inferenceMs": 3791.644707962405,
      "tokensPerSecond": 3.165693311502654,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 17335.09037393378,
      "residentMemoryBytes": 30586454016,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 13543.445665971376,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3747.506042011082,
      "inferenceMs": 3748.306291992776,
      "tokensPerSecond": 3.2023454336485337,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 17291.751957964152,
      "residentMemoryBytes": 30586454016,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 13543.445665971376,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3823.5879170242697,
      "inferenceMs": 3824.393374961801,
      "tokensPerSecond": 3.138625435081547,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 17367.839040933177,
      "residentMemoryBytes": 30586486784,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 13187.68395902589,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 4257.692749961279,
      "inferenceMs": 4258.468667045236,
      "tokensPerSecond": 2.820844797173472,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 17446.152626071125,
      "residentMemoryBytes": 30576836608,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 13187.68395902589,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 6362.028416013345,
      "inferenceMs": 6362.837957974989,
      "tokensPerSecond": 3.300993422183347,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 19550.52191700088,
      "residentMemoryBytes": 30585356288,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 13187.68395902589,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 2610.004917019978,
      "inferenceMs": 2610.765582998283,
      "tokensPerSecond": 3.0654595844773906,
      "inputTokens": 25,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 15798.449542024173,
      "residentMemoryBytes": 30585470976,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7380931729310785,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 13187.68395902589,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3319.3914580042474,
      "inferenceMs": 3320.153374981601,
      "tokensPerSecond": 3.0128271864800493,
      "inputTokens": 21,
      "outputTokens": 10,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 16507.83733400749,
      "residentMemoryBytes": 30585454592,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7613222236973526,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 13187.68395902589,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 5161.509750003461,
      "inferenceMs": 5162.286750040948,
      "tokensPerSecond": 3.2937932027297907,
      "inputTokens": 29,
      "outputTokens": 17,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 18349.970709066838,
      "residentMemoryBytes": 30585487360,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 13187.68395902589,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 7262.650666991249,
      "inferenceMs": 7263.428167032544,
      "tokensPerSecond": 3.3046945215034813,
      "inputTokens": 20,
      "outputTokens": 24,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 20451.112126058433,
      "residentMemoryBytes": 30585651200,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 13187.68395902589,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3757.741999987047,
      "inferenceMs": 3758.5494999657385,
      "tokensPerSecond": 3.193613659260168,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 16946.233458991628,
      "residentMemoryBytes": 30586617856,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 13187.68395902589,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3900.912708020769,
      "inferenceMs": 3901.7003750195727,
      "tokensPerSecond": 3.076407121640068,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 17089.384334045462,
      "residentMemoryBytes": 30586667008,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 13187.68395902589,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3766.034291998949,
      "inferenceMs": 3766.826167004183,
      "tokensPerSecond": 3.186598899368618,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 16954.510126030073,
      "residentMemoryBytes": 30586667008,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5703.028291987721,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 4072.6116249570623,
      "inferenceMs": 4073.330375016667,
      "tokensPerSecond": 2.949155816427165,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 9776.358667004388,
      "residentMemoryBytes": 30575460352,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5703.028291987721,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 6802.5834579602815,
      "inferenceMs": 6803.526541043539,
      "tokensPerSecond": 3.0872136002952653,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 12506.55483303126,
      "residentMemoryBytes": 30590566400,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5703.028291987721,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 2706.3983340049163,
      "inferenceMs": 2707.2866249945946,
      "tokensPerSecond": 2.9564776915384905,
      "inputTokens": 25,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 8410.314916982315,
      "residentMemoryBytes": 30590894080,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7380931729310785,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5703.028291987721,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3174.877249985002,
      "inferenceMs": 3175.6475000292994,
      "tokensPerSecond": 3.1499888864538876,
      "inputTokens": 21,
      "outputTokens": 10,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 8878.67579201702,
      "residentMemoryBytes": 30590910464,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7613222236973526,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5703.028291987721,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 5641.4926670258865,
      "inferenceMs": 5642.3134169890545,
      "tokensPerSecond": 3.013610150561966,
      "inputTokens": 29,
      "outputTokens": 17,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 11345.341708976775,
      "residentMemoryBytes": 30592253952,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5703.028291987721,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 7611.619082977995,
      "inferenceMs": 7612.41208401043,
      "tokensPerSecond": 3.153201445442238,
      "inputTokens": 20,
      "outputTokens": 24,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 13315.44037599815,
      "residentMemoryBytes": 30595137536,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5703.028291987721,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3963.0806670174934,
      "inferenceMs": 3963.8495839899406,
      "tokensPerSecond": 3.0281536590486993,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 9666.877875977661,
      "residentMemoryBytes": 30595137536,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5703.028291987721,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3755.355665984098,
      "inferenceMs": 3756.121499987785,
      "tokensPerSecond": 3.195646393696911,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 9459.149791975506,
      "residentMemoryBytes": 30595268608,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5703.028291987721,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3796.069457952399,
      "inferenceMs": 3796.8450839980505,
      "tokensPerSecond": 3.161377035232006,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 9499.873375985771,
      "residentMemoryBytes": 30595268608,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14750.21604099311,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1100.6679159472696,
      "inferenceMs": 1101.7945000203326,
      "tokensPerSecond": 10.954121900291895,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15088369664,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 15852.010541013442,
      "residentMemoryBytes": 709066752,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14750.21604099311,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1370.7772920024581,
      "inferenceMs": 1371.9755409983918,
      "tokensPerSecond": 15.343869352091632,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089172480,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 16122.191581991501,
      "residentMemoryBytes": 779190272,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14750.21604099311,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 632.7812920208089,
      "inferenceMs": 633.5513329831883,
      "tokensPerSecond": 12.683743345368104,
      "inputTokens": 25,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089385472,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 15383.767373976298,
      "residentMemoryBytes": 792657920,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7376087954634903,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14750.21604099311,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 805.7605000212789,
      "inferenceMs": 806.5663749584928,
      "tokensPerSecond": 13.684004610993481,
      "inputTokens": 21,
      "outputTokens": 11,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089598464,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 15556.782415951602,
      "residentMemoryBytes": 809517056,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7613222236973526,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14750.21604099311,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1105.965540977195,
      "inferenceMs": 1107.1145420428365,
      "tokensPerSecond": 15.391719840726267,
      "inputTokens": 29,
      "outputTokens": 17,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089598464,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 15857.330583035946,
      "residentMemoryBytes": 810893312,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14750.21604099311,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1569.7225000476465,
      "inferenceMs": 1570.6152910133824,
      "tokensPerSecond": 15.310607978720878,
      "inputTokens": 20,
      "outputTokens": 24,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089795072,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 16320.831332006492,
      "residentMemoryBytes": 828964864,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14750.21604099311,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 815.1705419877544,
      "inferenceMs": 816.2963750073686,
      "tokensPerSecond": 14.749428722001607,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089795072,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 15566.512416000478,
      "residentMemoryBytes": 829915136,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14750.21604099311,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 817.4676250200719,
      "inferenceMs": 818.2562080328353,
      "tokensPerSecond": 14.71498378652433,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089795072,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 15568.472249025945,
      "residentMemoryBytes": 830881792,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14750.21604099311,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 817.3582500312477,
      "inferenceMs": 818.4575829654932,
      "tokensPerSecond": 14.718565698217962,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089795072,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 15568.673623958603,
      "residentMemoryBytes": 831946752,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6493.908666016068,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1001.9266249728389,
      "inferenceMs": 1002.6766659575514,
      "tokensPerSecond": 12.03455723083486,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15088369664,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 7496.58533197362,
      "residentMemoryBytes": 689127424,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6493.908666016068,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1364.6453329711221,
      "inferenceMs": 1365.840542013757,
      "tokensPerSecond": 15.406880584288151,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089172480,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 7859.749208029825,
      "residentMemoryBytes": 758874112,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6493.908666016068,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 624.3911670171656,
      "inferenceMs": 625.1689579803497,
      "tokensPerSecond": 12.856627437454712,
      "inputTokens": 25,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089385472,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 7119.077623996418,
      "residentMemoryBytes": 772161536,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7376087954634903,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6493.908666016068,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 800.9874589624815,
      "inferenceMs": 802.150083007291,
      "tokensPerSecond": 13.766501147529143,
      "inputTokens": 21,
      "outputTokens": 11,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089598464,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 7296.058749023359,
      "residentMemoryBytes": 788971520,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7613222236973526,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6493.908666016068,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1109.4199999934062,
      "inferenceMs": 1110.5947499745525,
      "tokensPerSecond": 15.353086563860014,
      "inputTokens": 29,
      "outputTokens": 17,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089598464,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 7604.503415990621,
      "residentMemoryBytes": 790380544,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6493.908666016068,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1566.3357919547707,
      "inferenceMs": 1567.568083992228,
      "tokensPerSecond": 15.344786152979676,
      "inputTokens": 20,
      "outputTokens": 24,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089795072,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 8061.476750008296,
      "residentMemoryBytes": 808189952,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6493.908666016068,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 820.0086249853484,
      "inferenceMs": 820.7826249999925,
      "tokensPerSecond": 14.67297156073545,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089795072,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 7314.691291016061,
      "residentMemoryBytes": 809238528,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6493.908666016068,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 810.6508330092765,
      "inferenceMs": 811.8872919585556,
      "tokensPerSecond": 14.839482413745237,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089795072,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 7305.795957974624,
      "residentMemoryBytes": 810188800,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6493.908666016068,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 820.7072909572162,
      "inferenceMs": 822.0411249785684,
      "tokensPerSecond": 14.659235389816,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089795072,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 7315.949790994637,
      "residentMemoryBytes": 811237376,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6627.832625003066,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 996.2332919822074,
      "inferenceMs": 997.3602080135606,
      "tokensPerSecond": 12.105075578699417,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15088369664,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 7625.1928330166265,
      "residentMemoryBytes": 688488448,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6627.832625003066,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1400.1311250030994,
      "inferenceMs": 1401.1626670253463,
      "tokensPerSecond": 15.022719631505524,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089172480,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 8028.995292028412,
      "residentMemoryBytes": 757628928,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6627.832625003066,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 643.0982079473324,
      "inferenceMs": 644.205792050343,
      "tokensPerSecond": 12.483144497350485,
      "inputTokens": 25,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089385472,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 7272.038417053409,
      "residentMemoryBytes": 770752512,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7376087954634903,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6627.832625003066,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 821.7291660257615,
      "inferenceMs": 822.8254579589702,
      "tokensPerSecond": 13.422626337944987,
      "inputTokens": 21,
      "outputTokens": 11,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089598464,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 7450.658082962036,
      "residentMemoryBytes": 787382272,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7613222236973526,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6627.832625003066,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1130.3457079920918,
      "inferenceMs": 1131.4977079746313,
      "tokensPerSecond": 15.071065053667711,
      "inputTokens": 29,
      "outputTokens": 17,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089598464,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 7759.330332977697,
      "residentMemoryBytes": 788774912,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6627.832625003066,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1680.7207079837099,
      "inferenceMs": 1681.6629580571316,
      "tokensPerSecond": 14.299832527501323,
      "inputTokens": 20,
      "outputTokens": 24,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089795072,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 8309.495583060198,
      "residentMemoryBytes": 806551552,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6627.832625003066,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 861.4235839922912,
      "inferenceMs": 862.3916250071488,
      "tokensPerSecond": 13.96106616370498,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089795072,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 7490.224250010215,
      "residentMemoryBytes": 807567360,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6627.832625003066,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 823.9528750418685,
      "inferenceMs": 825.100084010046,
      "tokensPerSecond": 14.600716790325444,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089795072,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 7452.932709013112,
      "residentMemoryBytes": 808550400,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6627.832625003066,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 815.3278750251047,
      "inferenceMs": 816.5720410179347,
      "tokensPerSecond": 14.760097675191803,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089795072,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 7444.404666021001,
      "residentMemoryBytes": 809549824,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6785.672083962709,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1000.2681670011953,
      "inferenceMs": 1001.413166988641,
      "tokensPerSecond": 12.056788486566065,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15088369664,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 7787.08525095135,
      "residentMemoryBytes": 698220544,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6785.672083962709,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1386.9164580246434,
      "inferenceMs": 1388.1149999797344,
      "tokensPerSecond": 15.165191813070155,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089172480,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 8173.787083942443,
      "residentMemoryBytes": 768245760,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6785.672083962709,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 623.7327080452815,
      "inferenceMs": 624.5209580520168,
      "tokensPerSecond": 12.870833344778635,
      "inputTokens": 25,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089385472,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 7410.1930420147255,
      "residentMemoryBytes": 781647872,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7380931729310785,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6785.672083962709,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 739.0788340126164,
      "inferenceMs": 739.9030829546973,
      "tokensPerSecond": 13.566047573929175,
      "inputTokens": 21,
      "outputTokens": 10,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089598464,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 7525.575166917406,
      "residentMemoryBytes": 798474240,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7613222236973526,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6785.672083962709,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1100.000916980207,
      "inferenceMs": 1101.1598749901168,
      "tokensPerSecond": 15.482541887986288,
      "inputTokens": 29,
      "outputTokens": 17,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089598464,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 7886.8319589528255,
      "residentMemoryBytes": 799817728,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6785.672083962709,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1553.4557909704745,
      "inferenceMs": 1554.4695829739794,
      "tokensPerSecond": 15.470982222115397,
      "inputTokens": 20,
      "outputTokens": 24,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089795072,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 8340.141666936688,
      "residentMemoryBytes": 817774592,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6785.672083962709,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 807.5257909949869,
      "inferenceMs": 808.6852499982342,
      "tokensPerSecond": 14.897707072694653,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089795072,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 7594.357333960943,
      "residentMemoryBytes": 818757632,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6785.672083962709,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 814.2863340326585,
      "inferenceMs": 815.3080830234103,
      "tokensPerSecond": 14.774520811406727,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089795072,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 7600.980166986119,
      "residentMemoryBytes": 819724288,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 6785.672083962709,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 811.467542022001,
      "inferenceMs": 812.4139999854378,
      "tokensPerSecond": 14.826727302693396,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089795072,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 7598.086083948147,
      "residentMemoryBytes": 820723712,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 13974.678416969255,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 928.8477079826407,
      "inferenceMs": 929.9632499460131,
      "tokensPerSecond": 12.99955765830744,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15088369664,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 14904.641666915268,
      "residentMemoryBytes": 691339264,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 13974.678416969255,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1367.747749958653,
      "inferenceMs": 1368.9910419634543,
      "tokensPerSecond": 15.37766774301404,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089172480,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 15343.669458932709,
      "residentMemoryBytes": 760201216,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 13974.678416969255,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 623.4392080223188,
      "inferenceMs": 624.2779580061324,
      "tokensPerSecond": 12.87897990121978,
      "inputTokens": 25,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089385472,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 14598.956374975387,
      "residentMemoryBytes": 773193728,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7380931729310785,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 13974.678416969255,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 731.5755839808844,
      "inferenceMs": 732.3209579917602,
      "tokensPerSecond": 13.699513737248424,
      "inputTokens": 21,
      "outputTokens": 10,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089598464,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 14706.999374961015,
      "residentMemoryBytes": 789676032,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7613222236973526,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 13974.678416969255,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1096.1347909760661,
      "inferenceMs": 1097.3373330198228,
      "tokensPerSecond": 15.541776032038864,
      "inputTokens": 29,
      "outputTokens": 17,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089598464,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 15072.015749989077,
      "residentMemoryBytes": 791068672,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 13974.678416969255,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1546.990457980428,
      "inferenceMs": 1548.2812499976717,
      "tokensPerSecond": 15.536445135481786,
      "inputTokens": 20,
      "outputTokens": 24,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089795072,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 15522.959666966926,
      "residentMemoryBytes": 808747008,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 13974.678416969255,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 811.4250000216998,
      "inferenceMs": 812.454583006911,
      "tokensPerSecond": 14.826804391977772,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089795072,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 14787.132999976166,
      "residentMemoryBytes": 809730048,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 13974.678416969255,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 812.0097079663537,
      "inferenceMs": 812.9156249924563,
      "tokensPerSecond": 14.81748485315118,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089795072,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 14787.59404196171,
      "residentMemoryBytes": 810696704,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 13974.678416969255,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 811.4127499866299,
      "inferenceMs": 812.3217500397004,
      "tokensPerSecond": 14.828934340186464,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089795072,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 14787.000167008955,
      "residentMemoryBytes": 811679744,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7149.194958037697,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 953.1380420085043,
      "inferenceMs": 954.2899999651127,
      "tokensPerSecond": 12.655754208659385,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15088369664,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 8103.484958002809,
      "residentMemoryBytes": 697139200,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7149.194958037697,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1379.8312500002794,
      "inferenceMs": 1380.7465419522487,
      "tokensPerSecond": 15.243624887219482,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089172480,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 8529.941499989945,
      "residentMemoryBytes": 767033344,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7149.194958037697,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 647.2411670256406,
      "inferenceMs": 648.1717500137165,
      "tokensPerSecond": 12.402534334285262,
      "inputTokens": 25,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089385472,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 7797.366708051413,
      "residentMemoryBytes": 780500992,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7380931729310785,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7149.194958037697,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 750.6643750239164,
      "inferenceMs": 751.8726249691099,
      "tokensPerSecond": 13.350525838317095,
      "inputTokens": 21,
      "outputTokens": 10,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089598464,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 7901.067583006807,
      "residentMemoryBytes": 797310976,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7613222236973526,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7149.194958037697,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1103.6318749538623,
      "inferenceMs": 1104.7870829934254,
      "tokensPerSecond": 15.430885922360698,
      "inputTokens": 29,
      "outputTokens": 17,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089598464,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 8253.982041031122,
      "residentMemoryBytes": 798736384,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7149.194958037697,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1554.0143339894712,
      "inferenceMs": 1555.2298330003396,
      "tokensPerSecond": 15.465225393425694,
      "inputTokens": 20,
      "outputTokens": 24,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089795072,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 8704.424791038036,
      "residentMemoryBytes": 816578560,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7149.194958037697,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 810.3481250000186,
      "inferenceMs": 811.5132499951869,
      "tokensPerSecond": 14.847119526313909,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089795072,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 7960.708208032884,
      "residentMemoryBytes": 817545216,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7149.194958037697,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 823.1972909998149,
      "inferenceMs": 824.4241249631159,
      "tokensPerSecond": 14.613592251768287,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089795072,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 7973.619083000813,
      "residentMemoryBytes": 818511872,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7149.194958037697,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 815.9192079911008,
      "inferenceMs": 817.1553749707527,
      "tokensPerSecond": 14.744012446837322,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089795072,
      "memoryMeasurementScope": "Post-inference process RSS and MPS driver allocation, not peak or minimum",
      "totalMs": 7966.3503330084495,
      "residentMemoryBytes": 819494912,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14371.49866699474,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1071.35704200482,
      "inferenceMs": 1072.4655829835683,
      "tokensPerSecond": 11.25510612463049,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15088369664,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 15443.964249978308,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14371.49866699474,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1414.808583969716,
      "inferenceMs": 1415.6923749833368,
      "tokensPerSecond": 14.867838635466956,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089172480,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 15787.191041978076,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14371.49866699474,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 617.4583750544116,
      "inferenceMs": 618.7747920048423,
      "tokensPerSecond": 12.989609327290848,
      "inputTokens": 25,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089385472,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 14990.273458999582,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7376087954634903,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14371.49866699474,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 814.8602499859408,
      "inferenceMs": 815.9732079948299,
      "tokensPerSecond": 13.533151679258026,
      "inputTokens": 21,
      "outputTokens": 11,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089598464,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 15187.47187498957,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7613222236973526,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14371.49866699474,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1108.2339160493575,
      "inferenceMs": 1109.1525420197286,
      "tokensPerSecond": 15.363671655535644,
      "inputTokens": 29,
      "outputTokens": 17,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089598464,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 15480.651209014468,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14371.49866699474,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1552.4208749993704,
      "inferenceMs": 1553.4910839633085,
      "tokensPerSecond": 15.477121355406737,
      "inputTokens": 20,
      "outputTokens": 24,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089795072,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 15924.989750958048,
      "cpuThreads": 4,
      "precision": "bf16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "bf16",
      "actualPrecision": "bf16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7213.158041005954,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 5259.16504196357,
      "inferenceMs": 5260.12275001267,
      "tokensPerSecond": 2.2834153757356246,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 12473.280791018624,
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7213.158041005954,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 6964.37516604783,
      "inferenceMs": 6965.1402499875985,
      "tokensPerSecond": 3.0155016346001746,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 14178.298290993553,
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7213.158041005954,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 2694.291624997277,
      "inferenceMs": 2695.027374953497,
      "tokensPerSecond": 2.969691238420049,
      "inputTokens": 25,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 9908.185415959451,
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7380931729310785,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7213.158041005954,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3313.6002920218743,
      "inferenceMs": 3314.298708050046,
      "tokensPerSecond": 3.0181147624974365,
      "inputTokens": 21,
      "outputTokens": 10,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 10527.456749056,
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7613222236973526,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7213.158041005954,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 5495.09375001071,
      "inferenceMs": 5495.834124973044,
      "tokensPerSecond": 3.093919896872152,
      "inputTokens": 29,
      "outputTokens": 17,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 12708.992165978998,
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7213.158041005954,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 7526.038707990665,
      "inferenceMs": 7526.787166018039,
      "tokensPerSecond": 3.1890521964102443,
      "inputTokens": 20,
      "outputTokens": 24,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 14739.945207023993,
      "cpuThreads": 2,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14972.291791986208,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 4562.858375022188,
      "inferenceMs": 4563.663333014119,
      "tokensPerSecond": 2.632221417189013,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 19535.955125000328,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14972.291791986208,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 6970.68525000941,
      "inferenceMs": 6971.417874970939,
      "tokensPerSecond": 3.0127650856647485,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 21943.709666957147,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14972.291791986208,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 2836.880667018704,
      "inferenceMs": 2837.6411669887602,
      "tokensPerSecond": 2.820321876948597,
      "inputTokens": 25,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 17809.93295897497,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7380931729310785,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14972.291791986208,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3589.772167033516,
      "inferenceMs": 3590.518750017509,
      "tokensPerSecond": 2.7859089235012915,
      "inputTokens": 21,
      "outputTokens": 10,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 18562.810542003717,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7613222236973526,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14972.291791986208,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 5608.144541969523,
      "inferenceMs": 5608.870083000511,
      "tokensPerSecond": 3.0315189703860046,
      "inputTokens": 29,
      "outputTokens": 17,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 20581.16187498672,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 14972.291791986208,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 7638.016375014558,
      "inferenceMs": 7638.72591604013,
      "tokensPerSecond": 3.142285930948345,
      "inputTokens": 20,
      "outputTokens": 24,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 22611.01770802634,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "cpu",
      "actualDevice": "cpu",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7060.721374989953,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1048.951584030874,
      "inferenceMs": 1050.0572910532355,
      "tokensPerSecond": 11.494328115728372,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15088369664,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 8110.778666043188,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7060.721374989953,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1366.174582974054,
      "inferenceMs": 1367.287374974694,
      "tokensPerSecond": 15.393515994771525,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089172480,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 8428.008749964647,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7060.721374989953,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 622.6567919948138,
      "inferenceMs": 623.8601670484059,
      "tokensPerSecond": 12.8851251356781,
      "inputTokens": 25,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089385472,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 7684.581542038359,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7380931729310785,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7060.721374989953,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 751.4759170007892,
      "inferenceMs": 752.5361250154674,
      "tokensPerSecond": 13.339973687356897,
      "inputTokens": 21,
      "outputTokens": 10,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089598464,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 7813.25750000542,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7613222236973526,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7060.721374989953,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1100.4484579898417,
      "inferenceMs": 1101.509874977637,
      "tokensPerSecond": 15.474931341151343,
      "inputTokens": 29,
      "outputTokens": 17,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089598464,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 8162.23124996759,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 7060.721374989953,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1546.888459008187,
      "inferenceMs": 1548.019458015915,
      "tokensPerSecond": 15.532644302572256,
      "inputTokens": 20,
      "outputTokens": 24,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15089795072,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 8608.740833005868,
      "cpuThreads": 4,
      "precision": "fp16",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp16",
      "actualPrecision": "fp16",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "fail",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": false
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 21440.188082982786,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 2510.550415958278,
      "inferenceMs": 2511.3039170391858,
      "tokensPerSecond": 4.790001748869071,
      "inputTokens": 22,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 30114463744,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 23951.49200002197,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "acceptable"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 21440.188082982786,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 2446.162957989145,
      "inferenceMs": 2446.887083002366,
      "tokensPerSecond": 8.59086657829708,
      "inputTokens": 29,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 30132043776,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 23887.075165985152,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 21440.188082982786,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1058.4159999853,
      "inferenceMs": 1059.0835000039078,
      "tokensPerSecond": 7.566080551617521,
      "inputTokens": 25,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 30132256768,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 22499.271582986694,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7380931729310785,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 21440.188082982786,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1284.632208000403,
      "inferenceMs": 1285.285040969029,
      "tokensPerSecond": 7.789900415674963,
      "inputTokens": 21,
      "outputTokens": 10,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 30132469760,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 22725.473123951815,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.7613222236973526,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 21440.188082982786,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1990.059875010047,
      "inferenceMs": 1990.7416659989394,
      "tokensPerSecond": 8.547164140701268,
      "inputTokens": 29,
      "outputTokens": 17,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 30132469760,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 23430.929748981725,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 21440.188082982786,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 2799.802042020019,
      "inferenceMs": 2800.5338750081137,
      "tokensPerSecond": 8.57523600381525,
      "inputTokens": 20,
      "outputTokens": 24,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 30132666368,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "totalMs": 24240.7219579909,
      "cpuThreads": 4,
      "precision": "fp32",
      "artifactRevision": "397085d46158ad0b437ffd74228499e8c2a35c76",
      "requestedDevice": "mps",
      "actualDevice": "mps",
      "requestedPrecision": "fp32",
      "actualPrecision": "fp32",
      "fallback": false,
      "grade": "good"
    }
  }
]
Release validation still required
  • Sample publication excluded/pending
  • Complete model/runtime backup excluded/pending
  • Forced model-source and runtime-source recovery excluded/pending

Something not quite right?

Tell us what happened with HY-MT 1.5 Desktop. Loading, results, or the page itself—we’re here for all of it.

Attachments Optional

Up to 3 files · 50 MB each

Browser, OS, CPU/GPU, memory, screen, storage, and network details are included automatically. Model inputs, results, and browsing history are not collected.