A model is still running

A model is still running or downloading. Leaving will stop the current operation. Stay on this page to let it finish.

Skip to content
LocalAI.toolsBeta

Join the waitlist

Get an email when the new apps and model runs launch.

No account needed.

Hy-MT 2

Translate text locally with Tencent Hy-MT2 1.8B using explicit language selections and the official translation prompt.

Desktop inferencePrivate · Runs on your computer
About this model

Translate text locally with Tencent Hy-MT2 1.8B using explicit language selections and the official translation prompt.

Created by
Tencent Hunyuan
License
Apache-2.0
Model released
May 21, 2026
Source repository created
May 11, 2026
Model weights · BF16
4.1 GB

Best for

  • Draft multilingual text translations for human review.
  • Local short-form text translation.

Keep in mind

  • Translations can omit, invent or mistranslate facts, names, numbers, negation and idioms. Review important outputs.
  • Six short authored fixtures are a compatibility suite, not broad validation across all languages or domains.
  • The adapter limits text to16384 characters and input-plus-output to4096 tokens as a resource policy, below the underlying model context.
  • Language choices are accepted selections, not evidence of measured quality in every pair.
  • Source weights are BF16. CPU FP32 and GPU dtype casts have independent performance/memory needs.
  • App-owned pinned CPython and isolated model dependencies install on demand. Untested hardware/OS/provider combinations are unverified.
  • No dedicated content safety filter; review generated translations for consequential uses.
  • Text-only translation with genuine incremental text output. The official prompt specifies the target language; sourceLanguage is validated metadata while the checkpoint infers the source language.

Processed on your deviceYour input is processed locally in the desktop app. Model and runtime files download from external hosts.

Target platforms · verification required

  • macos · aarch64, x86_64
  • windows · x86_64

Recommended configuration

BF16 · mps (verified platform only)

Sources & technical details
Original model
https://huggingface.co/tencent/Hy-MT2-1.8B
Release verification
Verified
Release evidence
Original HY-MT2-1.8B release explicitly dated in upstream model-card news.
Source updated
May 26, 2026
Native model files
https://huggingface.co/tencent/Hy-MT2-1.8B
Files updated
May 26, 2026
Artifact revision
9a341cd1b679d3efd23b46e847b01745a71ed792

Desktop app

Sample result

Hy-MT2 1.8B — Translation

Run Hy-MT 2 on your computer

Download the desktop app to load this model, provide input and view results locally.

Public desktop downloads are coming soon.

Device requirements

macOS Apple SiliconCPU / FP32Tested24.0 GiBRAM estimate
macOS Apple SiliconMPS / BF16Tested16.0 GiBRAM estimate
macOS Apple SiliconMPS / FP32Tested24.0 GiBRAM estimate
Windows Intel / AMDCPU / FP32Unverified24.0 GiBRAM estimate
macOS Intel / AMDCPU / FP32Unverified24.0 GiBRAM estimate
If your device falls short

MemoryAdvisory only: close other apps, reduce image/tile/token/batch settings or choose a smaller verified model/precision. CPU can need more RAM than GPU.

GPUIf the selected GPU runtime fails, existing CPU fallback may recover when enabled, but CPU has its own memory/runtime needs.

PlatformUse a compatible OS/architecture/runtime or another model. These estimates do not add hardware detection or block runs.

Benchmark evidence

Saved desktop measurements apply to the recorded hardware and runtime. Fallback cases report the device that actually ran; they do not prove GPU support on the requested device.

Hy-MT2 1.8B — Translation

56 configurations tested
Passed56
Failed0
TestedNov 2, 2026
Runtimetransformers 5.22.1

Quality criteria

{ "minimumCharacterF2": 0.55, "requireSemanticAnchors": true, "rejectRepetitionAndTruncation": true }

PrecisionDeviceResultPlatformInference / load
BF16mpsAcceleratedPassedFastmacos · aarch64238 ms / 21948 ms
BF16mpsAcceleratedPassedFastmacos · aarch64256 ms / 24701 ms
BF16mpsAcceleratedPassedFastmacos · aarch64310 ms / 21948 ms
FP32mpsAcceleratedPassedFastmacos · aarch64343 ms / 20614 ms
FP32mpsAcceleratedPassedFastmacos · aarch64344 ms / 20389 ms
BF16mpsAcceleratedPassedFastmacos · aarch64355 ms / 24701 ms
BF16mpsAcceleratedPassedFastmacos · aarch64362 ms / 21948 ms
BF16mpsAcceleratedPassedFastmacos · aarch64390 ms / 24701 ms
BF16mpsAcceleratedPassedFastmacos · aarch64392 ms / 21948 ms
BF16mpsAcceleratedPassedFastmacos · aarch64397 ms / 21948 ms
BF16mpsAcceleratedPassedFastmacos · aarch64413 ms / 24701 ms
BF16mpsAcceleratedPassedFastmacos · aarch64447 ms / 24701 ms
FP32mpsAcceleratedPassedFastmacos · aarch64489 ms / 20389 ms
FP32mpsAcceleratedPassedFastmacos · aarch64551 ms / 20389 ms
BF16mpsAcceleratedPassedFastmacos · aarch64589 ms / 21948 ms
FP32mpsAcceleratedPassedFastmacos · aarch64592 ms / 20389 ms
FP32mpsAcceleratedPassedFastmacos · aarch64601 ms / 20614 ms
FP32mpsAcceleratedPassedFastmacos · aarch64621 ms / 20614 ms
FP32mpsAcceleratedPassedFastmacos · aarch64622 ms / 20389 ms
FP32mpsAcceleratedPassedFastmacos · aarch64632 ms / 20614 ms
FP32mpsAcceleratedPassedFastmacos · aarch64649 ms / 20614 ms
BF16mpsAcceleratedPassedFastmacos · aarch64652 ms / 24701 ms
BF16mpsAcceleratedPassedFastmacos · aarch64653 ms / 21948 ms
FP32mpsAcceleratedPassedFastmacos · aarch64656 ms / 20614 ms
FP32mpsAcceleratedPassedFastmacos · aarch64677 ms / 20389 ms
FP32mpsAcceleratedPassedFastmacos · aarch64733 ms / 20614 ms
FP32mpsAcceleratedPassedFastmacos · aarch64757 ms / 20389 ms
BF16mpsAcceleratedPassedFastmacos · aarch64770 ms / 24701 ms
FP32cpuPassedFastmacos · aarch64739 ms / 19427 ms
FP32cpuPassedFastmacos · aarch64742 ms / 21287 ms
FP32cpuPassedFastmacos · aarch64800 ms / 19837 ms
FP32cpuPassedFastmacos · aarch64850 ms / 19481 ms
FP32cpuPassedFastmacos · aarch641185 ms / 19427 ms
FP32cpuPassedFastmacos · aarch641212 ms / 21287 ms
FP32cpuPassedFastmacos · aarch641225 ms / 19427 ms
FP32cpuPassedFastmacos · aarch641244 ms / 19837 ms
FP32cpuPassedFastmacos · aarch641246 ms / 21287 ms
FP32cpuPassedFastmacos · aarch641248 ms / 19427 ms
FP32cpuPassedFastmacos · aarch641270 ms / 19837 ms
FP32cpuPassedFastmacos · aarch641334 ms / 21287 ms
FP32cpuPassedFastmacos · aarch641353 ms / 19837 ms
FP32cpuPassedFastmacos · aarch641362 ms / 19481 ms
FP32cpuPassedFastmacos · aarch641372 ms / 21287 ms
FP32cpuPassedFastmacos · aarch641384 ms / 19837 ms
FP32cpuPassedFastmacos · aarch641495 ms / 19427 ms
FP32cpuPassedFastmacos · aarch641553 ms / 19427 ms
FP32cpuPassedFastmacos · aarch641567 ms / 19837 ms
FP32cpuPassedFastmacos · aarch641570 ms / 21287 ms
FP32cpuPassedFastmacos · aarch641761 ms / 19481 ms
FP32cpuPassedFastmacos · aarch641775 ms / 19427 ms
FP32cpuPassedFastmacos · aarch641801 ms / 19481 ms
FP32cpuPassedFastmacos · aarch641834 ms / 19481 ms
FP32cpuPassedFastmacos · aarch641849 ms / 19481 ms
FP32cpuPassedFastmacos · aarch641898 ms / 19837 ms
FP32cpuPassedFastmacos · aarch641932 ms / 19481 ms
FP32cpuPassedFastmacos · aarch641968 ms / 21287 ms
Test setup & sample details
Quality gate
{ "minimumCharacterF2": 0.55, "requireSemanticAnchors": true, "rejectRepetitionAndTruncation": true }
Tested
2026-11-02T15:21:17.573Z
Environment
{ "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2", "runtime": "App-owned CPython3.13.18; Transformers5.22.1/Torch2.10.0; MPS op fallback disabled" }
Runtime
transformers 5.22.1
Artifact revision
9a341cd1b679d3efd23b46e847b01745a71ed792
Fixture
{
  "file": "model-samples/hunyuan-translation.json",
  "url": "https://assets.localai.tools/samples/v1/files/b63cba0b5f61f8423d4234d4d28e9cdde538b8345b4ccdc8b2044b1bc88e2fa8/hunyuan-translation.json",
  "size": 3099,
  "sha1": "dd6e22e1c2966dd5a249b11eb2fb48b076505088",
  "sha256": "b63cba0b5f61f8423d4234d4d28e9cdde538b8345b4ccdc8b2044b1bc88e2fa8"
}
Rerun when
model artifact revision change; runtime version change; fixture or quality-gate change; backend/platform behavior change
Per-case quality & failure details
[
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "mps",
      "driverAllocatedMemoryBytes": 4374102016,
      "durationMs": 659.7776250000001,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 653.4228340024129,
      "inputTokens": 33,
      "loadMs": 21948.349125000004,
      "maxNewTokens": 128,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "outputLimitReached": false,
      "outputTokens": 14,
      "parameterDtype": "torch.bfloat16",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "precision": "bf16",
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "bf16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 21.425636313083455,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 22608.612959000002,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "mps",
      "driverAllocatedMemoryBytes": 4374790144,
      "durationMs": 591.922042,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 589.3725420028204,
      "inputTokens": 39,
      "loadMs": 21948.349125000004,
      "maxNewTokens": 128,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "outputLimitReached": false,
      "outputTokens": 21,
      "parameterDtype": "torch.bfloat16",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "precision": "bf16",
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "bf16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 35.63111360539003,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 22540.609583,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "mps",
      "driverAllocatedMemoryBytes": 4375019520,
      "durationMs": 240.208417,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 237.9574579972541,
      "inputTokens": 36,
      "loadMs": 21948.349125000004,
      "maxNewTokens": 128,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "outputLimitReached": false,
      "outputTokens": 8,
      "parameterDtype": "torch.bfloat16",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "precision": "bf16",
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "bf16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 33.61945478545294,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 22188.885625000006,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "mps",
      "driverAllocatedMemoryBytes": 4375248896,
      "durationMs": 364.464292,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 362.15649999212474,
      "inputTokens": 32,
      "loadMs": 21948.349125000004,
      "maxNewTokens": 128,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "outputLimitReached": false,
      "outputTokens": 14,
      "parameterDtype": "torch.bfloat16",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "precision": "bf16",
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "bf16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 38.65732079999789,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 22313.242542000004,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.769566037344403,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "mps",
      "driverAllocatedMemoryBytes": 4375248896,
      "durationMs": 399.539042,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 397.1101670031203,
      "inputTokens": 33,
      "loadMs": 21948.349125000004,
      "maxNewTokens": 128,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "outputLimitReached": false,
      "outputTokens": 18,
      "parameterDtype": "torch.bfloat16",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "precision": "bf16",
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "bf16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 45.32747206106804,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 22348.222792000004,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "mps",
      "driverAllocatedMemoryBytes": 4375461888,
      "durationMs": 394.006292,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 391.5271250007208,
      "inputTokens": 31,
      "loadMs": 21948.349125000004,
      "maxNewTokens": 128,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "outputLimitReached": false,
      "outputTokens": 16,
      "parameterDtype": "torch.bfloat16",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "precision": "bf16",
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "bf16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 40.8656232948625,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 22342.842375000004,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "mps",
      "driverAllocatedMemoryBytes": 4375461888,
      "durationMs": 312.324875,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 309.8432499973569,
      "inputTokens": 33,
      "loadMs": 21948.349125000004,
      "maxNewTokens": 128,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "outputLimitReached": false,
      "outputTokens": 14,
      "parameterDtype": "torch.bfloat16",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "precision": "bf16",
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "bf16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 45.18413746344135,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 22261.096458000004,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "mps",
      "driverAllocatedMemoryBytes": 4374102016,
      "durationMs": 778.082833,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 769.989375010482,
      "inputTokens": 33,
      "loadMs": 24700.573875000002,
      "maxNewTokens": 128,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "outputLimitReached": false,
      "outputTokens": 14,
      "parameterDtype": "torch.bfloat16",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "precision": "bf16",
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "bf16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 18.182069070510764,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 25479.062416999994,
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "mps",
      "driverAllocatedMemoryBytes": 4374790144,
      "durationMs": 654.535584,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 651.9678749900777,
      "inputTokens": 39,
      "loadMs": 24700.573875000002,
      "maxNewTokens": 128,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "outputLimitReached": false,
      "outputTokens": 21,
      "parameterDtype": "torch.bfloat16",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "precision": "bf16",
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "bf16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 32.21017600034296,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 25355.421124999993,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "mps",
      "driverAllocatedMemoryBytes": 4375019520,
      "durationMs": 258.640458,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 256.1951669922564,
      "inputTokens": 36,
      "loadMs": 24700.573875000002,
      "maxNewTokens": 128,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "outputLimitReached": false,
      "outputTokens": 8,
      "parameterDtype": "torch.bfloat16",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "precision": "bf16",
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "bf16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 31.22619405323053,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 24959.517000000007,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "mps",
      "driverAllocatedMemoryBytes": 4375248896,
      "durationMs": 392.977625,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 390.1868749962887,
      "inputTokens": 32,
      "loadMs": 24700.573875000002,
      "maxNewTokens": 128,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "outputLimitReached": false,
      "outputTokens": 14,
      "parameterDtype": "torch.bfloat16",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "precision": "bf16",
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "bf16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 35.8802432812051,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 25093.88958300001,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.769566037344403,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "mps",
      "driverAllocatedMemoryBytes": 4375248896,
      "durationMs": 416.077792,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 413.22800000489224,
      "inputTokens": 33,
      "loadMs": 24700.573875000002,
      "maxNewTokens": 128,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "outputLimitReached": false,
      "outputTokens": 18,
      "parameterDtype": "torch.bfloat16",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "precision": "bf16",
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "bf16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 43.55948773990847,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 25116.964791000006,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "mps",
      "driverAllocatedMemoryBytes": 4375461888,
      "durationMs": 450.004625,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 447.3837499972433,
      "inputTokens": 31,
      "loadMs": 24700.573875000002,
      "maxNewTokens": 128,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "outputLimitReached": false,
      "outputTokens": 16,
      "parameterDtype": "torch.bfloat16",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "precision": "bf16",
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "bf16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 35.76348045743411,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 25150.878375,
      "grade": "excellent"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "mps",
      "driverAllocatedMemoryBytes": 4375461888,
      "durationMs": 357.838541,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 354.92241600877605,
      "inputTokens": 33,
      "loadMs": 24700.573875000002,
      "maxNewTokens": 128,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "outputLimitReached": false,
      "outputTokens": 14,
      "parameterDtype": "torch.bfloat16",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "precision": "bf16",
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "bf16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 39.4452403357184,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 25058.674874999997,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "cpu",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "cpu",
      "durationMs": 1358.75125,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1353.4542080014944,
      "inputTokens": 33,
      "loadMs": 19836.568667000014,
      "maxNewTokens": 128,
      "outputLimitReached": false,
      "outputTokens": 14,
      "parameterDtype": "torch.float32",
      "placement": {
        "cpu/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "PyTorch CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 10.343903707442273,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 21195.607707999996,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "cpu",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "cpu",
      "durationMs": 1899.234667,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1898.1381250050615,
      "inputTokens": 39,
      "loadMs": 19836.568667000014,
      "maxNewTokens": 128,
      "outputLimitReached": false,
      "outputTokens": 21,
      "parameterDtype": "torch.float32",
      "placement": {
        "cpu/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "PyTorch CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 11.063473054651384,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 21736.07412600002,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "cpu",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "cpu",
      "durationMs": 801.300792,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 800.2311659947736,
      "inputTokens": 36,
      "loadMs": 19836.568667000014,
      "maxNewTokens": 128,
      "outputLimitReached": false,
      "outputTokens": 8,
      "parameterDtype": "torch.float32",
      "placement": {
        "cpu/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "PyTorch CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 9.99711125978846,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 20638.127792000014,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "cpu",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "cpu",
      "durationMs": 1244.796834,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1243.75899998995,
      "inputTokens": 32,
      "loadMs": 19836.568667000014,
      "maxNewTokens": 128,
      "outputLimitReached": false,
      "outputTokens": 14,
      "parameterDtype": "torch.float32",
      "placement": {
        "cpu/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "PyTorch CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 11.256199955226956,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 21081.65066700001,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.769566037344403,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "cpu",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "cpu",
      "durationMs": 1568.0272909999999,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1566.976958012674,
      "inputTokens": 33,
      "loadMs": 19836.568667000014,
      "maxNewTokens": 128,
      "outputLimitReached": false,
      "outputTokens": 18,
      "parameterDtype": "torch.float32",
      "placement": {
        "cpu/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "PyTorch CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 11.48708658921736,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 21404.911542000016,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "cpu",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "cpu",
      "durationMs": 1384.9958339999998,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1383.80125000549,
      "inputTokens": 31,
      "loadMs": 19836.568667000014,
      "maxNewTokens": 128,
      "outputLimitReached": false,
      "outputTokens": 16,
      "parameterDtype": "torch.float32",
      "placement": {
        "cpu/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "PyTorch CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 11.562354059108214,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 21221.874708999996,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "cpu",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "cpu",
      "durationMs": 1270.922625,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1269.897124991985,
      "inputTokens": 33,
      "loadMs": 19836.568667000014,
      "maxNewTokens": 128,
      "outputLimitReached": false,
      "outputTokens": 14,
      "parameterDtype": "torch.float32",
      "placement": {
        "cpu/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "PyTorch CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 11.024515076438464,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 21107.77583400003,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "cpu",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "cpu",
      "durationMs": 1806.117833,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1800.6467079976574,
      "inputTokens": 33,
      "loadMs": 19480.776207999996,
      "maxNewTokens": 128,
      "outputLimitReached": false,
      "outputTokens": 14,
      "parameterDtype": "torch.float32",
      "placement": {
        "cpu/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "PyTorch CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 7.77498436412781,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 21287.190040999994,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "cpu",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "cpu",
      "durationMs": 1932.746791,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1931.6712919971903,
      "inputTokens": 39,
      "loadMs": 19480.776207999996,
      "maxNewTokens": 128,
      "outputLimitReached": false,
      "outputTokens": 21,
      "parameterDtype": "torch.float32",
      "placement": {
        "cpu/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "PyTorch CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 10.871414865977384,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 21413.791582999984,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "cpu",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "cpu",
      "durationMs": 851.316958,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 850.2449160005199,
      "inputTokens": 36,
      "loadMs": 19480.776207999996,
      "maxNewTokens": 128,
      "outputLimitReached": false,
      "outputTokens": 8,
      "parameterDtype": "torch.float32",
      "placement": {
        "cpu/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "PyTorch CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 9.409053614376576,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 20332.353166999994,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "cpu",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "cpu",
      "durationMs": 1762.56175,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1761.337041010847,
      "inputTokens": 32,
      "loadMs": 19480.776207999996,
      "maxNewTokens": 128,
      "outputLimitReached": false,
      "outputTokens": 14,
      "parameterDtype": "torch.float32",
      "placement": {
        "cpu/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "PyTorch CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 7.948507113644344,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 21243.611458,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.769566037344403,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "cpu",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "cpu",
      "durationMs": 1850.378667,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1849.1938750084955,
      "inputTokens": 33,
      "loadMs": 19480.776207999996,
      "maxNewTokens": 128,
      "outputLimitReached": false,
      "outputTokens": 18,
      "parameterDtype": "torch.float32",
      "placement": {
        "cpu/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "PyTorch CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 9.733971241883603,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 21331.400832999992,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "cpu",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "cpu",
      "durationMs": 1835.0864169999998,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1833.836332996725,
      "inputTokens": 31,
      "loadMs": 19480.776207999996,
      "maxNewTokens": 128,
      "outputLimitReached": false,
      "outputTokens": 16,
      "parameterDtype": "torch.float32",
      "placement": {
        "cpu/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "PyTorch CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 8.724878939362018,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 21316.205707999994,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "cpu",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "cpu",
      "durationMs": 1363.349125,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1362.1994999994058,
      "inputTokens": 33,
      "loadMs": 19480.776207999996,
      "maxNewTokens": 128,
      "outputLimitReached": false,
      "outputTokens": 14,
      "parameterDtype": "torch.float32",
      "placement": {
        "cpu/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "PyTorch CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 10.27749606427407,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 20844.401082999975,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "cpu",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 2,
      "decoding": "greedy",
      "device": "cpu",
      "durationMs": 1377.025792,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1371.6296659986256,
      "inputTokens": 33,
      "loadMs": 21287.11720799998,
      "maxNewTokens": 128,
      "outputLimitReached": false,
      "outputTokens": 14,
      "parameterDtype": "torch.float32",
      "placement": {
        "cpu/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "PyTorch CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 10.206836689994738,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 22664.436623999965,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "cpu",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 2,
      "decoding": "greedy",
      "device": "cpu",
      "durationMs": 1969.116708,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1968.03429198917,
      "inputTokens": 39,
      "loadMs": 21287.11720799998,
      "maxNewTokens": 128,
      "outputLimitReached": false,
      "outputTokens": 21,
      "parameterDtype": "torch.float32",
      "placement": {
        "cpu/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "PyTorch CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 10.67054577528447,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 23256.509373999987,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "cpu",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 2,
      "decoding": "greedy",
      "device": "cpu",
      "durationMs": 743.0032090000001,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 741.9097079982748,
      "inputTokens": 36,
      "loadMs": 21287.11720799998,
      "maxNewTokens": 128,
      "outputLimitReached": false,
      "outputTokens": 8,
      "parameterDtype": "torch.float32",
      "placement": {
        "cpu/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "PyTorch CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 10.7829833115199,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 22030.44070799998,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "cpu",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 2,
      "decoding": "greedy",
      "device": "cpu",
      "durationMs": 1213.374583,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1212.3512910038698,
      "inputTokens": 32,
      "loadMs": 21287.11720799998,
      "maxNewTokens": 128,
      "outputLimitReached": false,
      "outputTokens": 14,
      "parameterDtype": "torch.float32",
      "placement": {
        "cpu/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "PyTorch CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 11.547808051911671,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 22500.799207999982,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.769566037344403,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "cpu",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 2,
      "decoding": "greedy",
      "device": "cpu",
      "durationMs": 1570.908875,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1569.8936249973483,
      "inputTokens": 33,
      "loadMs": 21287.11720799998,
      "maxNewTokens": 128,
      "outputLimitReached": false,
      "outputTokens": 18,
      "parameterDtype": "torch.float32",
      "placement": {
        "cpu/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "PyTorch CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 11.465745011882827,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 22858.326,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "cpu",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 2,
      "decoding": "greedy",
      "device": "cpu",
      "durationMs": 1334.775958,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1333.7626660068054,
      "inputTokens": 31,
      "loadMs": 21287.11720799998,
      "maxNewTokens": 128,
      "outputLimitReached": false,
      "outputTokens": 16,
      "parameterDtype": "torch.float32",
      "placement": {
        "cpu/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "PyTorch CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 11.996137249742423,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 22622.18037399996,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "cpu",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 2,
      "decoding": "greedy",
      "device": "cpu",
      "durationMs": 1247.4299170000002,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1246.3203750085086,
      "inputTokens": 33,
      "loadMs": 21287.11720799998,
      "maxNewTokens": 128,
      "outputLimitReached": false,
      "outputTokens": 14,
      "parameterDtype": "torch.float32",
      "placement": {
        "cpu/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "PyTorch CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 11.233066778599705,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 22534.846290999965,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "cpu",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 2,
      "decoding": "greedy",
      "device": "cpu",
      "durationMs": 1252.09325,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1247.739915997954,
      "inputTokens": 33,
      "loadMs": 19426.93958300003,
      "maxNewTokens": 128,
      "outputLimitReached": false,
      "outputTokens": 14,
      "parameterDtype": "torch.float32",
      "placement": {
        "cpu/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "PyTorch CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 11.220287032977277,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 20679.309458000003,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "cpu",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 2,
      "decoding": "greedy",
      "device": "cpu",
      "durationMs": 1776.27025,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1775.1904169999762,
      "inputTokens": 39,
      "loadMs": 19426.93958300003,
      "maxNewTokens": 128,
      "outputLimitReached": false,
      "outputTokens": 21,
      "parameterDtype": "torch.float32",
      "placement": {
        "cpu/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "PyTorch CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 11.829716856791864,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 21203.49979100004,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "cpu",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 2,
      "decoding": "greedy",
      "device": "cpu",
      "durationMs": 740.449292,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 739.4447080005193,
      "inputTokens": 36,
      "loadMs": 19426.93958300003,
      "maxNewTokens": 128,
      "outputLimitReached": false,
      "outputTokens": 8,
      "parameterDtype": "torch.float32",
      "placement": {
        "cpu/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "PyTorch CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 10.818929276851868,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 20167.71320800006,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "cpu",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 2,
      "decoding": "greedy",
      "device": "cpu",
      "durationMs": 1185.9571660000001,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1184.974791991408,
      "inputTokens": 32,
      "loadMs": 19426.93958300003,
      "maxNewTokens": 128,
      "outputLimitReached": false,
      "outputTokens": 14,
      "parameterDtype": "torch.float32",
      "placement": {
        "cpu/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "PyTorch CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 11.81459731854069,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 20613.20424900006,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.769566037344403,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "cpu",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 2,
      "decoding": "greedy",
      "device": "cpu",
      "durationMs": 1553.7456670000001,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1552.6519999984885,
      "inputTokens": 33,
      "loadMs": 19426.93958300003,
      "maxNewTokens": 128,
      "outputLimitReached": false,
      "outputTokens": 18,
      "parameterDtype": "torch.float32",
      "placement": {
        "cpu/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "PyTorch CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 11.593067860678069,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 20981.01787500002,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "cpu",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 2,
      "decoding": "greedy",
      "device": "cpu",
      "durationMs": 1496.346084,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1495.246207996388,
      "inputTokens": 31,
      "loadMs": 19426.93958300003,
      "maxNewTokens": 128,
      "outputLimitReached": false,
      "outputTokens": 16,
      "parameterDtype": "torch.float32",
      "placement": {
        "cpu/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "PyTorch CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 10.700578884222558,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 20923.616791000008,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "cpu",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 2,
      "decoding": "greedy",
      "device": "cpu",
      "durationMs": 1226.307041,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1225.226875001681,
      "inputTokens": 33,
      "loadMs": 19426.93958300003,
      "maxNewTokens": 128,
      "outputLimitReached": false,
      "outputTokens": 14,
      "parameterDtype": "torch.float32",
      "placement": {
        "cpu/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "PyTorch CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 11.426455202413669,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 20653.54458300001,
      "grade": "good"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "mps",
      "driverAllocatedMemoryBytes": 7486275584,
      "durationMs": 662.381208,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 655.7098339981167,
      "inputTokens": 33,
      "loadMs": 20613.526916000003,
      "maxNewTokens": 128,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "outputLimitReached": false,
      "outputTokens": 14,
      "parameterDtype": "torch.float32",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 21.35090748088462,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 21276.195083,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "mps",
      "driverAllocatedMemoryBytes": 7486963712,
      "durationMs": 735.567875,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 733.0782920034835,
      "inputTokens": 39,
      "loadMs": 20613.526916000003,
      "maxNewTokens": 128,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "outputLimitReached": false,
      "outputTokens": 21,
      "parameterDtype": "torch.float32",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 28.646326359777422,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 21349.34904100001,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "mps",
      "driverAllocatedMemoryBytes": 7487193088,
      "durationMs": 345.366542,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 342.9617910005618,
      "inputTokens": 36,
      "loadMs": 20613.526916000003,
      "maxNewTokens": 128,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "outputLimitReached": false,
      "outputTokens": 8,
      "parameterDtype": "torch.float32",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 23.32621361890105,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 20959.145500000042,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "mps",
      "driverAllocatedMemoryBytes": 7487422464,
      "durationMs": 635.143834,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 632.427499993355,
      "inputTokens": 32,
      "loadMs": 20613.526916000003,
      "maxNewTokens": 128,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "outputLimitReached": false,
      "outputTokens": 14,
      "parameterDtype": "torch.float32",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 22.136924786077618,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 21248.94958200003,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.769566037344403,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "mps",
      "driverAllocatedMemoryBytes": 7487422464,
      "durationMs": 623.886709,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 621.4268329931656,
      "inputTokens": 33,
      "loadMs": 20613.526916000003,
      "maxNewTokens": 128,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "outputLimitReached": false,
      "outputTokens": 18,
      "parameterDtype": "torch.float32",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 28.96559827212669,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 21237.697083000035,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "mps",
      "driverAllocatedMemoryBytes": 7487635456,
      "durationMs": 651.96175,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 649.4119169947226,
      "inputTokens": 31,
      "loadMs": 20613.526916000003,
      "maxNewTokens": 128,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "outputLimitReached": false,
      "outputTokens": 16,
      "parameterDtype": "torch.float32",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 24.63767538181783,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 21265.76658299996,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "mps",
      "driverAllocatedMemoryBytes": 7487635456,
      "durationMs": 604.573,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 600.6057920021703,
      "inputTokens": 33,
      "loadMs": 20613.526916000003,
      "maxNewTokens": 128,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "outputLimitReached": false,
      "outputTokens": 14,
      "parameterDtype": "torch.float32",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 23.30979851747652,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 21218.51187399996,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "mps",
      "driverAllocatedMemoryBytes": 7486275584,
      "durationMs": 683.779083,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 676.9712079985766,
      "inputTokens": 33,
      "loadMs": 20389.204541000014,
      "maxNewTokens": 128,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "outputLimitReached": false,
      "outputTokens": 14,
      "parameterDtype": "torch.float32",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 20.6803477527355,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 21073.237916000013,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "mps",
      "driverAllocatedMemoryBytes": 7486963712,
      "durationMs": 759.264584,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 756.7442080035107,
      "inputTokens": 39,
      "loadMs": 20389.204541000014,
      "maxNewTokens": 128,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "outputLimitReached": false,
      "outputTokens": 21,
      "parameterDtype": "torch.float32",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 27.75046016593043,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 21148.765041000035,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "mps",
      "driverAllocatedMemoryBytes": 7487193088,
      "durationMs": 347.28325,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 344.3750000005821,
      "inputTokens": 36,
      "loadMs": 20389.204541000014,
      "maxNewTokens": 128,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "outputLimitReached": false,
      "outputTokens": 8,
      "parameterDtype": "torch.float32",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 23.230490018109556,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 20736.79116600001,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "mps",
      "driverAllocatedMemoryBytes": 7487422464,
      "durationMs": 554.4377499999999,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 551.4502919977531,
      "inputTokens": 32,
      "loadMs": 20389.204541000014,
      "maxNewTokens": 128,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "outputLimitReached": false,
      "outputTokens": 14,
      "parameterDtype": "torch.float32",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 25.387601027976324,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 20943.933958000038,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.769566037344403,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "mps",
      "driverAllocatedMemoryBytes": 7487422464,
      "durationMs": 624.1876669999999,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 622.2563329938566,
      "inputTokens": 33,
      "loadMs": 20389.204541000014,
      "maxNewTokens": 128,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "outputLimitReached": false,
      "outputTokens": 18,
      "parameterDtype": "torch.float32",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 28.926985625677368,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 21013.697082999977,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "mps",
      "driverAllocatedMemoryBytes": 7487635456,
      "durationMs": 594.683584,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 591.7933330056258,
      "inputTokens": 31,
      "loadMs": 20389.204541000014,
      "maxNewTokens": 128,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "outputLimitReached": false,
      "outputTokens": 16,
      "parameterDtype": "torch.float32",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 27.03646544772396,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 20984.17712399998,
      "grade": "excellent"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "artifactRevision": "9a341cd1b679d3efd23b46e847b01745a71ed792",
      "cpuThreads": 4,
      "decoding": "greedy",
      "device": "mps",
      "driverAllocatedMemoryBytes": 7487635456,
      "durationMs": 491.29566600000004,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 488.5365410009399,
      "inputTokens": 33,
      "loadMs": 20389.204541000014,
      "maxNewTokens": 128,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "outputLimitReached": false,
      "outputTokens": 14,
      "parameterDtype": "torch.float32",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "precision": "fp32",
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp32",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "tokensPerSecond": 28.657017080679463,
      "torchVersion": "2.10.0",
      "version": "5.22.1",
      "totalMs": 20880.79920800001,
      "grade": "excellent"
    }
  }
]

Hy-MT2 7B — Text Translation

83 configurations tested
Passed81
Failed2
TestedNov 6, 2026
Runtimetransformers 5.22.1

Quality criteria

{ "minimumCharacterF2": 0.55, "requireSemanticAnchors": true, "rejectRepetitionAndTruncation": true }

PrecisionDeviceResultPlatformInference / load
FP16mpsAcceleratedPassedFastmacos · aarch64659 ms / 4566 ms
FP16mpsAcceleratedPassedFastmacos · aarch64662 ms / 4574 ms
FP16mpsAcceleratedPassedFastmacos · aarch64667 ms / 4667 ms
FP16mpsAcceleratedPassedFastmacos · aarch64669 ms / 5109 ms
FP16mpsAcceleratedPassedFastmacos · aarch64846 ms / 4566 ms
FP16mpsAcceleratedPassedFastmacos · aarch64855 ms / 4667 ms
FP16mpsAcceleratedPassedFastmacos · aarch64902 ms / 4574 ms
FP16mpsAcceleratedPassedFastmacos · aarch64903 ms / 4574 ms
FP16mpsAcceleratedPassedFastmacos · aarch64913 ms / 4566 ms
FP16mpsAcceleratedPassedFastmacos · aarch64914 ms / 4667 ms
FP16mpsAcceleratedPassedFastmacos · aarch641020 ms / 4667 ms
FP16mpsAcceleratedPassedFastmacos · aarch641050 ms / 4574 ms
FP16mpsAcceleratedPassedFastmacos · aarch641056 ms / 4566 ms
FP16mpsAcceleratedPassedFastmacos · aarch641103 ms / 5109 ms
FP16mpsAcceleratedPassedFastmacos · aarch641151 ms / 4566 ms
FP16mpsAcceleratedPassedFastmacos · aarch641167 ms / 4667 ms
FP16mpsAcceleratedPassedFastmacos · aarch641170 ms / 4574 ms
FP16mpsAcceleratedPassedFastmacos · aarch641183 ms / 5109 ms
FP16mpsAcceleratedPassedFastmacos · aarch641459 ms / 4667 ms
FP16mpsAcceleratedPassedFastmacos · aarch641460 ms / 4566 ms
FP16mpsAcceleratedPassedFastmacos · aarch641468 ms / 4574 ms
FP16mpsAcceleratedPassedFastmacos · aarch641550 ms / 5109 ms
FP16mpsAcceleratedPassedFastmacos · aarch641688 ms / 4566 ms
FP16mpsAcceleratedPassedFastmacos · aarch641693 ms / 5109 ms
FP16mpsAcceleratedPassedFastmacos · aarch641709 ms / 4667 ms
FP16mpsAcceleratedPassedFastmacos · aarch641806 ms / 4574 ms
FP16mpsAcceleratedPassedFastmacos · aarch641849 ms / 5109 ms
FP32cpuPassedModeratemacos · aarch642985 ms / 3911 ms
FP32cpuPassedModeratemacos · aarch643035 ms / 4022 ms
FP32cpuPassedModeratemacos · aarch643047 ms / 3945 ms
FP32cpuPassedModeratemacos · aarch643080 ms / 3910 ms
FP32cpuPassedModeratemacos · aarch643086 ms / 12265 ms
FP32cpuPassedModeratemacos · aarch643128 ms / 4066 ms
FP32cpuPassedModeratemacos · aarch643234 ms / 3895 ms
FP32cpuPassedModeratemacos · aarch643842 ms / 4907 ms
FP32cpuPassedModeratemacos · aarch644022 ms / 3911 ms
FP32cpuPassedModeratemacos · aarch644085 ms / 4066 ms
FP32cpuPassedModeratemacos · aarch644165 ms / 4022 ms
FP32cpuPassedModeratemacos · aarch644181 ms / 3910 ms
FP32cpuPassedModeratemacos · aarch644190 ms / 4022 ms
FP32cpuPassedModeratemacos · aarch644196 ms / 3911 ms
FP32cpuPassedModeratemacos · aarch644201 ms / 3910 ms
FP32cpuPassedModeratemacos · aarch644213 ms / 3945 ms
FP32cpuPassedModeratemacos · aarch644232 ms / 3945 ms
FP32cpuPassedModeratemacos · aarch644238 ms / 12265 ms
FP32cpuPassedModeratemacos · aarch644252 ms / 4066 ms
FP32cpuPassedModeratemacos · aarch644344 ms / 3895 ms
FP32cpuPassedModeratemacos · aarch644364 ms / 3895 ms
FP32cpuPassedModeratemacos · aarch644539 ms / 4066 ms
FP32cpuPassedModeratemacos · aarch644617 ms / 3945 ms
FP32cpuPassedModeratemacos · aarch644804 ms / 4907 ms
FP32cpuPassedModeratemacos · aarch644841 ms / 3911 ms
FP32cpuPassedModeratemacos · aarch644929 ms / 3895 ms
FP32cpuPassedModeratemacos · aarch644971 ms / 3910 ms
FP32cpuPassedModeratemacos · aarch645608 ms / 3911 ms
FP32cpuPassedModeratemacos · aarch645795 ms / 3945 ms
FP32cpuPassedModeratemacos · aarch645818 ms / 12265 ms
FP32cpuPassedModeratemacos · aarch645826 ms / 4022 ms
FP32cpuPassedModeratemacos · aarch645876 ms / 3910 ms
FP32cpuPassedModeratemacos · aarch645994 ms / 4066 ms
FP32cpuPassedModeratemacos · aarch646270 ms / 3895 ms
FP32cpuPassedModeratemacos · aarch646487 ms / 4022 ms
FP32cpuPassedModeratemacos · aarch647108 ms / 4066 ms
FP32cpuPassedModeratemacos · aarch647239 ms / 3911 ms
FP32cpuPassedModeratemacos · aarch647282 ms / 12265 ms
FP32cpuPassedModeratemacos · aarch647286 ms / 3910 ms
FP32cpuPassedModeratemacos · aarch647335 ms / 3945 ms
FP32cpuPassedModeratemacos · aarch647364 ms / 4022 ms
FP32cpuPassedModeratemacos · aarch647403 ms / 4907 ms
FP32cpuPassedModeratemacos · aarch647847 ms / 3895 ms
FP32cpuPassedModeratemacos · aarch648218 ms / 3911 ms
FP32cpuPassedModeratemacos · aarch648310 ms / 4066 ms
FP32cpuPassedModeratemacos · aarch648373 ms / 3945 ms
FP32cpuPassedModeratemacos · aarch648399 ms / 3910 ms
FP32cpuPassedModeratemacos · aarch648407 ms / 4022 ms
FP32cpuPassedModeratemacos · aarch648443 ms / 12265 ms
FP32cpuPassedModeratemacos · aarch648783 ms / 3895 ms
FP32cpuPassedModeratemacos · aarch648898 ms / 12265 ms
FP32cpuPassedModeratemacos · aarch6410471 ms / 4907 ms
FP32cpuPassedModeratemacos · aarch6416483 ms / 4907 ms
FP32cpuPassedModeratemacos · aarch6417719 ms / 4907 ms
BF16mpsFailed—macos · aarch643042 ms / 23206 ms
FP32mpsFailed—macos · aarch643610 ms / 22668 ms
Test setup & sample details
Quality gate
{ "minimumCharacterF2": 0.55, "requireSemanticAnchors": true, "rejectRepetitionAndTruncation": true }
Tested
2026-11-06T13:34:21.657206+00:00
Environment
{ "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2", "runtime": "App-owned CPython3.13.18; Transformers5.22.1/Torch2.10.0; built-in HunYuanDenseV1ForCausalLM, SDPA; MPS CPU op fallback disabled" }
Runtime
transformers 5.22.1
Artifact revision
9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223
Fixture
{
  "file": "model-samples/hunyuan-translation.json",
  "url": "https://assets.localai.tools/samples/v1/files/b63cba0b5f61f8423d4234d4d28e9cdde538b8345b4ccdc8b2044b1bc88e2fa8/hunyuan-translation.json",
  "size": 3099,
  "sha1": "dd6e22e1c2966dd5a249b11eb2fb48b076505088",
  "sha256": "b63cba0b5f61f8423d4234d4d28e9cdde538b8345b4ccdc8b2044b1bc88e2fa8"
}
Rerun when
model artifact revision change; runtime version change; fixture or quality-gate change; backend/platform behavior change
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4022.0228329999372,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 6494.175249999898,
      "inferenceMs": 6487.1073750000505,
      "tokensPerSecond": 1.849822934370638,
      "inputTokens": 32,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 4,
      "totalMs": 10517.494333000286,
      "residentMemoryBytes": 30606082048,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4022.0228329999372,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 7364.277582999421,
      "inferenceMs": 7363.7570410010085,
      "tokensPerSecond": 2.851805115659454,
      "inputTokens": 39,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 4,
      "totalMs": 11387.184748999061,
      "residentMemoryBytes": 30612570112,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4022.0228329999372,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3035.7154169996647,
      "inferenceMs": 3035.2572909996525,
      "tokensPerSecond": 2.6356908930660126,
      "inputTokens": 35,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 4,
      "totalMs": 7058.587040999555,
      "residentMemoryBytes": 30616649728,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4022.0228329999372,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 4189.968167000188,
      "inferenceMs": 4189.6531250004045,
      "tokensPerSecond": 2.8641989305496125,
      "inputTokens": 31,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 4,
      "totalMs": 8212.78966599857,
      "residentMemoryBytes": 30619959296,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.8772489985666848,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4022.0228329999372,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 5826.339375000316,
      "inferenceMs": 5825.831374999325,
      "tokensPerSecond": 2.9180384576445055,
      "inputTokens": 32,
      "outputTokens": 17,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 4,
      "totalMs": 9849.2066660001,
      "residentMemoryBytes": 30619959296,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4022.0228329999372,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 8407.206874999247,
      "inferenceMs": 8406.83908399842,
      "tokensPerSecond": 2.9737693026127983,
      "inputTokens": 30,
      "outputTokens": 25,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 4,
      "totalMs": 12430.087041000661,
      "residentMemoryBytes": 30619959296,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4022.0228329999372,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 4165.448792000461,
      "inferenceMs": 4165.125915998942,
      "tokensPerSecond": 2.8810653608108225,
      "inputTokens": 32,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 4,
      "totalMs": 8188.344999998662,
      "residentMemoryBytes": 30619959296,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3945.2491249994637,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 4620.3236670007755,
      "inferenceMs": 4616.5452919995005,
      "tokensPerSecond": 2.599346316561882,
      "inputTokens": 32,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 4,
      "totalMs": 8566.323208999165,
      "residentMemoryBytes": 30603968512,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3945.2491249994637,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 7335.508834001303,
      "inferenceMs": 7335.092417000851,
      "tokensPerSecond": 2.8629496134673666,
      "inputTokens": 39,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 4,
      "totalMs": 11281.639124999856,
      "residentMemoryBytes": 30611259392,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3945.2491249994637,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3047.0905830006814,
      "inferenceMs": 3046.679834000315,
      "tokensPerSecond": 2.625809220490338,
      "inputTokens": 35,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 4,
      "totalMs": 6993.1555000002845,
      "residentMemoryBytes": 30620844032,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3945.2491249994637,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 4213.6778330004745,
      "inferenceMs": 4213.337375000265,
      "tokensPerSecond": 2.8480985337660614,
      "inputTokens": 31,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 4,
      "totalMs": 8159.799958000804,
      "residentMemoryBytes": 30621679616,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.8772489985666848,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3945.2491249994637,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 5795.280541999091,
      "inferenceMs": 5794.827292000264,
      "tokensPerSecond": 2.9336508481397594,
      "inputTokens": 32,
      "outputTokens": 17,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 4,
      "totalMs": 9741.396333000012,
      "residentMemoryBytes": 30621990912,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3945.2491249994637,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 8373.782374999792,
      "inferenceMs": 8373.424624998734,
      "tokensPerSecond": 2.9856362384110886,
      "inputTokens": 30,
      "outputTokens": 25,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 4,
      "totalMs": 12319.861582998783,
      "residentMemoryBytes": 30621990912,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3945.2491249994637,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 4231.950459001382,
      "inferenceMs": 4231.638958000985,
      "tokensPerSecond": 2.835780679566474,
      "inputTokens": 32,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 4,
      "totalMs": 8177.99579199891,
      "residentMemoryBytes": 30621990912,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3894.951875001425,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 4933.41379100093,
      "inferenceMs": 4929.271834000247,
      "tokensPerSecond": 2.4344366478692763,
      "inputTokens": 32,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 4,
      "totalMs": 8829.159250000885,
      "residentMemoryBytes": 30605721600,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3894.951875001425,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 7847.231791998638,
      "inferenceMs": 7846.855708001385,
      "tokensPerSecond": 2.6762311913785344,
      "inputTokens": 39,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 4,
      "totalMs": 11742.96379200132,
      "residentMemoryBytes": 30610014208,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3894.951875001425,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3234.80387500058,
      "inferenceMs": 3234.3485829987912,
      "tokensPerSecond": 2.4734501537810867,
      "inputTokens": 35,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 4,
      "totalMs": 7130.638041000566,
      "residentMemoryBytes": 30614192128,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3894.951875001425,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 4344.249707999552,
      "inferenceMs": 4343.851790999906,
      "tokensPerSecond": 2.76252519132052,
      "inputTokens": 31,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 4,
      "totalMs": 8240.01033300192,
      "residentMemoryBytes": 30620499968,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.8772489985666848,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3894.951875001425,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 6270.701542000097,
      "inferenceMs": 6270.2846660013165,
      "tokensPerSecond": 2.711200671984998,
      "inputTokens": 32,
      "outputTokens": 17,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 4,
      "totalMs": 10166.503417000058,
      "residentMemoryBytes": 30620745728,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3894.951875001425,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 8782.950540999082,
      "inferenceMs": 8782.525125001484,
      "tokensPerSecond": 2.8465617398385494,
      "inputTokens": 30,
      "outputTokens": 25,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 4,
      "totalMs": 12678.93712500154,
      "residentMemoryBytes": 30620745728,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3894.951875001425,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 4364.486624999699,
      "inferenceMs": 4364.040583999667,
      "tokensPerSecond": 2.7497452805541815,
      "inputTokens": 32,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 4,
      "totalMs": 8260.30216700019,
      "residentMemoryBytes": 30620909568,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3909.921165999549,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 4975.298750001457,
      "inferenceMs": 4970.807874999082,
      "tokensPerSecond": 2.414094509738463,
      "inputTokens": 32,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 2,
      "totalMs": 8886.006124999767,
      "residentMemoryBytes": 30604869632,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3909.921165999549,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 7286.183709000397,
      "inferenceMs": 7285.776500000793,
      "tokensPerSecond": 2.8823283283528824,
      "inputTokens": 39,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 2,
      "totalMs": 11196.97558199914,
      "residentMemoryBytes": 30612930560,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3909.921165999549,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3080.272249999325,
      "inferenceMs": 3079.848833000142,
      "tokensPerSecond": 2.5975300846850464,
      "inputTokens": 35,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 2,
      "totalMs": 6991.044791000604,
      "residentMemoryBytes": 30616076288,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3909.921165999549,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 4201.54999999977,
      "inferenceMs": 4201.203041999179,
      "tokensPerSecond": 2.856324695578078,
      "inputTokens": 31,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 2,
      "totalMs": 8112.319666000985,
      "residentMemoryBytes": 30618222592,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.8772489985666848,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3909.921165999549,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 5877.0284579986765,
      "inferenceMs": 5876.462333000745,
      "tokensPerSecond": 2.8928969568191127,
      "inputTokens": 32,
      "outputTokens": 17,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 2,
      "totalMs": 9787.875207999605,
      "residentMemoryBytes": 30618255360,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3909.921165999549,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 8399.129334000463,
      "inferenceMs": 8398.718917000224,
      "tokensPerSecond": 2.9766444438801707,
      "inputTokens": 30,
      "outputTokens": 25,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 2,
      "totalMs": 12309.84024999998,
      "residentMemoryBytes": 30618419200,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3909.921165999549,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 4181.007582999882,
      "inferenceMs": 4180.699792001178,
      "tokensPerSecond": 2.8703328622062942,
      "inputTokens": 32,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 2,
      "totalMs": 8091.7292489994,
      "residentMemoryBytes": 30618419200,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4066.322250000667,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 4542.732040999908,
      "inferenceMs": 4538.781625000411,
      "tokensPerSecond": 2.6438813301573885,
      "inputTokens": 32,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 2,
      "totalMs": 8609.964458000832,
      "residentMemoryBytes": 30606409728,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4066.322250000667,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 7108.121750001374,
      "inferenceMs": 7107.7397919998475,
      "tokensPerSecond": 2.9545257162673084,
      "inputTokens": 39,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 2,
      "totalMs": 11175.250416999916,
      "residentMemoryBytes": 30608703488,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4066.322250000667,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3128.2166670007427,
      "inferenceMs": 3127.816915999574,
      "tokensPerSecond": 2.5576944606565615,
      "inputTokens": 35,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 2,
      "totalMs": 7195.364000001064,
      "residentMemoryBytes": 30612783104,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4066.322250000667,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 4251.979165999728,
      "inferenceMs": 4251.642457998969,
      "tokensPerSecond": 2.822438650132351,
      "inputTokens": 31,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 2,
      "totalMs": 8319.148915999904,
      "residentMemoryBytes": 30616961024,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.8772489985666848,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4066.322250000667,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 5993.992457999411,
      "inferenceMs": 5993.595500000083,
      "tokensPerSecond": 2.836360912243705,
      "inputTokens": 32,
      "outputTokens": 17,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 2,
      "totalMs": 10061.203500001284,
      "residentMemoryBytes": 30617059328,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4066.322250000667,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 8310.341374999552,
      "inferenceMs": 8309.993082999426,
      "tokensPerSecond": 3.0084260901666657,
      "inputTokens": 30,
      "outputTokens": 25,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 2,
      "totalMs": 12377.470583000104,
      "residentMemoryBytes": 30617911296,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4066.322250000667,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 4085.2622920010617,
      "inferenceMs": 4084.950750000644,
      "tokensPerSecond": 2.9376119161285135,
      "inputTokens": 32,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 2,
      "totalMs": 8152.549792001082,
      "residentMemoryBytes": 30617911296,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3910.641000000396,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 4844.899125000666,
      "inferenceMs": 4840.623124999183,
      "tokensPerSecond": 2.4790196819964216,
      "inputTokens": 32,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 2,
      "totalMs": 8756.358333001117,
      "residentMemoryBytes": 30602002432,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3910.641000000396,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 7239.205292000406,
      "inferenceMs": 7238.761124999655,
      "tokensPerSecond": 2.9010489001322033,
      "inputTokens": 39,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 2,
      "totalMs": 11150.688458001241,
      "residentMemoryBytes": 30612013056,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3910.641000000396,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 2985.626042000149,
      "inferenceMs": 2985.2354169997852,
      "tokensPerSecond": 2.6798556503929403,
      "inputTokens": 35,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 2,
      "totalMs": 6897.072833999118,
      "residentMemoryBytes": 30612062208,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3910.641000000396,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 4196.046208000553,
      "inferenceMs": 4195.742042000347,
      "tokensPerSecond": 2.8600423667320887,
      "inputTokens": 31,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 2,
      "totalMs": 8107.486667000558,
      "residentMemoryBytes": 30616518656,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.8772489985666848,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3910.641000000396,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 5608.752417001597,
      "inferenceMs": 5608.377624999775,
      "tokensPerSecond": 3.0311796274596756,
      "inputTokens": 32,
      "outputTokens": 17,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 2,
      "totalMs": 9520.176083000479,
      "residentMemoryBytes": 30616518656,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3910.641000000396,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 8218.601124999623,
      "inferenceMs": 8218.29975000037,
      "tokensPerSecond": 3.041991745311903,
      "inputTokens": 30,
      "outputTokens": 25,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 2,
      "totalMs": 12130.04070900024,
      "residentMemoryBytes": 30616911872,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3910.641000000396,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 4022.1556659998896,
      "inferenceMs": 4021.8333749999147,
      "tokensPerSecond": 2.9837138640782834,
      "inputTokens": 32,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 2,
      "totalMs": 7933.63516700083,
      "residentMemoryBytes": 30616928256,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4574.295749998782,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1055.83895899872,
      "inferenceMs": 1050.4177920010989,
      "tokensPerSecond": 11.424025841317286,
      "inputTokens": 32,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15088517120,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "precision": "fp16",
      "cpuThreads": 4,
      "totalMs": 5631.066916997952,
      "residentMemoryBytes": 704987136,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "good",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4574.295749998782,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1470.30787499898,
      "inferenceMs": 1467.8658329994505,
      "tokensPerSecond": 14.306484644504877,
      "inputTokens": 39,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15097888768,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "precision": "fp16",
      "cpuThreads": 4,
      "totalMs": 6045.597374997669,
      "residentMemoryBytes": 773537792,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "good",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4574.295749998782,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 663.8540000003559,
      "inferenceMs": 661.5540000002511,
      "tokensPerSecond": 12.09273921705101,
      "inputTokens": 35,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15098265600,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "precision": "fp16",
      "cpuThreads": 4,
      "totalMs": 5238.965957998516,
      "residentMemoryBytes": 786661376,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "good",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4574.295749998782,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 905.468375000055,
      "inferenceMs": 903.4549170009996,
      "tokensPerSecond": 13.282345111180266,
      "inputTokens": 31,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15098609664,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "precision": "fp16",
      "cpuThreads": 4,
      "totalMs": 5480.917040998975,
      "residentMemoryBytes": 803717120,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "good",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.8772489985666848,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4574.295749998782,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1171.9027500002994,
      "inferenceMs": 1169.818624999607,
      "tokensPerSecond": 14.532167326371395,
      "inputTokens": 32,
      "outputTokens": 17,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15098609664,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "precision": "fp16",
      "cpuThreads": 4,
      "totalMs": 5747.296540999741,
      "residentMemoryBytes": 804962304,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "good",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4574.295749998782,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1807.6829590008856,
      "inferenceMs": 1805.7072499996139,
      "tokensPerSecond": 13.84499065394202,
      "inputTokens": 30,
      "outputTokens": 25,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15098937344,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "precision": "fp16",
      "cpuThreads": 4,
      "totalMs": 6383.049208998273,
      "residentMemoryBytes": 822607872,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "good",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4574.295749998782,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 904.1937920010241,
      "inferenceMs": 902.283333000014,
      "tokensPerSecond": 13.29959178133219,
      "inputTokens": 32,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15098937344,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "precision": "fp16",
      "cpuThreads": 4,
      "totalMs": 5479.58270899835,
      "residentMemoryBytes": 823541760,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "good",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4667.247833998772,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1026.13966699937,
      "inferenceMs": 1020.0371249993623,
      "tokensPerSecond": 11.764277697252933,
      "inputTokens": 32,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15088517120,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "precision": "fp16",
      "cpuThreads": 4,
      "totalMs": 5694.2606669981615,
      "residentMemoryBytes": 706691072,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "good",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4667.247833998772,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1461.0373329996946,
      "inferenceMs": 1459.3446250000852,
      "tokensPerSecond": 14.390021137055802,
      "inputTokens": 39,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15097888768,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "precision": "fp16",
      "cpuThreads": 4,
      "totalMs": 6129.278291999071,
      "residentMemoryBytes": 776830976,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "good",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4667.247833998772,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 668.9497909992497,
      "inferenceMs": 666.9504170004075,
      "tokensPerSecond": 11.9948946669526,
      "inputTokens": 35,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15098265600,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "precision": "fp16",
      "cpuThreads": 4,
      "totalMs": 5336.987749999025,
      "residentMemoryBytes": 790085632,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "good",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4667.247833998772,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 915.9534579994215,
      "inferenceMs": 913.7914590010041,
      "tokensPerSecond": 13.13209910401101,
      "inputTokens": 31,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15098609664,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "precision": "fp16",
      "cpuThreads": 4,
      "totalMs": 5584.252583998023,
      "residentMemoryBytes": 806993920,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "good",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.8772489985666848,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4667.247833998772,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1169.325166998533,
      "inferenceMs": 1167.1897910000553,
      "tokensPerSecond": 14.56489778361949,
      "inputTokens": 32,
      "outputTokens": 17,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15098609664,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "precision": "fp16",
      "cpuThreads": 4,
      "totalMs": 5837.45987599832,
      "residentMemoryBytes": 808435712,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "good",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4667.247833998772,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1710.9502919993247,
      "inferenceMs": 1708.9125829988916,
      "tokensPerSecond": 14.62918597985197,
      "inputTokens": 30,
      "outputTokens": 25,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15098937344,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "precision": "fp16",
      "cpuThreads": 4,
      "totalMs": 6379.165791999185,
      "residentMemoryBytes": 826671104,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "good",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4667.247833998772,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 856.6502920002677,
      "inferenceMs": 855.3774170013639,
      "tokensPerSecond": 14.028895036845315,
      "inputTokens": 32,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15098937344,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "precision": "fp16",
      "cpuThreads": 4,
      "totalMs": 5524.878291998903,
      "residentMemoryBytes": 827637760,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "good",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4565.782250001575,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1061.2369589998707,
      "inferenceMs": 1056.0281660000328,
      "tokensPerSecond": 11.3633332768509,
      "inputTokens": 32,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15088517120,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "precision": "fp16",
      "cpuThreads": 4,
      "totalMs": 5627.958292001495,
      "residentMemoryBytes": 702185472,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "good",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4565.782250001575,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1461.458624999068,
      "inferenceMs": 1459.5982920000097,
      "tokensPerSecond": 14.387520261636386,
      "inputTokens": 39,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15097888768,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "precision": "fp16",
      "cpuThreads": 4,
      "totalMs": 6028.138041001512,
      "residentMemoryBytes": 770736128,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "good",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4565.782250001575,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 660.0942080003733,
      "inferenceMs": 658.8478329995269,
      "tokensPerSecond": 12.142409216978853,
      "inputTokens": 35,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15098265600,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "precision": "fp16",
      "cpuThreads": 4,
      "totalMs": 5226.789625001402,
      "residentMemoryBytes": 783794176,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "good",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4565.782250001575,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 914.5162089989753,
      "inferenceMs": 912.7977909993206,
      "tokensPerSecond": 13.146394654244876,
      "inputTokens": 31,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15098609664,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "precision": "fp16",
      "cpuThreads": 4,
      "totalMs": 5481.202334001864,
      "residentMemoryBytes": 800702464,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "good",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.8772489985666848,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4565.782250001575,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1153.0442090006545,
      "inferenceMs": 1151.0140419995878,
      "tokensPerSecond": 14.769585235004534,
      "inputTokens": 32,
      "outputTokens": 17,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15098609664,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "precision": "fp16",
      "cpuThreads": 4,
      "totalMs": 5719.807541001501,
      "residentMemoryBytes": 802062336,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "good",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4565.782250001575,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1689.92791600067,
      "inferenceMs": 1688.0604160014627,
      "tokensPerSecond": 14.809896472318167,
      "inputTokens": 30,
      "outputTokens": 25,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15098937344,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "precision": "fp16",
      "cpuThreads": 4,
      "totalMs": 6256.759125000826,
      "residentMemoryBytes": 819494912,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "good",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4565.782250001575,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 847.2154589999263,
      "inferenceMs": 845.6187919982767,
      "tokensPerSecond": 14.19079154052723,
      "inputTokens": 32,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15098937344,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "precision": "fp16",
      "cpuThreads": 4,
      "totalMs": 5414.094375002605,
      "residentMemoryBytes": 820412416,
      "resourceMeasurementScope": "Post-inference process RSS; MPS driver allocation where available, not peak or a validated minimum",
      "grade": "good",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "fallback": false
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "fail",
    "quality": {
      "passed": false,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": false
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 23205.868708000708,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.bfloat16": 354
      },
      "parameterDtype": "torch.bfloat16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3050.0502079994476,
      "inferenceMs": 3041.5640420014824,
      "tokensPerSecond": 3.9453385936610013,
      "inputTokens": 32,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15088517120,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "precision": "bf16",
      "cpuThreads": 4,
      "totalMs": 26256.862499001727,
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "mps",
      "requestedPrecision": "bf16",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 12264.60891700117,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 8450.321665999581,
      "inferenceMs": 8442.504000000554,
      "tokensPerSecond": 1.4213792495685182,
      "inputTokens": 32,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 4,
      "totalMs": 20715.750749999643,
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 12264.60891700117,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 7282.779500001197,
      "inferenceMs": 7282.349166000131,
      "tokensPerSecond": 2.883684855162522,
      "inputTokens": 39,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 4,
      "totalMs": 19548.181125001065,
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 12264.60891700117,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3086.745249998785,
      "inferenceMs": 3086.4015410006687,
      "tokensPerSecond": 2.5920152947455604,
      "inputTokens": 35,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 4,
      "totalMs": 15352.081042001373,
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 12264.60891700117,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 4238.076084000568,
      "inferenceMs": 4237.744000000021,
      "tokensPerSecond": 2.831695354886926,
      "inputTokens": 31,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 4,
      "totalMs": 16503.47645900183,
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.8772489985666848,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 12264.60891700117,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 5818.490167001073,
      "inferenceMs": 5818.08970900056,
      "tokensPerSecond": 2.921921257711285,
      "inputTokens": 32,
      "outputTokens": 17,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 4,
      "totalMs": 18083.896167001512,
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 12264.60891700117,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 8898.271042000488,
      "inferenceMs": 8897.999082999377,
      "tokensPerSecond": 2.8096204289080338,
      "inputTokens": 30,
      "outputTokens": 25,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 4,
      "totalMs": 21163.626667001154,
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4907.014500000514,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 4808.753625000463,
      "inferenceMs": 4804.160792000403,
      "tokensPerSecond": 2.497834797699043,
      "inputTokens": 32,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 2,
      "totalMs": 9716.658749999624,
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4907.014500000514,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 7402.894999999262,
      "inferenceMs": 7402.5348750001285,
      "tokensPerSecond": 2.8368660674495825,
      "inputTokens": 39,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 2,
      "totalMs": 12310.727000001862,
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4907.014500000514,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3842.158749999726,
      "inferenceMs": 3841.842165998969,
      "tokensPerSecond": 2.082334373546502,
      "inputTokens": 35,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 2,
      "totalMs": 8750.225834000958,
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4907.014500000514,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 10471.883041000183,
      "inferenceMs": 10471.499625000433,
      "tokensPerSecond": 1.1459676674533141,
      "inputTokens": 31,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 2,
      "totalMs": 15380.878500000108,
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.8772489985666848,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4907.014500000514,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 16483.510458998353,
      "inferenceMs": 16482.942583001204,
      "tokensPerSecond": 1.03136924213593,
      "inputTokens": 32,
      "outputTokens": 17,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 2,
      "totalMs": 21391.742499999964,
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4907.014500000514,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 17719.418915999995,
      "inferenceMs": 17719.05975000118,
      "tokensPerSecond": 1.4109100794695575,
      "inputTokens": 30,
      "outputTokens": 25,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "precision": "fp32",
      "cpuThreads": 2,
      "totalMs": 22627.242042000944,
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "fail",
    "quality": {
      "passed": false,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": false
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 22668.143791999682,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float32": 354
      },
      "parameterDtype": "torch.float32",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 3616.6300829991087,
      "inferenceMs": 3609.568040999875,
      "tokensPerSecond": 3.324497519840606,
      "inputTokens": 32,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 30131388416,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "precision": "fp32",
      "cpuThreads": 4,
      "totalMs": 26285.963624999567,
      "grade": "acceptable",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "mps",
      "requestedPrecision": "fp32",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5109.491499999422,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1555.3694169993832,
      "inferenceMs": 1550.3912500007573,
      "tokensPerSecond": 7.739981762664192,
      "inputTokens": 32,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15088517120,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "precision": "fp16",
      "cpuThreads": 4,
      "totalMs": 6665.8748329991795,
      "grade": "good",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5109.491499999422,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1694.610124999599,
      "inferenceMs": 1693.4167920007894,
      "tokensPerSecond": 12.400963601635414,
      "inputTokens": 39,
      "outputTokens": 21,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15097888768,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "precision": "fp16",
      "cpuThreads": 4,
      "totalMs": 6805.068833000405,
      "grade": "good",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5109.491499999422,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 672.4714169995423,
      "inferenceMs": 669.121916000222,
      "tokensPerSecond": 11.955967677491744,
      "inputTokens": 35,
      "outputTokens": 8,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15098265600,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "precision": "fp16",
      "cpuThreads": 4,
      "totalMs": 5782.86295900034,
      "grade": "good",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5109.491499999422,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1104.9929999990127,
      "inferenceMs": 1103.0618340009823,
      "tokensPerSecond": 10.878809899961885,
      "inputTokens": 31,
      "outputTokens": 12,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15098609664,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "precision": "fp16",
      "cpuThreads": 4,
      "totalMs": 6215.443333998337,
      "grade": "good",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 0.8772489985666848,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5109.491499999422,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1184.4537500001024,
      "inferenceMs": 1182.711792000191,
      "tokensPerSecond": 14.37374693901526,
      "inputTokens": 32,
      "outputTokens": 17,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15098609664,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "precision": "fp16",
      "cpuThreads": 4,
      "totalMs": 6294.840291999208,
      "grade": "good",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "characterF2": 1,
      "threshold": 0.55,
      "anchorsPassed": true,
      "repetitive": false,
      "metric": "best authored reference character-ngram F2 orders1–6",
      "speedPassed": true
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 5109.491499999422,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 354
      },
      "parameterDtype": "torch.float16",
      "accelerationDetails": "All parameters and model inputs on selected device; tokenization/decoding on CPU; MPS CPU fallback disabled",
      "durationMs": 1850.5546249998588,
      "inferenceMs": 1849.0752080015227,
      "tokensPerSecond": 13.520272129450028,
      "inputTokens": 30,
      "outputTokens": 25,
      "maxNewTokens": 128,
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 15098937344,
      "memoryMeasurementScope": "Post-inference driver allocation, not peak",
      "precision": "fp16",
      "cpuThreads": 4,
      "totalMs": 6960.866916999294,
      "grade": "good",
      "artifactRevision": "9b0eb4e8f001def3e5ff6469a0ac96fdb39ec223",
      "modelId": "hy-mt2-7b-native",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "fallback": false
    }
  }
]

Something not quite right?

Tell us what happened with Hy-MT 2. Loading, results, or the page itself—we’re here for all of it.

Attachments Optional

Up to 3 files · 50 MB each

Browser, OS, CPU/GPU, memory, screen, storage, and network details are included automatically. Model inputs, results, and browsing history are not collected.