A model is still running

A model is still running or downloading. Leaving will stop the current operation. Stay on this page to let it finish.

Skip to content
LocalAI.toolsBeta

Join the waitlist

Get an email when the new apps and model runs launch.

No account needed.

EttinX NLI Desktop

Compare two English statements and score contradiction, entailment and neutral relationships locally with the original XS · 32M checkpoint.

Desktop inferencePrivate · Runs on your computer
About this model

Compare two English statements and score contradiction, entailment and neutral relationships locally with the original XS · 32M checkpoint.

Created by
D. Lee Miller
License
MIT
Model released
Unknown
Source repository created
Sep 8, 2025
Model weights · FP32
128.1 MB

Best for

  • Compare whether one English statement entails, contradicts, or is neutral toward another.
  • Prototype lightweight claim consistency and LLM-output evaluation.
  • Benchmark browser NLI across ONNX precisions and backends.

Keep in mind

  • Designed primarily for English and may degrade on other languages or specialized domains.
  • Labels are probabilistic model judgments, not logical proofs or factuality guarantees.
  • Long inputs are truncated at the configured token limit and local attention can miss distant relationships.
  • Ambiguity, unstated world knowledge, negation, numerical reasoning, and adversarial phrasing can cause errors.
  • Do not use the classification alone for safety-critical or high-impact decisions.
  • Six short English sanity fixtures only, not full MNLI/SNLI or factuality validation.
  • CPU FP32 is faster than Metal on the measured short pairs; GPU support is independently verified but not the default.
  • Tokenizer/preprocessing, softmax/label sorting and serialization run on CPU; only model forward/parameters use MPS.
  • Windows, Intel Mac and other Apple hardware/OS remain unverified.

Processed on your deviceYour input is processed locally in the desktop app. Model and runtime files download from external hosts.

Target platforms · verification required

  • macos · aarch64, x86_64
  • windows · x86_64

Recommended configuration

FP32 · cpu (verified platform only)

Sources & technical details
Original model
dleemiller/EttinX-nli-xs
Release verification
Unknown
Release evidence
Exact original checkpoint public release date has not been verified. Repository creation, training, paper, artifact conversion and update dates are not release evidence.
Source updated
Sep 9, 2025
Native model files
dleemiller/EttinX-nli-xs
Files updated
Sep 9, 2025
Artifact revision
802360b6ffe6d4627f1d28fd3225522e247cd3e8

Desktop app

Sample result

EttinX NLI XS Desktop

Run EttinX NLI Desktop on your computer

Download the desktop app to load this model, provide input and view results locally.

Public desktop downloads are coming soon.

Device requirements

macOS Apple SiliconCPU / FP32Tested8.0 GiBRAM estimate
macOS Apple SiliconMPS / FP32Tested8.0 GiBRAM estimate
macOS Apple SiliconMPS / FP16Tested8.0 GiBRAM estimate
Windows Intel / AMDCPU / FP32Unverified8.0 GiBRAM estimate
macOS Intel / AMDCPU / FP32Unverified8.0 GiBRAM estimate
If your device falls short

MemoryAdvisory only: close other apps, reduce image/tile/token/batch settings or choose a smaller verified model/precision. CPU can need more RAM than GPU.

GPUIf the selected GPU runtime fails, existing CPU fallback may recover when enabled, but CPU has its own memory/runtime needs.

PlatformUse a compatible OS/architecture/runtime or another model. These estimates do not add hardware detection or block runs.

Benchmark evidence

Saved desktop measurements apply to the recorded hardware and runtime. Fallback cases report the device that actually ran; they do not prove GPU support on the requested device.

EttinX NLI XS Desktop

72 configurations tested
Passed72
Failed0
TestedNov 12, 2026
Runtimetransformers 5.22.1

Quality criteria

{ "file": "research/ettinx-nli-family-expansion/quality-gates.json", "sha256": "b98612c89959b9c069902c28bfd95bd8d943b61cefa34ea3c6da14ace9f0c18c", "description": "All six expected labels, per-case minimum margins, complete finite normalized three-label probabilities, <=1000ms/pair." }

PrecisionDeviceResultPlatformInference / load
FP32mpsAcceleratedPassedVery fastmacos · aarch647 ms / 1779 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch647 ms / 1778 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch649 ms / 1836 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6410 ms / 1771 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6411 ms / 1764 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6411 ms / 1729 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6421 ms / 1729 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6422 ms / 1764 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6422 ms / 1764 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6422 ms / 1764 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6422 ms / 1836 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6423 ms / 1771 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6423 ms / 1764 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6424 ms / 1778 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6424 ms / 1778 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6424 ms / 1771 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6425 ms / 1778 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6426 ms / 1729 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6426 ms / 1836 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6426 ms / 1778 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6426 ms / 1771 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6426 ms / 1779 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6426 ms / 1729 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6427 ms / 1779 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6428 ms / 1771 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6428 ms / 1779 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6428 ms / 1779 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6431 ms / 1836 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6472 ms / 1778 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6472 ms / 1764 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6478 ms / 1771 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6482 ms / 1779 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch64100 ms / 1729 ms
FP16mpsAcceleratedPassedFastmacos · aarch64106 ms / 1836 ms
FP32mpsAcceleratedPassedFastmacos · aarch64402 ms / 1729 ms
FP16mpsAcceleratedPassedFastmacos · aarch64418 ms / 1836 ms
FP32cpuPassedVery fastmacos · aarch643 ms / 1583 ms
FP32cpuPassedVery fastmacos · aarch643 ms / 1605 ms
FP32cpuPassedVery fastmacos · aarch643 ms / 1601 ms
FP32cpuPassedVery fastmacos · aarch643 ms / 1583 ms
FP32cpuPassedVery fastmacos · aarch644 ms / 1583 ms
FP32cpuPassedVery fastmacos · aarch644 ms / 1605 ms
FP32cpuPassedVery fastmacos · aarch644 ms / 1601 ms
FP32cpuPassedVery fastmacos · aarch644 ms / 1605 ms
FP32cpuPassedVery fastmacos · aarch644 ms / 1682 ms
FP32cpuPassedVery fastmacos · aarch644 ms / 1682 ms
FP32cpuPassedVery fastmacos · aarch644 ms / 1601 ms
FP32cpuPassedVery fastmacos · aarch644 ms / 1682 ms
FP32cpuPassedVery fastmacos · aarch644 ms / 1611 ms
FP32cpuPassedVery fastmacos · aarch644 ms / 1605 ms
FP32cpuPassedVery fastmacos · aarch644 ms / 1677 ms
FP32cpuPassedVery fastmacos · aarch644 ms / 1611 ms
FP32cpuPassedVery fastmacos · aarch644 ms / 1611 ms
FP32cpuPassedVery fastmacos · aarch644 ms / 1677 ms
FP32cpuPassedVery fastmacos · aarch644 ms / 1601 ms
FP32cpuPassedVery fastmacos · aarch644 ms / 1677 ms
FP32cpuPassedVery fastmacos · aarch644 ms / 1583 ms
FP32cpuPassedVery fastmacos · aarch644 ms / 1611 ms
FP32cpuPassedVery fastmacos · aarch644 ms / 1682 ms
FP32cpuPassedVery fastmacos · aarch644 ms / 1677 ms
FP32cpuPassedVery fastmacos · aarch645 ms / 1583 ms
FP32cpuPassedVery fastmacos · aarch645 ms / 1605 ms
FP32cpuPassedVery fastmacos · aarch645 ms / 1601 ms
FP32cpuPassedVery fastmacos · aarch645 ms / 1682 ms
FP32cpuPassedVery fastmacos · aarch646 ms / 1611 ms
FP32cpuPassedVery fastmacos · aarch646 ms / 1677 ms
FP32cpuPassedVery fastmacos · aarch6466 ms / 1682 ms
FP32cpuPassedVery fastmacos · aarch6466 ms / 1605 ms
FP32cpuPassedVery fastmacos · aarch6467 ms / 1601 ms
FP32cpuPassedVery fastmacos · aarch6468 ms / 1583 ms
FP32cpuPassedVery fastmacos · aarch6468 ms / 1611 ms
FP32cpuPassedVery fastmacos · aarch6470 ms / 1677 ms
Test setup & sample details
Quality gate
{ "file": "research/ettinx-nli-family-expansion/quality-gates.json", "sha256": "b98612c89959b9c069902c28bfd95bd8d943b61cefa34ea3c6da14ace9f0c18c", "description": "All six expected labels, per-case minimum margins, complete finite normalized three-label probabilities, <=1000ms/pair." }
Tested
2026-11-12T22:03:07.787398+00:00
Environment
{ "os": "macos", "osVersion": "27.2", "architecture": "aarch64", "hardware": "Apple M3 Max;Mac15,11;96GiB RAM", "cpu": "Apple M3 Max", "gpu": "Apple M3 Max", "driver": "OS-managed Metal/MPS macOS27.2 build26B118", "runtime": "App-owned CPython3.13.18;Torch2.10.0;Transformers5.22.1", "timingScope": "Fresh helper/model process per cold repetition; runtime/tokenizer/model startup in load. OS files/driver cache may be warm. Five warm pairs follow six fixtures. Snapshot loads, not Internet transfer.", "concurrentUploadActivity": "none; uploads excluded" }
Runtime
transformers 5.22.1
Artifact revision
802360b6ffe6d4627f1d28fd3225522e247cd3e8
Fixture
{
  "file": "model-samples/native/ettinx-nli/quality-gates.json",
  "url": "https://assets.localai.tools/samples/v1/files/b98612c89959b9c069902c28bfd95bd8d943b61cefa34ea3c6da14ace9f0c18c/ettinx-nli-quality-gates.json",
  "size": 2262,
  "sha1": "9104fbed107fd4340666e85c8fcaf3376ba1a2a4",
  "sha256": "b98612c89959b9c069902c28bfd95bd8d943b61cefa34ea3c6da14ace9f0c18c"
}
Rerun when
artifact-revision; runtime-version; fixture-checksum; quality-gate; backend-behavior
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.934897530823946,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 67.37174998852424,
      "inferenceMs": 66.93391699809581,
      "pairsPerSecond": 14.940108764715635,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1601.1102499905974,
      "totalMs": 1668.0441669886932,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9997462833562167,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 5.500166997080669,
      "inferenceMs": 5.259833007585257,
      "pairsPerSecond": 190.12010429948825,
      "inputTokens": 26,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1601.1102499905974,
      "totalMs": 1606.3700829981826,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9828259916976094,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 4.119207995245233,
      "inferenceMs": 3.9364160038530827,
      "pairsPerSecond": 254.0381908368348,
      "inputTokens": 18,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1601.1102499905974,
      "totalMs": 1605.0466659944504,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9937077884096652,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 3.6121250013820827,
      "inferenceMs": 3.4612089802976698,
      "pairsPerSecond": 288.9163889532028,
      "inputTokens": 16,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1601.1102499905974,
      "totalMs": 1604.571458970895,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9895947277545929,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 3.6834999918937683,
      "inferenceMs": 3.545208979630843,
      "pairsPerSecond": 282.07081888417434,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1601.1102499905974,
      "totalMs": 1604.6554589702282,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9861082676798105,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 3.8606250018347055,
      "inferenceMs": 3.7147499970160425,
      "pairsPerSecond": 269.1971198070591,
      "inputTokens": 15,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1601.1102499905974,
      "totalMs": 1604.8249999876134,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.934897530823946,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 66.45591699634679,
      "inferenceMs": 65.99516599089839,
      "pairsPerSecond": 15.152624968591084,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1605.0223339989316,
      "totalMs": 1671.01749998983,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9997462833562167,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 5.384291987866163,
      "inferenceMs": 5.161624983884394,
      "pairsPerSecond": 193.7374379429339,
      "inputTokens": 26,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1605.0223339989316,
      "totalMs": 1610.183958982816,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9828259916976094,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 4.005166993010789,
      "inferenceMs": 3.834792005363852,
      "pairsPerSecond": 260.77033607070905,
      "inputTokens": 18,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1605.0223339989316,
      "totalMs": 1608.8571260042954,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9937077884096652,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 3.6725419922731817,
      "inferenceMs": 3.5236670228186995,
      "pairsPerSecond": 283.79526031380414,
      "inputTokens": 16,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1605.0223339989316,
      "totalMs": 1608.5460010217503,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9895947277545929,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 3.4848749928642064,
      "inferenceMs": 3.3582499891053885,
      "pairsPerSecond": 297.7741392821063,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1605.0223339989316,
      "totalMs": 1608.380583988037,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9861082676798105,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 3.7288340099621564,
      "inferenceMs": 3.6121250013820827,
      "pairsPerSecond": 276.84534716195503,
      "inputTokens": 15,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1605.0223339989316,
      "totalMs": 1608.6344590003137,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.934897530823946,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 68.40912500047125,
      "inferenceMs": 67.79166599153541,
      "pairsPerSecond": 14.751075746167114,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1583.1185409915633,
      "totalMs": 1650.9102069830988,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9997462833562167,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 5.390290985815227,
      "inferenceMs": 5.148083000676706,
      "pairsPerSecond": 194.2470624247029,
      "inputTokens": 26,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1583.1185409915633,
      "totalMs": 1588.26662399224,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9828259916976094,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 4.207457997836173,
      "inferenceMs": 4.032083001220599,
      "pairsPerSecond": 248.01076756040942,
      "inputTokens": 18,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1583.1185409915633,
      "totalMs": 1587.150623992784,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9937077884096652,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 3.4230419842060655,
      "inferenceMs": 3.250667010433972,
      "pairsPerSecond": 307.6291717331261,
      "inputTokens": 16,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1583.1185409915633,
      "totalMs": 1586.3692080019973,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9895947277545929,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 3.6484999873209745,
      "inferenceMs": 3.511957998853177,
      "pairsPerSecond": 284.74144631756644,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1583.1185409915633,
      "totalMs": 1586.6304989904165,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9861082676798105,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 3.5977500083390623,
      "inferenceMs": 3.462416003458202,
      "pairsPerSecond": 288.8156706187867,
      "inputTokens": 15,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1583.1185409915633,
      "totalMs": 1586.5809569950216,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.934897530823946,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 70.29912498546764,
      "inferenceMs": 69.76466698688455,
      "pairsPerSecond": 14.333903438369392,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1677.209833986126,
      "totalMs": 1746.9745009730104,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9997462833562167,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 6.056458019884303,
      "inferenceMs": 5.774833989562467,
      "pairsPerSecond": 173.16515103419718,
      "inputTokens": 26,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1677.209833986126,
      "totalMs": 1682.9846679756884,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9828259916976094,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 4.596874990966171,
      "inferenceMs": 4.4043330126442015,
      "pairsPerSecond": 227.04913482453415,
      "inputTokens": 18,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1677.209833986126,
      "totalMs": 1681.61416699877,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9937077884096652,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 4.046582995215431,
      "inferenceMs": 3.8731660170014948,
      "pairsPerSecond": 258.1867122685782,
      "inputTokens": 16,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1677.209833986126,
      "totalMs": 1681.0830000031274,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9895947277545929,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 4.026959009934217,
      "inferenceMs": 3.848916996503249,
      "pairsPerSecond": 259.81334513280035,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1677.209833986126,
      "totalMs": 1681.0587509826291,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9861082676798105,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 4.178834002232179,
      "inferenceMs": 4.003749985713512,
      "pairsPerSecond": 249.76584541199546,
      "inputTokens": 15,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1677.209833986126,
      "totalMs": 1681.2135839718394,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.934897530823946,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 66.21812502271496,
      "inferenceMs": 65.72812498779967,
      "pairsPerSecond": 15.214187232415622,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1681.628250022186,
      "totalMs": 1747.3563750099856,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9997462833562167,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 5.722582980524749,
      "inferenceMs": 5.475792015204206,
      "pairsPerSecond": 182.62198367348097,
      "inputTokens": 26,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1681.628250022186,
      "totalMs": 1687.1040420373902,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9828259916976094,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 4.437750001670793,
      "inferenceMs": 4.246666008839384,
      "pairsPerSecond": 235.47884338408346,
      "inputTokens": 18,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1681.628250022186,
      "totalMs": 1685.8749160310254,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9937077884096652,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 3.889958985382691,
      "inferenceMs": 3.728791984030977,
      "pairsPerSecond": 268.18336991782496,
      "inputTokens": 16,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1681.628250022186,
      "totalMs": 1685.357042006217,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9895947277545929,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 3.8690420042257756,
      "inferenceMs": 3.7107919924892485,
      "pairsPerSecond": 269.4842508079217,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1681.628250022186,
      "totalMs": 1685.3390420146752,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9861082676798105,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 3.767624992178753,
      "inferenceMs": 3.638916998170316,
      "pairsPerSecond": 274.8070375067115,
      "inputTokens": 15,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1681.628250022186,
      "totalMs": 1685.2671670203563,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.934897530823946,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 68.71270798728801,
      "inferenceMs": 68.19349998841062,
      "pairsPerSecond": 14.664154210737804,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1611.1022499972023,
      "totalMs": 1679.295749985613,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9997462833562167,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 5.953874991973862,
      "inferenceMs": 5.723000009311363,
      "pairsPerSecond": 174.73353108037614,
      "inputTokens": 26,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1611.1022499972023,
      "totalMs": 1616.8252500065137,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9828259916976094,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 4.296625003917143,
      "inferenceMs": 4.123833001358435,
      "pairsPerSecond": 242.49284577493543,
      "inputTokens": 18,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1611.1022499972023,
      "totalMs": 1615.2260829985607,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9937077884096652,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 3.903042001184076,
      "inferenceMs": 3.7576250033453107,
      "pairsPerSecond": 266.1255444888001,
      "inputTokens": 16,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1611.1022499972023,
      "totalMs": 1614.8598750005476,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9895947277545929,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 4.021666012704372,
      "inferenceMs": 3.860541997710243,
      "pairsPerSecond": 259.03098595821984,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1611.1022499972023,
      "totalMs": 1614.9627919949125,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9861082676798105,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 4.011374985566363,
      "inferenceMs": 3.858499985653907,
      "pairsPerSecond": 259.1680714573148,
      "inputTokens": 15,
      "placement": {
        "cpu/torch.float32": 66
      },
      "loadMs": 1611.1022499972023,
      "totalMs": 1614.9607499828562,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9348973371088505,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 403.3629169862252,
      "inferenceMs": 402.47854200424626,
      "pairsPerSecond": 2.4846045084049466,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float32": 66
      },
      "driverAllocatedMemoryBytes": 1110212608,
      "loadMs": 1729.397125018295,
      "totalMs": 2131.875667022541,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9997462833562167,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 27.251167019130662,
      "inferenceMs": 26.461375004146248,
      "pairsPerSecond": 37.79093111538268,
      "inputTokens": 26,
      "placement": {
        "mps:0/torch.float32": 66
      },
      "driverAllocatedMemoryBytes": 1110310912,
      "loadMs": 1729.397125018295,
      "totalMs": 1755.8585000224411,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9828259600326419,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 21.842249989276752,
      "inferenceMs": 21.240040980046615,
      "pairsPerSecond": 47.080888447410395,
      "inputTokens": 18,
      "placement": {
        "mps:0/torch.float32": 66
      },
      "driverAllocatedMemoryBytes": 1110376448,
      "loadMs": 1729.397125018295,
      "totalMs": 1750.6371659983415,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9937077942304313,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 26.210625015664846,
      "inferenceMs": 25.67866598838009,
      "pairsPerSecond": 38.94283295138899,
      "inputTokens": 16,
      "placement": {
        "mps:0/torch.float32": 66
      },
      "driverAllocatedMemoryBytes": 1110441984,
      "loadMs": 1729.397125018295,
      "totalMs": 1755.075791006675,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9895947277545929,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 11.26433297758922,
      "inferenceMs": 10.644458001479506,
      "pairsPerSecond": 93.94560059901659,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float32": 66
      },
      "driverAllocatedMemoryBytes": 1110441984,
      "loadMs": 1729.397125018295,
      "totalMs": 1740.0415830197744,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9861082504503429,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 100.18437501275912,
      "inferenceMs": 99.57466699415818,
      "pairsPerSecond": 10.042714981498936,
      "inputTokens": 15,
      "placement": {
        "mps:0/torch.float32": 66
      },
      "driverAllocatedMemoryBytes": 1110507520,
      "loadMs": 1729.397125018295,
      "totalMs": 1828.971792012453,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9348973371088505,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 73.4710420074407,
      "inferenceMs": 72.34912499552593,
      "pairsPerSecond": 13.821867231453595,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float32": 66
      },
      "driverAllocatedMemoryBytes": 1110212608,
      "loadMs": 1764.048167009605,
      "totalMs": 1836.397292005131,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9997462833562167,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 22.692291997373104,
      "inferenceMs": 22.011541994288564,
      "pairsPerSecond": 45.43071086339497,
      "inputTokens": 26,
      "placement": {
        "mps:0/torch.float32": 66
      },
      "driverAllocatedMemoryBytes": 1110310912,
      "loadMs": 1764.048167009605,
      "totalMs": 1786.0597090038937,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9828259600326419,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 22.883750003529713,
      "inferenceMs": 22.322416974930093,
      "pairsPerSecond": 44.798016322474496,
      "inputTokens": 18,
      "placement": {
        "mps:0/torch.float32": 66
      },
      "driverAllocatedMemoryBytes": 1110376448,
      "loadMs": 1764.048167009605,
      "totalMs": 1786.3705839845352,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9937077942304313,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 24.19745799852535,
      "inferenceMs": 23.286207986529917,
      "pairsPerSecond": 42.943874785386164,
      "inputTokens": 16,
      "placement": {
        "mps:0/torch.float32": 66
      },
      "driverAllocatedMemoryBytes": 1110441984,
      "loadMs": 1764.048167009605,
      "totalMs": 1787.334374996135,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9895947277545929,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 11.087499995483086,
      "inferenceMs": 10.568375000730157,
      "pairsPerSecond": 94.62192625932664,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float32": 66
      },
      "driverAllocatedMemoryBytes": 1110441984,
      "loadMs": 1764.048167009605,
      "totalMs": 1774.6165420103353,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9861082504503429,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 22.737207997124642,
      "inferenceMs": 22.126541007310152,
      "pairsPerSecond": 45.19459230747457,
      "inputTokens": 15,
      "placement": {
        "mps:0/torch.float32": 66
      },
      "driverAllocatedMemoryBytes": 1110507520,
      "loadMs": 1764.048167009605,
      "totalMs": 1786.1747080169152,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9348973371088505,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 82.78416702523828,
      "inferenceMs": 81.90141702652909,
      "pairsPerSecond": 12.209800957120011,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float32": 66
      },
      "driverAllocatedMemoryBytes": 1110212608,
      "loadMs": 1779.3654589913785,
      "totalMs": 1861.2668760179076,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9997462833562167,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 28.70383401750587,
      "inferenceMs": 27.881999994860962,
      "pairsPerSecond": 35.86543290238555,
      "inputTokens": 26,
      "placement": {
        "mps:0/torch.float32": 66
      },
      "driverAllocatedMemoryBytes": 1110310912,
      "loadMs": 1779.3654589913785,
      "totalMs": 1807.2474589862395,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9828259600326419,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 28.429958008928224,
      "inferenceMs": 27.80250000068918,
      "pairsPerSecond": 35.967988489352095,
      "inputTokens": 18,
      "placement": {
        "mps:0/torch.float32": 66
      },
      "driverAllocatedMemoryBytes": 1110376448,
      "loadMs": 1779.3654589913785,
      "totalMs": 1807.1679589920677,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9937077942304313,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 30.076707975240424,
      "inferenceMs": 26.437124994117767,
      "pairsPerSecond": 37.82559564334242,
      "inputTokens": 16,
      "placement": {
        "mps:0/torch.float32": 66
      },
      "driverAllocatedMemoryBytes": 1110441984,
      "loadMs": 1779.3654589913785,
      "totalMs": 1805.8025839854963,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9895947277545929,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 8.605667011579499,
      "inferenceMs": 7.2109170141629875,
      "pairsPerSecond": 138.6786171628236,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float32": 66
      },
      "driverAllocatedMemoryBytes": 1110441984,
      "loadMs": 1779.3654589913785,
      "totalMs": 1786.5763760055415,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9861082504503429,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 27.578957990044728,
      "inferenceMs": 26.97937499033287,
      "pairsPerSecond": 37.0653508599927,
      "inputTokens": 15,
      "placement": {
        "mps:0/torch.float32": 66
      },
      "driverAllocatedMemoryBytes": 1110507520,
      "loadMs": 1779.3654589913785,
      "totalMs": 1806.3448339817114,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9351372569799423,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 419.44487500586547,
      "inferenceMs": 418.47995901480317,
      "pairsPerSecond": 2.3896006928365865,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float16": 66
      },
      "driverAllocatedMemoryBytes": 1126989824,
      "loadMs": 1835.9750410018023,
      "totalMs": 2254.4550000166055,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9997481048776535,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 32.0542500121519,
      "inferenceMs": 31.36416699271649,
      "pairsPerSecond": 31.883518546251327,
      "inputTokens": 26,
      "placement": {
        "mps:0/torch.float16": 66
      },
      "driverAllocatedMemoryBytes": 1127088128,
      "loadMs": 1835.9750410018023,
      "totalMs": 1867.3392079945188,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9828035058453679,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 22.955624997848645,
      "inferenceMs": 22.33862501452677,
      "pairsPerSecond": 44.76551261994423,
      "inputTokens": 18,
      "placement": {
        "mps:0/torch.float16": 66
      },
      "driverAllocatedMemoryBytes": 1127153664,
      "loadMs": 1835.9750410018023,
      "totalMs": 1858.313666016329,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9937266162596643,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 26.538250007433817,
      "inferenceMs": 25.913374993251637,
      "pairsPerSecond": 38.59011032952752,
      "inputTokens": 16,
      "placement": {
        "mps:0/torch.float16": 66
      },
      "driverAllocatedMemoryBytes": 1127219200,
      "loadMs": 1835.9750410018023,
      "totalMs": 1861.888415995054,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9896271242760122,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.83908399939537,
      "inferenceMs": 9.17779200244695,
      "pairsPerSecond": 108.95866889698347,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float16": 66
      },
      "driverAllocatedMemoryBytes": 1127219200,
      "loadMs": 1835.9750410018023,
      "totalMs": 1845.1528330042493,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9860793617554009,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 107.3271659843158,
      "inferenceMs": 106.46091599483043,
      "pairsPerSecond": 9.393118504151875,
      "inputTokens": 15,
      "placement": {
        "mps:0/torch.float16": 66
      },
      "driverAllocatedMemoryBytes": 1127284736,
      "loadMs": 1835.9750410018023,
      "totalMs": 1942.4359569966327,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9351372569799423,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 79.29158399929293,
      "inferenceMs": 78.38454202283174,
      "pairsPerSecond": 12.757617435702073,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float16": 66
      },
      "driverAllocatedMemoryBytes": 1126989824,
      "loadMs": 1770.9310000063851,
      "totalMs": 1849.3155420292169,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9997481048776535,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 27.11816699593328,
      "inferenceMs": 26.393790991278365,
      "pairsPerSecond": 37.887698676194056,
      "inputTokens": 26,
      "placement": {
        "mps:0/torch.float16": 66
      },
      "driverAllocatedMemoryBytes": 1127088128,
      "loadMs": 1770.9310000063851,
      "totalMs": 1797.3247909976635,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9828035058453679,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 23.44195800833404,
      "inferenceMs": 22.785167006077245,
      "pairsPerSecond": 43.88820146603624,
      "inputTokens": 18,
      "placement": {
        "mps:0/torch.float16": 66
      },
      "driverAllocatedMemoryBytes": 1127153664,
      "loadMs": 1770.9310000063851,
      "totalMs": 1793.7161670124624,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9937266162596643,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 28.265749977435917,
      "inferenceMs": 27.661959000397474,
      "pairsPerSecond": 36.15072959892794,
      "inputTokens": 16,
      "placement": {
        "mps:0/torch.float16": 66
      },
      "driverAllocatedMemoryBytes": 1127219200,
      "loadMs": 1770.9310000063851,
      "totalMs": 1798.5929590067826,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9896271242760122,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 10.620999993989244,
      "inferenceMs": 9.977375011658296,
      "pairsPerSecond": 100.22676293429151,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float16": 66
      },
      "driverAllocatedMemoryBytes": 1127219200,
      "loadMs": 1770.9310000063851,
      "totalMs": 1780.9083750180434,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9860793617554009,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 25.31062500202097,
      "inferenceMs": 24.12587497383356,
      "pairsPerSecond": 41.4492739054886,
      "inputTokens": 15,
      "placement": {
        "mps:0/torch.float16": 66
      },
      "driverAllocatedMemoryBytes": 1127284736,
      "loadMs": 1770.9310000063851,
      "totalMs": 1795.0568749802187,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9351372569799423,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 73.04449999355711,
      "inferenceMs": 71.97845799964853,
      "pairsPerSecond": 13.89304561101994,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float16": 66
      },
      "driverAllocatedMemoryBytes": 1126989824,
      "loadMs": 1777.745250001317,
      "totalMs": 1849.7237080009654,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9997481048776535,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 27.022750000469387,
      "inferenceMs": 26.262792001944035,
      "pairsPerSecond": 38.07668278094643,
      "inputTokens": 26,
      "placement": {
        "mps:0/torch.float16": 66
      },
      "driverAllocatedMemoryBytes": 1127088128,
      "loadMs": 1777.745250001317,
      "totalMs": 1804.008042003261,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9828035058453679,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 24.61175000644289,
      "inferenceMs": 23.88275001430884,
      "pairsPerSecond": 41.87122502228057,
      "inputTokens": 18,
      "placement": {
        "mps:0/torch.float16": 66
      },
      "driverAllocatedMemoryBytes": 1127153664,
      "loadMs": 1777.745250001317,
      "totalMs": 1801.6280000156257,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9937266162596643,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 24.653667001985013,
      "inferenceMs": 24.02399998391047,
      "pairsPerSecond": 41.62504165291905,
      "inputTokens": 16,
      "placement": {
        "mps:0/torch.float16": 66
      },
      "driverAllocatedMemoryBytes": 1127219200,
      "loadMs": 1777.745250001317,
      "totalMs": 1801.7692499852274,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9896271242760122,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 7.966167002450675,
      "inferenceMs": 7.31566600734368,
      "pairsPerSecond": 136.6929544071819,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float16": 66
      },
      "driverAllocatedMemoryBytes": 1127219200,
      "loadMs": 1777.745250001317,
      "totalMs": 1785.0609160086606,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9860793617554009,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 25.47954098554328,
      "inferenceMs": 24.715749983442947,
      "pairsPerSecond": 40.46003057442719,
      "inputTokens": 15,
      "placement": {
        "mps:0/torch.float16": 66
      },
      "driverAllocatedMemoryBytes": 1127284736,
      "loadMs": 1777.745250001317,
      "totalMs": 1802.4609999847598,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "802360b6ffe6d4627f1d28fd3225522e247cd3e8",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  }
]

EttinX NLI XXS Desktop

72 configurations tested
Passed72
Failed0
TestedNov 12, 2026
Runtimetransformers 5.22.1

Quality criteria

{ "file": "research/ettinx-nli-family-expansion/quality-gates.json", "sha256": "b98612c89959b9c069902c28bfd95bd8d943b61cefa34ea3c6da14ace9f0c18c", "description": "All six expected labels, per-case minimum margins, complete finite normalized three-label probabilities, <=1000ms/pair." }

PrecisionDeviceResultPlatformInference / load
FP32mpsAcceleratedPassedVery fastmacos · aarch644 ms / 1717 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch645 ms / 1699 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch646 ms / 1798 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch648 ms / 1703 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch649 ms / 1739 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6410 ms / 1709 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6421 ms / 1717 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6421 ms / 1709 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6422 ms / 1709 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6423 ms / 1798 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6423 ms / 1739 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6423 ms / 1699 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6423 ms / 1709 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6423 ms / 1717 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6423 ms / 1717 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6424 ms / 1699 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6424 ms / 1699 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6424 ms / 1798 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6424 ms / 1798 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6424 ms / 1798 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6425 ms / 1709 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6425 ms / 1717 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6427 ms / 1739 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6427 ms / 1703 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6427 ms / 1699 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6443 ms / 1739 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6447 ms / 1703 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6466 ms / 1717 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6468 ms / 1703 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6469 ms / 1709 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6470 ms / 1699 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6473 ms / 1798 ms
FP32mpsAcceleratedPassedFastmacos · aarch64222 ms / 1739 ms
FP16mpsAcceleratedPassedFastmacos · aarch64243 ms / 1703 ms
FP32mpsAcceleratedPassedFastmacos · aarch64554 ms / 1739 ms
FP16mpsAcceleratedPassedFastmacos · aarch64566 ms / 1703 ms
FP32cpuPassedVery fastmacos · aarch642 ms / 10387 ms
FP32cpuPassedVery fastmacos · aarch642 ms / 1586 ms
FP32cpuPassedVery fastmacos · aarch642 ms / 1612 ms
FP32cpuPassedVery fastmacos · aarch642 ms / 1623 ms
FP32cpuPassedVery fastmacos · aarch642 ms / 1586 ms
FP32cpuPassedVery fastmacos · aarch642 ms / 1612 ms
FP32cpuPassedVery fastmacos · aarch642 ms / 1612 ms
FP32cpuPassedVery fastmacos · aarch642 ms / 10387 ms
FP32cpuPassedVery fastmacos · aarch642 ms / 1586 ms
FP32cpuPassedVery fastmacos · aarch642 ms / 1614 ms
FP32cpuPassedVery fastmacos · aarch642 ms / 1614 ms
FP32cpuPassedVery fastmacos · aarch642 ms / 1623 ms
FP32cpuPassedVery fastmacos · aarch642 ms / 10387 ms
FP32cpuPassedVery fastmacos · aarch642 ms / 1614 ms
FP32cpuPassedVery fastmacos · aarch642 ms / 1623 ms
FP32cpuPassedVery fastmacos · aarch642 ms / 1612 ms
FP32cpuPassedVery fastmacos · aarch642 ms / 1586 ms
FP32cpuPassedVery fastmacos · aarch642 ms / 1681 ms
FP32cpuPassedVery fastmacos · aarch642 ms / 10387 ms
FP32cpuPassedVery fastmacos · aarch642 ms / 1681 ms
FP32cpuPassedVery fastmacos · aarch643 ms / 1614 ms
FP32cpuPassedVery fastmacos · aarch643 ms / 1623 ms
FP32cpuPassedVery fastmacos · aarch643 ms / 1681 ms
FP32cpuPassedVery fastmacos · aarch643 ms / 1681 ms
FP32cpuPassedVery fastmacos · aarch643 ms / 1612 ms
FP32cpuPassedVery fastmacos · aarch643 ms / 1586 ms
FP32cpuPassedVery fastmacos · aarch644 ms / 10387 ms
FP32cpuPassedVery fastmacos · aarch644 ms / 1614 ms
FP32cpuPassedVery fastmacos · aarch644 ms / 1623 ms
FP32cpuPassedVery fastmacos · aarch644 ms / 1681 ms
FP32cpuPassedVery fastmacos · aarch6422 ms / 1612 ms
FP32cpuPassedVery fastmacos · aarch6423 ms / 1623 ms
FP32cpuPassedVery fastmacos · aarch6423 ms / 1586 ms
FP32cpuPassedVery fastmacos · aarch6423 ms / 1614 ms
FP32cpuPassedVery fastmacos · aarch6423 ms / 10387 ms
FP32cpuPassedVery fastmacos · aarch6424 ms / 1681 ms
Test setup & sample details
Quality gate
{ "file": "research/ettinx-nli-family-expansion/quality-gates.json", "sha256": "b98612c89959b9c069902c28bfd95bd8d943b61cefa34ea3c6da14ace9f0c18c", "description": "All six expected labels, per-case minimum margins, complete finite normalized three-label probabilities, <=1000ms/pair." }
Tested
2026-11-12T22:03:07.787398+00:00
Environment
{ "os": "macos", "osVersion": "27.2", "architecture": "aarch64", "hardware": "Apple M3 Max;Mac15,11;96GiB RAM", "cpu": "Apple M3 Max", "gpu": "Apple M3 Max", "driver": "OS-managed Metal/MPS macOS27.2 build26B118", "runtime": "App-owned CPython3.13.18;Torch2.10.0;Transformers5.22.1", "timingScope": "Fresh helper/model process per cold repetition; runtime/tokenizer/model startup in load. OS files/driver cache may be warm. Five warm pairs follow six fixtures. Snapshot loads, not Internet transfer.", "concurrentUploadActivity": "none; uploads excluded" }
Runtime
transformers 5.22.1
Artifact revision
04ded685adf9835dff917af61327bb4aa1dbec01
Fixture
{
  "file": "model-samples/native/ettinx-nli/quality-gates.json",
  "url": "https://assets.localai.tools/samples/v1/files/b98612c89959b9c069902c28bfd95bd8d943b61cefa34ea3c6da14ace9f0c18c/ettinx-nli-quality-gates.json",
  "size": 2262,
  "sha1": "9104fbed107fd4340666e85c8fcaf3376ba1a2a4",
  "sha256": "b98612c89959b9c069902c28bfd95bd8d943b61cefa34ea3c6da14ace9f0c18c"
}
Rerun when
artifact-revision; runtime-version; fixture-checksum; quality-gate; backend-behavior
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9350983984768391,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 24.097957997582853,
      "inferenceMs": 23.155000002589077,
      "pairsPerSecond": 43.18721657906219,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 10387.186416017357,
      "totalMs": 10410.341416019946,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.999408537405543,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 3.7819579883944243,
      "inferenceMs": 3.5465830005705357,
      "pairsPerSecond": 281.96153870898576,
      "inputTokens": 26,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 10387.186416017357,
      "totalMs": 10390.732999017928,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9768406730145216,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 2.58004100760445,
      "inferenceMs": 2.419374999590218,
      "pairsPerSecond": 413.32988898760016,
      "inputTokens": 18,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 10387.186416017357,
      "totalMs": 10389.605791016947,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9936191779561341,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 1.8935840053018183,
      "inferenceMs": 1.764917018590495,
      "pairsPerSecond": 566.5988765854975,
      "inputTokens": 16,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 10387.186416017357,
      "totalMs": 10388.951333035948,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.8207429498434067,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 2.1315830235835165,
      "inferenceMs": 1.9974579918198287,
      "pairsPerSecond": 500.6363107986705,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 10387.186416017357,
      "totalMs": 10389.183874009177,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9908071178942919,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 2.1797079825773835,
      "inferenceMs": 2.064542000880465,
      "pairsPerSecond": 484.36893004527406,
      "inputTokens": 15,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 10387.186416017357,
      "totalMs": 10389.250958018238,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9350983984768391,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 23.26845799689181,
      "inferenceMs": 22.794874996179715,
      "pairsPerSecond": 43.869510149434646,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1585.6987080187537,
      "totalMs": 1608.4935830149334,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.999408537405543,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 3.6877499951515347,
      "inferenceMs": 3.4679579839576036,
      "pairsPerSecond": 288.35412788329364,
      "inputTokens": 26,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1585.6987080187537,
      "totalMs": 1589.1666660027113,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9768406730145216,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 2.352707990212366,
      "inferenceMs": 2.2236660006456077,
      "pairsPerSecond": 449.70782469564455,
      "inputTokens": 18,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1585.6987080187537,
      "totalMs": 1587.9223740193993,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9936191779561341,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 1.8783329869620502,
      "inferenceMs": 1.7728749953676015,
      "pairsPerSecond": 564.0555609464458,
      "inputTokens": 16,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1585.6987080187537,
      "totalMs": 1587.4715830141213,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.8207429498434067,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 2.105916995787993,
      "inferenceMs": 2.004207985009998,
      "pairsPerSecond": 498.950212492548,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1585.6987080187537,
      "totalMs": 1587.7029160037637,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9908071178942919,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 2.0364579977467656,
      "inferenceMs": 1.9127920095343143,
      "pairsPerSecond": 522.7959940314988,
      "inputTokens": 15,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1585.6987080187537,
      "totalMs": 1587.611500028288,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9350983984768391,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 22.57716699386947,
      "inferenceMs": 22.135249979328364,
      "pairsPerSecond": 45.17681078523525,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1612.2754579992034,
      "totalMs": 1634.4107079785317,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.999408537405543,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 3.649292018963024,
      "inferenceMs": 3.4455419809091836,
      "pairsPerSecond": 290.2301018361493,
      "inputTokens": 26,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1612.2754579992034,
      "totalMs": 1615.7209999801125,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9768406730145216,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 2.285916998516768,
      "inferenceMs": 2.1607920061796904,
      "pairsPerSecond": 462.79327077297626,
      "inputTokens": 18,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1612.2754579992034,
      "totalMs": 1614.436250005383,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9936191779561341,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 1.95537501713261,
      "inferenceMs": 1.8093750113621354,
      "pairsPerSecond": 552.6770258903814,
      "inputTokens": 16,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1612.2754579992034,
      "totalMs": 1614.0848330105655,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.8207429498434067,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 2.0876670023426414,
      "inferenceMs": 1.9934169831685722,
      "pairsPerSecond": 501.6511891107108,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1612.2754579992034,
      "totalMs": 1614.268874982372,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9908071178942919,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 2.029082999797538,
      "inferenceMs": 1.923417003126815,
      "pairsPerSecond": 519.9080586135735,
      "inputTokens": 15,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1612.2754579992034,
      "totalMs": 1614.1988750023302,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9350983984768391,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 23.31858398974873,
      "inferenceMs": 22.842333011794835,
      "pairsPerSecond": 43.77836534839245,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1613.9189589885063,
      "totalMs": 1636.7612920003012,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.999408537405543,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 3.922290983609855,
      "inferenceMs": 3.6986659979447722,
      "pairsPerSecond": 270.3677489548035,
      "inputTokens": 26,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1613.9189589885063,
      "totalMs": 1617.6176249864511,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9768406730145216,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 2.6880829827859998,
      "inferenceMs": 2.526542026316747,
      "pairsPerSecond": 395.79788880766165,
      "inputTokens": 18,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1613.9189589885063,
      "totalMs": 1616.445501014823,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9936191779561341,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 2.1497500129044056,
      "inferenceMs": 2.021666005020961,
      "pairsPerSecond": 494.6415468808517,
      "inputTokens": 16,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1613.9189589885063,
      "totalMs": 1615.9406249935273,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.8207429498434067,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 2.1741249947808683,
      "inferenceMs": 2.065250009763986,
      "pairsPerSecond": 484.20287871795176,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1613.9189589885063,
      "totalMs": 1615.9842089982703,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9908071178942919,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 2.1328330039978027,
      "inferenceMs": 2.006208000238985,
      "pairsPerSecond": 498.45280244165974,
      "inputTokens": 15,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1613.9189589885063,
      "totalMs": 1615.9251669887453,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9350983984768391,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 23.26866702060215,
      "inferenceMs": 22.739042004104704,
      "pairsPerSecond": 43.97722647328267,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1622.9219579836354,
      "totalMs": 1645.66099998774,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.999408537405543,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 4.198999988147989,
      "inferenceMs": 3.9350420120172203,
      "pairsPerSecond": 254.12689291400224,
      "inputTokens": 26,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1622.9219579836354,
      "totalMs": 1626.8569999956526,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9768406730145216,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 2.7622080233413726,
      "inferenceMs": 2.596332982648164,
      "pairsPerSecond": 385.15860896241315,
      "inputTokens": 18,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1622.9219579836354,
      "totalMs": 1625.5182909662835,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9936191779561341,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 1.9673750211950392,
      "inferenceMs": 1.8412499921396375,
      "pairsPerSecond": 543.1093030653285,
      "inputTokens": 16,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1622.9219579836354,
      "totalMs": 1624.763207975775,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.8207429498434067,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 2.1747499995399266,
      "inferenceMs": 2.0687499782070518,
      "pairsPerSecond": 483.38369089273993,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1622.9219579836354,
      "totalMs": 1624.9907079618424,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9908071178942919,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 2.1874999802093953,
      "inferenceMs": 2.058790996670723,
      "pairsPerSecond": 485.72196090671804,
      "inputTokens": 15,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1622.9219579836354,
      "totalMs": 1624.980748980306,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9350983984768391,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 24.119584006257355,
      "inferenceMs": 23.622250009793788,
      "pairsPerSecond": 42.33296995778979,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1681.1614580219612,
      "totalMs": 1704.783708031755,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.999408537405543,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 4.225834010867402,
      "inferenceMs": 3.972915990743786,
      "pairsPerSecond": 251.70429033229718,
      "inputTokens": 26,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1681.1614580219612,
      "totalMs": 1685.134374012705,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9768406730145216,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 2.9890410078223795,
      "inferenceMs": 2.8310839843470603,
      "pairsPerSecond": 353.2215948127842,
      "inputTokens": 18,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1681.1614580219612,
      "totalMs": 1683.9925420063082,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9936191779561341,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 2.4772089964244515,
      "inferenceMs": 2.308916999027133,
      "pairsPerSecond": 433.1034854961664,
      "inputTokens": 16,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1681.1614580219612,
      "totalMs": 1683.4703750209883,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.8207429498434067,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 2.81649999669753,
      "inferenceMs": 2.622041996801272,
      "pairsPerSecond": 381.38214461093213,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1681.1614580219612,
      "totalMs": 1683.7835000187624,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9908071178942919,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 2.5808749778661877,
      "inferenceMs": 2.4289170105475932,
      "pairsPerSecond": 411.70612073508124,
      "inputTokens": 15,
      "placement": {
        "cpu/torch.float32": 48
      },
      "loadMs": 1681.1614580219612,
      "totalMs": 1683.5903750325087,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9350983835756779,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 555.3770830156282,
      "inferenceMs": 554.4127080065664,
      "pairsPerSecond": 1.8037104589387516,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float32": 48
      },
      "driverAllocatedMemoryBytes": 1110212608,
      "loadMs": 1738.9782500104047,
      "totalMs": 2293.390958016971,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9994085376674775,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 30.477457999950275,
      "inferenceMs": 26.561375008895993,
      "pairsPerSecond": 37.64865334212095,
      "inputTokens": 26,
      "placement": {
        "mps:0/torch.float32": 48
      },
      "driverAllocatedMemoryBytes": 1110294528,
      "loadMs": 1738.9782500104047,
      "totalMs": 1765.5396250193007,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9768408071249723,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 23.18108401959762,
      "inferenceMs": 22.60216700960882,
      "pairsPerSecond": 44.24354530142494,
      "inputTokens": 18,
      "placement": {
        "mps:0/torch.float32": 48
      },
      "driverAllocatedMemoryBytes": 1110360064,
      "loadMs": 1738.9782500104047,
      "totalMs": 1761.5804170200136,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9936191795859486,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 43.72479100129567,
      "inferenceMs": 43.070374988019466,
      "pairsPerSecond": 23.217815035930425,
      "inputTokens": 16,
      "placement": {
        "mps:0/torch.float32": 48
      },
      "driverAllocatedMemoryBytes": 1110425600,
      "loadMs": 1738.9782500104047,
      "totalMs": 1782.0486249984242,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.8207431510090828,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.917582996422425,
      "inferenceMs": 9.303375001763925,
      "pairsPerSecond": 107.48787400383189,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float32": 48
      },
      "driverAllocatedMemoryBytes": 1110425600,
      "loadMs": 1738.9782500104047,
      "totalMs": 1748.2816250121687,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9908071213867515,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 222.5927910185419,
      "inferenceMs": 221.98145900620148,
      "pairsPerSecond": 4.504880743089733,
      "inputTokens": 15,
      "placement": {
        "mps:0/torch.float32": 48
      },
      "driverAllocatedMemoryBytes": 1110491136,
      "loadMs": 1738.9782500104047,
      "totalMs": 1960.9597090166062,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9350983835756779,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 66.47625000914559,
      "inferenceMs": 65.61154199880548,
      "pairsPerSecond": 15.24122082084591,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float32": 48
      },
      "driverAllocatedMemoryBytes": 1110212608,
      "loadMs": 1716.7617499944754,
      "totalMs": 1782.3732919932809,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9994085376674775,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 26.018042000941932,
      "inferenceMs": 25.337041006423533,
      "pairsPerSecond": 39.46790786447701,
      "inputTokens": 26,
      "placement": {
        "mps:0/torch.float32": 48
      },
      "driverAllocatedMemoryBytes": 1110294528,
      "loadMs": 1716.7617499944754,
      "totalMs": 1742.098791000899,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9768408071249723,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 21.45870801177807,
      "inferenceMs": 20.856417017057538,
      "pairsPerSecond": 47.946874057137634,
      "inputTokens": 18,
      "placement": {
        "mps:0/torch.float32": 48
      },
      "driverAllocatedMemoryBytes": 1110360064,
      "loadMs": 1716.7617499944754,
      "totalMs": 1737.618167011533,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9936191795859486,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 23.38495900039561,
      "inferenceMs": 22.831125010270625,
      "pairsPerSecond": 43.79985653576633,
      "inputTokens": 16,
      "placement": {
        "mps:0/torch.float32": 48
      },
      "driverAllocatedMemoryBytes": 1110425600,
      "loadMs": 1716.7617499944754,
      "totalMs": 1739.592875004746,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.8207431510090828,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 4.656082979636267,
      "inferenceMs": 4.01108298683539,
      "pairsPerSecond": 249.3092272790313,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float32": 48
      },
      "driverAllocatedMemoryBytes": 1110425600,
      "loadMs": 1716.7617499944754,
      "totalMs": 1720.7728329813108,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9908071213867515,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 23.487666010623798,
      "inferenceMs": 23.005499999271706,
      "pairsPerSecond": 43.46786638115483,
      "inputTokens": 15,
      "placement": {
        "mps:0/torch.float32": 48
      },
      "driverAllocatedMemoryBytes": 1110491136,
      "loadMs": 1716.7617499944754,
      "totalMs": 1739.767249993747,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9350983835756779,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 70.25395799428225,
      "inferenceMs": 69.47154199588113,
      "pairsPerSecond": 14.394383243419135,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float32": 48
      },
      "driverAllocatedMemoryBytes": 1110212608,
      "loadMs": 1708.8568750186823,
      "totalMs": 1778.3284170145635,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9994085376674775,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 25.35354101564735,
      "inferenceMs": 24.622208991786465,
      "pairsPerSecond": 40.61374023482549,
      "inputTokens": 26,
      "placement": {
        "mps:0/torch.float32": 48
      },
      "driverAllocatedMemoryBytes": 1110294528,
      "loadMs": 1708.8568750186823,
      "totalMs": 1733.4790840104688,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9768408071249723,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 21.49729200755246,
      "inferenceMs": 20.942292001564056,
      "pairsPerSecond": 47.75026534465835,
      "inputTokens": 18,
      "placement": {
        "mps:0/torch.float32": 48
      },
      "driverAllocatedMemoryBytes": 1110360064,
      "loadMs": 1708.8568750186823,
      "totalMs": 1729.7991670202464,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9936191795859486,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 23.198415990918875,
      "inferenceMs": 22.629959014011547,
      "pairsPerSecond": 44.18920950677997,
      "inputTokens": 16,
      "placement": {
        "mps:0/torch.float32": 48
      },
      "driverAllocatedMemoryBytes": 1110425600,
      "loadMs": 1708.8568750186823,
      "totalMs": 1731.4868340326939,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.8207431510090828,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 10.633124999003485,
      "inferenceMs": 10.110584000358358,
      "pairsPerSecond": 98.90625506543995,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float32": 48
      },
      "driverAllocatedMemoryBytes": 1110425600,
      "loadMs": 1708.8568750186823,
      "totalMs": 1718.9674590190407,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9908071213867515,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 22.70829101325944,
      "inferenceMs": 22.120000008726493,
      "pairsPerSecond": 45.20795658252681,
      "inputTokens": 15,
      "placement": {
        "mps:0/torch.float32": 48
      },
      "driverAllocatedMemoryBytes": 1110491136,
      "loadMs": 1708.8568750186823,
      "totalMs": 1730.9768750274088,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9347764737904072,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 567.3714580188971,
      "inferenceMs": 566.3940829981584,
      "pairsPerSecond": 1.7655551673608345,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float16": 48
      },
      "driverAllocatedMemoryBytes": 1101824000,
      "loadMs": 1702.5347909948323,
      "totalMs": 2268.9288739929907,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9994080024480354,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 47.96075000194833,
      "inferenceMs": 47.36220798804425,
      "pairsPerSecond": 21.113880506846986,
      "inputTokens": 26,
      "placement": {
        "mps:0/torch.float16": 48
      },
      "driverAllocatedMemoryBytes": 1101905920,
      "loadMs": 1702.5347909948323,
      "totalMs": 1749.8969989828765,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9769495939835906,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 27.198625000892207,
      "inferenceMs": 26.56854101223871,
      "pairsPerSecond": 37.63849883737889,
      "inputTokens": 18,
      "placement": {
        "mps:0/torch.float16": 48
      },
      "driverAllocatedMemoryBytes": 1101971456,
      "loadMs": 1702.5347909948323,
      "totalMs": 1729.103332007071,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.993615276645869,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 69.7937500081025,
      "inferenceMs": 68.35162499919534,
      "pairsPerSecond": 14.630230078828008,
      "inputTokens": 16,
      "placement": {
        "mps:0/torch.float16": 48
      },
      "driverAllocatedMemoryBytes": 1102036992,
      "loadMs": 1702.5347909948323,
      "totalMs": 1770.8864159940276,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.8206676840782166,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 8.29233301919885,
      "inferenceMs": 7.6804170093964785,
      "pairsPerSecond": 130.20126365229473,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float16": 48
      },
      "driverAllocatedMemoryBytes": 1102036992,
      "loadMs": 1702.5347909948323,
      "totalMs": 1710.2152080042288,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9907999183051288,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 243.53587499354035,
      "inferenceMs": 242.9982080066111,
      "pairsPerSecond": 4.1152566852377515,
      "inputTokens": 15,
      "placement": {
        "mps:0/torch.float16": 48
      },
      "driverAllocatedMemoryBytes": 1102102528,
      "loadMs": 1702.5347909948323,
      "totalMs": 1945.5329990014434,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9347764737904072,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 74.32691700523719,
      "inferenceMs": 73.4237500000745,
      "pairsPerSecond": 13.619571323978757,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float16": 48
      },
      "driverAllocatedMemoryBytes": 1101824000,
      "loadMs": 1797.8989169932902,
      "totalMs": 1871.3226669933647,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9994080024480354,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 25.472417008131742,
      "inferenceMs": 23.74145801877603,
      "pairsPerSecond": 42.1204122850899,
      "inputTokens": 26,
      "placement": {
        "mps:0/torch.float16": 48
      },
      "driverAllocatedMemoryBytes": 1101905920,
      "loadMs": 1797.8989169932902,
      "totalMs": 1821.6403750120662,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9769495939835906,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 23.10370802297257,
      "inferenceMs": 22.51895802328363,
      "pairsPerSecond": 44.40702802350105,
      "inputTokens": 18,
      "placement": {
        "mps:0/torch.float16": 48
      },
      "driverAllocatedMemoryBytes": 1101971456,
      "loadMs": 1797.8989169932902,
      "totalMs": 1820.4178750165738,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.993615276645869,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 25.614708982175216,
      "inferenceMs": 23.936290992423892,
      "pairsPerSecond": 41.77756697211407,
      "inputTokens": 16,
      "placement": {
        "mps:0/torch.float16": 48
      },
      "driverAllocatedMemoryBytes": 1102036992,
      "loadMs": 1797.8989169932902,
      "totalMs": 1821.835207985714,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.8206676840782166,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 6.822833005571738,
      "inferenceMs": 6.102415994973853,
      "pairsPerSecond": 163.86952328776542,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float16": 48
      },
      "driverAllocatedMemoryBytes": 1102036992,
      "loadMs": 1797.8989169932902,
      "totalMs": 1804.001332988264,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9907999183051288,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 24.74141598213464,
      "inferenceMs": 24.16783300577663,
      "pairsPerSecond": 41.377313380185086,
      "inputTokens": 15,
      "placement": {
        "mps:0/torch.float16": 48
      },
      "driverAllocatedMemoryBytes": 1102102528,
      "loadMs": 1797.8989169932902,
      "totalMs": 1822.0667499990668,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9347764737904072,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 70.83483299356885,
      "inferenceMs": 69.93904200498946,
      "pairsPerSecond": 14.298165535762699,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float16": 48
      },
      "driverAllocatedMemoryBytes": 1101824000,
      "loadMs": 1699.4528750074096,
      "totalMs": 1769.391917012399,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9994080024480354,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 27.361290995031595,
      "inferenceMs": 26.621167024131864,
      "pairsPerSecond": 37.56409323052999,
      "inputTokens": 26,
      "placement": {
        "mps:0/torch.float16": 48
      },
      "driverAllocatedMemoryBytes": 1101905920,
      "loadMs": 1699.4528750074096,
      "totalMs": 1726.0740420315415,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9769495939835906,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 23.28304198454134,
      "inferenceMs": 22.61487499345094,
      "pairsPerSecond": 44.218683512050845,
      "inputTokens": 18,
      "placement": {
        "mps:0/torch.float16": 48
      },
      "driverAllocatedMemoryBytes": 1101971456,
      "loadMs": 1699.4528750074096,
      "totalMs": 1722.0677500008605,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.993615276645869,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 24.307332991156727,
      "inferenceMs": 23.73883401742205,
      "pairsPerSecond": 42.1250681169132,
      "inputTokens": 16,
      "placement": {
        "mps:0/torch.float16": 48
      },
      "driverAllocatedMemoryBytes": 1102036992,
      "loadMs": 1699.4528750074096,
      "totalMs": 1723.1917090248317,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.8206676840782166,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 5.859209020854905,
      "inferenceMs": 5.298291012877598,
      "pairsPerSecond": 188.74010460533043,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float16": 48
      },
      "driverAllocatedMemoryBytes": 1102036992,
      "loadMs": 1699.4528750074096,
      "totalMs": 1704.7511660202872,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9907999183051288,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 24.171249999199063,
      "inferenceMs": 23.518874979345128,
      "pairsPerSecond": 42.519040595191115,
      "inputTokens": 15,
      "placement": {
        "mps:0/torch.float16": 48
      },
      "driverAllocatedMemoryBytes": 1102102528,
      "loadMs": 1699.4528750074096,
      "totalMs": 1722.9717499867547,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "04ded685adf9835dff917af61327bb4aa1dbec01",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  }
]

EttinX NLI S Desktop

72 configurations tested
Passed72
Failed0
TestedNov 12, 2026
Runtimetransformers 5.22.1

Quality criteria

{ "file": "research/ettinx-nli-family-expansion/quality-gates.json", "sha256": "b98612c89959b9c069902c28bfd95bd8d943b61cefa34ea3c6da14ace9f0c18c", "description": "All six expected labels, per-case minimum margins, complete finite normalized three-label probabilities, <=1000ms/pair." }

PrecisionDeviceResultPlatformInference / load
FP16mpsAcceleratedPassedVery fastmacos · aarch649 ms / 1900 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6411 ms / 1849 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6411 ms / 1932 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6411 ms / 1799 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6412 ms / 1842 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6413 ms / 1859 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6424 ms / 1849 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6424 ms / 1842 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6425 ms / 1849 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6425 ms / 1842 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6426 ms / 1842 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6426 ms / 1900 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6426 ms / 1900 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6426 ms / 1849 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6427 ms / 1799 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6427 ms / 1842 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6427 ms / 1849 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6428 ms / 1900 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6429 ms / 1859 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6429 ms / 1900 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6429 ms / 1799 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6429 ms / 1799 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6430 ms / 1799 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6430 ms / 1859 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6431 ms / 1932 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6431 ms / 1932 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6431 ms / 1859 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6433 ms / 1932 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6479 ms / 1842 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6485 ms / 1799 ms
FP32mpsAcceleratedPassedVery fastmacos · aarch6486 ms / 1849 ms
FP16mpsAcceleratedPassedVery fastmacos · aarch6488 ms / 1900 ms
FP32mpsAcceleratedPassedFastmacos · aarch64181 ms / 1932 ms
FP16mpsAcceleratedPassedFastmacos · aarch64186 ms / 1859 ms
FP32mpsAcceleratedPassedFastmacos · aarch64196 ms / 1932 ms
FP16mpsAcceleratedPassedFastmacos · aarch64258 ms / 1859 ms
FP32cpuPassedVery fastmacos · aarch648 ms / 1619 ms
FP32cpuPassedVery fastmacos · aarch648 ms / 1685 ms
FP32cpuPassedVery fastmacos · aarch649 ms / 1685 ms
FP32cpuPassedVery fastmacos · aarch649 ms / 1622 ms
FP32cpuPassedVery fastmacos · aarch649 ms / 1619 ms
FP32cpuPassedVery fastmacos · aarch649 ms / 1619 ms
FP32cpuPassedVery fastmacos · aarch649 ms / 1708 ms
FP32cpuPassedVery fastmacos · aarch649 ms / 1663 ms
FP32cpuPassedVery fastmacos · aarch649 ms / 1664 ms
FP32cpuPassedVery fastmacos · aarch649 ms / 1685 ms
FP32cpuPassedVery fastmacos · aarch649 ms / 1708 ms
FP32cpuPassedVery fastmacos · aarch649 ms / 1708 ms
FP32cpuPassedVery fastmacos · aarch649 ms / 1663 ms
FP32cpuPassedVery fastmacos · aarch649 ms / 1622 ms
FP32cpuPassedVery fastmacos · aarch649 ms / 1708 ms
FP32cpuPassedVery fastmacos · aarch6410 ms / 1663 ms
FP32cpuPassedVery fastmacos · aarch6410 ms / 1619 ms
FP32cpuPassedVery fastmacos · aarch6410 ms / 1622 ms
FP32cpuPassedVery fastmacos · aarch6410 ms / 1685 ms
FP32cpuPassedVery fastmacos · aarch6410 ms / 1622 ms
FP32cpuPassedVery fastmacos · aarch6410 ms / 1664 ms
FP32cpuPassedVery fastmacos · aarch6410 ms / 1663 ms
FP32cpuPassedVery fastmacos · aarch6410 ms / 1664 ms
FP32cpuPassedVery fastmacos · aarch6410 ms / 1664 ms
FP32cpuPassedVery fastmacos · aarch6411 ms / 1619 ms
FP32cpuPassedVery fastmacos · aarch6411 ms / 1685 ms
FP32cpuPassedVery fastmacos · aarch6411 ms / 1622 ms
FP32cpuPassedVery fastmacos · aarch6411 ms / 1663 ms
FP32cpuPassedVery fastmacos · aarch6411 ms / 1664 ms
FP32cpuPassedVery fastmacos · aarch6412 ms / 1708 ms
FP32cpuPassedFastmacos · aarch64216 ms / 1619 ms
FP32cpuPassedFastmacos · aarch64217 ms / 1685 ms
FP32cpuPassedFastmacos · aarch64218 ms / 1622 ms
FP32cpuPassedFastmacos · aarch64221 ms / 1663 ms
FP32cpuPassedFastmacos · aarch64223 ms / 1664 ms
FP32cpuPassedFastmacos · aarch64226 ms / 1708 ms
Test setup & sample details
Quality gate
{ "file": "research/ettinx-nli-family-expansion/quality-gates.json", "sha256": "b98612c89959b9c069902c28bfd95bd8d943b61cefa34ea3c6da14ace9f0c18c", "description": "All six expected labels, per-case minimum margins, complete finite normalized three-label probabilities, <=1000ms/pair." }
Tested
2026-11-12T22:03:07.787398+00:00
Environment
{ "os": "macos", "osVersion": "27.2", "architecture": "aarch64", "hardware": "Apple M3 Max;Mac15,11;96GiB RAM", "cpu": "Apple M3 Max", "gpu": "Apple M3 Max", "driver": "OS-managed Metal/MPS macOS27.2 build26B118", "runtime": "App-owned CPython3.13.18;Torch2.10.0;Transformers5.22.1", "timingScope": "Fresh helper/model process per cold repetition; runtime/tokenizer/model startup in load. OS files/driver cache may be warm. Five warm pairs follow six fixtures. Snapshot loads, not Internet transfer.", "concurrentUploadActivity": "none; uploads excluded" }
Runtime
transformers 5.22.1
Artifact revision
db0dfbf65c68dcd4747f384257a44f18776be709
Fixture
{
  "file": "model-samples/native/ettinx-nli/quality-gates.json",
  "url": "https://assets.localai.tools/samples/v1/files/b98612c89959b9c069902c28bfd95bd8d943b61cefa34ea3c6da14ace9f0c18c/ettinx-nli-quality-gates.json",
  "size": 2262,
  "sha1": "9104fbed107fd4340666e85c8fcaf3376ba1a2a4",
  "sha256": "b98612c89959b9c069902c28bfd95bd8d943b61cefa34ea3c6da14ace9f0c18c"
}
Rerun when
artifact-revision; runtime-version; fixture-checksum; quality-gate; backend-behavior
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9052759148180485,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 216.26566699706018,
      "inferenceMs": 215.80570799415,
      "pairsPerSecond": 4.6337977308139955,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1618.503875011811,
      "totalMs": 1834.309583005961,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9997192201844882,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 11.050541012082249,
      "inferenceMs": 10.772582987556234,
      "pairsPerSecond": 92.82824752012893,
      "inputTokens": 26,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1618.503875011811,
      "totalMs": 1629.2764579993673,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9867861345410347,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.75666698650457,
      "inferenceMs": 9.566708002239466,
      "pairsPerSecond": 104.52916507600216,
      "inputTokens": 18,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1618.503875011811,
      "totalMs": 1628.0705830140505,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9980845952522941,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 8.473416994092986,
      "inferenceMs": 8.296917018014938,
      "pairsPerSecond": 120.5266965824437,
      "inputTokens": 16,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1618.503875011811,
      "totalMs": 1626.800792029826,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9949608948081732,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.125957993092015,
      "inferenceMs": 8.972791983978823,
      "pairsPerSecond": 111.44803109060464,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1618.503875011811,
      "totalMs": 1627.4766669957899,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9816788006573915,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.125749988015741,
      "inferenceMs": 8.963917003711686,
      "pairsPerSecond": 111.55837337471223,
      "inputTokens": 15,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1618.503875011811,
      "totalMs": 1627.4677920155227,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9052759148180485,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 226.32870799861848,
      "inferenceMs": 225.72666601627134,
      "pairsPerSecond": 4.430136756318882,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1707.8512080188375,
      "totalMs": 1933.5778740351088,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9997192201844882,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 11.902750004082918,
      "inferenceMs": 11.625749990344048,
      "pairsPerSecond": 86.01595603127248,
      "inputTokens": 26,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1707.8512080188375,
      "totalMs": 1719.4769580091815,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9867861345410347,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.559083002386615,
      "inferenceMs": 9.345124999526888,
      "pairsPerSecond": 107.0076644293818,
      "inputTokens": 18,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1707.8512080188375,
      "totalMs": 1717.1963330183644,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9980845952522941,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.142291994066909,
      "inferenceMs": 8.97495899698697,
      "pairsPerSecond": 111.42112184977282,
      "inputTokens": 16,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1707.8512080188375,
      "totalMs": 1716.8261670158245,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9949608948081732,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.384417004184797,
      "inferenceMs": 9.215833008056507,
      "pairsPerSecond": 108.50891060263324,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1707.8512080188375,
      "totalMs": 1717.067041026894,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9816788006573915,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.681583003839478,
      "inferenceMs": 9.492166980635375,
      "pairsPerSecond": 105.35002197496775,
      "inputTokens": 15,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1707.8512080188375,
      "totalMs": 1717.3433749994729,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9052759148180485,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 217.0542500098236,
      "inferenceMs": 216.5741250209976,
      "pairsPerSecond": 4.617356759045184,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1685.442875023,
      "totalMs": 1902.0170000439975,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9997192201844882,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 11.32845800020732,
      "inferenceMs": 11.07570799649693,
      "pairsPerSecond": 90.28768186343338,
      "inputTokens": 26,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1685.442875023,
      "totalMs": 1696.5185830194969,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9867861345410347,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.7996249969583,
      "inferenceMs": 9.581041987985373,
      "pairsPerSecond": 104.37278129602188,
      "inputTokens": 18,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1685.442875023,
      "totalMs": 1695.0239170109853,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9980845952522941,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 8.537499990779907,
      "inferenceMs": 8.368624985450879,
      "pairsPerSecond": 119.49394335850057,
      "inputTokens": 16,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1685.442875023,
      "totalMs": 1693.8115000084508,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9949608948081732,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.302124992245808,
      "inferenceMs": 9.153999999398366,
      "pairsPerSecond": 109.24186148849941,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1685.442875023,
      "totalMs": 1694.5968750223983,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9816788006573915,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 8.963000000221655,
      "inferenceMs": 8.80262500140816,
      "pairsPerSecond": 113.6024765158154,
      "inputTokens": 15,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1685.442875023,
      "totalMs": 1694.245500024408,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9052759148180485,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 218.09249999932945,
      "inferenceMs": 217.5912079983391,
      "pairsPerSecond": 4.595773924871235,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1622.0134579925798,
      "totalMs": 1839.6046659909189,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9997192201844882,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 11.42470899503678,
      "inferenceMs": 11.174124985700473,
      "pairsPerSecond": 89.4924659675545,
      "inputTokens": 26,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1622.0134579925798,
      "totalMs": 1633.1875829782803,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9867861345410347,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.775666985660791,
      "inferenceMs": 9.569167013978586,
      "pairsPerSecond": 104.50230396639598,
      "inputTokens": 18,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1622.0134579925798,
      "totalMs": 1631.5826250065584,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9980845952522941,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.796792000997812,
      "inferenceMs": 9.620416996767744,
      "pairsPerSecond": 103.94559823508469,
      "inputTokens": 16,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1622.0134579925798,
      "totalMs": 1631.6338749893475,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9949608948081732,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 8.981082995887846,
      "inferenceMs": 8.81854200270027,
      "pairsPerSecond": 113.3974300619985,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1622.0134579925798,
      "totalMs": 1630.83199999528,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9816788006573915,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.612291003577411,
      "inferenceMs": 9.453374979784712,
      "pairsPerSecond": 105.7823266440208,
      "inputTokens": 15,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1622.0134579925798,
      "totalMs": 1631.4668329723645,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9052759148180485,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 223.87854199041612,
      "inferenceMs": 223.3853750221897,
      "pairsPerSecond": 4.47656879910185,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1664.0752090024762,
      "totalMs": 1887.4605840246659,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9997192201844882,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 11.762124981032684,
      "inferenceMs": 11.46958299796097,
      "pairsPerSecond": 87.18712791718558,
      "inputTokens": 26,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1664.0752090024762,
      "totalMs": 1675.5447920004372,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9867861345410347,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.99645801493898,
      "inferenceMs": 9.775207989150658,
      "pairsPerSecond": 102.29961358468111,
      "inputTokens": 18,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1664.0752090024762,
      "totalMs": 1673.8504169916268,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9980845952522941,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.828500013099983,
      "inferenceMs": 9.645832993555814,
      "pairsPerSecond": 103.67170991536757,
      "inputTokens": 16,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1664.0752090024762,
      "totalMs": 1673.721041996032,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9949608948081732,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.30150001659058,
      "inferenceMs": 9.132917009992525,
      "pairsPerSecond": 109.49404214511947,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1664.0752090024762,
      "totalMs": 1673.2081260124687,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9816788006573915,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.846875007497147,
      "inferenceMs": 9.67812500311993,
      "pairsPerSecond": 103.3257991271688,
      "inputTokens": 15,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1664.0752090024762,
      "totalMs": 1673.7533340055961,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9052759148180485,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 221.1161660088692,
      "inferenceMs": 220.66424999502487,
      "pairsPerSecond": 4.531771684912921,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1662.9761250223964,
      "totalMs": 1883.6403750174213,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9997192201844882,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 11.51258300524205,
      "inferenceMs": 11.225582973565906,
      "pairsPerSecond": 89.08223317709273,
      "inputTokens": 26,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1662.9761250223964,
      "totalMs": 1674.2017079959624,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9867861345410347,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.875458985334262,
      "inferenceMs": 9.65029199142009,
      "pairsPerSecond": 103.62380753754218,
      "inputTokens": 18,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1662.9761250223964,
      "totalMs": 1672.6264170138165,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9980845952522941,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.589207998942584,
      "inferenceMs": 9.405166987562552,
      "pairsPerSecond": 106.32453430358078,
      "inputTokens": 16,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1662.9761250223964,
      "totalMs": 1672.381292009959,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9949608948081732,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.238375001586974,
      "inferenceMs": 9.055417001945898,
      "pairsPerSecond": 110.43113749318361,
      "inputTokens": 14,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1662.9761250223964,
      "totalMs": 1672.0315420243423,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9816788006573915,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 9.717584005557,
      "inferenceMs": 9.559458994772285,
      "pairsPerSecond": 104.60843030414829,
      "inputTokens": 15,
      "placement": {
        "cpu/torch.float32": 120
      },
      "loadMs": 1662.9761250223964,
      "totalMs": 1672.5355840171687,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9052758924663067,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 196.74625000334345,
      "inferenceMs": 195.81829197704792,
      "pairsPerSecond": 5.10677521442793,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float32": 120
      },
      "driverAllocatedMemoryBytes": 1118601216,
      "loadMs": 1932.2858749947045,
      "totalMs": 2128.1041669717524,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9997192201844882,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 31.34529199451208,
      "inferenceMs": 30.58983400114812,
      "pairsPerSecond": 32.69059910434517,
      "inputTokens": 26,
      "placement": {
        "mps:0/torch.float32": 120
      },
      "driverAllocatedMemoryBytes": 1118699520,
      "loadMs": 1932.2858749947045,
      "totalMs": 1962.8757089958526,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9867861312814057,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 31.456790980882943,
      "inferenceMs": 30.69375001359731,
      "pairsPerSecond": 32.57992260825089,
      "inputTokens": 18,
      "placement": {
        "mps:0/torch.float32": 120
      },
      "driverAllocatedMemoryBytes": 1118781440,
      "loadMs": 1932.2858749947045,
      "totalMs": 1962.9796250083018,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9980845952522941,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 33.592040999792516,
      "inferenceMs": 32.87445902242325,
      "pairsPerSecond": 30.41875150912484,
      "inputTokens": 16,
      "placement": {
        "mps:0/torch.float32": 120
      },
      "driverAllocatedMemoryBytes": 1118863360,
      "loadMs": 1932.2858749947045,
      "totalMs": 1965.1603340171278,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9949608956230804,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 11.940833006519824,
      "inferenceMs": 11.138917005155236,
      "pairsPerSecond": 89.77533449052426,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float32": 120
      },
      "driverAllocatedMemoryBytes": 1118863360,
      "loadMs": 1932.2858749947045,
      "totalMs": 1943.4247919998597,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9816788006573915,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 181.9457079982385,
      "inferenceMs": 180.74508299469016,
      "pairsPerSecond": 5.532653964530684,
      "inputTokens": 15,
      "placement": {
        "mps:0/torch.float32": 120
      },
      "driverAllocatedMemoryBytes": 1118928896,
      "loadMs": 1932.2858749947045,
      "totalMs": 2113.0309579893947,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9052758924663067,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 80.29054102371447,
      "inferenceMs": 78.7571250111796,
      "pairsPerSecond": 12.697263896543324,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float32": 120
      },
      "driverAllocatedMemoryBytes": 1118601216,
      "loadMs": 1841.5201659954619,
      "totalMs": 1920.2772910066415,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9997192201844882,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 25.851458980469033,
      "inferenceMs": 25.164708000374958,
      "pairsPerSecond": 39.738192073800334,
      "inputTokens": 26,
      "placement": {
        "mps:0/torch.float32": 120
      },
      "driverAllocatedMemoryBytes": 1118699520,
      "loadMs": 1841.5201659954619,
      "totalMs": 1866.6848739958368,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9867861312814057,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 24.797292018774897,
      "inferenceMs": 24.249957990832627,
      "pairsPerSecond": 41.23718483875464,
      "inputTokens": 18,
      "placement": {
        "mps:0/torch.float32": 120
      },
      "driverAllocatedMemoryBytes": 1118781440,
      "loadMs": 1841.5201659954619,
      "totalMs": 1865.7701239862945,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9980845952522941,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 26.20116699836217,
      "inferenceMs": 25.588916993001476,
      "pairsPerSecond": 39.079418651187865,
      "inputTokens": 16,
      "placement": {
        "mps:0/torch.float32": 120
      },
      "driverAllocatedMemoryBytes": 1118863360,
      "loadMs": 1841.5201659954619,
      "totalMs": 1867.1090829884633,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9949608956230804,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 12.279999995371327,
      "inferenceMs": 11.710041988408193,
      "pairsPerSecond": 85.39679029246037,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float32": 120
      },
      "driverAllocatedMemoryBytes": 1118863360,
      "loadMs": 1841.5201659954619,
      "totalMs": 1853.23020798387,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9816788006573915,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 27.458124997792765,
      "inferenceMs": 26.85716698761098,
      "pairsPerSecond": 37.23400909936974,
      "inputTokens": 15,
      "placement": {
        "mps:0/torch.float32": 120
      },
      "driverAllocatedMemoryBytes": 1118928896,
      "loadMs": 1841.5201659954619,
      "totalMs": 1868.3773329830728,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9052758924663067,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 87.4681249842979,
      "inferenceMs": 85.83658401039429,
      "pairsPerSecond": 11.650044226817158,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float32": 120
      },
      "driverAllocatedMemoryBytes": 1118601216,
      "loadMs": 1848.542375024408,
      "totalMs": 1934.3789590348024,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9997192201844882,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 29.201375000411645,
      "inferenceMs": 24.94608299457468,
      "pairsPerSecond": 40.086453661582134,
      "inputTokens": 26,
      "placement": {
        "mps:0/torch.float32": 120
      },
      "driverAllocatedMemoryBytes": 1118699520,
      "loadMs": 1848.542375024408,
      "totalMs": 1873.4884580189828,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9867861312814057,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 24.36587499687448,
      "inferenceMs": 23.7409999826923,
      "pairsPerSecond": 42.12122491592694,
      "inputTokens": 18,
      "placement": {
        "mps:0/torch.float32": 120
      },
      "driverAllocatedMemoryBytes": 1118781440,
      "loadMs": 1848.542375024408,
      "totalMs": 1872.2833750071004,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9980845952522941,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 27.868042001500726,
      "inferenceMs": 27.29774999897927,
      "pairsPerSecond": 36.63305583930516,
      "inputTokens": 16,
      "placement": {
        "mps:0/torch.float32": 120
      },
      "driverAllocatedMemoryBytes": 1118863360,
      "loadMs": 1848.542375024408,
      "totalMs": 1875.8401250233874,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9949608956230804,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 12.351915996987373,
      "inferenceMs": 10.702416999265552,
      "pairsPerSecond": 93.43683768522797,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float32": 120
      },
      "driverAllocatedMemoryBytes": 1118863360,
      "loadMs": 1848.542375024408,
      "totalMs": 1859.2447920236737,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9816788006573915,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 26.917624985799193,
      "inferenceMs": 26.296375028323382,
      "pairsPerSecond": 38.02805515676274,
      "inputTokens": 15,
      "placement": {
        "mps:0/torch.float32": 120
      },
      "driverAllocatedMemoryBytes": 1118928896,
      "loadMs": 1848.542375024408,
      "totalMs": 1874.8387500527315,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9052134975790977,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 259.67624998884276,
      "inferenceMs": 258.0718329991214,
      "pairsPerSecond": 3.874890135737535,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float16": 120
      },
      "driverAllocatedMemoryBytes": 1126989824,
      "loadMs": 1858.6206249892712,
      "totalMs": 2116.6924579883926,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9997197498159949,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 29.855917004169896,
      "inferenceMs": 28.8093329872936,
      "pairsPerSecond": 34.710973712617765,
      "inputTokens": 26,
      "placement": {
        "mps:0/torch.float16": 120
      },
      "driverAllocatedMemoryBytes": 1127088128,
      "loadMs": 1858.6206249892712,
      "totalMs": 1887.4299579765648,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9867661232128739,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 31.502500001806766,
      "inferenceMs": 30.892667011357844,
      "pairsPerSecond": 32.370141420044604,
      "inputTokens": 18,
      "placement": {
        "mps:0/torch.float16": 120
      },
      "driverAllocatedMemoryBytes": 1127170048,
      "loadMs": 1858.6206249892712,
      "totalMs": 1889.513292000629,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9980876952176914,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 30.657541996333748,
      "inferenceMs": 29.975084005855024,
      "pairsPerSecond": 33.36104078322751,
      "inputTokens": 16,
      "placement": {
        "mps:0/torch.float16": 120
      },
      "driverAllocatedMemoryBytes": 1127251968,
      "loadMs": 1858.6206249892712,
      "totalMs": 1888.5957089951262,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9949509820435196,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 13.746499986154959,
      "inferenceMs": 13.158666988601908,
      "pairsPerSecond": 75.99553973561335,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float16": 120
      },
      "driverAllocatedMemoryBytes": 1127251968,
      "loadMs": 1858.6206249892712,
      "totalMs": 1871.779291977873,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9817087044939399,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 186.7764589842409,
      "inferenceMs": 186.18420799612068,
      "pairsPerSecond": 5.371024807973166,
      "inputTokens": 15,
      "placement": {
        "mps:0/torch.float16": 120
      },
      "driverAllocatedMemoryBytes": 1127317504,
      "loadMs": 1858.6206249892712,
      "totalMs": 2044.8048329853918,
      "grade": "good",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9052134975790977,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 86.25554200261831,
      "inferenceMs": 85.31774999573827,
      "pairsPerSecond": 11.7208904366319,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float16": 120
      },
      "driverAllocatedMemoryBytes": 1126989824,
      "loadMs": 1798.9354169985745,
      "totalMs": 1884.2531669943128,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9997197498159949,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 30.238250008551404,
      "inferenceMs": 29.32112498092465,
      "pairsPerSecond": 34.10510342459803,
      "inputTokens": 26,
      "placement": {
        "mps:0/torch.float16": 120
      },
      "driverAllocatedMemoryBytes": 1127088128,
      "loadMs": 1798.9354169985745,
      "totalMs": 1828.2565419794992,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9867661232128739,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 30.04337500897236,
      "inferenceMs": 29.15558297536336,
      "pairsPerSecond": 34.298748230999394,
      "inputTokens": 18,
      "placement": {
        "mps:0/torch.float16": 120
      },
      "driverAllocatedMemoryBytes": 1127170048,
      "loadMs": 1798.9354169985745,
      "totalMs": 1828.0909999739379,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9980876952176914,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 31.11800001352094,
      "inferenceMs": 29.847625002730638,
      "pairsPerSecond": 33.5035032069893,
      "inputTokens": 16,
      "placement": {
        "mps:0/torch.float16": 120
      },
      "driverAllocatedMemoryBytes": 1127251968,
      "loadMs": 1798.9354169985745,
      "totalMs": 1828.7830420013051,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9949509820435196,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 11.797791987191886,
      "inferenceMs": 11.278333986410871,
      "pairsPerSecond": 88.66557784198339,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float16": 120
      },
      "driverAllocatedMemoryBytes": 1127251968,
      "loadMs": 1798.9354169985745,
      "totalMs": 1810.2137509849854,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9817087044939399,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 27.16250001685694,
      "inferenceMs": 26.553292002063245,
      "pairsPerSecond": 37.66011385414276,
      "inputTokens": 15,
      "placement": {
        "mps:0/torch.float16": 120
      },
      "driverAllocatedMemoryBytes": 1127317504,
      "loadMs": 1798.9354169985745,
      "totalMs": 1825.4887090006378,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9052134975790977,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 89.42645898787305,
      "inferenceMs": 88.48787500755861,
      "pairsPerSecond": 11.300983325846397,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float16": 120
      },
      "driverAllocatedMemoryBytes": 1126989824,
      "loadMs": 1899.735042010434,
      "totalMs": 1988.2229170179926,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9997197498159949,
      "minimumMargin": 0.3,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 28.764208022039384,
      "inferenceMs": 27.856375003466383,
      "pairsPerSecond": 35.898425400848545,
      "inputTokens": 26,
      "placement": {
        "mps:0/torch.float16": 120
      },
      "driverAllocatedMemoryBytes": 1127088128,
      "loadMs": 1899.735042010434,
      "totalMs": 1927.5914170139004,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "entailment",
      "actualLabel": "entailment",
      "finite": true,
      "allLabels": true,
      "margin": 0.9867661232128739,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 26.816708006663248,
      "inferenceMs": 26.164834009250626,
      "pairsPerSecond": 38.21923730326163,
      "inputTokens": 18,
      "placement": {
        "mps:0/torch.float16": 120
      },
      "driverAllocatedMemoryBytes": 1127170048,
      "loadMs": 1899.735042010434,
      "totalMs": 1925.8998760196846,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9980876952176914,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 29.8120410006959,
      "inferenceMs": 28.873917006421834,
      "pairsPerSecond": 34.633333599233886,
      "inputTokens": 16,
      "placement": {
        "mps:0/torch.float16": 120
      },
      "driverAllocatedMemoryBytes": 1127251968,
      "loadMs": 1899.735042010434,
      "totalMs": 1928.6089590168558,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "contradiction",
      "actualLabel": "contradiction",
      "finite": true,
      "allLabels": true,
      "margin": 0.9949509820435196,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 10.069416981423274,
      "inferenceMs": 9.434875013539568,
      "pairsPerSecond": 105.9897453400225,
      "inputTokens": 14,
      "placement": {
        "mps:0/torch.float16": 120
      },
      "driverAllocatedMemoryBytes": 1127251968,
      "loadMs": 1899.735042010434,
      "totalMs": 1909.1699170239735,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "expectedLabel": "neutral",
      "actualLabel": "neutral",
      "finite": true,
      "allLabels": true,
      "margin": 0.9817087044939399,
      "minimumMargin": 0.15,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 26.70245899935253,
      "inferenceMs": 26.100458024302498,
      "pairsPerSecond": 38.3135038882799,
      "inputTokens": 15,
      "placement": {
        "mps:0/torch.float16": 120
      },
      "driverAllocatedMemoryBytes": 1127317504,
      "loadMs": 1899.735042010434,
      "totalMs": 1925.8355000347365,
      "grade": "excellent",
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "db0dfbf65c68dcd4747f384257a44f18776be709",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false
    }
  }
]
Release validation still required
  • Sample publication, complete dependency backup and forced model/runtime-source recovery pending under user upload exclusion.

Something not quite right?

Tell us what happened with EttinX NLI Desktop. Loading, results, or the page itself—we’re here for all of it.

Attachments Optional

Up to 3 files · 50 MB each

Browser, OS, CPU/GPU, memory, screen, storage, and network details are included automatically. Model inputs, results, and browsing history are not collected.