A model is still running

A model is still running or downloading. Leaving will stop the current operation. Stay on this page to let it finish.

Skip to content
LocalAI.toolsBeta

Join the waitlist

Get an email when the new apps and model runs launch.

No account needed.

Liquid LFM3 Desktop

Local text chat with Liquid LFM3 1.7B.

Desktop inferencePrivate · Runs on your computer
About this model

Answer questions about text, still images and WAV audio, transcribe speech, and read short text aloud.

Created by
Liquid AI
Model released
Unknown
Source repository created
Oct 21, 2026
Model weights · Q4
1.3 GB

Best for

  • Ask about an attached still image or short audio clip.
  • Transcribe speech and synthesize short passages.

Keep in mind

  • Manually confirm perception and factual answers. Noise, accents and low-quality images can reduce accuracy.
  • The checkpoint also accepts video upstream. DesktopAI excludes video and sampled frames by product policy.
  • Live audio and broad language coverage require separate qualification.
  • The Liquid AI license places conditions on commercial use and redistribution.
  • Combined PNG and WAV input is unavailable. Both devices missed the exact audio word in two mixed-attachment prompts.

Processed on your deviceYour input is processed locally in the desktop app. Model and runtime files download from external hosts.

Target platforms · verification required

  • macos · aarch64, x86_64
  • windows · x86_64

Recommended configuration

Q4 · metal (verified platform only)

Sources & technical details
Original model
https://huggingface.co/LiquidAI/LFM3-Omni-2B
Release verification
Unknown
Release evidence
Exact checkpoint date pending provenance review.
Source updated
Nov 3, 2026
Native model files
LiquidAI/LFM3-Omni-2B-GGUF
Files updated
Nov 3, 2026
Artifact revision
d508b88f08a5b4286f2d2805a6a8d69a0ff44c72

Desktop app

Sample result

LFM3 Omni 2B Desktop

Run Liquid LFM3 Desktop on your computer

Download the desktop app to load this model, provide input and view results locally.

Public desktop downloads are coming soon.

Device requirements

macOS Apple SiliconCPU / Q4TestedNot establishedRAM estimate
macOS Apple SiliconMETAL / Q4TestedNot establishedRAM estimate
If your device falls short

MemoryShow advisory requirements.

GPUUse verified CPU recovery when permitted.

PlatformNo verified configuration.

Benchmark evidence

Saved desktop measurements apply to the recorded hardware and runtime. Fallback cases report the device that actually ran; they do not prove GPU support on the requested device.

LFM3 Omni 2B Desktop

10 configurations tested
Passed10
Failed0
TestedNov 15, 2026
Runtimellama.cpp b12359

Quality criteria

No explicit quality gate recorded. Review per-case evidence; full certification is pending.

PrecisionDeviceResultPlatformInference / load
Q4metalAcceleratedPassedVery fastmacos · aarch6488 ms / 754 ms
Q4metalAcceleratedPassedFastmacos · aarch64228 ms / 754 ms
Q4metalAcceleratedPassedFastmacos · aarch64264 ms / 754 ms
Q4metalAcceleratedPassedFastmacos · aarch64394 ms / 754 ms
Q4metalAcceleratedPassedFastmacos · aarch64625 ms / 754 ms
Q4cpuPassedFastmacos · aarch64107 ms / 7635 ms
Q4cpuPassedFastmacos · aarch64477 ms / 7635 ms
Q4cpuPassedFastmacos · aarch64640 ms / 7635 ms
Q4cpuPassedFastmacos · aarch64846 ms / 7635 ms
Q4cpuPassedFastmacos · aarch64933 ms / 7635 ms
Test setup & sample details
Quality gate
No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
Tested
2026-11-15T22:36:20.409Z
Environment
{ "scope": "DesktopAI TypeScript client using native JSON-lines transport with verified cached sources.", "hardware": "Apple M3 Max", "os": "macOS", "arch": "arm64", "runtime": { "name": "llama.cpp", "version": "b12359" }, "speedGradePolicy": "Short requests completing in 2 seconds or less are interactive." }
Runtime
llama.cpp b12359
Artifact revision
d508b88f08a5b4286f2d2805a6a8d69a0ff44c72
Fixture
{
  "file": "model-samples/voxtral-realtime/jfk.wav",
  "url": "https://assets.localai.tools/samples/v1/files/4d968ac99a1d0d4bc42ae8dd1552f4235e7b952a8121b39797f3dc2ca16123e9/jfk.wav",
  "size": 352078,
  "sha1": "069db705beb788608e93b99f1a4300451ebfb972",
  "sha256": "4d968ac99a1d0d4bc42ae8dd1552f4235e7b952a8121b39797f3dc2ca16123e9"
}
Rerun when
artifactRevision; runtimeVersion; fixtureChecksum; qualityGate; nativeBackend
Per-case quality & failure details
[
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 107.171584,
      "loadMs": 7634.52925,
      "offloadedLayers": 0,
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q4",
      "selectionReason": "Exact Q4_0 package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "d508b88f08a5b4286f2d2805a6a8d69a0ff44c72",
      "totalMs": 7741.700833999999,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 18.661663151306975
      },
      "grade": "good",
      "platform": "macos",
      "arch": "aarch64"
    }
  },
  {
    "precision": "q4",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 87.70400000000001,
      "loadMs": 753.985542,
      "offloadedLayers": 25,
      "precision": "q4",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q4",
      "selectionReason": "Exact Q4_0 package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "d508b88f08a5b4286f2d2805a6a8d69a0ff44c72",
      "totalMs": 841.6895420000001,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 22.80397701359117
      },
      "grade": "good",
      "platform": "macos",
      "arch": "aarch64"
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 477.129041,
      "loadMs": 7634.52925,
      "offloadedLayers": 0,
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q4",
      "selectionReason": "Exact Q4_0 package with verified layer placement",
      "version": "b12359",
      "visualizationMs": 0.018292,
      "artifactRevision": "d508b88f08a5b4286f2d2805a6a8d69a0ff44c72",
      "totalMs": 8111.658291,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 6.287607213579775
      },
      "grade": "good",
      "platform": "macos",
      "arch": "aarch64"
    }
  },
  {
    "precision": "q4",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 228.486834,
      "loadMs": 753.985542,
      "offloadedLayers": 25,
      "precision": "q4",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q4",
      "selectionReason": "Exact Q4_0 package with verified layer placement",
      "version": "b12359",
      "visualizationMs": 0.036875000000000005,
      "artifactRevision": "d508b88f08a5b4286f2d2805a6a8d69a0ff44c72",
      "totalMs": 982.4723759999999,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 13.12985937736789
      },
      "grade": "good",
      "platform": "macos",
      "arch": "aarch64"
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "normalizedWordMatch",
      "score": 1,
      "passed": true
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 845.560792,
      "loadMs": 7634.52925,
      "offloadedLayers": 0,
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q4",
      "selectionReason": "Exact Q4_0 package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "d508b88f08a5b4286f2d2805a6a8d69a0ff44c72",
      "totalMs": 8480.090042,
      "performanceMetric": {
        "name": "audioSecondsPerSecond",
        "value": 13.009117858908482
      },
      "grade": "good",
      "platform": "macos",
      "arch": "aarch64"
    }
  },
  {
    "precision": "q4",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "normalizedWordMatch",
      "score": 1,
      "passed": true
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 393.675,
      "loadMs": 753.985542,
      "offloadedLayers": 25,
      "precision": "q4",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q4",
      "selectionReason": "Exact Q4_0 package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "d508b88f08a5b4286f2d2805a6a8d69a0ff44c72",
      "totalMs": 1147.660542,
      "performanceMetric": {
        "name": "audioSecondsPerSecond",
        "value": 27.941830189877436
      },
      "grade": "good",
      "platform": "macos",
      "arch": "aarch64"
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 639.875333,
      "loadMs": 7634.52925,
      "offloadedLayers": 0,
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q4",
      "selectionReason": "Exact Q4_0 package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "d508b88f08a5b4286f2d2805a6a8d69a0ff44c72",
      "totalMs": 8274.404583,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 14.065239798828127
      },
      "grade": "good",
      "platform": "macos",
      "arch": "aarch64"
    }
  },
  {
    "precision": "q4",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 264.48062500000003,
      "loadMs": 753.985542,
      "offloadedLayers": 25,
      "precision": "q4",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q4",
      "selectionReason": "Exact Q4_0 package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "d508b88f08a5b4286f2d2805a6a8d69a0ff44c72",
      "totalMs": 1018.466167,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 34.028957697751956
      },
      "grade": "good",
      "platform": "macos",
      "arch": "aarch64"
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "independentWhisperWordMatch",
      "score": 1,
      "passed": true
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 932.528125,
      "loadMs": 7634.52925,
      "offloadedLayers": 0,
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q4",
      "selectionReason": "Exact Q4_0 package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "d508b88f08a5b4286f2d2805a6a8d69a0ff44c72",
      "totalMs": 8567.057375,
      "performanceMetric": {
        "name": "audioSecondsPerSecond",
        "value": 3.2599552962544696
      },
      "grade": "good",
      "platform": "macos",
      "arch": "aarch64"
    }
  },
  {
    "precision": "q4",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "independentWhisperWordMatch",
      "score": 1,
      "passed": true
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 624.8902499999999,
      "loadMs": 753.985542,
      "offloadedLayers": 25,
      "precision": "q4",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q4",
      "selectionReason": "Exact Q4_0 package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "d508b88f08a5b4286f2d2805a6a8d69a0ff44c72",
      "totalMs": 1378.8757919999998,
      "performanceMetric": {
        "name": "audioSecondsPerSecond",
        "value": 4.608809306914295
      },
      "grade": "good",
      "platform": "macos",
      "arch": "aarch64"
    }
  }
]

LFM3-VL 2B Desktop

78 configurations tested
Passed75
Failed3
TestedNov 16, 2026
Runtimellama.cpp b12359

Quality criteria

No explicit quality gate recorded. Review per-case evidence; full certification is pending.

PrecisionDeviceResultPlatformInference / load
Q5_K_MmetalAcceleratedPassedVery fastmacos · aarch6464 ms / 2474 ms
FP16metalAcceleratedPassedFastmacos · aarch64117 ms / 2055 ms
Q5_K_MmetalAcceleratedPassedFastmacos · aarch64133 ms / 2474 ms
FP16metalAcceleratedPassedFastmacos · aarch64136 ms / 2055 ms
FP16metalAcceleratedPassedFastmacos · aarch64138 ms / 2055 ms
BF16metalAcceleratedPassedFastmacos · aarch64139 ms / 3233 ms
Q5_K_MmetalAcceleratedPassedFastmacos · aarch64160 ms / 2474 ms
BF16metalAcceleratedPassedFastmacos · aarch64169 ms / 3233 ms
BF16metalAcceleratedPassedFastmacos · aarch64186 ms / 3233 ms
FP16metalAcceleratedPassedFastmacos · aarch64221 ms / 2055 ms
BF16metalAcceleratedPassedFastmacos · aarch64231 ms / 3233 ms
Q5_K_MmetalAcceleratedPassedFastmacos · aarch64246 ms / 2474 ms
FP16metalAcceleratedPassedFastmacos · aarch64323 ms / 2055 ms
Q5_K_MmetalAcceleratedPassedFastmacos · aarch64329 ms / 2474 ms
BF16metalAcceleratedPassedFastmacos · aarch64349 ms / 3233 ms
NVFP4metalAcceleratedPassedFastmacos · aarch64402 ms / 2785 ms
Q4_K_MmetalAcceleratedPassedFastmacos · aarch64543 ms / 6791 ms
Q4_K_MmetalAcceleratedPassedFastmacos · aarch64590 ms / 6791 ms
Q4_K_MmetalAcceleratedPassedFastmacos · aarch64618 ms / 6791 ms
Q6_KmetalAcceleratedPassedFastmacos · aarch64695 ms / 2187 ms
Q4metalAcceleratedPassedFastmacos · aarch64731 ms / 2902 ms
Q4_K_MmetalAcceleratedPassedFastmacos · aarch64744 ms / 6791 ms
Q8_0metalAcceleratedPassedFastmacos · aarch64759 ms / 2178 ms
NVFP4metalAcceleratedPassedFastmacos · aarch64800 ms / 2785 ms
NVFP4metalAcceleratedPassedFastmacos · aarch64805 ms / 2785 ms
Q8_0metalAcceleratedPassedFastmacos · aarch64819 ms / 2178 ms
Q6_KmetalAcceleratedPassedFastmacos · aarch64829 ms / 2187 ms
Q4metalAcceleratedPassedFastmacos · aarch64879 ms / 2902 ms
Q4metalAcceleratedPassedFastmacos · aarch64880 ms / 2902 ms
Q4metalAcceleratedPassedFastmacos · aarch64888 ms / 2902 ms
Q4metalAcceleratedPassedFastmacos · aarch64925 ms / 2902 ms
NVFP4metalAcceleratedPassedFastmacos · aarch64954 ms / 2785 ms
Q6_KmetalAcceleratedPassedFastmacos · aarch64975 ms / 2187 ms
Q6_KmetalAcceleratedPassedFastmacos · aarch64996 ms / 2187 ms
Q8_0metalAcceleratedPassedFastmacos · aarch64999 ms / 2178 ms
Q8_0metalAcceleratedPassedFastmacos · aarch641040 ms / 2178 ms
Q4_K_MmetalAcceleratedPassedFastmacos · aarch641141 ms / 6791 ms
NVFP4metalAcceleratedPassedFastmacos · aarch641149 ms / 2785 ms
Q6_KmetalAcceleratedPassedFastmacos · aarch641352 ms / 2187 ms
Q8_0metalAcceleratedPassedFastmacos · aarch641449 ms / 2178 ms
Q8_0cpuPassedFastmacos · aarch64120 ms / 3520 ms
Q5_K_McpuPassedFastmacos · aarch64152 ms / 2893 ms
Q6_KcpuPassedFastmacos · aarch64157 ms / 3147 ms
NVFP4cpuPassedFastmacos · aarch64247 ms / 3251 ms
FP16cpuPassedFastmacos · aarch64306 ms / 2542 ms
Q8_0cpuPassedFastmacos · aarch64484 ms / 3520 ms
Q8_0cpuPassedFastmacos · aarch64498 ms / 3520 ms
Q6_KcpuPassedFastmacos · aarch64656 ms / 3147 ms
Q6_KcpuPassedFastmacos · aarch64659 ms / 3147 ms
Q5_K_McpuPassedFastmacos · aarch64659 ms / 2893 ms
Q5_K_McpuPassedFastmacos · aarch64663 ms / 2893 ms
NVFP4cpuPassedFastmacos · aarch64761 ms / 3251 ms
NVFP4cpuPassedFastmacos · aarch64788 ms / 3251 ms
Q4_K_McpuPassedFastmacos · aarch64815 ms / 2799 ms
FP16cpuPassedFastmacos · aarch64911 ms / 2542 ms
FP16cpuPassedFastmacos · aarch64954 ms / 2542 ms
Q4cpuPassedFastmacos · aarch641143 ms / 7105 ms
Q8_0cpuPassedFastmacos · aarch641295 ms / 3520 ms
Q8_0cpuPassedFastmacos · aarch641335 ms / 3520 ms
Q4_K_McpuPassedFastmacos · aarch641381 ms / 2799 ms
NVFP4cpuPassedFastmacos · aarch641401 ms / 3251 ms
NVFP4cpuPassedFastmacos · aarch641421 ms / 3251 ms
Q4_K_McpuPassedFastmacos · aarch641631 ms / 2799 ms
Q6_KcpuPassedFastmacos · aarch641645 ms / 3147 ms
Q6_KcpuPassedFastmacos · aarch641656 ms / 3147 ms
Q5_K_McpuPassedFastmacos · aarch641687 ms / 2893 ms
Q5_K_McpuPassedFastmacos · aarch641693 ms / 2893 ms
FP16cpuPassedFastmacos · aarch641727 ms / 2542 ms
FP16cpuPassedFastmacos · aarch641809 ms / 2542 ms
Q4cpuPassedModeratemacos · aarch642016 ms / 7105 ms
Q4_K_McpuPassedModeratemacos · aarch642046 ms / 2799 ms
Q4cpuPassedModeratemacos · aarch642118 ms / 7105 ms
Q4cpuPassedModeratemacos · aarch642239 ms / 7105 ms
Q4_K_McpuPassedModeratemacos · aarch642541 ms / 2799 ms
Q4cpuPassedModeratemacos · aarch643452 ms / 7105 ms
BF16cpuFailed—macos · aarch6439035 ms / 3972 ms
BF16cpuFailed—macos · aarch6439429 ms / 3972 ms
BF16cpuFailed—macos · aarch6443616 ms / 3972 ms
Test setup & sample details
Quality gate
No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
Tested
2026-11-16T01:57:03.731Z
Environment
{ "scope": "DesktopAI TypeScript client using native JSON-lines transport with verified cached sources.", "hardware": "Apple M3 Max", "os": "macOS", "arch": "arm64", "runtime": { "name": "llama.cpp", "version": "b12359" }, "speedGradePolicy": "Short requests completing in 2 seconds or less are interactive." }
Runtime
llama.cpp b12359
Artifact revision
1add221a5ea6c047fcf1eaaf2d247939d7cf8849
Fixture
{
  "file": "model-samples/lfm3-vl-2b-native/vl-samples/desktopai-arithmetic-input.json",
  "url": "https://assets.localai.tools/samples/v1/files/712f8b4b7e032c2e4b12e94dbb2e8d7c9681d69c344dbcf7874175c28b7678b5/desktopai-arithmetic-input.json",
  "size": 219,
  "sha1": "30f7cac931b376f1ddcdf626b6cad3d91a3c6e93",
  "sha256": "712f8b4b7e032c2e4b12e94dbb2e8d7c9681d69c344dbcf7874175c28b7678b5"
}
Rerun when
artifactRevision; runtimeVersion; fixtureChecksum; qualityGate; nativeBackend
Per-case quality & failure details
[
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "15"
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1143.015709,
      "loadMs": 7105.099291,
      "offloadedLayers": 0,
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q4",
      "selectionReason": "Exact QAD Q4_0 package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 8248.115,
      "charactersPerSecond": 1.749757229277065,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 1.749757229277065
      },
      "grade": "good"
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "red"
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 2015.819,
      "loadMs": 7105.099291,
      "offloadedLayers": 0,
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q4",
      "selectionReason": "Exact QAD Q4_0 package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 9120.918291,
      "charactersPerSecond": 1.4882288538802344,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 1.4882288538802344
      },
      "grade": "moderate"
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "blue"
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 2117.605291,
      "loadMs": 7105.099291,
      "offloadedLayers": 0,
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q4",
      "selectionReason": "Exact QAD Q4_0 package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 9222.704582,
      "charactersPerSecond": 1.888926145491011,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 1.888926145491011
      },
      "grade": "moderate"
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "localai2026"
      ]
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 2239.10725,
      "loadMs": 7105.099291,
      "offloadedLayers": 0,
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q4",
      "selectionReason": "Exact QAD Q4_0 package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 9344.206541,
      "charactersPerSecond": 5.805885358997431,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 5.805885358997431
      },
      "grade": "moderate"
    }
  },
  {
    "precision": "q4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "3"
      ]
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 3451.848541,
      "loadMs": 7105.099291,
      "offloadedLayers": 0,
      "precision": "q4",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q4",
      "selectionReason": "Exact QAD Q4_0 package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 10556.947832,
      "charactersPerSecond": 0.28969984868174437,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 0.28969984868174437
      },
      "grade": "moderate"
    }
  },
  {
    "precision": "q4",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "15"
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 879.5343340000001,
      "loadMs": 2901.979416,
      "offloadedLayers": 25,
      "precision": "q4",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q4",
      "selectionReason": "Exact QAD Q4_0 package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3781.51375,
      "charactersPerSecond": 2.2739305592588726,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 2.2739305592588726
      },
      "grade": "good"
    }
  },
  {
    "precision": "q4",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "red"
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 924.6751250000001,
      "loadMs": 2901.979416,
      "offloadedLayers": 25,
      "precision": "q4",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q4",
      "selectionReason": "Exact QAD Q4_0 package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3826.6545410000003,
      "charactersPerSecond": 3.2443827230671958,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 3.2443827230671958
      },
      "grade": "good"
    }
  },
  {
    "precision": "q4",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "blue"
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 879.015125,
      "loadMs": 2901.979416,
      "offloadedLayers": 25,
      "precision": "q4",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q4",
      "selectionReason": "Exact QAD Q4_0 package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3780.994541,
      "charactersPerSecond": 4.550547409522674,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 4.550547409522674
      },
      "grade": "good"
    }
  },
  {
    "precision": "q4",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "localai2026"
      ]
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 731.3244589999999,
      "loadMs": 2901.979416,
      "offloadedLayers": 25,
      "precision": "q4",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q4",
      "selectionReason": "Exact QAD Q4_0 package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3633.303875,
      "charactersPerSecond": 17.775967752775518,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 17.775967752775518
      },
      "grade": "good"
    }
  },
  {
    "precision": "q4",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "3"
      ]
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 887.623542,
      "loadMs": 2901.979416,
      "offloadedLayers": 25,
      "precision": "q4",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q4",
      "selectionReason": "Exact QAD Q4_0 package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3789.6029580000004,
      "charactersPerSecond": 1.1266037376011822,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 1.1266037376011822
      },
      "grade": "good"
    }
  },
  {
    "precision": "q4_k_m",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "15"
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 815.113167,
      "loadMs": 2799.4535,
      "offloadedLayers": 0,
      "precision": "q4_k_m",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q4_k_m",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3614.566667,
      "charactersPerSecond": 2.453647028376368,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 2.453647028376368
      },
      "grade": "good"
    }
  },
  {
    "precision": "q4_k_m",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "red"
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1631.486708,
      "loadMs": 2799.4535,
      "offloadedLayers": 0,
      "precision": "q4_k_m",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q4_k_m",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 4430.940208,
      "charactersPerSecond": 1.8388136325533584,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 1.8388136325533584
      },
      "grade": "good"
    }
  },
  {
    "precision": "q4_k_m",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "blue"
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1380.7507079999998,
      "loadMs": 2799.4535,
      "offloadedLayers": 0,
      "precision": "q4_k_m",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q4_k_m",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 4180.204208,
      "charactersPerSecond": 2.896974795539993,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 2.896974795539993
      },
      "grade": "good"
    }
  },
  {
    "precision": "q4_k_m",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "localai2026"
      ]
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 2046.1294169999999,
      "loadMs": 2799.4535,
      "offloadedLayers": 0,
      "precision": "q4_k_m",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q4_k_m",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 4845.582917,
      "charactersPerSecond": 6.353459312979518,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 6.353459312979518
      },
      "grade": "moderate"
    }
  },
  {
    "precision": "q4_k_m",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "3",
        "three"
      ]
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 2541.326666,
      "loadMs": 2799.4535,
      "offloadedLayers": 0,
      "precision": "q4_k_m",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q4_k_m",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 5340.7801660000005,
      "charactersPerSecond": 0.3934952611086323,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 0.3934952611086323
      },
      "grade": "moderate"
    }
  },
  {
    "precision": "q4_k_m",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "15"
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 542.529083,
      "loadMs": 6790.742958,
      "offloadedLayers": 25,
      "precision": "q4_k_m",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q4_k_m",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 7333.272041,
      "charactersPerSecond": 3.6864383176302455,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 3.6864383176302455
      },
      "grade": "good"
    }
  },
  {
    "precision": "q4_k_m",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "red"
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 744.162208,
      "loadMs": 6790.742958,
      "offloadedLayers": 25,
      "precision": "q4_k_m",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q4_k_m",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 7534.905166,
      "charactersPerSecond": 4.031379137167901,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 4.031379137167901
      },
      "grade": "good"
    }
  },
  {
    "precision": "q4_k_m",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "blue"
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 617.913791,
      "loadMs": 6790.742958,
      "offloadedLayers": 25,
      "precision": "q4_k_m",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q4_k_m",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 7408.656749,
      "charactersPerSecond": 6.473394927027936,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 6.473394927027936
      },
      "grade": "good"
    }
  },
  {
    "precision": "q4_k_m",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "localai2026"
      ]
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 590.235708,
      "loadMs": 6790.742958,
      "offloadedLayers": 25,
      "precision": "q4_k_m",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q4_k_m",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 7380.978666,
      "charactersPerSecond": 22.025099165975906,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 22.025099165975906
      },
      "grade": "good"
    }
  },
  {
    "precision": "q4_k_m",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "3",
        "three"
      ]
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1140.5493330000002,
      "loadMs": 6790.742958,
      "offloadedLayers": 25,
      "precision": "q4_k_m",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q4_k_m",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 7931.292291,
      "charactersPerSecond": 0.8767704921361783,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 0.8767704921361783
      },
      "grade": "good"
    }
  },
  {
    "precision": "q5_k_m",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "15"
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 152.28225,
      "loadMs": 2893.1373750000002,
      "offloadedLayers": 0,
      "precision": "q5_k_m",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q5_k_m",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3045.4196250000005,
      "charactersPerSecond": 13.133507023963725,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 13.133507023963725
      },
      "grade": "good"
    }
  },
  {
    "precision": "q5_k_m",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "red"
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 662.718042,
      "loadMs": 2893.1373750000002,
      "offloadedLayers": 0,
      "precision": "q5_k_m",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q5_k_m",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3555.855417,
      "charactersPerSecond": 4.526812022419635,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 4.526812022419635
      },
      "grade": "good"
    }
  },
  {
    "precision": "q5_k_m",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "blue"
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 659.1525419999999,
      "loadMs": 2893.1373750000002,
      "offloadedLayers": 0,
      "precision": "q5_k_m",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q5_k_m",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3552.289917,
      "charactersPerSecond": 6.068398049203003,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 6.068398049203003
      },
      "grade": "good"
    }
  },
  {
    "precision": "q5_k_m",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "localai2026"
      ]
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1693.2307500000002,
      "loadMs": 2893.1373750000002,
      "offloadedLayers": 0,
      "precision": "q5_k_m",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q5_k_m",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 4586.368125000001,
      "charactersPerSecond": 7.677630470625459,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 7.677630470625459
      },
      "grade": "good"
    }
  },
  {
    "precision": "q5_k_m",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "3",
        "three"
      ]
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1686.535958,
      "loadMs": 2893.1373750000002,
      "offloadedLayers": 0,
      "precision": "q5_k_m",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q5_k_m",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 4579.673333000001,
      "charactersPerSecond": 0.5929313248594252,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 0.5929313248594252
      },
      "grade": "good"
    }
  },
  {
    "precision": "q5_k_m",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "15"
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 63.886083,
      "loadMs": 2473.8635000000004,
      "offloadedLayers": 25,
      "precision": "q5_k_m",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q5_k_m",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 2537.7495830000003,
      "charactersPerSecond": 31.30572271898404,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 31.30572271898404
      },
      "grade": "good"
    }
  },
  {
    "precision": "q5_k_m",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "red"
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 160.091625,
      "loadMs": 2473.8635000000004,
      "offloadedLayers": 25,
      "precision": "q5_k_m",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q5_k_m",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 2633.9551250000004,
      "charactersPerSecond": 18.739268840578013,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 18.739268840578013
      },
      "grade": "good"
    }
  },
  {
    "precision": "q5_k_m",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "blue"
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 133.380958,
      "loadMs": 2473.8635000000004,
      "offloadedLayers": 25,
      "precision": "q5_k_m",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q5_k_m",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 2607.2444580000006,
      "charactersPerSecond": 29.989288276067114,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 29.989288276067114
      },
      "grade": "good"
    }
  },
  {
    "precision": "q5_k_m",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "localai2026"
      ]
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 328.665416,
      "loadMs": 2473.8635000000004,
      "offloadedLayers": 25,
      "precision": "q5_k_m",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q5_k_m",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 2802.528916,
      "charactersPerSecond": 39.55390304892925,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 39.55390304892925
      },
      "grade": "good"
    }
  },
  {
    "precision": "q5_k_m",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "3",
        "three"
      ]
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 245.967584,
      "loadMs": 2473.8635000000004,
      "offloadedLayers": 25,
      "precision": "q5_k_m",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q5_k_m",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 2719.8310840000004,
      "charactersPerSecond": 4.0655763809917325,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 4.0655763809917325
      },
      "grade": "good"
    }
  },
  {
    "precision": "q6_k",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "15"
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 995.9282499999999,
      "loadMs": 2187.437667,
      "offloadedLayers": 25,
      "precision": "q6_k",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q6_k",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3183.365917,
      "charactersPerSecond": 2.008176793860401,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 2.008176793860401
      },
      "grade": "good"
    }
  },
  {
    "precision": "q6_k",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "red"
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 829.275584,
      "loadMs": 2187.437667,
      "offloadedLayers": 25,
      "precision": "q6_k",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q6_k",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3016.713251,
      "charactersPerSecond": 3.617615251047835,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 3.617615251047835
      },
      "grade": "good"
    }
  },
  {
    "precision": "q6_k",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "blue"
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 975.455916,
      "loadMs": 2187.437667,
      "offloadedLayers": 25,
      "precision": "q6_k",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q6_k",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3162.893583,
      "charactersPerSecond": 4.100646614972193,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 4.100646614972193
      },
      "grade": "good"
    }
  },
  {
    "precision": "q6_k",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "localai2026"
      ]
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 694.940333,
      "loadMs": 2187.437667,
      "offloadedLayers": 25,
      "precision": "q6_k",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q6_k",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 2882.378,
      "charactersPerSecond": 18.706641970081193,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 18.706641970081193
      },
      "grade": "good"
    }
  },
  {
    "precision": "q6_k",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "3",
        "three"
      ]
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1351.511792,
      "loadMs": 2187.437667,
      "offloadedLayers": 25,
      "precision": "q6_k",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q6_k",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3538.9494590000004,
      "charactersPerSecond": 0.7399121531305144,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 0.7399121531305144
      },
      "grade": "good"
    }
  },
  {
    "precision": "q8_0",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "15"
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1039.566417,
      "loadMs": 2177.748666,
      "offloadedLayers": 25,
      "precision": "q8_0",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q8_0",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3217.315083,
      "charactersPerSecond": 1.9238790011817013,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 1.9238790011817013
      },
      "grade": "good"
    }
  },
  {
    "precision": "q8_0",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "red"
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 819.099209,
      "loadMs": 2177.748666,
      "offloadedLayers": 25,
      "precision": "q8_0",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q8_0",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 2996.847875,
      "charactersPerSecond": 3.662560001324577,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 3.662560001324577
      },
      "grade": "good"
    }
  },
  {
    "precision": "q8_0",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "blue"
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 998.7096250000001,
      "loadMs": 2177.748666,
      "offloadedLayers": 25,
      "precision": "q8_0",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q8_0",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3176.458291,
      "charactersPerSecond": 4.005168168875913,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 4.005168168875913
      },
      "grade": "good"
    }
  },
  {
    "precision": "q8_0",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "localai2026"
      ]
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 758.8914589999999,
      "loadMs": 2177.748666,
      "offloadedLayers": 25,
      "precision": "q8_0",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q8_0",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 2936.640125,
      "charactersPerSecond": 17.130249452445085,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 17.130249452445085
      },
      "grade": "good"
    }
  },
  {
    "precision": "q8_0",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "3",
        "three"
      ]
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1449.183166,
      "loadMs": 2177.748666,
      "offloadedLayers": 25,
      "precision": "q8_0",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "q8_0",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3626.931832,
      "charactersPerSecond": 0.6900438974599571,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 0.6900438974599571
      },
      "grade": "good"
    }
  },
  {
    "precision": "q6_k",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "15"
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 156.909417,
      "loadMs": 3146.91175,
      "offloadedLayers": 0,
      "precision": "q6_k",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q6_k",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3303.821167,
      "charactersPerSecond": 12.746207577840915,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 12.746207577840915
      },
      "grade": "good"
    }
  },
  {
    "precision": "q6_k",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "red"
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 656.397708,
      "loadMs": 3146.91175,
      "offloadedLayers": 0,
      "precision": "q6_k",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q6_k",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3803.309458,
      "charactersPerSecond": 4.57039987104891,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 4.57039987104891
      },
      "grade": "good"
    }
  },
  {
    "precision": "q6_k",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "blue"
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 658.636416,
      "loadMs": 3146.91175,
      "offloadedLayers": 0,
      "precision": "q6_k",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q6_k",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3805.5481660000005,
      "charactersPerSecond": 6.073153416406298,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 6.073153416406298
      },
      "grade": "good"
    }
  },
  {
    "precision": "q6_k",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "localai2026"
      ]
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1656.087375,
      "loadMs": 3146.91175,
      "offloadedLayers": 0,
      "precision": "q6_k",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q6_k",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 4802.999125,
      "charactersPerSecond": 7.849827367955148,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 7.849827367955148
      },
      "grade": "good"
    }
  },
  {
    "precision": "q6_k",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "3",
        "three"
      ]
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1644.980041,
      "loadMs": 3146.91175,
      "offloadedLayers": 0,
      "precision": "q6_k",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q6_k",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 4791.891791,
      "charactersPerSecond": 0.6079101114151452,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 0.6079101114151452
      },
      "grade": "good"
    }
  },
  {
    "precision": "q8_0",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "15"
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 120.412959,
      "loadMs": 3520.119667,
      "offloadedLayers": 0,
      "precision": "q8_0",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q8_0",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3640.532626,
      "charactersPerSecond": 16.609507951714733,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 16.609507951714733
      },
      "grade": "good"
    }
  },
  {
    "precision": "q8_0",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "red"
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 498.074833,
      "loadMs": 3520.119667,
      "offloadedLayers": 0,
      "precision": "q8_0",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q8_0",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 4018.1945,
      "charactersPerSecond": 6.023191298244133,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 6.023191298244133
      },
      "grade": "good"
    }
  },
  {
    "precision": "q8_0",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "blue"
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 483.856792,
      "loadMs": 3520.119667,
      "offloadedLayers": 0,
      "precision": "q8_0",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q8_0",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 4003.976459,
      "charactersPerSecond": 8.266908858437601,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 8.266908858437601
      },
      "grade": "good"
    }
  },
  {
    "precision": "q8_0",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "localai2026"
      ]
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1334.946417,
      "loadMs": 3520.119667,
      "offloadedLayers": 0,
      "precision": "q8_0",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q8_0",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 4855.066084,
      "charactersPerSecond": 9.738218579000836,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 9.738218579000836
      },
      "grade": "good"
    }
  },
  {
    "precision": "q8_0",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "3",
        "three"
      ]
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1294.901709,
      "loadMs": 3520.119667,
      "offloadedLayers": 0,
      "precision": "q8_0",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "q8_0",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 4815.021376,
      "charactersPerSecond": 0.7722593869864142,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 0.7722593869864142
      },
      "grade": "good"
    }
  },
  {
    "precision": "nvfp4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "15"
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 247.41470800000002,
      "loadMs": 3250.873542,
      "offloadedLayers": 0,
      "precision": "nvfp4",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "nvfp4",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3498.2882499999996,
      "charactersPerSecond": 8.08359380154554,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 8.08359380154554
      },
      "grade": "good"
    }
  },
  {
    "precision": "nvfp4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "red"
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 760.594625,
      "loadMs": 3250.873542,
      "offloadedLayers": 0,
      "precision": "nvfp4",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "nvfp4",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 4011.468167,
      "charactersPerSecond": 3.9442824093057456,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 3.9442824093057456
      },
      "grade": "good"
    }
  },
  {
    "precision": "nvfp4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "blue"
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 787.9119999999999,
      "loadMs": 3250.873542,
      "offloadedLayers": 0,
      "precision": "nvfp4",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "nvfp4",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 4038.7855419999996,
      "charactersPerSecond": 5.076709074109799,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 5.076709074109799
      },
      "grade": "good"
    }
  },
  {
    "precision": "nvfp4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "localai2026"
      ]
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1420.912959,
      "loadMs": 3250.873542,
      "offloadedLayers": 0,
      "precision": "nvfp4",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "nvfp4",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 4671.786501,
      "charactersPerSecond": 9.14904739073465,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 9.14904739073465
      },
      "grade": "good"
    }
  },
  {
    "precision": "nvfp4",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "3",
        "three"
      ]
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1400.7547499999998,
      "loadMs": 3250.873542,
      "offloadedLayers": 0,
      "precision": "nvfp4",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "nvfp4",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 4651.628291999999,
      "charactersPerSecond": 0.7139008452407534,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 0.7139008452407534
      },
      "grade": "good"
    }
  },
  {
    "precision": "nvfp4",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "15"
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 401.596,
      "loadMs": 2784.6329170000004,
      "offloadedLayers": 25,
      "precision": "nvfp4",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "nvfp4",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3186.2289170000004,
      "charactersPerSecond": 4.980129284156217,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 4.980129284156217
      },
      "grade": "good"
    }
  },
  {
    "precision": "nvfp4",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "red"
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 800.3299579999999,
      "loadMs": 2784.6329170000004,
      "offloadedLayers": 25,
      "precision": "nvfp4",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "nvfp4",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3584.962875,
      "charactersPerSecond": 3.748453959535525,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 3.748453959535525
      },
      "grade": "good"
    }
  },
  {
    "precision": "nvfp4",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "blue"
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 805.1493330000001,
      "loadMs": 2784.6329170000004,
      "offloadedLayers": 25,
      "precision": "nvfp4",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "nvfp4",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3589.78225,
      "charactersPerSecond": 4.968022497262629,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 4.968022497262629
      },
      "grade": "good"
    }
  },
  {
    "precision": "nvfp4",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "localai2026"
      ]
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1149.2737909999998,
      "loadMs": 2784.6329170000004,
      "offloadedLayers": 25,
      "precision": "nvfp4",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "nvfp4",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3933.9067080000004,
      "charactersPerSecond": 11.31149087519738,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 11.31149087519738
      },
      "grade": "good"
    }
  },
  {
    "precision": "nvfp4",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "3",
        "three"
      ]
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 954.409958,
      "loadMs": 2784.6329170000004,
      "offloadedLayers": 25,
      "precision": "nvfp4",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "nvfp4",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3739.042875,
      "charactersPerSecond": 1.0477677769577505,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 1.0477677769577505
      },
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "15"
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 39035.376291,
      "loadMs": 3971.8835,
      "offloadedLayers": 0,
      "precision": "bf16",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "bf16",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 43007.259791000004,
      "charactersPerSecond": 0.05123557629086107,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 0.05123557629086107
      },
      "grade": "moderate"
    }
  },
  {
    "precision": "bf16",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "red"
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 39428.568833,
      "loadMs": 3971.8835,
      "offloadedLayers": 0,
      "precision": "bf16",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "bf16",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 43400.452332999994,
      "charactersPerSecond": 0.07608696153052176,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 0.07608696153052176
      },
      "grade": "moderate"
    }
  },
  {
    "precision": "bf16",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "blue"
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 43616.044167,
      "loadMs": 3971.8835,
      "offloadedLayers": 0,
      "precision": "bf16",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "bf16",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 47587.927666999996,
      "charactersPerSecond": 0.09170937154879372,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 0.09170937154879372
      },
      "grade": "moderate"
    }
  },
  {
    "precision": "bf16",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "15"
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 186.08333399999998,
      "loadMs": 3232.747792,
      "offloadedLayers": 25,
      "precision": "bf16",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "bf16",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3418.831126,
      "charactersPerSecond": 10.747872778332745,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 10.747872778332745
      },
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "red"
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 169.448625,
      "loadMs": 3232.747792,
      "offloadedLayers": 25,
      "precision": "bf16",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "bf16",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3402.196417,
      "charactersPerSecond": 17.704481225504193,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 17.704481225504193
      },
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "blue"
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 139.03975,
      "loadMs": 3232.747792,
      "offloadedLayers": 25,
      "precision": "bf16",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "bf16",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3371.787542,
      "charactersPerSecond": 28.768751382248603,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 28.768751382248603
      },
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "localai2026"
      ]
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 349.22225000000003,
      "loadMs": 3232.747792,
      "offloadedLayers": 25,
      "precision": "bf16",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "bf16",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3581.970042,
      "charactersPerSecond": 37.225577694433845,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 37.225577694433845
      },
      "grade": "good"
    }
  },
  {
    "precision": "bf16",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "3",
        "three"
      ]
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 230.592916,
      "loadMs": 3232.747792,
      "offloadedLayers": 25,
      "precision": "bf16",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "bf16",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3463.340708,
      "charactersPerSecond": 4.336646664375414,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 4.336646664375414
      },
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "15"
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 306.45054100000004,
      "loadMs": 2542.124958,
      "offloadedLayers": 0,
      "precision": "fp16",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp16",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 2848.575499,
      "charactersPerSecond": 6.5263386172322,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 6.5263386172322
      },
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "red"
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 910.6492910000001,
      "loadMs": 2542.124958,
      "offloadedLayers": 0,
      "precision": "fp16",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp16",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3452.774249,
      "charactersPerSecond": 3.294352754292102,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 3.294352754292102
      },
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "blue"
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 953.506333,
      "loadMs": 2542.124958,
      "offloadedLayers": 0,
      "precision": "fp16",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp16",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 3495.6312909999997,
      "charactersPerSecond": 4.1950429289912226,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 4.1950429289912226
      },
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "localai2026"
      ]
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1808.945417,
      "loadMs": 2542.124958,
      "offloadedLayers": 0,
      "precision": "fp16",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp16",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 4351.070374999999,
      "charactersPerSecond": 7.186507607045172,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 7.186507607045172
      },
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "3",
        "three"
      ]
    },
    "runtime": {
      "acceleration": "cpu",
      "cpuThreads": 4,
      "device": "cpu",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 1727.031792,
      "loadMs": 2542.124958,
      "offloadedLayers": 0,
      "precision": "fp16",
      "provider": "llama.cpp CPU",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp16",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 4269.15675,
      "charactersPerSecond": 0.5790281363853433,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 0.5790281363853433
      },
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "15"
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 116.532125,
      "loadMs": 2055.0385420000002,
      "offloadedLayers": 25,
      "precision": "fp16",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "fp16",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 2171.5706670000004,
      "charactersPerSecond": 17.16264935527435,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 17.16264935527435
      },
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "red"
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 135.75216700000001,
      "loadMs": 2055.0385420000002,
      "offloadedLayers": 25,
      "precision": "fp16",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "fp16",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 2190.7907090000003,
      "charactersPerSecond": 22.09909474225925,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 22.09909474225925
      },
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": "blue"
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 137.64950000000002,
      "loadMs": 2055.0385420000002,
      "offloadedLayers": 25,
      "precision": "fp16",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "fp16",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 2192.688042,
      "charactersPerSecond": 29.059313691658883,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 29.059313691658883
      },
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "localai2026"
      ]
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 323.403167,
      "loadMs": 2055.0385420000002,
      "offloadedLayers": 25,
      "precision": "fp16",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "fp16",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 2378.441709,
      "charactersPerSecond": 40.19750369358628,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 40.19750369358628
      },
      "grade": "good"
    }
  },
  {
    "precision": "fp16",
    "device": "metal",
    "status": "pass",
    "quality": {
      "metric": "frozenAnswerMatch",
      "score": 1,
      "passed": true,
      "expected": [
        "3",
        "three"
      ]
    },
    "runtime": {
      "acceleration": "partial",
      "cpuThreads": 4,
      "device": "metal",
      "engine": "llama.cpp",
      "fallback": false,
      "fallbackReason": null,
      "inferenceMs": 220.765,
      "loadMs": 2055.0385420000002,
      "offloadedLayers": 25,
      "precision": "fp16",
      "provider": "llama.cpp Metal",
      "requestedDevice": "metal",
      "requestedPrecision": "fp16",
      "selectionReason": "Pinned package with verified layer placement",
      "version": "b12359",
      "artifactRevision": "1add221a5ea6c047fcf1eaaf2d247939d7cf8849",
      "totalMs": 2275.803542,
      "charactersPerSecond": 4.529703530903903,
      "performanceMetric": {
        "name": "charactersPerSecond",
        "value": 4.529703530903903
      },
      "grade": "good"
    }
  }
]
Release validation still required
  • Full production lint fails on seven pre-existing errors in vendored generated DuckDB worker; changed catalog sources pass.
  • Strict native Clippy retains pre-existing findings in unrelated Gradium/GPA/NeuDecide/NeuTTS/Voxtral adapters; LFM3 findings are repaired without behavior changes.

Something not quite right?

Tell us what happened with Liquid LFM3 Desktop. Loading, results, or the page itself—we’re here for all of it.

Attachments Optional

Up to 3 files · 50 MB each

Browser, OS, CPU/GPU, memory, screen, storage, and network details are included automatically. Model inputs, results, and browsing history are not collected.