macOS Apple SiliconCPU / Q4TestedNot establishedRAM estimate
Liquid LFM3 Desktop
Local text chat with Liquid LFM3 1.7B.
Desktop app
Run Liquid LFM3 Desktop on your computer
Download the desktop app to load this model, provide input and view results locally.
Public desktop downloads are coming soon.Device requirements
macOS Apple SiliconMETAL / Q4TestedNot establishedRAM estimate
macOS Apple SiliconCPU / Q4_K_MTestedNot establishedRAM estimate
macOS Apple SiliconMETAL / Q4_K_MTestedNot establishedRAM estimate
macOS Apple SiliconCPU / Q5_K_MTestedNot establishedRAM estimate
macOS Apple SiliconMETAL / Q5_K_MTestedNot establishedRAM estimate
macOS Apple SiliconCPU / Q6_KTestedNot establishedRAM estimate
macOS Apple SiliconMETAL / Q6_KTestedNot establishedRAM estimate
macOS Apple SiliconCPU / Q8_0TestedNot establishedRAM estimate
macOS Apple SiliconMETAL / Q8_0TestedNot establishedRAM estimate
macOS Apple SiliconMETAL / FP16TestedNot establishedRAM estimate
macOS Apple SiliconMETAL / BF16TestedNot establishedRAM estimate
If your device falls short
MemoryShow advisory requirements.
GPUUse verified CPU recovery when permitted.
PlatformNo verified configuration.
Benchmark evidence
Saved desktop measurements apply to the recorded hardware and runtime. Fallback cases report the device that actually ran; they do not prove GPU support on the requested device.
LFM3 1.7B Desktop
54 configurations testedPassed54
Failed0
TestedNov 16, 2026
Runtimellama.cpp b12359
Quality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
Q4 | metalAccelerated | Passed | Not measured | macos · aarch64 | 671 ms / 524 ms |
Q4 | metalAccelerated | Passed | Not measured | macos · aarch64 | 472 ms / 524 ms |
Q4 | metalAccelerated | Passed | Not measured | macos · aarch64 | 397 ms / 524 ms |
Q4_K_M | metalAccelerated | Passed | Not measured | macos · aarch64 | 1495 ms / 6414 ms |
Q4_K_M | metalAccelerated | Passed | Not measured | macos · aarch64 | 555 ms / 6414 ms |
Q4_K_M | metalAccelerated | Passed | Not measured | macos · aarch64 | 464 ms / 6414 ms |
Q5_K_M | metalAccelerated | Passed | Not measured | macos · aarch64 | 689 ms / 2684 ms |
Q5_K_M | metalAccelerated | Passed | Not measured | macos · aarch64 | 817 ms / 2684 ms |
Q5_K_M | metalAccelerated | Passed | Not measured | macos · aarch64 | 575 ms / 2684 ms |
Q6_K | metalAccelerated | Passed | Not measured | macos · aarch64 | 909 ms / 1944 ms |
Q6_K | metalAccelerated | Passed | Not measured | macos · aarch64 | 690 ms / 1944 ms |
Q6_K | metalAccelerated | Passed | Not measured | macos · aarch64 | 718 ms / 1944 ms |
Q8_0 | metalAccelerated | Passed | Not measured | macos · aarch64 | 1119 ms / 1820 ms |
Q8_0 | metalAccelerated | Passed | Not measured | macos · aarch64 | 969 ms / 1820 ms |
Q8_0 | metalAccelerated | Passed | Not measured | macos · aarch64 | 575 ms / 1820 ms |
FP16 | metalAccelerated | Passed | Not measured | macos · aarch64 | 1685 ms / 1959 ms |
FP16 | metalAccelerated | Passed | Not measured | macos · aarch64 | 1063 ms / 1959 ms |
FP16 | metalAccelerated | Passed | Not measured | macos · aarch64 | 912 ms / 1959 ms |
BF16 | metalAccelerated | Passed | Not measured | macos · aarch64 | 1701 ms / 1867 ms |
BF16 | metalAccelerated | Passed | Not measured | macos · aarch64 | 1069 ms / 1867 ms |
BF16 | metalAccelerated | Passed | Not measured | macos · aarch64 | 929 ms / 1867 ms |
Q4_K_M | metalAccelerated | Passed | Not measured | macos · aarch64 | 964 ms / 1837 ms |
Q4_K_M | metalAccelerated | Passed | Not measured | macos · aarch64 | 581 ms / 1837 ms |
Q4_K_M | metalAccelerated | Passed | Not measured | macos · aarch64 | 482 ms / 1837 ms |
Q4_K_M | metalAccelerated | Passed | Not measured | macos · aarch64 | 648 ms / 2541 ms |
Q4_K_M | metalAccelerated | Passed | Not measured | macos · aarch64 | 608 ms / 2541 ms |
Q4_K_M | metalAccelerated | Passed | Not measured | macos · aarch64 | 604 ms / 2541 ms |
Q4_K_M | metalAccelerated | Passed | Not measured | macos · aarch64 | 606 ms / 2541 ms |
Q4_K_M | metalAccelerated | Passed | Not measured | macos · aarch64 | 606 ms / 2541 ms |
Q4_K_M | metalAccelerated | Passed | Not measured | macos · aarch64 | 608 ms / 2541 ms |
Q4 | cpu | Passed | Not measured | macos · aarch64 | 1345 ms / 2721 ms |
Q4 | cpu | Passed | Not measured | macos · aarch64 | 1373 ms / 2721 ms |
Q4 | cpu | Passed | Not measured | macos · aarch64 | 825 ms / 2721 ms |
Q4_K_M | cpu | Passed | Not measured | macos · aarch64 | 1369 ms / 2603 ms |
Q4_K_M | cpu | Passed | Not measured | macos · aarch64 | 933 ms / 2603 ms |
Q4_K_M | cpu | Passed | Not measured | macos · aarch64 | 789 ms / 2603 ms |
Q5_K_M | cpu | Passed | Not measured | macos · aarch64 | 1358 ms / 2698 ms |
Q5_K_M | cpu | Passed | Not measured | macos · aarch64 | 1148 ms / 2698 ms |
Q5_K_M | cpu | Passed | Not measured | macos · aarch64 | 1126 ms / 2698 ms |
Q6_K | cpu | Passed | Not measured | macos · aarch64 | 1821 ms / 3229 ms |
Q6_K | cpu | Passed | Not measured | macos · aarch64 | 1689 ms / 3229 ms |
Q6_K | cpu | Passed | Not measured | macos · aarch64 | 1353 ms / 3229 ms |
Q8_0 | cpu | Passed | Not measured | macos · aarch64 | 1830 ms / 3252 ms |
Q8_0 | cpu | Passed | Not measured | macos · aarch64 | 1511 ms / 3252 ms |
Q8_0 | cpu | Passed | Not measured | macos · aarch64 | 999 ms / 3252 ms |
Q4_K_M | cpu | Passed | Not measured | macos · aarch64 | 971 ms / 6902 ms |
Q4_K_M | cpu | Passed | Not measured | macos · aarch64 | 946 ms / 6902 ms |
Q4_K_M | cpu | Passed | Not measured | macos · aarch64 | 952 ms / 6902 ms |
Q4_K_M | cpu | Passed | Not measured | macos · aarch64 | 1412 ms / 8399 ms |
Q4_K_M | cpu | Passed | Not measured | macos · aarch64 | 1427 ms / 8399 ms |
Q4_K_M | cpu | Passed | Not measured | macos · aarch64 | 1427 ms / 8399 ms |
Q4_K_M | cpu | Passed | Not measured | macos · aarch64 | 1410 ms / 8399 ms |
Q4_K_M | cpu | Passed | Not measured | macos · aarch64 | 1409 ms / 8399 ms |
Q4_K_M | cpu | Passed | Not measured | macos · aarch64 | 1433 ms / 8399 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-16T02:46:43.608Z
- Environment
- { "scope": "DesktopAI TypeScript client using native JSON-lines transport with verified cached sources.", "hardware": "Apple M3 Max", "os": "macOS", "arch": "arm64", "runtime": { "name": "llama.cpp", "version": "b12359" }, "speedGradePolicy": "Short requests completing in 2 seconds or less are interactive." }
- Runtime
- llama.cpp b12359
- Artifact revision
- cd0c74c575b5c416f3011b5b4c298077f26ca8c2
- Fixture
{ "file": "model-samples/lfm3-1.7b-native/desktop-samples/lfm3-short-chat-fixtures.json", "url": "https://assets.localai.tools/samples/v1/files/9d508a32edbc0d766ac2c288b1bfbd94bce1e6905b117c8f4795e4971e2728ce/lfm3-short-chat-fixtures.json", "size": 528, "sha1": "95b313bbbad0375e460f02a514fe9efc41756091", "sha256": "9d508a32edbc0d766ac2c288b1bfbd94bce1e6905b117c8f4795e4971e2728ce" }- Rerun when
- artifactRevision; runtimeVersion; fixtureChecksum; qualityGate; nativeBackend
Per-case quality & failure details
[
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "cpu",
"cpuThreads": 4,
"device": "cpu",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"loadMs": 2721.013084,
"offloadedLayers": 0,
"precision": "q4",
"provider": "llama.cpp CPU",
"requestedDevice": "cpu",
"requestedPrecision": "q4",
"selectionReason": "Exact QAD Q4_0 package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"inferenceMs": 1345.196333,
"totalMs": 4066.209417,
"performanceScore": 0.7433859098989976,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "cpu",
"cpuThreads": 4,
"device": "cpu",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"loadMs": 2721.013084,
"offloadedLayers": 0,
"precision": "q4",
"provider": "llama.cpp CPU",
"requestedDevice": "cpu",
"requestedPrecision": "q4",
"selectionReason": "Exact QAD Q4_0 package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"inferenceMs": 1372.521125,
"totalMs": 4093.5342090000004,
"performanceScore": 0.7285862357856241,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "cpu",
"cpuThreads": 4,
"device": "cpu",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"loadMs": 2721.013084,
"offloadedLayers": 0,
"precision": "q4",
"provider": "llama.cpp CPU",
"requestedDevice": "cpu",
"requestedPrecision": "q4",
"selectionReason": "Exact QAD Q4_0 package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"inferenceMs": 824.861375,
"totalMs": 3545.874459,
"performanceScore": 1.2123249194447976,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q4",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"loadMs": 523.755542,
"offloadedLayers": 25,
"precision": "q4",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "q4",
"selectionReason": "Exact QAD Q4_0 package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"inferenceMs": 670.679625,
"totalMs": 1194.435167,
"performanceScore": 1.491024868990168,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q4",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"loadMs": 523.755542,
"offloadedLayers": 25,
"precision": "q4",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "q4",
"selectionReason": "Exact QAD Q4_0 package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"inferenceMs": 471.6545,
"totalMs": 995.410042,
"performanceScore": 2.120196033325241,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q4",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"loadMs": 523.755542,
"offloadedLayers": 25,
"precision": "q4",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "q4",
"selectionReason": "Exact QAD Q4_0 package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"inferenceMs": 397.141459,
"totalMs": 920.897001,
"performanceScore": 2.517994476119402,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q4_k_m",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "cpu",
"cpuThreads": 4,
"device": "cpu",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 1369.33025,
"loadMs": 2603.140167,
"offloadedLayers": 0,
"precision": "q4_k_m",
"provider": "llama.cpp CPU",
"requestedDevice": "cpu",
"requestedPrecision": "q4_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 3972.470417,
"performanceScore": 0.7302840202354399,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q4_k_m",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "cpu",
"cpuThreads": 4,
"device": "cpu",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 933.1112919999999,
"loadMs": 2603.140167,
"offloadedLayers": 0,
"precision": "q4_k_m",
"provider": "llama.cpp CPU",
"requestedDevice": "cpu",
"requestedPrecision": "q4_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 3536.251459,
"performanceScore": 1.0716835264704954,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q4_k_m",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "cpu",
"cpuThreads": 4,
"device": "cpu",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 788.830291,
"loadMs": 2603.140167,
"offloadedLayers": 0,
"precision": "q4_k_m",
"provider": "llama.cpp CPU",
"requestedDevice": "cpu",
"requestedPrecision": "q4_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 3391.970458,
"performanceScore": 1.267699797293915,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q4_k_m",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 1495.251084,
"loadMs": 6414.436334,
"offloadedLayers": 25,
"precision": "q4_k_m",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "q4_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 7909.6874179999995,
"performanceScore": 0.668783999356726,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q4_k_m",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 555.1485,
"loadMs": 6414.436334,
"offloadedLayers": 25,
"precision": "q4_k_m",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "q4_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 6969.584834,
"performanceScore": 1.8013198270372701,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q4_k_m",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 463.657166,
"loadMs": 6414.436334,
"offloadedLayers": 25,
"precision": "q4_k_m",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "q4_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 6878.0935,
"performanceScore": 2.1567659756605595,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q5_k_m",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "cpu",
"cpuThreads": 4,
"device": "cpu",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 1357.953458,
"loadMs": 2697.8429579999997,
"offloadedLayers": 0,
"precision": "q5_k_m",
"provider": "llama.cpp CPU",
"requestedDevice": "cpu",
"requestedPrecision": "q5_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 4055.7964159999997,
"performanceScore": 0.7364022633535648,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q5_k_m",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "cpu",
"cpuThreads": 4,
"device": "cpu",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 1148.213334,
"loadMs": 2697.8429579999997,
"offloadedLayers": 0,
"precision": "q5_k_m",
"provider": "llama.cpp CPU",
"requestedDevice": "cpu",
"requestedPrecision": "q5_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 3846.0562919999998,
"performanceScore": 0.8709182957459018,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q5_k_m",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "cpu",
"cpuThreads": 4,
"device": "cpu",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 1126.4690420000002,
"loadMs": 2697.8429579999997,
"offloadedLayers": 0,
"precision": "q5_k_m",
"provider": "llama.cpp CPU",
"requestedDevice": "cpu",
"requestedPrecision": "q5_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 3824.312,
"performanceScore": 0.8877296780606952,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q5_k_m",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 688.950667,
"loadMs": 2683.9547079999998,
"offloadedLayers": 25,
"precision": "q5_k_m",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "q5_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 3372.905375,
"performanceScore": 1.4514827372973573,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q5_k_m",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 816.791167,
"loadMs": 2683.9547079999998,
"offloadedLayers": 25,
"precision": "q5_k_m",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "q5_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 3500.7458749999996,
"performanceScore": 1.2243031516524714,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q5_k_m",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 575.203542,
"loadMs": 2683.9547079999998,
"offloadedLayers": 25,
"precision": "q5_k_m",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "q5_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 3259.1582499999995,
"performanceScore": 1.7385150246519172,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q6_k",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "cpu",
"cpuThreads": 4,
"device": "cpu",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 1820.9170829999998,
"loadMs": 3228.923583,
"offloadedLayers": 0,
"precision": "q6_k",
"provider": "llama.cpp CPU",
"requestedDevice": "cpu",
"requestedPrecision": "q6_k",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 5049.840666,
"performanceScore": 0.5491738252861457,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q6_k",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "cpu",
"cpuThreads": 4,
"device": "cpu",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 1689.215042,
"loadMs": 3228.923583,
"offloadedLayers": 0,
"precision": "q6_k",
"provider": "llama.cpp CPU",
"requestedDevice": "cpu",
"requestedPrecision": "q6_k",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 4918.138625,
"performanceScore": 0.5919909396591793,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q6_k",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "cpu",
"cpuThreads": 4,
"device": "cpu",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 1353.0702909999998,
"loadMs": 3228.923583,
"offloadedLayers": 0,
"precision": "q6_k",
"provider": "llama.cpp CPU",
"requestedDevice": "cpu",
"requestedPrecision": "q6_k",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 4581.993874,
"performanceScore": 0.7390599044643426,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q6_k",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 909.254,
"loadMs": 1943.6027500000002,
"offloadedLayers": 25,
"precision": "q6_k",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "q6_k",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 2852.8567500000004,
"performanceScore": 1.099802695396446,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q6_k",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 689.924083,
"loadMs": 1943.6027500000002,
"offloadedLayers": 25,
"precision": "q6_k",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "q6_k",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 2633.5268330000004,
"performanceScore": 1.4494348358614988,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q6_k",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 718.15,
"loadMs": 1943.6027500000002,
"offloadedLayers": 25,
"precision": "q6_k",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "q6_k",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 2661.75275,
"performanceScore": 1.392466754856228,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q8_0",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "cpu",
"cpuThreads": 4,
"device": "cpu",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 1829.904583,
"loadMs": 3251.848125,
"offloadedLayers": 0,
"precision": "q8_0",
"provider": "llama.cpp CPU",
"requestedDevice": "cpu",
"requestedPrecision": "q8_0",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 5081.752708,
"performanceScore": 0.5464765809595221,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q8_0",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "cpu",
"cpuThreads": 4,
"device": "cpu",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 1510.524417,
"loadMs": 3251.848125,
"offloadedLayers": 0,
"precision": "q8_0",
"provider": "llama.cpp CPU",
"requestedDevice": "cpu",
"requestedPrecision": "q8_0",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 4762.372542,
"performanceScore": 0.6620217381100434,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q8_0",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "cpu",
"cpuThreads": 4,
"device": "cpu",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 998.826209,
"loadMs": 3251.848125,
"offloadedLayers": 0,
"precision": "q8_0",
"provider": "llama.cpp CPU",
"requestedDevice": "cpu",
"requestedPrecision": "q8_0",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 4250.674334,
"performanceScore": 1.0011751704044443,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q8_0",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 1119.1196670000002,
"loadMs": 1819.6787080000001,
"offloadedLayers": 25,
"precision": "q8_0",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "q8_0",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 2938.7983750000003,
"performanceScore": 0.8935594909887326,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q8_0",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 969.155208,
"loadMs": 1819.6787080000001,
"offloadedLayers": 25,
"precision": "q8_0",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "q8_0",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 2788.833916,
"performanceScore": 1.031826472937862,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q8_0",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 575.034833,
"loadMs": 1819.6787080000001,
"offloadedLayers": 25,
"precision": "q8_0",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "q8_0",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 2394.713541,
"performanceScore": 1.739025086155085,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "fp16",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 1684.881375,
"loadMs": 1958.9232920000002,
"offloadedLayers": 25,
"precision": "fp16",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "fp16",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 3643.8046670000003,
"performanceScore": 0.5935135937982577,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "fp16",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 1063.4110839999998,
"loadMs": 1958.9232920000002,
"offloadedLayers": 25,
"precision": "fp16",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "fp16",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 3022.334376,
"performanceScore": 0.9403701118466057,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "fp16",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 911.92525,
"loadMs": 1958.9232920000002,
"offloadedLayers": 25,
"precision": "fp16",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "fp16",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 2870.848542,
"performanceScore": 1.0965811068396232,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "bf16",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 1701.3125,
"loadMs": 1866.731708,
"offloadedLayers": 25,
"precision": "bf16",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "bf16",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 3568.0442080000003,
"performanceScore": 0.5877814922302634,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "bf16",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 1068.7720000000002,
"loadMs": 1866.731708,
"offloadedLayers": 25,
"precision": "bf16",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "bf16",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 2935.503708,
"performanceScore": 0.9356532543891493,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "bf16",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 928.919625,
"loadMs": 1866.731708,
"offloadedLayers": 25,
"precision": "bf16",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "bf16",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2",
"totalMs": 2795.651333,
"performanceScore": 1.0765194028493046,
"performanceMetric": "short requests per second",
"grade": "interactive"
}
},
{
"precision": "q4_k_m",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "cpu",
"cpuThreads": 4,
"device": "cpu",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 970.993916,
"loadMs": 6901.847167,
"offloadedLayers": 0,
"precision": "q4_k_m",
"provider": "llama.cpp CPU",
"requestedDevice": "cpu",
"requestedPrecision": "q4_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"totalMs": 7872.841083,
"performanceScore": 1.0298725702829223,
"performanceMetric": "short requests per second",
"grade": "interactive",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2"
}
},
{
"precision": "q4_k_m",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "cpu",
"cpuThreads": 4,
"device": "cpu",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 946.308833,
"loadMs": 6901.847167,
"offloadedLayers": 0,
"precision": "q4_k_m",
"provider": "llama.cpp CPU",
"requestedDevice": "cpu",
"requestedPrecision": "q4_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"totalMs": 7848.156,
"performanceScore": 1.0567374678621435,
"performanceMetric": "short requests per second",
"grade": "interactive",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2"
}
},
{
"precision": "q4_k_m",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "cpu",
"cpuThreads": 4,
"device": "cpu",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 951.663041,
"loadMs": 6901.847167,
"offloadedLayers": 0,
"precision": "q4_k_m",
"provider": "llama.cpp CPU",
"requestedDevice": "cpu",
"requestedPrecision": "q4_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"totalMs": 7853.510208,
"performanceScore": 1.0507920943837514,
"performanceMetric": "short requests per second",
"grade": "interactive",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2"
}
},
{
"precision": "q4_k_m",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 963.8613750000001,
"loadMs": 1836.7214999999999,
"offloadedLayers": 25,
"precision": "q4_k_m",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "q4_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"totalMs": 2800.582875,
"performanceScore": 1.0374935918559864,
"performanceMetric": "short requests per second",
"grade": "interactive",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2"
}
},
{
"precision": "q4_k_m",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 581.002875,
"loadMs": 1836.7214999999999,
"offloadedLayers": 25,
"precision": "q4_k_m",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "q4_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"totalMs": 2417.724375,
"performanceScore": 1.7211618789321825,
"performanceMetric": "short requests per second",
"grade": "interactive",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2"
}
},
{
"precision": "q4_k_m",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"exactMatch": true,
"streamParity": true
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 481.570291,
"loadMs": 1836.7214999999999,
"offloadedLayers": 25,
"precision": "q4_k_m",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "q4_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"totalMs": 2318.2917909999996,
"performanceScore": 2.0765400579912434,
"performanceMetric": "short requests per second",
"grade": "interactive",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2"
}
},
{
"precision": "q4_k_m",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"streamParity": true,
"expected": {
"city": "Paris",
"temperature_celsius": 21,
"condition": "overcast"
},
"metric": "allWeatherFacts",
"score": 1
},
"runtime": {
"acceleration": "cpu",
"cpuThreads": 4,
"device": "cpu",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 1412.2444999999998,
"loadMs": 8398.992084,
"offloadedLayers": 0,
"precision": "q4_k_m",
"provider": "llama.cpp CPU",
"requestedDevice": "cpu",
"requestedPrecision": "q4_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"totalMs": 9811.236583999998,
"performanceMetric": {
"name": "allWeatherFacts",
"value": 1
},
"grade": "interactive",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2"
}
},
{
"precision": "q4_k_m",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"streamParity": true,
"expected": {
"city": "Paris",
"temperature_celsius": 21,
"condition": "overcast"
},
"metric": "allWeatherFacts",
"score": 1
},
"runtime": {
"acceleration": "cpu",
"cpuThreads": 4,
"device": "cpu",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 1426.763291,
"loadMs": 8398.992084,
"offloadedLayers": 0,
"precision": "q4_k_m",
"provider": "llama.cpp CPU",
"requestedDevice": "cpu",
"requestedPrecision": "q4_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"totalMs": 9825.755374999999,
"performanceMetric": {
"name": "allWeatherFacts",
"value": 1
},
"grade": "interactive",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2"
}
},
{
"precision": "q4_k_m",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"streamParity": true,
"expected": {
"city": "Paris",
"temperature_celsius": 21,
"condition": "overcast"
},
"metric": "allWeatherFacts",
"score": 1
},
"runtime": {
"acceleration": "cpu",
"cpuThreads": 4,
"device": "cpu",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 1427.291041,
"loadMs": 8398.992084,
"offloadedLayers": 0,
"precision": "q4_k_m",
"provider": "llama.cpp CPU",
"requestedDevice": "cpu",
"requestedPrecision": "q4_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"totalMs": 9826.283125,
"performanceMetric": {
"name": "allWeatherFacts",
"value": 1
},
"grade": "interactive",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2"
}
},
{
"precision": "q4_k_m",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"streamParity": true,
"expected": {
"city": "Paris",
"temperature_celsius": 21,
"condition": "overcast"
},
"metric": "allWeatherFacts",
"score": 1
},
"runtime": {
"acceleration": "cpu",
"cpuThreads": 4,
"device": "cpu",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 1409.9322089999998,
"loadMs": 8398.992084,
"offloadedLayers": 0,
"precision": "q4_k_m",
"provider": "llama.cpp CPU",
"requestedDevice": "cpu",
"requestedPrecision": "q4_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"totalMs": 9808.924293,
"performanceMetric": {
"name": "allWeatherFacts",
"value": 1
},
"grade": "interactive",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2"
}
},
{
"precision": "q4_k_m",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"streamParity": true,
"expected": {
"city": "Paris",
"temperature_celsius": 21,
"condition": "overcast"
},
"metric": "allWeatherFacts",
"score": 1
},
"runtime": {
"acceleration": "cpu",
"cpuThreads": 4,
"device": "cpu",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 1408.966167,
"loadMs": 8398.992084,
"offloadedLayers": 0,
"precision": "q4_k_m",
"provider": "llama.cpp CPU",
"requestedDevice": "cpu",
"requestedPrecision": "q4_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"totalMs": 9807.958251,
"performanceMetric": {
"name": "allWeatherFacts",
"value": 1
},
"grade": "interactive",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2"
}
},
{
"precision": "q4_k_m",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"streamParity": true,
"expected": {
"city": "Paris",
"temperature_celsius": 21,
"condition": "overcast"
},
"metric": "allWeatherFacts",
"score": 1
},
"runtime": {
"acceleration": "cpu",
"cpuThreads": 4,
"device": "cpu",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 1433.300416,
"loadMs": 8398.992084,
"offloadedLayers": 0,
"precision": "q4_k_m",
"provider": "llama.cpp CPU",
"requestedDevice": "cpu",
"requestedPrecision": "q4_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"totalMs": 9832.2925,
"performanceMetric": {
"name": "allWeatherFacts",
"value": 1
},
"grade": "interactive",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2"
}
},
{
"precision": "q4_k_m",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"streamParity": true,
"expected": {
"city": "Paris",
"temperature_celsius": 21,
"condition": "overcast"
},
"metric": "allWeatherFacts",
"score": 1
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 647.887834,
"loadMs": 2540.7353329999996,
"offloadedLayers": 25,
"precision": "q4_k_m",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "q4_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"totalMs": 3188.6231669999997,
"performanceMetric": {
"name": "allWeatherFacts",
"value": 1
},
"grade": "interactive",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2"
}
},
{
"precision": "q4_k_m",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"streamParity": true,
"expected": {
"city": "Paris",
"temperature_celsius": 21,
"condition": "overcast"
},
"metric": "allWeatherFacts",
"score": 1
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 607.6971249999999,
"loadMs": 2540.7353329999996,
"offloadedLayers": 25,
"precision": "q4_k_m",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "q4_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"totalMs": 3148.4324579999993,
"performanceMetric": {
"name": "allWeatherFacts",
"value": 1
},
"grade": "interactive",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2"
}
},
{
"precision": "q4_k_m",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"streamParity": true,
"expected": {
"city": "Paris",
"temperature_celsius": 21,
"condition": "overcast"
},
"metric": "allWeatherFacts",
"score": 1
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 603.7017089999999,
"loadMs": 2540.7353329999996,
"offloadedLayers": 25,
"precision": "q4_k_m",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "q4_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"totalMs": 3144.4370419999996,
"performanceMetric": {
"name": "allWeatherFacts",
"value": 1
},
"grade": "interactive",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2"
}
},
{
"precision": "q4_k_m",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"streamParity": true,
"expected": {
"city": "Paris",
"temperature_celsius": 21,
"condition": "overcast"
},
"metric": "allWeatherFacts",
"score": 1
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 605.924625,
"loadMs": 2540.7353329999996,
"offloadedLayers": 25,
"precision": "q4_k_m",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "q4_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"totalMs": 3146.6599579999997,
"performanceMetric": {
"name": "allWeatherFacts",
"value": 1
},
"grade": "interactive",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2"
}
},
{
"precision": "q4_k_m",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"streamParity": true,
"expected": {
"city": "Paris",
"temperature_celsius": 21,
"condition": "overcast"
},
"metric": "allWeatherFacts",
"score": 1
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 605.512917,
"loadMs": 2540.7353329999996,
"offloadedLayers": 25,
"precision": "q4_k_m",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "q4_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"totalMs": 3146.2482499999996,
"performanceMetric": {
"name": "allWeatherFacts",
"value": 1
},
"grade": "interactive",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2"
}
},
{
"precision": "q4_k_m",
"device": "metal",
"status": "pass",
"quality": {
"passed": true,
"streamParity": true,
"expected": {
"city": "Paris",
"temperature_celsius": 21,
"condition": "overcast"
},
"metric": "allWeatherFacts",
"score": 1
},
"runtime": {
"acceleration": "partial",
"cpuThreads": 4,
"device": "metal",
"engine": "llama.cpp",
"fallback": false,
"fallbackReason": null,
"inferenceMs": 607.563792,
"loadMs": 2540.7353329999996,
"offloadedLayers": 25,
"precision": "q4_k_m",
"provider": "llama.cpp Metal",
"requestedDevice": "metal",
"requestedPrecision": "q4_k_m",
"selectionReason": "Pinned package with verified layer placement",
"version": "b12359",
"totalMs": 3148.2991249999995,
"performanceMetric": {
"name": "allWeatherFacts",
"value": 1
},
"grade": "interactive",
"artifactRevision": "cd0c74c575b5c416f3011b5b4c298077f26ca8c2"
}
}
]Release validation still required
- Public verified model and runtime archives, forced-source recovery, packaged desktop app smoke and catalog acceptance.