Liquid LFM3 Desktop
Answer a text question or ask about a single PNG image with Liquid AI LFM3-VL on Apple Silicon.
Desktop app
Run Liquid LFM3 Desktop on your computer
Download the desktop app to load this model, provide input and view results locally.
Public desktop downloads are coming soon.Device requirements
macOS Apple SiliconMETAL / Q8TestedNot establishedRAM estimate
macOS Apple SiliconMETAL / Q4TestedNot establishedRAM estimate
macOS Apple SiliconMETAL / Q5TestedNot establishedRAM estimate
macOS Apple SiliconMETAL / Q6TestedNot establishedRAM estimate
macOS Apple SiliconMETAL / BF16TestedNot establishedRAM estimate
macOS Apple SiliconCPU / BF16TestedNot establishedRAM estimate
If your device falls short
MemoryShow advisory requirements.
GPUUse the verified BF16 CPU profile when recovery is enabled.
PlatformNo verified configuration.
Benchmark evidence
Saved desktop measurements apply to the recorded hardware and runtime. Fallback cases report the device that actually ran; they do not prove GPU support on the requested device.
LFM3-VL 2B MLX Desktop
30 configurations testedPassed30
Failed0
TestedNov 16, 2026
RuntimeMLX-VLM 0.7.12
Quality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
Q4 | metalAccelerated | Passed | Very fast | macos · aarch64 | 84 ms / 563 ms |
Q6 | metalAccelerated | Passed | Very fast | macos · aarch64 | 89 ms / 549 ms |
Q5 | metalAccelerated | Passed | Very fast | macos · aarch64 | 89 ms / 544 ms |
Q4 | metalAccelerated | Passed | Very fast | macos · aarch64 | 90 ms / 563 ms |
Q8 | metalAccelerated | Passed | Very fast | macos · aarch64 | 91 ms / 598 ms |
Q6 | metalAccelerated | Passed | Very fast | macos · aarch64 | 92 ms / 549 ms |
Q5 | metalAccelerated | Passed | Fast | macos · aarch64 | 113 ms / 544 ms |
BF16 | metalAccelerated | Passed | Fast | macos · aarch64 | 117 ms / 627 ms |
BF16 | metalAccelerated | Passed | Fast | macos · aarch64 | 123 ms / 627 ms |
Q5 | metalAccelerated | Passed | Fast | macos · aarch64 | 129 ms / 544 ms |
Q4 | metalAccelerated | Passed | Fast | macos · aarch64 | 130 ms / 563 ms |
Q6 | metalAccelerated | Passed | Fast | macos · aarch64 | 168 ms / 549 ms |
Q4 | metalAccelerated | Passed | Fast | macos · aarch64 | 175 ms / 563 ms |
Q6 | metalAccelerated | Passed | Fast | macos · aarch64 | 180 ms / 549 ms |
BF16 | metalAccelerated | Passed | Fast | macos · aarch64 | 188 ms / 627 ms |
Q8 | metalAccelerated | Passed | Fast | macos · aarch64 | 192 ms / 598 ms |
BF16 | metalAccelerated | Passed | Fast | macos · aarch64 | 200 ms / 627 ms |
Q4 | metalAccelerated | Passed | Fast | macos · aarch64 | 201 ms / 563 ms |
Q5 | metalAccelerated | Passed | Fast | macos · aarch64 | 202 ms / 544 ms |
Q5 | metalAccelerated | Passed | Fast | macos · aarch64 | 205 ms / 544 ms |
Q8 | metalAccelerated | Passed | Fast | macos · aarch64 | 229 ms / 598 ms |
Q6 | metalAccelerated | Passed | Fast | macos · aarch64 | 270 ms / 549 ms |
BF16 | metalAccelerated | Passed | Fast | macos · aarch64 | 288 ms / 627 ms |
Q8 | metalAccelerated | Passed | Fast | macos · aarch64 | 472 ms / 598 ms |
Q8 | metalAccelerated | Passed | Fast | macos · aarch64 | 539 ms / 598 ms |
BF16 | cpu | Passed | Fast | macos · aarch64 | 1078 ms / 618 ms |
BF16 | cpu | Passed | Fast | macos · aarch64 | 1622 ms / 618 ms |
BF16 | cpu | Passed | Fast | macos · aarch64 | 1626 ms / 618 ms |
BF16 | cpu | Passed | Moderate | macos · aarch64 | 2029 ms / 618 ms |
BF16 | cpu | Passed | Moderate | macos · aarch64 | 3149 ms / 618 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-16T01:19:50.059Z
- Environment
- { "scope": "Managed DesktopAI sidecar with staged weights and concurrent persistent backup mirror.", "hardware": "Apple M3 Max", "os": "macOS", "arch": "arm64", "runtime": { "name": "MLX-VLM", "version": "0.7.12" }, "speedGradePolicy": "Two seconds is the interactive grade. The BF16 CPU fallback must match or improve the exact GGUF CPU matrix maximum and total." }
- Runtime
- MLX-VLM 0.7.12
- Artifact revision
- 6990b981e8235d1a41a3aa3c50b3402631659abb
- Fixture
{ "file": "model-samples/lfm3-vl-2b-mlx-native/vl-samples/desktopai-arithmetic-input.json", "url": "https://assets.localai.tools/samples/v1/files/712f8b4b7e032c2e4b12e94dbb2e8d7c9681d69c344dbcf7874175c28b7678b5/desktopai-arithmetic-input.json", "size": 219, "sha1": "30f7cac931b376f1ddcdf626b6cad3d91a3c6e93", "sha256": "712f8b4b7e032c2e4b12e94dbb2e8d7c9681d69c344dbcf7874175c28b7678b5" }- Rerun when
- artifactRevision; runtimeVersion; fixtureChecksum; qualityGate; nativeBackend
Per-case quality & failure details
[
{
"precision": "q8",
"device": "metal",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": "15"
},
"runtime": {
"acceleration": "gpu",
"artifactRevision": "6990b981e8235d1a41a3aa3c50b3402631659abb",
"device": "metal",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(gpu, 0)",
"inferenceMs": 472.3175,
"loadMs": 598.3486670302227,
"mlxVersion": "0.32.4",
"precision": "q8",
"provider": "MLX Metal",
"requestedDevice": "metal",
"requestedPrecision": "q8",
"version": "0.7.12",
"totalMs": 1070.6661670302228,
"charactersPerSecond": 4.23443975715488,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 4.23443975715488
},
"grade": "good"
}
},
{
"precision": "q8",
"device": "metal",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": "red"
},
"runtime": {
"acceleration": "gpu",
"artifactRevision": "6990b981e8235d1a41a3aa3c50b3402631659abb",
"device": "metal",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(gpu, 0)",
"inferenceMs": 539.4216250000001,
"loadMs": 598.3486670302227,
"mlxVersion": "0.32.4",
"precision": "q8",
"provider": "MLX Metal",
"requestedDevice": "metal",
"requestedPrecision": "q8",
"version": "0.7.12",
"totalMs": 1137.7702920302227,
"charactersPerSecond": 5.561512295692631,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 5.561512295692631
},
"grade": "good"
}
},
{
"precision": "q8",
"device": "metal",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": "blue"
},
"runtime": {
"acceleration": "gpu",
"artifactRevision": "6990b981e8235d1a41a3aa3c50b3402631659abb",
"device": "metal",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(gpu, 0)",
"inferenceMs": 91.205708,
"loadMs": 598.3486670302227,
"mlxVersion": "0.32.4",
"precision": "q8",
"provider": "MLX Metal",
"requestedDevice": "metal",
"requestedPrecision": "q8",
"version": "0.7.12",
"totalMs": 689.5543750302227,
"charactersPerSecond": 43.85690421919646,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 43.85690421919646
},
"grade": "good"
}
},
{
"precision": "q8",
"device": "metal",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": [
"localai2026"
]
},
"runtime": {
"acceleration": "gpu",
"artifactRevision": "6990b981e8235d1a41a3aa3c50b3402631659abb",
"device": "metal",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(gpu, 0)",
"inferenceMs": 229.239417,
"loadMs": 598.3486670302227,
"mlxVersion": "0.32.4",
"precision": "q8",
"provider": "MLX Metal",
"requestedDevice": "metal",
"requestedPrecision": "q8",
"version": "0.7.12",
"totalMs": 827.5880840302227,
"charactersPerSecond": 56.70927002924632,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 56.70927002924632
},
"grade": "good"
}
},
{
"precision": "q8",
"device": "metal",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": [
"3"
]
},
"runtime": {
"acceleration": "gpu",
"artifactRevision": "6990b981e8235d1a41a3aa3c50b3402631659abb",
"device": "metal",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(gpu, 0)",
"inferenceMs": 191.791125,
"loadMs": 598.3486670302227,
"mlxVersion": "0.32.4",
"precision": "q8",
"provider": "MLX Metal",
"requestedDevice": "metal",
"requestedPrecision": "q8",
"version": "0.7.12",
"totalMs": 790.1397920302227,
"charactersPerSecond": 5.214005601145518,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 5.214005601145518
},
"grade": "good"
}
},
{
"precision": "q4",
"device": "metal",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": "15"
},
"runtime": {
"acceleration": "gpu",
"artifactRevision": "d0d7edaf1b462161720898e46687c6fdaae6caee",
"device": "metal",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(gpu, 0)",
"inferenceMs": 130.28637500000002,
"loadMs": 563.4360839612782,
"mlxVersion": "0.32.4",
"precision": "q4",
"provider": "MLX Metal",
"requestedDevice": "metal",
"requestedPrecision": "q4",
"version": "0.7.12",
"totalMs": 693.7224589612782,
"charactersPerSecond": 15.350799344904635,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 15.350799344904635
},
"grade": "good"
}
},
{
"precision": "q4",
"device": "metal",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": "red"
},
"runtime": {
"acceleration": "gpu",
"artifactRevision": "d0d7edaf1b462161720898e46687c6fdaae6caee",
"device": "metal",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(gpu, 0)",
"inferenceMs": 90.442458,
"loadMs": 563.4360839612782,
"mlxVersion": "0.32.4",
"precision": "q4",
"provider": "MLX Metal",
"requestedDevice": "metal",
"requestedPrecision": "q4",
"version": "0.7.12",
"totalMs": 653.8785419612782,
"charactersPerSecond": 33.170261692799194,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 33.170261692799194
},
"grade": "good"
}
},
{
"precision": "q4",
"device": "metal",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": "blue"
},
"runtime": {
"acceleration": "gpu",
"artifactRevision": "d0d7edaf1b462161720898e46687c6fdaae6caee",
"device": "metal",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(gpu, 0)",
"inferenceMs": 83.517834,
"loadMs": 563.4360839612782,
"mlxVersion": "0.32.4",
"precision": "q4",
"provider": "MLX Metal",
"requestedDevice": "metal",
"requestedPrecision": "q4",
"version": "0.7.12",
"totalMs": 646.9539179612782,
"charactersPerSecond": 47.89396238412984,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 47.89396238412984
},
"grade": "good"
}
},
{
"precision": "q4",
"device": "metal",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": [
"localai2026"
]
},
"runtime": {
"acceleration": "gpu",
"artifactRevision": "d0d7edaf1b462161720898e46687c6fdaae6caee",
"device": "metal",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(gpu, 0)",
"inferenceMs": 200.67475000000002,
"loadMs": 563.4360839612782,
"mlxVersion": "0.32.4",
"precision": "q4",
"provider": "MLX Metal",
"requestedDevice": "metal",
"requestedPrecision": "q4",
"version": "0.7.12",
"totalMs": 764.1108339612782,
"charactersPerSecond": 64.78144360463884,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 64.78144360463884
},
"grade": "good"
}
},
{
"precision": "q4",
"device": "metal",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": [
"3"
]
},
"runtime": {
"acceleration": "gpu",
"artifactRevision": "d0d7edaf1b462161720898e46687c6fdaae6caee",
"device": "metal",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(gpu, 0)",
"inferenceMs": 175.089958,
"loadMs": 563.4360839612782,
"mlxVersion": "0.32.4",
"precision": "q4",
"provider": "MLX Metal",
"requestedDevice": "metal",
"requestedPrecision": "q4",
"version": "0.7.12",
"totalMs": 738.5260419612782,
"charactersPerSecond": 5.711349819388271,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 5.711349819388271
},
"grade": "good"
}
},
{
"precision": "q5",
"device": "metal",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": "15"
},
"runtime": {
"acceleration": "gpu",
"artifactRevision": "a533a106dda473d565d84b8a6ee19f502d6dbf95",
"device": "metal",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(gpu, 0)",
"inferenceMs": 128.81725,
"loadMs": 544.1885840264149,
"mlxVersion": "0.32.4",
"precision": "q5",
"provider": "MLX Metal",
"requestedDevice": "metal",
"requestedPrecision": "q5",
"version": "0.7.12",
"totalMs": 673.005834026415,
"charactersPerSecond": 15.525870952842107,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 15.525870952842107
},
"grade": "good"
}
},
{
"precision": "q5",
"device": "metal",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": "red"
},
"runtime": {
"acceleration": "gpu",
"artifactRevision": "a533a106dda473d565d84b8a6ee19f502d6dbf95",
"device": "metal",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(gpu, 0)",
"inferenceMs": 89.386708,
"loadMs": 544.1885840264149,
"mlxVersion": "0.32.4",
"precision": "q5",
"provider": "MLX Metal",
"requestedDevice": "metal",
"requestedPrecision": "q5",
"version": "0.7.12",
"totalMs": 633.5752920264149,
"charactersPerSecond": 33.56203698652824,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 33.56203698652824
},
"grade": "good"
}
},
{
"precision": "q5",
"device": "metal",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": "blue"
},
"runtime": {
"acceleration": "gpu",
"artifactRevision": "a533a106dda473d565d84b8a6ee19f502d6dbf95",
"device": "metal",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(gpu, 0)",
"inferenceMs": 112.729625,
"loadMs": 544.1885840264149,
"mlxVersion": "0.32.4",
"precision": "q5",
"provider": "MLX Metal",
"requestedDevice": "metal",
"requestedPrecision": "q5",
"version": "0.7.12",
"totalMs": 656.9182090264148,
"charactersPerSecond": 35.48313054354612,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 35.48313054354612
},
"grade": "good"
}
},
{
"precision": "q5",
"device": "metal",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": [
"localai2026"
]
},
"runtime": {
"acceleration": "gpu",
"artifactRevision": "a533a106dda473d565d84b8a6ee19f502d6dbf95",
"device": "metal",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(gpu, 0)",
"inferenceMs": 205.08166599999998,
"loadMs": 544.1885840264149,
"mlxVersion": "0.32.4",
"precision": "q5",
"provider": "MLX Metal",
"requestedDevice": "metal",
"requestedPrecision": "q5",
"version": "0.7.12",
"totalMs": 749.2702500264149,
"charactersPerSecond": 63.38938167198233,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 63.38938167198233
},
"grade": "good"
}
},
{
"precision": "q5",
"device": "metal",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": [
"3"
]
},
"runtime": {
"acceleration": "gpu",
"artifactRevision": "a533a106dda473d565d84b8a6ee19f502d6dbf95",
"device": "metal",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(gpu, 0)",
"inferenceMs": 202.098667,
"loadMs": 544.1885840264149,
"mlxVersion": "0.32.4",
"precision": "q5",
"provider": "MLX Metal",
"requestedDevice": "metal",
"requestedPrecision": "q5",
"version": "0.7.12",
"totalMs": 746.2872510264149,
"charactersPerSecond": 4.948078158278995,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 4.948078158278995
},
"grade": "good"
}
},
{
"precision": "q6",
"device": "metal",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": "15"
},
"runtime": {
"acceleration": "gpu",
"artifactRevision": "dc1e22c62368e1ca6187eccda650a55819797906",
"device": "metal",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(gpu, 0)",
"inferenceMs": 167.857709,
"loadMs": 548.7807500176132,
"mlxVersion": "0.32.4",
"precision": "q6",
"provider": "MLX Metal",
"requestedDevice": "metal",
"requestedPrecision": "q6",
"version": "0.7.12",
"totalMs": 716.6384590176132,
"charactersPerSecond": 11.914853431009236,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 11.914853431009236
},
"grade": "good"
}
},
{
"precision": "q6",
"device": "metal",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": "red"
},
"runtime": {
"acceleration": "gpu",
"artifactRevision": "dc1e22c62368e1ca6187eccda650a55819797906",
"device": "metal",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(gpu, 0)",
"inferenceMs": 92.268375,
"loadMs": 548.7807500176132,
"mlxVersion": "0.32.4",
"precision": "q6",
"provider": "MLX Metal",
"requestedDevice": "metal",
"requestedPrecision": "q6",
"version": "0.7.12",
"totalMs": 641.0491250176132,
"charactersPerSecond": 32.513848867502,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 32.513848867502
},
"grade": "good"
}
},
{
"precision": "q6",
"device": "metal",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": "blue"
},
"runtime": {
"acceleration": "gpu",
"artifactRevision": "dc1e22c62368e1ca6187eccda650a55819797906",
"device": "metal",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(gpu, 0)",
"inferenceMs": 88.61120799999999,
"loadMs": 548.7807500176132,
"mlxVersion": "0.32.4",
"precision": "q6",
"provider": "MLX Metal",
"requestedDevice": "metal",
"requestedPrecision": "q6",
"version": "0.7.12",
"totalMs": 637.3919580176132,
"charactersPerSecond": 45.14101647276945,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 45.14101647276945
},
"grade": "good"
}
},
{
"precision": "q6",
"device": "metal",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": [
"localai2026"
]
},
"runtime": {
"acceleration": "gpu",
"artifactRevision": "dc1e22c62368e1ca6187eccda650a55819797906",
"device": "metal",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(gpu, 0)",
"inferenceMs": 269.563334,
"loadMs": 548.7807500176132,
"mlxVersion": "0.32.4",
"precision": "q6",
"provider": "MLX Metal",
"requestedDevice": "metal",
"requestedPrecision": "q6",
"version": "0.7.12",
"totalMs": 818.3440840176131,
"charactersPerSecond": 48.226143396787045,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 48.226143396787045
},
"grade": "good"
}
},
{
"precision": "q6",
"device": "metal",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": [
"3"
]
},
"runtime": {
"acceleration": "gpu",
"artifactRevision": "dc1e22c62368e1ca6187eccda650a55819797906",
"device": "metal",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(gpu, 0)",
"inferenceMs": 179.834,
"loadMs": 548.7807500176132,
"mlxVersion": "0.32.4",
"precision": "q6",
"provider": "MLX Metal",
"requestedDevice": "metal",
"requestedPrecision": "q6",
"version": "0.7.12",
"totalMs": 728.6147500176132,
"charactersPerSecond": 5.560683741672876,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 5.560683741672876
},
"grade": "good"
}
},
{
"precision": "bf16",
"device": "metal",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": "15"
},
"runtime": {
"acceleration": "gpu",
"artifactRevision": "e4b38de5f5930b0461a42ba52335f77a938979f2",
"device": "metal",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(gpu, 0)",
"inferenceMs": 188.381041,
"loadMs": 627.3590420023538,
"mlxVersion": "0.32.4",
"precision": "bf16",
"provider": "MLX Metal",
"requestedDevice": "metal",
"requestedPrecision": "bf16",
"version": "0.7.12",
"totalMs": 815.7400830023538,
"charactersPerSecond": 10.616779636545271,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 10.616779636545271
},
"grade": "good"
}
},
{
"precision": "bf16",
"device": "metal",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": "red"
},
"runtime": {
"acceleration": "gpu",
"artifactRevision": "e4b38de5f5930b0461a42ba52335f77a938979f2",
"device": "metal",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(gpu, 0)",
"inferenceMs": 122.603291,
"loadMs": 627.3590420023538,
"mlxVersion": "0.32.4",
"precision": "bf16",
"provider": "MLX Metal",
"requestedDevice": "metal",
"requestedPrecision": "bf16",
"version": "0.7.12",
"totalMs": 749.9623330023538,
"charactersPerSecond": 24.469163719267534,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 24.469163719267534
},
"grade": "good"
}
},
{
"precision": "bf16",
"device": "metal",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": "blue"
},
"runtime": {
"acceleration": "gpu",
"artifactRevision": "e4b38de5f5930b0461a42ba52335f77a938979f2",
"device": "metal",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(gpu, 0)",
"inferenceMs": 117.18008400000001,
"loadMs": 627.3590420023538,
"mlxVersion": "0.32.4",
"precision": "bf16",
"provider": "MLX Metal",
"requestedDevice": "metal",
"requestedPrecision": "bf16",
"version": "0.7.12",
"totalMs": 744.5391260023538,
"charactersPerSecond": 34.135493536597906,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 34.135493536597906
},
"grade": "good"
}
},
{
"precision": "bf16",
"device": "metal",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": [
"localai2026"
]
},
"runtime": {
"acceleration": "gpu",
"artifactRevision": "e4b38de5f5930b0461a42ba52335f77a938979f2",
"device": "metal",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(gpu, 0)",
"inferenceMs": 287.836625,
"loadMs": 627.3590420023538,
"mlxVersion": "0.32.4",
"precision": "bf16",
"provider": "MLX Metal",
"requestedDevice": "metal",
"requestedPrecision": "bf16",
"version": "0.7.12",
"totalMs": 915.1956670023538,
"charactersPerSecond": 45.164509554682276,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 45.164509554682276
},
"grade": "good"
}
},
{
"precision": "bf16",
"device": "metal",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": [
"3"
]
},
"runtime": {
"acceleration": "gpu",
"artifactRevision": "e4b38de5f5930b0461a42ba52335f77a938979f2",
"device": "metal",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(gpu, 0)",
"inferenceMs": 199.77375,
"loadMs": 627.3590420023538,
"mlxVersion": "0.32.4",
"precision": "bf16",
"provider": "MLX Metal",
"requestedDevice": "metal",
"requestedPrecision": "bf16",
"version": "0.7.12",
"totalMs": 827.1327920023539,
"charactersPerSecond": 5.005662655879464,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 5.005662655879464
},
"grade": "good"
}
},
{
"precision": "bf16",
"device": "cpu",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": "15"
},
"runtime": {
"acceleration": "cpu",
"artifactRevision": "e4b38de5f5930b0461a42ba52335f77a938979f2",
"device": "cpu",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(cpu, 0)",
"inferenceMs": 1626.3776669999997,
"loadMs": 617.5710830138996,
"mlxVersion": "0.32.4",
"precision": "bf16",
"provider": "MLX CPU",
"requestedDevice": "cpu",
"requestedPrecision": "bf16",
"version": "0.7.12",
"totalMs": 2243.9487500138994,
"charactersPerSecond": 1.2297266745485878,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 1.2297266745485878
},
"grade": "good"
}
},
{
"precision": "bf16",
"device": "cpu",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": "red"
},
"runtime": {
"acceleration": "cpu",
"artifactRevision": "e4b38de5f5930b0461a42ba52335f77a938979f2",
"device": "cpu",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(cpu, 0)",
"inferenceMs": 1621.830125,
"loadMs": 617.5710830138996,
"mlxVersion": "0.32.4",
"precision": "bf16",
"provider": "MLX CPU",
"requestedDevice": "cpu",
"requestedPrecision": "bf16",
"version": "0.7.12",
"totalMs": 2239.4012080138996,
"charactersPerSecond": 1.8497621629762242,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 1.8497621629762242
},
"grade": "good"
}
},
{
"precision": "bf16",
"device": "cpu",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": "blue"
},
"runtime": {
"acceleration": "cpu",
"artifactRevision": "e4b38de5f5930b0461a42ba52335f77a938979f2",
"device": "cpu",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(cpu, 0)",
"inferenceMs": 1077.9563329999999,
"loadMs": 617.5710830138996,
"mlxVersion": "0.32.4",
"precision": "bf16",
"provider": "MLX CPU",
"requestedDevice": "cpu",
"requestedPrecision": "bf16",
"version": "0.7.12",
"totalMs": 1695.5274160138995,
"charactersPerSecond": 3.7107254510651875,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 3.7107254510651875
},
"grade": "good"
}
},
{
"precision": "bf16",
"device": "cpu",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": [
"localai2026"
]
},
"runtime": {
"acceleration": "cpu",
"artifactRevision": "e4b38de5f5930b0461a42ba52335f77a938979f2",
"device": "cpu",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(cpu, 0)",
"inferenceMs": 3148.829208,
"loadMs": 617.5710830138996,
"mlxVersion": "0.32.4",
"precision": "bf16",
"provider": "MLX CPU",
"requestedDevice": "cpu",
"requestedPrecision": "bf16",
"version": "0.7.12",
"totalMs": 3766.4002910138997,
"charactersPerSecond": 4.12851861478287,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 4.12851861478287
},
"grade": "moderate"
}
},
{
"precision": "bf16",
"device": "cpu",
"status": "pass",
"quality": {
"metric": "frozenAnswerMatch",
"score": 1,
"passed": true,
"expected": [
"3"
]
},
"runtime": {
"acceleration": "cpu",
"artifactRevision": "e4b38de5f5930b0461a42ba52335f77a938979f2",
"device": "cpu",
"engine": "MLX-VLM",
"fallback": false,
"fallbackReason": null,
"generationStreamDevice": "Device(cpu, 0)",
"inferenceMs": 2029.4370000000001,
"loadMs": 617.5710830138996,
"mlxVersion": "0.32.4",
"precision": "bf16",
"provider": "MLX CPU",
"requestedDevice": "cpu",
"requestedPrecision": "bf16",
"version": "0.7.12",
"totalMs": 2647.0080830138995,
"charactersPerSecond": 0.4927474959804123,
"performanceMetric": {
"name": "charactersPerSecond",
"value": 0.4927474959804123
},
"grade": "moderate"
}
}
]Release validation still required
- Packaged inference page replay, complete backup and independent model/runtime source recovery.