SmolLM3 Desktop
General chat, drafting and short multilingual replies locally. Strict-format output is not guaranteed.
Desktop app
Run SmolLM3 Desktop on your computer
Download the desktop app to load this model, provide input and view results locally.
Public desktop downloads are coming soon.Device requirements
If your device falls short
MemoryAdvisory only: close other apps, reduce image/tile/token/batch settings or choose a smaller verified model/precision. CPU can need more RAM than GPU.
GPUIf the selected GPU runtime fails, existing CPU fallback may recover when enabled, but CPU has its own memory/runtime needs.
PlatformUse a compatible OS/architecture/runtime or another model. These estimates do not add hardware detection or block runs.
Benchmark evidence
Saved desktop measurements apply to the recorded hardware and runtime. Fallback cases report the device that actually ran; they do not prove GPU support on the requested device.
SmolLM3 3B Base Desktop
21 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 260 ms / 1654 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 264 ms / 1876 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 266 ms / 1654 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 267 ms / 1876 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 274 ms / 1876 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 276 ms / 1654 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 448 ms / 2509 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 486 ms / 2509 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 652 ms / 2509 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1141 ms / 1623 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1148 ms / 1623 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1149 ms / 1623 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1152 ms / 1478 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1167 ms / 1507 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1184 ms / 1430 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1188 ms / 1507 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1200 ms / 1478 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1252 ms / 1430 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1259 ms / 1507 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1329 ms / 1430 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1383 ms / 1478 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-07T11:51:08.117204+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
- Runtime
- transformers 5.22.1
- Artifact revision
- d78a42f79198603e614095753484a04c10c2b940
- Fixture
{ "file": "model-samples/desktop/smollm3/smollm3-3b-base-native-geography.json", "size": 152, "sha1": "f033f82b9e5c7821b37f56df58bd3f18c26c24ef", "sha256": "48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21", "url": "https://assets.localai.tools/samples/v1/files/48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21/smollm3-3b-base-native-geography.json" }- Rerun when
- Artifact/runtime revisions; Generation/preprocessing changes; Fixture/quality scope; OS/backend behavior
Per-case quality & failure details
[
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedPrefix": " France",
"prefixPassed": true,
"streamParity": true,
"finalParity": true,
"repetitionPassed": true,
"passed": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d78a42f79198603e614095753484a04c10c2b940",
"cpuThreads": 4,
"durationMs": 653.2365409948397,
"inferenceMs": 651.743542002805,
"tokensPerSecond": 12.274766813056614,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"placement": {
"mps:0/torch.float16": 326
},
"decoding": "greedy",
"enableThinking": false,
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 6504185856,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"torchVersion": "2.10.0",
"loadMs": 2509.2017499991925,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"totalMs": 3162.4382909940323,
"grade": "acceptable",
"loadTimingScope": "Load from pinned local snapshot. Network transfer and runtime provisioning excluded."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedPrefix": " Celsius",
"prefixPassed": true,
"streamParity": true,
"finalParity": true,
"repetitionPassed": true,
"passed": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d78a42f79198603e614095753484a04c10c2b940",
"cpuThreads": 4,
"durationMs": 486.47320800228044,
"inferenceMs": 485.66837499674875,
"tokensPerSecond": 16.47214521648348,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"placement": {
"mps:0/torch.float16": 326
},
"decoding": "greedy",
"enableThinking": false,
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 6504267776,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"torchVersion": "2.10.0",
"loadMs": 2509.2017499991925,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"totalMs": 2995.674958001473,
"grade": "acceptable",
"loadTimingScope": "Load from pinned local snapshot. Network transfer and runtime provisioning excluded."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedPrefix": " x",
"prefixPassed": true,
"streamParity": true,
"finalParity": true,
"repetitionPassed": true,
"passed": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d78a42f79198603e614095753484a04c10c2b940",
"cpuThreads": 4,
"durationMs": 449.03729199722875,
"inferenceMs": 448.14449999830686,
"tokensPerSecond": 17.851384988614665,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"placement": {
"mps:0/torch.float16": 326
},
"decoding": "greedy",
"enableThinking": false,
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 6512754688,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"torchVersion": "2.10.0",
"loadMs": 2509.2017499991925,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"totalMs": 2958.2390419964213,
"grade": "acceptable",
"loadTimingScope": "Load from pinned local snapshot. Network transfer and runtime provisioning excluded."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedPrefix": " France",
"prefixPassed": true,
"streamParity": true,
"finalParity": true,
"repetitionPassed": true,
"passed": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d78a42f79198603e614095753484a04c10c2b940",
"cpuThreads": 4,
"durationMs": 277.80287499626866,
"inferenceMs": 276.3670420026756,
"tokensPerSecond": 28.947011705985368,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"placement": {
"mps:0/torch.float16": 326
},
"decoding": "greedy",
"enableThinking": false,
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 6504202240,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"torchVersion": "2.10.0",
"loadMs": 1654.1304589991341,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"totalMs": 1931.9333339954028,
"grade": "fast",
"loadTimingScope": "Load from pinned local snapshot. Network transfer and runtime provisioning excluded."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedPrefix": " Celsius",
"prefixPassed": true,
"streamParity": true,
"finalParity": true,
"repetitionPassed": true,
"passed": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d78a42f79198603e614095753484a04c10c2b940",
"cpuThreads": 4,
"durationMs": 260.59574999817414,
"inferenceMs": 259.74349999887636,
"tokensPerSecond": 30.799615774926448,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"placement": {
"mps:0/torch.float16": 326
},
"decoding": "greedy",
"enableThinking": false,
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 6504267776,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"torchVersion": "2.10.0",
"loadMs": 1654.1304589991341,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"totalMs": 1914.7262089973083,
"grade": "fast",
"loadTimingScope": "Load from pinned local snapshot. Network transfer and runtime provisioning excluded."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedPrefix": " x",
"prefixPassed": true,
"streamParity": true,
"finalParity": true,
"repetitionPassed": true,
"passed": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d78a42f79198603e614095753484a04c10c2b940",
"cpuThreads": 4,
"durationMs": 267.21795900084544,
"inferenceMs": 266.35383399843704,
"tokensPerSecond": 30.035235010159244,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"placement": {
"mps:0/torch.float16": 326
},
"decoding": "greedy",
"enableThinking": false,
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 6512754688,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"torchVersion": "2.10.0",
"loadMs": 1654.1304589991341,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"totalMs": 1921.3484179999796,
"grade": "fast",
"loadTimingScope": "Load from pinned local snapshot. Network transfer and runtime provisioning excluded."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedPrefix": " France",
"prefixPassed": true,
"streamParity": true,
"finalParity": true,
"repetitionPassed": true,
"passed": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d78a42f79198603e614095753484a04c10c2b940",
"cpuThreads": 4,
"durationMs": 268.126167000446,
"inferenceMs": 266.76662499812664,
"tokensPerSecond": 29.98875890136624,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"placement": {
"mps:0/torch.float16": 326
},
"decoding": "greedy",
"enableThinking": false,
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 6504202240,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"torchVersion": "2.10.0",
"loadMs": 1876.1434170010034,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"totalMs": 2144.2695840014494,
"grade": "fast",
"loadTimingScope": "Load from pinned local snapshot. Network transfer and runtime provisioning excluded."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedPrefix": " Celsius",
"prefixPassed": true,
"streamParity": true,
"finalParity": true,
"repetitionPassed": true,
"passed": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d78a42f79198603e614095753484a04c10c2b940",
"cpuThreads": 4,
"durationMs": 264.63254199916264,
"inferenceMs": 263.59941600094317,
"tokensPerSecond": 30.349080894668507,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"placement": {
"mps:0/torch.float16": 326
},
"decoding": "greedy",
"enableThinking": false,
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 6504267776,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"torchVersion": "2.10.0",
"loadMs": 1876.1434170010034,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"totalMs": 2140.775959000166,
"grade": "fast",
"loadTimingScope": "Load from pinned local snapshot. Network transfer and runtime provisioning excluded."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedPrefix": " x",
"prefixPassed": true,
"streamParity": true,
"finalParity": true,
"repetitionPassed": true,
"passed": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d78a42f79198603e614095753484a04c10c2b940",
"cpuThreads": 4,
"durationMs": 274.46900000359165,
"inferenceMs": 273.54350000678096,
"tokensPerSecond": 29.245805511012637,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"placement": {
"mps:0/torch.float16": 326
},
"decoding": "greedy",
"enableThinking": false,
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 6512754688,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"torchVersion": "2.10.0",
"loadMs": 1876.1434170010034,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"totalMs": 2150.612417004595,
"grade": "fast",
"loadTimingScope": "Load from pinned local snapshot. Network transfer and runtime provisioning excluded."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedPrefix": " France",
"prefixPassed": true,
"streamParity": true,
"finalParity": true,
"repetitionPassed": true,
"passed": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d78a42f79198603e614095753484a04c10c2b940",
"cpuThreads": 4,
"durationMs": 1152.7111670002341,
"inferenceMs": 1152.498082999955,
"tokensPerSecond": 6.941443216266341,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"placement": {
"cpu/torch.float32": 326
},
"decoding": "greedy",
"enableThinking": false,
"outputLimitReached": true,
"torchVersion": "2.10.0",
"loadMs": 1477.8738749955664,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"totalMs": 2630.5850419958006,
"grade": "acceptable",
"loadTimingScope": "Load from pinned local snapshot. Network transfer and runtime provisioning excluded."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedPrefix": " Celsius",
"prefixPassed": true,
"streamParity": true,
"finalParity": true,
"repetitionPassed": true,
"passed": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d78a42f79198603e614095753484a04c10c2b940",
"cpuThreads": 4,
"durationMs": 1199.8001669999212,
"inferenceMs": 1199.6416249967297,
"tokensPerSecond": 6.668658233680254,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"placement": {
"cpu/torch.float32": 326
},
"decoding": "greedy",
"enableThinking": false,
"outputLimitReached": true,
"torchVersion": "2.10.0",
"loadMs": 1477.8738749955664,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"totalMs": 2677.6740419954876,
"grade": "acceptable",
"loadTimingScope": "Load from pinned local snapshot. Network transfer and runtime provisioning excluded."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedPrefix": " x",
"prefixPassed": true,
"streamParity": true,
"finalParity": true,
"repetitionPassed": true,
"passed": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d78a42f79198603e614095753484a04c10c2b940",
"cpuThreads": 4,
"durationMs": 1383.1105830031447,
"inferenceMs": 1382.92429199646,
"tokensPerSecond": 5.78484306501753,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"placement": {
"cpu/torch.float32": 326
},
"decoding": "greedy",
"enableThinking": false,
"outputLimitReached": true,
"torchVersion": "2.10.0",
"loadMs": 1477.8738749955664,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"totalMs": 2860.984457998711,
"grade": "acceptable",
"loadTimingScope": "Load from pinned local snapshot. Network transfer and runtime provisioning excluded."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedPrefix": " France",
"prefixPassed": true,
"streamParity": true,
"finalParity": true,
"repetitionPassed": true,
"passed": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d78a42f79198603e614095753484a04c10c2b940",
"cpuThreads": 4,
"durationMs": 1329.5288329973118,
"inferenceMs": 1329.2707910004538,
"tokensPerSecond": 6.018337312579427,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"placement": {
"cpu/torch.float32": 326
},
"decoding": "greedy",
"enableThinking": false,
"outputLimitReached": true,
"torchVersion": "2.10.0",
"loadMs": 1430.105667001044,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"totalMs": 2759.6344999983557,
"grade": "acceptable",
"loadTimingScope": "Load from pinned local snapshot. Network transfer and runtime provisioning excluded."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedPrefix": " Celsius",
"prefixPassed": true,
"streamParity": true,
"finalParity": true,
"repetitionPassed": true,
"passed": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d78a42f79198603e614095753484a04c10c2b940",
"cpuThreads": 4,
"durationMs": 1184.4219999984489,
"inferenceMs": 1184.263834002195,
"tokensPerSecond": 6.755251465346339,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"placement": {
"cpu/torch.float32": 326
},
"decoding": "greedy",
"enableThinking": false,
"outputLimitReached": true,
"torchVersion": "2.10.0",
"loadMs": 1430.105667001044,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"totalMs": 2614.527666999493,
"grade": "acceptable",
"loadTimingScope": "Load from pinned local snapshot. Network transfer and runtime provisioning excluded."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedPrefix": " x",
"prefixPassed": true,
"streamParity": true,
"finalParity": true,
"repetitionPassed": true,
"passed": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d78a42f79198603e614095753484a04c10c2b940",
"cpuThreads": 4,
"durationMs": 1251.8304999975953,
"inferenceMs": 1251.6379160006181,
"tokensPerSecond": 6.391624844318034,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"placement": {
"cpu/torch.float32": 326
},
"decoding": "greedy",
"enableThinking": false,
"outputLimitReached": true,
"torchVersion": "2.10.0",
"loadMs": 1430.105667001044,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"totalMs": 2681.9361669986392,
"grade": "acceptable",
"loadTimingScope": "Load from pinned local snapshot. Network transfer and runtime provisioning excluded."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedPrefix": " France",
"prefixPassed": true,
"streamParity": true,
"finalParity": true,
"repetitionPassed": true,
"passed": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d78a42f79198603e614095753484a04c10c2b940",
"cpuThreads": 4,
"durationMs": 1167.1460830038995,
"inferenceMs": 1166.93933300121,
"tokensPerSecond": 6.855540621314978,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"placement": {
"cpu/torch.float32": 326
},
"decoding": "greedy",
"enableThinking": false,
"outputLimitReached": true,
"torchVersion": "2.10.0",
"loadMs": 1506.6771249985322,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"totalMs": 2673.823208002432,
"grade": "acceptable",
"loadTimingScope": "Load from pinned local snapshot. Network transfer and runtime provisioning excluded."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedPrefix": " Celsius",
"prefixPassed": true,
"streamParity": true,
"finalParity": true,
"repetitionPassed": true,
"passed": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d78a42f79198603e614095753484a04c10c2b940",
"cpuThreads": 4,
"durationMs": 1187.807290996716,
"inferenceMs": 1187.57020799967,
"tokensPerSecond": 6.736443829687435,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"placement": {
"cpu/torch.float32": 326
},
"decoding": "greedy",
"enableThinking": false,
"outputLimitReached": true,
"torchVersion": "2.10.0",
"loadMs": 1506.6771249985322,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"totalMs": 2694.484415995248,
"grade": "acceptable",
"loadTimingScope": "Load from pinned local snapshot. Network transfer and runtime provisioning excluded."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedPrefix": " x",
"prefixPassed": true,
"streamParity": true,
"finalParity": true,
"repetitionPassed": true,
"passed": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d78a42f79198603e614095753484a04c10c2b940",
"cpuThreads": 4,
"durationMs": 1259.3396669981303,
"inferenceMs": 1259.167249998427,
"tokensPerSecond": 6.353405395518343,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"placement": {
"cpu/torch.float32": 326
},
"decoding": "greedy",
"enableThinking": false,
"outputLimitReached": true,
"torchVersion": "2.10.0",
"loadMs": 1506.6771249985322,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"totalMs": 2766.0167919966625,
"grade": "acceptable",
"loadTimingScope": "Load from pinned local snapshot. Network transfer and runtime provisioning excluded."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedPrefix": " France",
"prefixPassed": true,
"streamParity": true,
"finalParity": true,
"repetitionPassed": true,
"passed": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d78a42f79198603e614095753484a04c10c2b940",
"cpuThreads": 1,
"durationMs": 1149.2468340002233,
"inferenceMs": 1149.0534170006867,
"tokensPerSecond": 6.962252478115401,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"placement": {
"cpu/torch.float32": 326
},
"decoding": "greedy",
"enableThinking": false,
"outputLimitReached": true,
"torchVersion": "2.10.0",
"loadMs": 1622.9159170034109,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"totalMs": 2772.162751003634,
"grade": "acceptable",
"loadTimingScope": "Load from pinned local snapshot. Network transfer and runtime provisioning excluded."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedPrefix": " Celsius",
"prefixPassed": true,
"streamParity": true,
"finalParity": true,
"repetitionPassed": true,
"passed": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d78a42f79198603e614095753484a04c10c2b940",
"cpuThreads": 1,
"durationMs": 1148.1494999970892,
"inferenceMs": 1148.0004170007305,
"tokensPerSecond": 6.968638583686951,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"placement": {
"cpu/torch.float32": 326
},
"decoding": "greedy",
"enableThinking": false,
"outputLimitReached": true,
"torchVersion": "2.10.0",
"loadMs": 1622.9159170034109,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"totalMs": 2771.0654170005,
"grade": "acceptable",
"loadTimingScope": "Load from pinned local snapshot. Network transfer and runtime provisioning excluded."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedPrefix": " x",
"prefixPassed": true,
"streamParity": true,
"finalParity": true,
"repetitionPassed": true,
"passed": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "d78a42f79198603e614095753484a04c10c2b940",
"cpuThreads": 1,
"durationMs": 1141.495625000971,
"inferenceMs": 1141.3215420034248,
"tokensPerSecond": 7.009418210013943,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"placement": {
"cpu/torch.float32": 326
},
"decoding": "greedy",
"enableThinking": false,
"outputLimitReached": true,
"torchVersion": "2.10.0",
"loadMs": 1622.9159170034109,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"totalMs": 2764.4115420043818,
"grade": "acceptable",
"loadTimingScope": "Load from pinned local snapshot. Network transfer and runtime provisioning excluded."
}
}
]Release validation still required
- Public backup and source-outage recovery pending.