Granite Speech 5
Transcribe complete WAV recordings locally with the multilingual Granite5 TurboCTC checkpoint. Returns punctuated text and complete native per-frame CTC token IDs.
Desktop app
Run Granite Speech 5 on your computer
Download the desktop app to load this model, provide input and view results locally.
Public desktop downloads are coming soon.Device requirements
If your device falls short
MemoryAdvisory only: close other apps, reduce image/tile/token/batch settings or choose a smaller verified model/precision. CPU can need more RAM than GPU.
GPUIf the selected GPU runtime fails, existing CPU fallback may recover when enabled, but CPU has its own memory/runtime needs.
PlatformUse a compatible OS/architecture/runtime or another model. These estimates do not add hardware detection or block runs.
Benchmark evidence
Saved desktop measurements apply to the recorded hardware and runtime. Fallback cases report the device that actually ran; they do not prove GPU support on the requested device.
Granite Speech5 TurboCTC ·470M multilingual
35 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | Task metric | |
|---|---|---|---|---|---|---|
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 44 ms / 0 ms | 0.004 RTF |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 47 ms / 0 ms | 0.005 RTF |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 50 ms / 0 ms | 0.005 RTF |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 54 ms / 0 ms | 0.005 RTF |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 56 ms / 0 ms | 0.006 RTF |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 158 ms / 462 ms | 0.020 RTF |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 161 ms / 441 ms | 0.021 RTF |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 210 ms / 486 ms | 0.033 RTF |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 401 ms / 0 ms | 0.037 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 527 ms / 0 ms | 0.175 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 433 ms / 0 ms | 0.231 RTF |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 256 ms / 0 ms | 0.022 RTF |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 250 ms / 0 ms | 0.023 RTF |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 250 ms / 0 ms | 0.023 RTF |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 253 ms / 0 ms | 0.023 RTF |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 256 ms / 0 ms | 0.024 RTF |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 263 ms / 0 ms | 0.024 RTF |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 280 ms / 210 ms | 0.026 RTF |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 292 ms / 220 ms | 0.027 RTF |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 299 ms / 229 ms | 0.028 RTF |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 150 ms / 0 ms | 0.045 RTF |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 604 ms / 0 ms | 0.055 RTF |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 622 ms / 0 ms | 0.057 RTF |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 622 ms / 0 ms | 0.057 RTF |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 641 ms / 0 ms | 0.059 RTF |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 641 ms / 207 ms | 0.059 RTF |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 657 ms / 0 ms | 0.060 RTF |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 678 ms / 218 ms | 0.062 RTF |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 688 ms / 210 ms | 0.063 RTF |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 753 ms / 0 ms | 0.066 RTF |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 132 ms / 0 ms | 0.067 RTF |
FP32 | cpu | Passed | Fast | macos · aarch64 | 570 ms / 0 ms | 0.168 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 615 ms / 0 ms | 0.308 RTF |
FP16 | mps | Failed | — | Not recorded | — / — | — |
BF16 | mps | Failed | — | Not recorded | — / — | — |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-02T23:08:52.489605+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2", "runtime": "Transformers5.22.1/Torch2.10.0", "interpreter": "App-ownedCPython3.13.18 private development venv" }
- Runtime
- transformers 5.22.1
- Artifact revision
- 1dd23f46f6d4e6eae2531982ba2e33220086dd97
- Fixture
{ "file": "model-samples/desktop/granite-speech5/granite-speech5-jfk.wav", "size": 3880888, "sha1": "6fb9dfd0ec18bb61a6af995c0df0cd32ee2afe34", "sha256": "c72e27f68334ae37e037ef819fa5aa1fc1552aa840b93e308df4ffe52317bedb", "url": "https://assets.localai.tools/samples/v1/files/c72e27f68334ae37e037ef819fa5aa1fc1552aa840b93e308df4ffe52317bedb/granite-speech5-jfk.wav" }- Rerun when
- Exact multilingual artifact/tokenizer/runtime revision; Decode/resample/feature extraction/greedyCTC/native output; Fixture/reference/quality gate; Relevant OS/backend behavior
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 228.75025001121685,
"provider": "PyTorch CPU",
"placement": {
"cpu/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "cpu",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 305.26266699598636,
"inferenceMs": 298.69683300785255,
"audioSeconds": 11,
"realTimeFactor": 0.02775115154508967,
"audioSecondsPerSecond": 36.82663752819507,
"activationPlacement": {
"audioEncoder": "cpu"
},
"predictionDevice": "cpu",
"predictionShape": [
1,
137
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": null,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 534.0129170072032,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3410378752,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"provider": "PyTorch CPU",
"placement": {
"cpu/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "cpu",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 151.6457499965327,
"inferenceMs": 150.4200839990517,
"audioSeconds": 3.4,
"realTimeFactor": 0.04460169117545079,
"audioSecondsPerSecond": 22.603364588078772,
"activationPlacement": {
"audioEncoder": "cpu"
},
"predictionDevice": "cpu",
"predictionShape": [
1,
42
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": null,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 151.6457499965327,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3412115456,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"provider": "PyTorch CPU",
"placement": {
"cpu/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "cpu",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 257.9129590012599,
"inferenceMs": 255.58337499387562,
"audioSeconds": 11.5,
"realTimeFactor": 0.022427213826196512,
"audioSecondsPerSecond": 44.9951018929755,
"activationPlacement": {
"audioEncoder": "cpu"
},
"predictionDevice": "cpu",
"predictionShape": [
1,
143
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": null,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 257.9129590012599,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3457318912,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 0,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"provider": "PyTorch CPU",
"placement": {
"cpu/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "cpu",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 133.14541600993834,
"inferenceMs": 132.02345800527837,
"audioSeconds": 2,
"realTimeFactor": 0.06657270800496917,
"audioSecondsPerSecond": 15.148823021436378,
"activationPlacement": {
"audioEncoder": "cpu"
},
"predictionDevice": "cpu",
"predictionShape": [
1,
25
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": null,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 133.14541600993834,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3457712128,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"provider": "PyTorch CPU",
"placement": {
"cpu/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "cpu",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 265.6615000014426,
"inferenceMs": 263.16729199606925,
"audioSeconds": 11,
"realTimeFactor": 0.0241510454546766,
"audioSecondsPerSecond": 41.79850739264475,
"activationPlacement": {
"audioEncoder": "cpu"
},
"predictionDevice": "cpu",
"predictionShape": [
1,
137
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": null,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 265.6615000014426,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3498491904,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"provider": "PyTorch CPU",
"placement": {
"cpu/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "cpu",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 258.7837920000311,
"inferenceMs": 256.33795800968073,
"audioSeconds": 11,
"realTimeFactor": 0.0235257992727301,
"audioSecondsPerSecond": 42.91209965706514,
"activationPlacement": {
"audioEncoder": "cpu"
},
"predictionDevice": "cpu",
"predictionShape": [
1,
137
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": null,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 258.7837920000311,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3545022464,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"provider": "PyTorch CPU",
"placement": {
"cpu/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "cpu",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 252.50008299190085,
"inferenceMs": 250.10508400737308,
"audioSeconds": 11,
"realTimeFactor": 0.022954552999263713,
"audioSecondsPerSecond": 43.981512985460625,
"activationPlacement": {
"audioEncoder": "cpu"
},
"predictionDevice": "cpu",
"predictionShape": [
1,
137
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": null,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 252.50008299190085,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3577823232,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"provider": "PyTorch CPU",
"placement": {
"cpu/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "cpu",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 252.09549999271985,
"inferenceMs": 249.79441599862184,
"audioSeconds": 11,
"realTimeFactor": 0.022917772726610896,
"audioSecondsPerSecond": 44.03621256313708,
"activationPlacement": {
"audioEncoder": "cpu"
},
"predictionDevice": "cpu",
"predictionShape": [
1,
137
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": null,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 252.09549999271985,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3609706496,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"provider": "PyTorch CPU",
"placement": {
"cpu/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "cpu",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 255.0442909996491,
"inferenceMs": 252.62858299538493,
"audioSeconds": 11,
"realTimeFactor": 0.023185844636331734,
"audioSecondsPerSecond": 43.542183032396416,
"activationPlacement": {
"audioEncoder": "cpu"
},
"predictionDevice": "cpu",
"predictionShape": [
1,
137
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": null,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 255.0442909996491,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3640770560,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 209.83783299743664,
"provider": "PyTorch CPU",
"placement": {
"cpu/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "cpu",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 283.5699580027722,
"inferenceMs": 280.03216700744815,
"audioSeconds": 11,
"realTimeFactor": 0.02577908709116111,
"audioSecondsPerSecond": 39.28120157605832,
"activationPlacement": {
"audioEncoder": "cpu"
},
"predictionDevice": "cpu",
"predictionShape": [
1,
137
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": null,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 493.4077910002088,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3407282176,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 219.62604099826422,
"provider": "PyTorch CPU",
"placement": {
"cpu/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "cpu",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 295.8606250031153,
"inferenceMs": 292.3435840057209,
"audioSeconds": 11,
"realTimeFactor": 0.02689642045482866,
"audioSecondsPerSecond": 37.62695883137541,
"activationPlacement": {
"audioEncoder": "cpu"
},
"predictionDevice": "cpu",
"predictionShape": [
1,
137
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": null,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 515.4866660013795,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3389620224,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 206.75054199818987,
"provider": "PyTorch CPU",
"placement": {
"cpu/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "cpu",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 644.9002919980558,
"inferenceMs": 640.5599169956986,
"audioSeconds": 11,
"realTimeFactor": 0.058627299272550525,
"audioSecondsPerSecond": 17.172476310399336,
"activationPlacement": {
"audioEncoder": "cpu"
},
"predictionDevice": "cpu",
"predictionShape": [
1,
137
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": null,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 851.6508339962456,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3409821696,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"provider": "PyTorch CPU",
"placement": {
"cpu/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "cpu",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 572.1612089982955,
"inferenceMs": 569.5837090024725,
"audioSeconds": 3.4,
"realTimeFactor": 0.16828270852891045,
"audioSecondsPerSecond": 5.969271849355581,
"activationPlacement": {
"audioEncoder": "cpu"
},
"predictionDevice": "cpu",
"predictionShape": [
1,
42
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": null,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 572.1612089982955,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3411591168,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"provider": "PyTorch CPU",
"placement": {
"cpu/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "cpu",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 756.0626670019701,
"inferenceMs": 752.8549999988172,
"audioSeconds": 11.5,
"realTimeFactor": 0.06574457973930174,
"audioSecondsPerSecond": 15.275185792772934,
"activationPlacement": {
"audioEncoder": "cpu"
},
"predictionDevice": "cpu",
"predictionShape": [
1,
143
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": null,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 756.0626670019701,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3457286144,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 0,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"provider": "PyTorch CPU",
"placement": {
"cpu/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "cpu",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 616.9154590024846,
"inferenceMs": 614.518540998688,
"audioSeconds": 2,
"realTimeFactor": 0.3084577295012423,
"audioSecondsPerSecond": 3.2545804016746014,
"activationPlacement": {
"audioEncoder": "cpu"
},
"predictionDevice": "cpu",
"predictionShape": [
1,
25
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": null,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 616.9154590024846,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "moderate",
"sampledPeakHelperRssBytes": 3457400832,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"provider": "PyTorch CPU",
"placement": {
"cpu/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "cpu",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 624.8692500084871,
"inferenceMs": 621.8220000009751,
"audioSeconds": 11,
"realTimeFactor": 0.05680629545531701,
"audioSecondsPerSecond": 17.689949857005303,
"activationPlacement": {
"audioEncoder": "cpu"
},
"predictionDevice": "cpu",
"predictionShape": [
1,
137
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": null,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 624.8692500084871,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3498557440,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"provider": "PyTorch CPU",
"placement": {
"cpu/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "cpu",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 606.7210420005722,
"inferenceMs": 603.6057080054889,
"audioSeconds": 11,
"realTimeFactor": 0.05515645836368838,
"audioSecondsPerSecond": 18.223817061551003,
"activationPlacement": {
"audioEncoder": "cpu"
},
"predictionDevice": "cpu",
"predictionShape": [
1,
137
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": null,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 606.7210420005722,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3545120768,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"provider": "PyTorch CPU",
"placement": {
"cpu/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "cpu",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 644.5386669947766,
"inferenceMs": 641.0184579872293,
"audioSeconds": 11,
"realTimeFactor": 0.058594424272252414,
"audioSecondsPerSecond": 17.16019228921977,
"activationPlacement": {
"audioEncoder": "cpu"
},
"predictionDevice": "cpu",
"predictionShape": [
1,
137
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": null,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 644.5386669947766,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3575840768,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"provider": "PyTorch CPU",
"placement": {
"cpu/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "cpu",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 624.8146249999991,
"inferenceMs": 621.5802910010098,
"audioSeconds": 11,
"realTimeFactor": 0.056801329545454464,
"audioSecondsPerSecond": 17.696828807562902,
"activationPlacement": {
"audioEncoder": "cpu"
},
"predictionDevice": "cpu",
"predictionShape": [
1,
137
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": null,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 624.8146249999991,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3606577152,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"provider": "PyTorch CPU",
"placement": {
"cpu/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "cpu",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 661.0943330015289,
"inferenceMs": 657.4017919920152,
"audioSeconds": 11,
"realTimeFactor": 0.06009948481832081,
"audioSecondsPerSecond": 16.732537291491905,
"activationPlacement": {
"audioEncoder": "cpu"
},
"predictionDevice": "cpu",
"predictionShape": [
1,
137
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": null,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 661.0943330015289,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3638689792,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 210.44583299953956,
"provider": "PyTorch CPU",
"placement": {
"cpu/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "cpu",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 693.7018329917919,
"inferenceMs": 688.487499995972,
"audioSeconds": 11,
"realTimeFactor": 0.0630638029992538,
"audioSecondsPerSecond": 15.977051144812876,
"activationPlacement": {
"audioEncoder": "cpu"
},
"predictionDevice": "cpu",
"predictionShape": [
1,
137
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": null,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 904.1476659913315,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3413966848,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 217.72091599996202,
"provider": "PyTorch CPU",
"placement": {
"cpu/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "cpu",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 681.679458997678,
"inferenceMs": 677.5467920087976,
"audioSeconds": 11,
"realTimeFactor": 0.06197085990887982,
"audioSecondsPerSecond": 16.235041077217836,
"activationPlacement": {
"audioEncoder": "cpu"
},
"predictionDevice": "cpu",
"predictionShape": [
1,
137
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": null,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 899.40037499764,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3417587712,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 486.05141599546187,
"provider": "Metal/MPS",
"placement": {
"mps:0/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "partial",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 357.6175000052899,
"inferenceMs": 210.01533399976324,
"audioSeconds": 11,
"realTimeFactor": 0.03251068181866272,
"audioSecondsPerSecond": 52.377127853018585,
"activationPlacement": {
"audioEncoder": "mps:0"
},
"predictionDevice": "mps:0",
"predictionShape": [
1,
137
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": 2249605120,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 843.6689160007518,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3228663808,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"provider": "Metal/MPS",
"placement": {
"mps:0/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "partial",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 596.502957996563,
"inferenceMs": 526.8656670086784,
"audioSeconds": 3.4,
"realTimeFactor": 0.17544204646957737,
"audioSecondsPerSecond": 6.453257847116456,
"activationPlacement": {
"audioEncoder": "mps:0"
},
"predictionDevice": "mps:0",
"predictionShape": [
1,
42
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": 2251259904,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 596.502957996563,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3228663808,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"provider": "Metal/MPS",
"placement": {
"mps:0/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "partial",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 430.63287500990555,
"inferenceMs": 401.0233330045594,
"audioSeconds": 11.5,
"realTimeFactor": 0.03744633695738309,
"audioSecondsPerSecond": 28.67663563074833,
"activationPlacement": {
"audioEncoder": "mps:0"
},
"predictionDevice": "mps:0",
"predictionShape": [
1,
143
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": 2252881920,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 430.63287500990555,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3228663808,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 0,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"provider": "Metal/MPS",
"placement": {
"mps:0/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "partial",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 462.3122909979429,
"inferenceMs": 432.5345420074882,
"audioSeconds": 2,
"realTimeFactor": 0.23115614549897145,
"audioSecondsPerSecond": 4.623908163999016,
"activationPlacement": {
"audioEncoder": "mps:0"
},
"predictionDevice": "mps:0",
"predictionShape": [
1,
25
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": 2253438976,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 462.3122909979429,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3228663808,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"provider": "Metal/MPS",
"placement": {
"mps:0/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "partial",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 50.30150001402944,
"inferenceMs": 46.69841699069366,
"audioSeconds": 11,
"realTimeFactor": 0.0045728636376390405,
"audioSecondsPerSecond": 235.55402321650746,
"activationPlacement": {
"audioEncoder": "mps:0"
},
"predictionDevice": "mps:0",
"predictionShape": [
1,
137
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": 2253438976,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 50.30150001402944,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3228663808,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"provider": "Metal/MPS",
"placement": {
"mps:0/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "partial",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 61.46966699452605,
"inferenceMs": 56.17979199450929,
"audioSeconds": 11,
"realTimeFactor": 0.005588151544956913,
"audioSecondsPerSecond": 195.79994174907378,
"activationPlacement": {
"audioEncoder": "mps:0"
},
"predictionDevice": "mps:0",
"predictionShape": [
1,
137
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": 2253438976,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 61.46966699452605,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3228663808,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"provider": "Metal/MPS",
"placement": {
"mps:0/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "partial",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 58.26783399970736,
"inferenceMs": 54.16037498798687,
"audioSeconds": 11,
"realTimeFactor": 0.005297075818155215,
"audioSecondsPerSecond": 203.10051402782705,
"activationPlacement": {
"audioEncoder": "mps:0"
},
"predictionDevice": "mps:0",
"predictionShape": [
1,
137
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": 2253438976,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 58.26783399970736,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3228663808,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"provider": "Metal/MPS",
"placement": {
"mps:0/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "partial",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 54.701875007594936,
"inferenceMs": 50.12783300480805,
"audioSeconds": 11,
"realTimeFactor": 0.004972897727963176,
"audioSecondsPerSecond": 219.4389691440467,
"activationPlacement": {
"audioEncoder": "mps:0"
},
"predictionDevice": "mps:0",
"predictionShape": [
1,
137
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": 2253438976,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 54.701875007594936,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3228663808,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"provider": "Metal/MPS",
"placement": {
"mps:0/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "partial",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 48.19945900817402,
"inferenceMs": 44.222750002518296,
"audioSeconds": 11,
"realTimeFactor": 0.004381769000743093,
"audioSecondsPerSecond": 248.74074993919638,
"activationPlacement": {
"audioEncoder": "mps:0"
},
"predictionDevice": "mps:0",
"predictionShape": [
1,
137
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": 2253438976,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 48.19945900817402,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3228663808,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 461.70749999873806,
"provider": "Metal/MPS",
"placement": {
"mps:0/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "partial",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 221.07533300004434,
"inferenceMs": 158.02233399881516,
"audioSeconds": 11,
"realTimeFactor": 0.020097757545458575,
"audioSecondsPerSecond": 69.61041342474081,
"activationPlacement": {
"audioEncoder": "mps:0"
},
"predictionDevice": "mps:0",
"predictionShape": [
1,
137
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": 2249605120,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 682.7828329987824,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3231989760,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"maximumConsecutiveRepeatedWords": 1,
"completeNativeTokenIds": true,
"silenceNoHallucination": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 440.8940830035135,
"provider": "Metal/MPS",
"placement": {
"mps:0/torch.float32": 502
},
"parameterDtype": "torch.float32",
"acceleration": "partial",
"accelerationDetails": "Neural audio encoder/head and logits on selected device; WAV decode/resample/token decoding on CPU; log-mel frontend on selected device",
"attentionImplementation": "sdpa",
"durationMs": 226.36016699834727,
"inferenceMs": 161.15945800265763,
"audioSeconds": 11,
"realTimeFactor": 0.02057819699984975,
"audioSecondsPerSecond": 68.25537971105986,
"activationPlacement": {
"audioEncoder": "mps:0"
},
"predictionDevice": "mps:0",
"predictionShape": [
1,
137
],
"decoding": "greedyCTC",
"mode": "offline",
"streaming": false,
"driverAllocatedMemoryBytes": 2249605120,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 667.2542500018608,
"artifactRevision": "1dd23f46f6d4e6eae2531982ba2e33220086dd97",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3228401664,
"rssScope": "Helper RSS sampled at100ms; includes runtime, not isolated tensor/VRAM minimum"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false
},
"runtime": "Not embedded",
"failure": "{'ok': False, 'error': 'Wrong parameter placement', 'code': 'runtime_error', 'traceback': 'Traceback (most recent call last):\\n File \"/Users/pengzhang/Desktop/hubters-webai/webai-js/localai-desktop/crates/desktopai/src/models/granite_speech5/runtime.py\", line 66, in <module>\\n with contextlib.redirect_stdout(sys.stderr):data=init(r)if r[\\'type\\']==\\'init\\'else{\\'alive\\':model is not None}if r[\\'type\\']==\\'health\\'else generate(r)if r[\\'type\\']==\\'generate\\'else(_ for _ in()).throw(InvalidInput(\\'Unknown operation\\'))\\n ~~~~^^^\\n File \"/Users/pengzhang/Desktop/hubters-webai/webai-js/localai-desktop/crates/desktopai/src/models/granite_speech5/runtime.py\", line 19, in init\\n start=time.perf_counter();processor=AutoProcessor.from_pretrained(r[\\'path\\'],trust_remote_code=False,local_files_only=True);model=AutoModelForCTC.from_pretrained(r[\\'path\\'],trust_remote_code=False,local_files_only=True,use_safetensors=True,dtype=dtype,attn_implementation=\\'sdpa\\').eval().requires_grad_(False).to(device);sync();assert all(p.device.type==device and p.dtype==dtype for p in model.parameters()),\\'Wrong parameter placement\\'\\n ~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\nAssertionError: Wrong parameter placement\\n'}"
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false
},
"runtime": "Not embedded",
"failure": "{'ok': False, 'error': 'Wrong parameter placement', 'code': 'runtime_error', 'traceback': 'Traceback (most recent call last):\\n File \"/Users/pengzhang/Desktop/hubters-webai/webai-js/localai-desktop/crates/desktopai/src/models/granite_speech5/runtime.py\", line 66, in <module>\\n with contextlib.redirect_stdout(sys.stderr):data=init(r)if r[\\'type\\']==\\'init\\'else{\\'alive\\':model is not None}if r[\\'type\\']==\\'health\\'else generate(r)if r[\\'type\\']==\\'generate\\'else(_ for _ in()).throw(InvalidInput(\\'Unknown operation\\'))\\n ~~~~^^^\\n File \"/Users/pengzhang/Desktop/hubters-webai/webai-js/localai-desktop/crates/desktopai/src/models/granite_speech5/runtime.py\", line 19, in init\\n start=time.perf_counter();processor=AutoProcessor.from_pretrained(r[\\'path\\'],trust_remote_code=False,local_files_only=True);model=AutoModelForCTC.from_pretrained(r[\\'path\\'],trust_remote_code=False,local_files_only=True,use_safetensors=True,dtype=dtype,attn_implementation=\\'sdpa\\').eval().requires_grad_(False).to(device);sync();assert all(p.device.type==device and p.dtype==dtype for p in model.parameters()),\\'Wrong parameter placement\\'\\n ~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\nAssertionError: Wrong parameter placement\\n'}"
}
]