VibeVoice ASR
Transcribe WAV recordings with speaker labels, segment timestamps and optional recognition context.
Desktop app
Run VibeVoice ASR on your computer
Download the desktop app to load this model, provide input and view results locally.
Public desktop downloads are coming soon.Device requirements
If your device falls short
MemoryAdvisory only: close other apps, reduce image/tile/token/batch settings or choose a smaller verified model/precision. CPU can need more RAM than GPU.
GPUIf the selected GPU runtime fails, existing CPU fallback may recover when enabled, but CPU has its own memory/runtime needs.
PlatformUse a compatible OS/architecture/runtime or another model. These estimates do not add hardware detection or block runs.
Benchmark evidence
Saved desktop measurements apply to the recorded hardware and runtime. Fallback cases report the device that actually ran; they do not prove GPU support on the requested device.
VibeVoice ASR Offline 7B
22 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | Task metric | |
|---|---|---|---|---|---|---|
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 3517 ms / 0 ms | 0.320 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 3525 ms / 0 ms | 0.320 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 3526 ms / 0 ms | 0.321 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 3541 ms / 0 ms | 0.322 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 3554 ms / 0 ms | 0.323 RTF |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 3598 ms / 0 ms | 0.327 RTF |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 3635 ms / 0 ms | 0.330 RTF |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 3635 ms / 0 ms | 0.330 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 3637 ms / 0 ms | 0.331 RTF |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 3725 ms / 0 ms | 0.339 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 3773 ms / 2927 ms | 0.343 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 3794 ms / 2964 ms | 0.345 RTF |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 3814 ms / 2556 ms | 0.347 RTF |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 3814 ms / 2408 ms | 0.347 RTF |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 3845 ms / 2563 ms | 0.350 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 3960 ms / 2969 ms | 0.360 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 12993 ms / 0 ms | 0.365 RTF |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 13135 ms / 0 ms | 0.369 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 2499 ms / 0 ms | 0.735 RTF |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 2656 ms / 0 ms | 0.781 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1844 ms / 0 ms | 0.922 RTF |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1922 ms / 0 ms | 0.961 RTF |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-07T10:41:57.359638+00:00
- Environment
- { "hardware": "Apple M3 Max", "os": "macos", "osVersion": "27.2", "runtime": "Transformers5.22.1/Torch2.10.0", "provisioning": "Task-private research venv from app-owned CPython3.13.18, no system Python", "concurrency": "Prior persistent archive services may transfer; no builds overlap timed cases" }
- Runtime
- transformers 5.22.1
- Artifact revision
- f22241c2062b3b25272bf117397e03d73381037a
- Fixture
{ "file": "model-samples/desktop/granite-speech5/granite-speech5-jfk.wav", "size": 3880888, "sha1": "6fb9dfd0ec18bb61a6af995c0df0cd32ee2afe34", "sha256": "c72e27f68334ae37e037ef819fa5aa1fc1552aa840b93e308df4ffe52317bedb", "url": "https://assets.localai.tools/samples/v1/files/c72e27f68334ae37e037ef819fa5aa1fc1552aa840b93e308df4ffe52317bedb/granite-speech5-jfk.wav" }- Rerun when
- Checkpoint, runtime, fixture checksum, frozen quality gate or relevant backend changes
Per-case quality & failure details
[
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"speakerCount": 1,
"expectedSpeakerCount": 1,
"speakerContent": {
"0": " And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country."
},
"plainTranscript": "And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country.",
"permutationInvariantSpeakerWer": null,
"maximumSpeakerWer": null,
"nativeStructure": true,
"timestampsOrderedAndInRange": true,
"maximumConsecutiveRepeatedWords": 1,
"completeGeneration": true,
"scope": "English authored/retained audio only; speaker labels not identities; timestamps checked against duration/order, not broad alignment accuracy"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 2926.805958006298,
"placement": {
"mps:0/torch.bfloat16": 901
},
"checkpointLoading": {
"missing_keys": [],
"unexpected_keys": [],
"mismatched_keys": [],
"error_msgs": []
},
"acceleration": "partial",
"provider": "Metal/MPS",
"durationMs": 3784.6821249986533,
"inferenceMs": 3772.9644580031163,
"audioSeconds": 11,
"realTimeFactor": 0.3429967689093742,
"tokensPerSecond": 14.312353217496277,
"outputTokens": 54,
"outputLimitReached": false,
"eosReached": true,
"activationPlacement": {
"audioProjector": [
{
"device": "mps:0",
"dtype": "torch.bfloat16"
}
],
"decoderLayer": [
{
"device": "mps:0",
"dtype": "torch.bfloat16"
}
],
"logits": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"totalMs": 6711.4880830049515,
"firstChunkMs": 1069.4975829974283,
"sampledPeakHelperRssBytes": 819281920,
"memoryScope": "250ms helper process RSS sampling; not peak or total Metal allocations",
"grade": "moderate",
"fallback": false,
"artifactRevision": "f22241c2062b3b25272bf117397e03d73381037a"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"speakerCount": 1,
"expectedSpeakerCount": 1,
"speakerContent": {
"0": " The quick brown fox jumps over the lazy dog."
},
"plainTranscript": "The quick brown fox jumps over the lazy dog.",
"permutationInvariantSpeakerWer": null,
"maximumSpeakerWer": null,
"nativeStructure": true,
"timestampsOrderedAndInRange": true,
"maximumConsecutiveRepeatedWords": 1,
"completeGeneration": true,
"scope": "English authored/retained audio only; speaker labels not identities; timestamps checked against duration/order, not broad alignment accuracy"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"mps:0/torch.bfloat16": 901
},
"checkpointLoading": {
"missing_keys": [],
"unexpected_keys": [],
"mismatched_keys": [],
"error_msgs": []
},
"acceleration": "partial",
"provider": "Metal/MPS",
"durationMs": 2506.4065420010593,
"inferenceMs": 2498.87633400067,
"audioSeconds": 3.4,
"realTimeFactor": 0.7349636276472559,
"tokensPerSecond": 14.806655093957156,
"outputTokens": 37,
"outputLimitReached": false,
"eosReached": true,
"activationPlacement": {
"audioProjector": [
{
"device": "mps:0",
"dtype": "torch.bfloat16"
}
],
"decoderLayer": [
{
"device": "mps:0",
"dtype": "torch.bfloat16"
}
],
"logits": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"totalMs": 2506.4065420010593,
"firstChunkMs": 631.0126249954919,
"sampledPeakHelperRssBytes": 875151360,
"memoryScope": "250ms helper process RSS sampling; not peak or total Metal allocations",
"grade": "moderate",
"fallback": false,
"artifactRevision": "f22241c2062b3b25272bf117397e03d73381037a"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.35,
"speakerCount": 2,
"expectedSpeakerCount": 2,
"speakerContent": {
"0": " And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country. And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country.",
"1": " The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog."
},
"plainTranscript": "And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country. The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country. The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog.",
"permutationInvariantSpeakerWer": 0,
"maximumSpeakerWer": 0.35,
"nativeStructure": true,
"timestampsOrderedAndInRange": true,
"maximumConsecutiveRepeatedWords": 1,
"completeGeneration": true,
"scope": "English authored/retained audio only; speaker labels not identities; timestamps checked against duration/order, not broad alignment accuracy"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"mps:0/torch.bfloat16": 901
},
"checkpointLoading": {
"missing_keys": [],
"unexpected_keys": [],
"mismatched_keys": [],
"error_msgs": []
},
"acceleration": "partial",
"provider": "Metal/MPS",
"durationMs": 13000.557375002245,
"inferenceMs": 12992.859042002237,
"audioSeconds": 35.644375,
"realTimeFactor": 0.36451358852560156,
"tokensPerSecond": 14.623417323760979,
"outputTokens": 190,
"outputLimitReached": false,
"eosReached": true,
"activationPlacement": {
"audioProjector": [
{
"device": "mps:0",
"dtype": "torch.bfloat16"
}
],
"decoderLayer": [
{
"device": "mps:0",
"dtype": "torch.bfloat16"
}
],
"logits": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"totalMs": 13000.557375002245,
"firstChunkMs": 1886.1577920033596,
"sampledPeakHelperRssBytes": 993476608,
"memoryScope": "250ms helper process RSS sampling; not peak or total Metal allocations",
"grade": "moderate",
"fallback": false,
"artifactRevision": "f22241c2062b3b25272bf117397e03d73381037a"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"speakerCount": 0,
"expectedSpeakerCount": 0,
"speakerContent": {},
"plainTranscript": "",
"permutationInvariantSpeakerWer": null,
"maximumSpeakerWer": null,
"nativeStructure": true,
"timestampsOrderedAndInRange": true,
"maximumConsecutiveRepeatedWords": 0,
"completeGeneration": true,
"scope": "English authored/retained audio only; speaker labels not identities; timestamps checked against duration/order, not broad alignment accuracy"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"mps:0/torch.bfloat16": 901
},
"checkpointLoading": {
"missing_keys": [],
"unexpected_keys": [],
"mismatched_keys": [],
"error_msgs": []
},
"acceleration": "partial",
"provider": "Metal/MPS",
"durationMs": 1846.8199160051881,
"inferenceMs": 1843.648125002801,
"audioSeconds": 2,
"realTimeFactor": 0.9218240625014005,
"tokensPerSecond": 14.102474136685112,
"outputTokens": 26,
"outputLimitReached": false,
"eosReached": true,
"activationPlacement": {
"audioProjector": [
{
"device": "mps:0",
"dtype": "torch.bfloat16"
}
],
"decoderLayer": [
{
"device": "mps:0",
"dtype": "torch.bfloat16"
}
],
"logits": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"totalMs": 1846.8199160051881,
"firstChunkMs": 561.7607079984737,
"sampledPeakHelperRssBytes": 1048346624,
"memoryScope": "250ms helper process RSS sampling; not peak or total Metal allocations",
"grade": "moderate",
"fallback": false,
"artifactRevision": "f22241c2062b3b25272bf117397e03d73381037a"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"speakerCount": 1,
"expectedSpeakerCount": 1,
"speakerContent": {
"0": " And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country."
},
"plainTranscript": "And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country.",
"permutationInvariantSpeakerWer": null,
"maximumSpeakerWer": null,
"nativeStructure": true,
"timestampsOrderedAndInRange": true,
"maximumConsecutiveRepeatedWords": 1,
"completeGeneration": true,
"scope": "English authored/retained audio only; speaker labels not identities; timestamps checked against duration/order, not broad alignment accuracy"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"mps:0/torch.bfloat16": 901
},
"checkpointLoading": {
"missing_keys": [],
"unexpected_keys": [],
"mismatched_keys": [],
"error_msgs": []
},
"acceleration": "partial",
"provider": "Metal/MPS",
"durationMs": 3546.2714999957825,
"inferenceMs": 3540.8004999990226,
"audioSeconds": 11,
"realTimeFactor": 0.3218909545453657,
"tokensPerSecond": 15.250788628168943,
"outputTokens": 54,
"outputLimitReached": false,
"eosReached": true,
"activationPlacement": {
"audioProjector": [
{
"device": "mps:0",
"dtype": "torch.bfloat16"
}
],
"decoderLayer": [
{
"device": "mps:0",
"dtype": "torch.bfloat16"
}
],
"logits": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"totalMs": 3546.2714999957825,
"firstChunkMs": 834.4090839964338,
"sampledPeakHelperRssBytes": 1084981248,
"memoryScope": "250ms helper process RSS sampling; not peak or total Metal allocations",
"grade": "moderate",
"fallback": false,
"artifactRevision": "f22241c2062b3b25272bf117397e03d73381037a"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"speakerCount": 1,
"expectedSpeakerCount": 1,
"speakerContent": {
"0": " And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country."
},
"plainTranscript": "And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country.",
"permutationInvariantSpeakerWer": null,
"maximumSpeakerWer": null,
"nativeStructure": true,
"timestampsOrderedAndInRange": true,
"maximumConsecutiveRepeatedWords": 1,
"completeGeneration": true,
"scope": "English authored/retained audio only; speaker labels not identities; timestamps checked against duration/order, not broad alignment accuracy"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"mps:0/torch.bfloat16": 901
},
"checkpointLoading": {
"missing_keys": [],
"unexpected_keys": [],
"mismatched_keys": [],
"error_msgs": []
},
"acceleration": "partial",
"provider": "Metal/MPS",
"durationMs": 3560.035041999072,
"inferenceMs": 3553.592500000377,
"audioSeconds": 11,
"realTimeFactor": 0.32305386363639793,
"tokensPerSecond": 15.195889793214688,
"outputTokens": 54,
"outputLimitReached": false,
"eosReached": true,
"activationPlacement": {
"audioProjector": [
{
"device": "mps:0",
"dtype": "torch.bfloat16"
}
],
"decoderLayer": [
{
"device": "mps:0",
"dtype": "torch.bfloat16"
}
],
"logits": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"totalMs": 3560.035041999072,
"firstChunkMs": 819.6622920004302,
"sampledPeakHelperRssBytes": 1113210880,
"memoryScope": "250ms helper process RSS sampling; not peak or total Metal allocations",
"grade": "moderate",
"fallback": false,
"artifactRevision": "f22241c2062b3b25272bf117397e03d73381037a"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"speakerCount": 1,
"expectedSpeakerCount": 1,
"speakerContent": {
"0": " And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country."
},
"plainTranscript": "And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country.",
"permutationInvariantSpeakerWer": null,
"maximumSpeakerWer": null,
"nativeStructure": true,
"timestampsOrderedAndInRange": true,
"maximumConsecutiveRepeatedWords": 1,
"completeGeneration": true,
"scope": "English authored/retained audio only; speaker labels not identities; timestamps checked against duration/order, not broad alignment accuracy"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"mps:0/torch.bfloat16": 901
},
"checkpointLoading": {
"missing_keys": [],
"unexpected_keys": [],
"mismatched_keys": [],
"error_msgs": []
},
"acceleration": "partial",
"provider": "Metal/MPS",
"durationMs": 3531.695416997536,
"inferenceMs": 3525.631834003434,
"audioSeconds": 11,
"realTimeFactor": 0.3205119849094031,
"tokensPerSecond": 15.316403567493827,
"outputTokens": 54,
"outputLimitReached": false,
"eosReached": true,
"activationPlacement": {
"audioProjector": [
{
"device": "mps:0",
"dtype": "torch.bfloat16"
}
],
"decoderLayer": [
{
"device": "mps:0",
"dtype": "torch.bfloat16"
}
],
"logits": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"totalMs": 3531.695416997536,
"firstChunkMs": 834.3855409984826,
"sampledPeakHelperRssBytes": 1140998144,
"memoryScope": "250ms helper process RSS sampling; not peak or total Metal allocations",
"grade": "moderate",
"fallback": false,
"artifactRevision": "f22241c2062b3b25272bf117397e03d73381037a"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"speakerCount": 1,
"expectedSpeakerCount": 1,
"speakerContent": {
"0": " And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country."
},
"plainTranscript": "And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country.",
"permutationInvariantSpeakerWer": null,
"maximumSpeakerWer": null,
"nativeStructure": true,
"timestampsOrderedAndInRange": true,
"maximumConsecutiveRepeatedWords": 1,
"completeGeneration": true,
"scope": "English authored/retained audio only; speaker labels not identities; timestamps checked against duration/order, not broad alignment accuracy"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"mps:0/torch.bfloat16": 901
},
"checkpointLoading": {
"missing_keys": [],
"unexpected_keys": [],
"mismatched_keys": [],
"error_msgs": []
},
"acceleration": "partial",
"provider": "Metal/MPS",
"durationMs": 3531.1235419940203,
"inferenceMs": 3524.844999999914,
"audioSeconds": 11,
"realTimeFactor": 0.32044045454544673,
"tokensPerSecond": 15.319822573758938,
"outputTokens": 54,
"outputLimitReached": false,
"eosReached": true,
"activationPlacement": {
"audioProjector": [
{
"device": "mps:0",
"dtype": "torch.bfloat16"
}
],
"decoderLayer": [
{
"device": "mps:0",
"dtype": "torch.bfloat16"
}
],
"logits": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"totalMs": 3531.1235419940203,
"firstChunkMs": 823.0626660006237,
"sampledPeakHelperRssBytes": 1168703488,
"memoryScope": "250ms helper process RSS sampling; not peak or total Metal allocations",
"grade": "moderate",
"fallback": false,
"artifactRevision": "f22241c2062b3b25272bf117397e03d73381037a"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"speakerCount": 1,
"expectedSpeakerCount": 1,
"speakerContent": {
"0": " And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country."
},
"plainTranscript": "And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country.",
"permutationInvariantSpeakerWer": null,
"maximumSpeakerWer": null,
"nativeStructure": true,
"timestampsOrderedAndInRange": true,
"maximumConsecutiveRepeatedWords": 1,
"completeGeneration": true,
"scope": "English authored/retained audio only; speaker labels not identities; timestamps checked against duration/order, not broad alignment accuracy"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"mps:0/torch.bfloat16": 901
},
"checkpointLoading": {
"missing_keys": [],
"unexpected_keys": [],
"mismatched_keys": [],
"error_msgs": []
},
"acceleration": "partial",
"provider": "Metal/MPS",
"durationMs": 3642.5747920002323,
"inferenceMs": 3637.209417000122,
"audioSeconds": 11,
"realTimeFactor": 0.33065540154546563,
"tokensPerSecond": 14.846546846493604,
"outputTokens": 54,
"outputLimitReached": false,
"eosReached": true,
"activationPlacement": {
"audioProjector": [
{
"device": "mps:0",
"dtype": "torch.bfloat16"
}
],
"decoderLayer": [
{
"device": "mps:0",
"dtype": "torch.bfloat16"
}
],
"logits": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"totalMs": 3642.5747920002323,
"firstChunkMs": 839.9297910000314,
"sampledPeakHelperRssBytes": 1196425216,
"memoryScope": "250ms helper process RSS sampling; not peak or total Metal allocations",
"grade": "moderate",
"fallback": false,
"artifactRevision": "f22241c2062b3b25272bf117397e03d73381037a"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"speakerCount": 1,
"expectedSpeakerCount": 1,
"speakerContent": {
"0": " And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country."
},
"plainTranscript": "And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country.",
"permutationInvariantSpeakerWer": null,
"maximumSpeakerWer": null,
"nativeStructure": true,
"timestampsOrderedAndInRange": true,
"maximumConsecutiveRepeatedWords": 1,
"completeGeneration": true,
"scope": "English authored/retained audio only; speaker labels not identities; timestamps checked against duration/order, not broad alignment accuracy"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 2968.7941250012955,
"placement": {
"mps:0/torch.bfloat16": 901
},
"checkpointLoading": {
"missing_keys": [],
"unexpected_keys": [],
"mismatched_keys": [],
"error_msgs": []
},
"acceleration": "partial",
"provider": "Metal/MPS",
"durationMs": 3970.5451670015464,
"inferenceMs": 3959.504874997947,
"audioSeconds": 11,
"realTimeFactor": 0.359954988636177,
"tokensPerSecond": 13.638068825468487,
"outputTokens": 54,
"outputLimitReached": false,
"eosReached": true,
"activationPlacement": {
"audioProjector": [
{
"device": "mps:0",
"dtype": "torch.bfloat16"
}
],
"decoderLayer": [
{
"device": "mps:0",
"dtype": "torch.bfloat16"
}
],
"logits": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"totalMs": 6939.339292002842,
"firstChunkMs": 990.1293330040062,
"sampledPeakHelperRssBytes": 818937856,
"memoryScope": "250ms helper process RSS sampling; not peak or total Metal allocations",
"grade": "moderate",
"fallback": false,
"artifactRevision": "f22241c2062b3b25272bf117397e03d73381037a"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"speakerCount": 1,
"expectedSpeakerCount": 1,
"speakerContent": {
"0": " And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country."
},
"plainTranscript": "And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country.",
"permutationInvariantSpeakerWer": null,
"maximumSpeakerWer": null,
"nativeStructure": true,
"timestampsOrderedAndInRange": true,
"maximumConsecutiveRepeatedWords": 1,
"completeGeneration": true,
"scope": "English authored/retained audio only; speaker labels not identities; timestamps checked against duration/order, not broad alignment accuracy"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 2964.385500003118,
"placement": {
"mps:0/torch.bfloat16": 901
},
"checkpointLoading": {
"missing_keys": [],
"unexpected_keys": [],
"mismatched_keys": [],
"error_msgs": []
},
"acceleration": "partial",
"provider": "Metal/MPS",
"durationMs": 3806.3306660042144,
"inferenceMs": 3794.007958997099,
"audioSeconds": 11,
"realTimeFactor": 0.34490981445428176,
"tokensPerSecond": 14.232969615138671,
"outputTokens": 54,
"outputLimitReached": false,
"eosReached": true,
"activationPlacement": {
"audioProjector": [
{
"device": "mps:0",
"dtype": "torch.bfloat16"
}
],
"decoderLayer": [
{
"device": "mps:0",
"dtype": "torch.bfloat16"
}
],
"logits": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"totalMs": 6770.7161660073325,
"firstChunkMs": 1026.9616250006948,
"sampledPeakHelperRssBytes": 819544064,
"memoryScope": "250ms helper process RSS sampling; not peak or total Metal allocations",
"grade": "moderate",
"fallback": false,
"artifactRevision": "f22241c2062b3b25272bf117397e03d73381037a"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"speakerCount": 1,
"expectedSpeakerCount": 1,
"speakerContent": {
"0": " And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country."
},
"plainTranscript": "And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country.",
"permutationInvariantSpeakerWer": null,
"maximumSpeakerWer": null,
"nativeStructure": true,
"timestampsOrderedAndInRange": true,
"maximumConsecutiveRepeatedWords": 1,
"completeGeneration": true,
"scope": "English authored/retained audio only; speaker labels not identities; timestamps checked against duration/order, not broad alignment accuracy"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 2556.4549999980954,
"placement": {
"mps:0/torch.float16": 901
},
"checkpointLoading": {
"missing_keys": [],
"unexpected_keys": [],
"mismatched_keys": [],
"error_msgs": []
},
"acceleration": "partial",
"provider": "Metal/MPS",
"durationMs": 3825.6102919986006,
"inferenceMs": 3813.6481670007925,
"audioSeconds": 11,
"realTimeFactor": 0.34669528790916293,
"tokensPerSecond": 14.15967012040017,
"outputTokens": 54,
"outputLimitReached": false,
"eosReached": true,
"activationPlacement": {
"audioProjector": [
{
"device": "mps:0",
"dtype": "torch.float16"
}
],
"decoderLayer": [
{
"device": "mps:0",
"dtype": "torch.float16"
}
],
"logits": {
"device": "mps:0",
"dtype": "torch.float16"
}
},
"totalMs": 6382.065291996696,
"firstChunkMs": 1028.946124999493,
"sampledPeakHelperRssBytes": 27057307648,
"memoryScope": "250ms helper process RSS sampling; not peak or total Metal allocations",
"grade": "moderate",
"fallback": false,
"artifactRevision": "f22241c2062b3b25272bf117397e03d73381037a"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"speakerCount": 1,
"expectedSpeakerCount": 1,
"speakerContent": {
"0": " The quick brown fox jumps over the lazy dog."
},
"plainTranscript": "The quick brown fox jumps over the lazy dog.",
"permutationInvariantSpeakerWer": null,
"maximumSpeakerWer": null,
"nativeStructure": true,
"timestampsOrderedAndInRange": true,
"maximumConsecutiveRepeatedWords": 1,
"completeGeneration": true,
"scope": "English authored/retained audio only; speaker labels not identities; timestamps checked against duration/order, not broad alignment accuracy"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float16": 901
},
"checkpointLoading": {
"missing_keys": [],
"unexpected_keys": [],
"mismatched_keys": [],
"error_msgs": []
},
"acceleration": "partial",
"provider": "Metal/MPS",
"durationMs": 2659.1533749960945,
"inferenceMs": 2655.5562499997905,
"audioSeconds": 3.4,
"realTimeFactor": 0.7810459558822913,
"tokensPerSecond": 13.933050749726322,
"outputTokens": 37,
"outputLimitReached": false,
"eosReached": true,
"activationPlacement": {
"audioProjector": [
{
"device": "mps:0",
"dtype": "torch.float16"
}
],
"decoderLayer": [
{
"device": "mps:0",
"dtype": "torch.float16"
}
],
"logits": {
"device": "mps:0",
"dtype": "torch.float16"
}
},
"totalMs": 2659.1533749960945,
"firstChunkMs": 654.0704590006499,
"sampledPeakHelperRssBytes": 27057307648,
"memoryScope": "250ms helper process RSS sampling; not peak or total Metal allocations",
"grade": "moderate",
"fallback": false,
"artifactRevision": "f22241c2062b3b25272bf117397e03d73381037a"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.35,
"speakerCount": 2,
"expectedSpeakerCount": 2,
"speakerContent": {
"0": " And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country. And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country.",
"1": " The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog."
},
"plainTranscript": "And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country. The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog. And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country. The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog.",
"permutationInvariantSpeakerWer": 0,
"maximumSpeakerWer": 0.35,
"nativeStructure": true,
"timestampsOrderedAndInRange": true,
"maximumConsecutiveRepeatedWords": 1,
"completeGeneration": true,
"scope": "English authored/retained audio only; speaker labels not identities; timestamps checked against duration/order, not broad alignment accuracy"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float16": 901
},
"checkpointLoading": {
"missing_keys": [],
"unexpected_keys": [],
"mismatched_keys": [],
"error_msgs": []
},
"acceleration": "partial",
"provider": "Metal/MPS",
"durationMs": 13142.771042003005,
"inferenceMs": 13135.461042002134,
"audioSeconds": 35.644375,
"realTimeFactor": 0.36851427587107743,
"tokensPerSecond": 14.388531882942742,
"outputTokens": 189,
"outputLimitReached": false,
"eosReached": true,
"activationPlacement": {
"audioProjector": [
{
"device": "mps:0",
"dtype": "torch.float16"
}
],
"decoderLayer": [
{
"device": "mps:0",
"dtype": "torch.float16"
}
],
"logits": {
"device": "mps:0",
"dtype": "torch.float16"
}
},
"totalMs": 13142.771042003005,
"firstChunkMs": 1912.1562500004075,
"sampledPeakHelperRssBytes": 27057307648,
"memoryScope": "250ms helper process RSS sampling; not peak or total Metal allocations",
"grade": "moderate",
"fallback": false,
"artifactRevision": "f22241c2062b3b25272bf117397e03d73381037a"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"speakerCount": 0,
"expectedSpeakerCount": 0,
"speakerContent": {},
"plainTranscript": "",
"permutationInvariantSpeakerWer": null,
"maximumSpeakerWer": null,
"nativeStructure": true,
"timestampsOrderedAndInRange": true,
"maximumConsecutiveRepeatedWords": 0,
"completeGeneration": true,
"scope": "English authored/retained audio only; speaker labels not identities; timestamps checked against duration/order, not broad alignment accuracy"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float16": 901
},
"checkpointLoading": {
"missing_keys": [],
"unexpected_keys": [],
"mismatched_keys": [],
"error_msgs": []
},
"acceleration": "partial",
"provider": "Metal/MPS",
"durationMs": 1927.0129170035943,
"inferenceMs": 1921.870416001184,
"audioSeconds": 2,
"realTimeFactor": 0.960935208000592,
"tokensPerSecond": 13.528487552297065,
"outputTokens": 26,
"outputLimitReached": false,
"eosReached": true,
"activationPlacement": {
"audioProjector": [
{
"device": "mps:0",
"dtype": "torch.float16"
}
],
"decoderLayer": [
{
"device": "mps:0",
"dtype": "torch.float16"
}
],
"logits": {
"device": "mps:0",
"dtype": "torch.float16"
}
},
"totalMs": 1927.0129170035943,
"firstChunkMs": 578.177000003052,
"sampledPeakHelperRssBytes": 27057307648,
"memoryScope": "250ms helper process RSS sampling; not peak or total Metal allocations",
"grade": "moderate",
"fallback": false,
"artifactRevision": "f22241c2062b3b25272bf117397e03d73381037a"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"speakerCount": 1,
"expectedSpeakerCount": 1,
"speakerContent": {
"0": " And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country."
},
"plainTranscript": "And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country.",
"permutationInvariantSpeakerWer": null,
"maximumSpeakerWer": null,
"nativeStructure": true,
"timestampsOrderedAndInRange": true,
"maximumConsecutiveRepeatedWords": 1,
"completeGeneration": true,
"scope": "English authored/retained audio only; speaker labels not identities; timestamps checked against duration/order, not broad alignment accuracy"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float16": 901
},
"checkpointLoading": {
"missing_keys": [],
"unexpected_keys": [],
"mismatched_keys": [],
"error_msgs": []
},
"acceleration": "partial",
"provider": "Metal/MPS",
"durationMs": 3640.9709170038695,
"inferenceMs": 3635.1297910005087,
"audioSeconds": 11,
"realTimeFactor": 0.3304663446364099,
"tokensPerSecond": 14.855040426255977,
"outputTokens": 54,
"outputLimitReached": false,
"eosReached": true,
"activationPlacement": {
"audioProjector": [
{
"device": "mps:0",
"dtype": "torch.float16"
}
],
"decoderLayer": [
{
"device": "mps:0",
"dtype": "torch.float16"
}
],
"logits": {
"device": "mps:0",
"dtype": "torch.float16"
}
},
"totalMs": 3640.9709170038695,
"firstChunkMs": 850.1699999978882,
"sampledPeakHelperRssBytes": 27057307648,
"memoryScope": "250ms helper process RSS sampling; not peak or total Metal allocations",
"grade": "moderate",
"fallback": false,
"artifactRevision": "f22241c2062b3b25272bf117397e03d73381037a"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"speakerCount": 1,
"expectedSpeakerCount": 1,
"speakerContent": {
"0": " And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country."
},
"plainTranscript": "And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country.",
"permutationInvariantSpeakerWer": null,
"maximumSpeakerWer": null,
"nativeStructure": true,
"timestampsOrderedAndInRange": true,
"maximumConsecutiveRepeatedWords": 1,
"completeGeneration": true,
"scope": "English authored/retained audio only; speaker labels not identities; timestamps checked against duration/order, not broad alignment accuracy"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float16": 901
},
"checkpointLoading": {
"missing_keys": [],
"unexpected_keys": [],
"mismatched_keys": [],
"error_msgs": []
},
"acceleration": "partial",
"provider": "Metal/MPS",
"durationMs": 3730.5401250050636,
"inferenceMs": 3724.88941699703,
"audioSeconds": 11,
"realTimeFactor": 0.3386263106360936,
"tokensPerSecond": 14.497074665785457,
"outputTokens": 54,
"outputLimitReached": false,
"eosReached": true,
"activationPlacement": {
"audioProjector": [
{
"device": "mps:0",
"dtype": "torch.float16"
}
],
"decoderLayer": [
{
"device": "mps:0",
"dtype": "torch.float16"
}
],
"logits": {
"device": "mps:0",
"dtype": "torch.float16"
}
},
"totalMs": 3730.5401250050636,
"firstChunkMs": 900.3970839985413,
"sampledPeakHelperRssBytes": 27057307648,
"memoryScope": "250ms helper process RSS sampling; not peak or total Metal allocations",
"grade": "moderate",
"fallback": false,
"artifactRevision": "f22241c2062b3b25272bf117397e03d73381037a"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"speakerCount": 1,
"expectedSpeakerCount": 1,
"speakerContent": {
"0": " And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country."
},
"plainTranscript": "And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country.",
"permutationInvariantSpeakerWer": null,
"maximumSpeakerWer": null,
"nativeStructure": true,
"timestampsOrderedAndInRange": true,
"maximumConsecutiveRepeatedWords": 1,
"completeGeneration": true,
"scope": "English authored/retained audio only; speaker labels not identities; timestamps checked against duration/order, not broad alignment accuracy"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float16": 901
},
"checkpointLoading": {
"missing_keys": [],
"unexpected_keys": [],
"mismatched_keys": [],
"error_msgs": []
},
"acceleration": "partial",
"provider": "Metal/MPS",
"durationMs": 3642.1954590041423,
"inferenceMs": 3635.180417004449,
"audioSeconds": 11,
"realTimeFactor": 0.3304709470004044,
"tokensPerSecond": 14.854833544822629,
"outputTokens": 54,
"outputLimitReached": false,
"eosReached": true,
"activationPlacement": {
"audioProjector": [
{
"device": "mps:0",
"dtype": "torch.float16"
}
],
"decoderLayer": [
{
"device": "mps:0",
"dtype": "torch.float16"
}
],
"logits": {
"device": "mps:0",
"dtype": "torch.float16"
}
},
"totalMs": 3642.1954590041423,
"firstChunkMs": 885.0290830014274,
"sampledPeakHelperRssBytes": 27057307648,
"memoryScope": "250ms helper process RSS sampling; not peak or total Metal allocations",
"grade": "moderate",
"fallback": false,
"artifactRevision": "f22241c2062b3b25272bf117397e03d73381037a"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"speakerCount": 1,
"expectedSpeakerCount": 1,
"speakerContent": {
"0": " And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country."
},
"plainTranscript": "And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country.",
"permutationInvariantSpeakerWer": null,
"maximumSpeakerWer": null,
"nativeStructure": true,
"timestampsOrderedAndInRange": true,
"maximumConsecutiveRepeatedWords": 1,
"completeGeneration": true,
"scope": "English authored/retained audio only; speaker labels not identities; timestamps checked against duration/order, not broad alignment accuracy"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float16": 901
},
"checkpointLoading": {
"missing_keys": [],
"unexpected_keys": [],
"mismatched_keys": [],
"error_msgs": []
},
"acceleration": "partial",
"provider": "Metal/MPS",
"durationMs": 3602.3030419964925,
"inferenceMs": 3597.9209170036484,
"audioSeconds": 11,
"realTimeFactor": 0.3270837197276044,
"tokensPerSecond": 15.008667851702324,
"outputTokens": 54,
"outputLimitReached": false,
"eosReached": true,
"activationPlacement": {
"audioProjector": [
{
"device": "mps:0",
"dtype": "torch.float16"
}
],
"decoderLayer": [
{
"device": "mps:0",
"dtype": "torch.float16"
}
],
"logits": {
"device": "mps:0",
"dtype": "torch.float16"
}
},
"totalMs": 3602.3030419964925,
"firstChunkMs": 890.9544999987702,
"sampledPeakHelperRssBytes": 27057307648,
"memoryScope": "250ms helper process RSS sampling; not peak or total Metal allocations",
"grade": "moderate",
"fallback": false,
"artifactRevision": "f22241c2062b3b25272bf117397e03d73381037a"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"speakerCount": 1,
"expectedSpeakerCount": 1,
"speakerContent": {
"0": " And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country."
},
"plainTranscript": "And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country.",
"permutationInvariantSpeakerWer": null,
"maximumSpeakerWer": null,
"nativeStructure": true,
"timestampsOrderedAndInRange": true,
"maximumConsecutiveRepeatedWords": 1,
"completeGeneration": true,
"scope": "English authored/retained audio only; speaker labels not identities; timestamps checked against duration/order, not broad alignment accuracy"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float16": 901
},
"checkpointLoading": {
"missing_keys": [],
"unexpected_keys": [],
"mismatched_keys": [],
"error_msgs": []
},
"acceleration": "partial",
"provider": "Metal/MPS",
"durationMs": 3521.5032089981833,
"inferenceMs": 3517.0416250039125,
"audioSeconds": 11,
"realTimeFactor": 0.3197310568185375,
"tokensPerSecond": 15.353813163908724,
"outputTokens": 54,
"outputLimitReached": false,
"eosReached": true,
"activationPlacement": {
"audioProjector": [
{
"device": "mps:0",
"dtype": "torch.float16"
}
],
"decoderLayer": [
{
"device": "mps:0",
"dtype": "torch.float16"
}
],
"logits": {
"device": "mps:0",
"dtype": "torch.float16"
}
},
"totalMs": 3521.5032089981833,
"firstChunkMs": 861.2561250047293,
"sampledPeakHelperRssBytes": 27057307648,
"memoryScope": "250ms helper process RSS sampling; not peak or total Metal allocations",
"grade": "moderate",
"fallback": false,
"artifactRevision": "f22241c2062b3b25272bf117397e03d73381037a"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"speakerCount": 1,
"expectedSpeakerCount": 1,
"speakerContent": {
"0": " And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country."
},
"plainTranscript": "And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country.",
"permutationInvariantSpeakerWer": null,
"maximumSpeakerWer": null,
"nativeStructure": true,
"timestampsOrderedAndInRange": true,
"maximumConsecutiveRepeatedWords": 1,
"completeGeneration": true,
"scope": "English authored/retained audio only; speaker labels not identities; timestamps checked against duration/order, not broad alignment accuracy"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 2562.756500003161,
"placement": {
"mps:0/torch.float16": 901
},
"checkpointLoading": {
"missing_keys": [],
"unexpected_keys": [],
"mismatched_keys": [],
"error_msgs": []
},
"acceleration": "partial",
"provider": "Metal/MPS",
"durationMs": 3857.920959002513,
"inferenceMs": 3844.768249997287,
"audioSeconds": 11,
"realTimeFactor": 0.3495243863633897,
"tokensPerSecond": 14.045059803029247,
"outputTokens": 54,
"outputLimitReached": false,
"eosReached": true,
"activationPlacement": {
"audioProjector": [
{
"device": "mps:0",
"dtype": "torch.float16"
}
],
"decoderLayer": [
{
"device": "mps:0",
"dtype": "torch.float16"
}
],
"logits": {
"device": "mps:0",
"dtype": "torch.float16"
}
},
"totalMs": 6420.677459005674,
"firstChunkMs": 1104.1120420049992,
"sampledPeakHelperRssBytes": 30220124160,
"memoryScope": "250ms helper process RSS sampling; not peak or total Metal allocations",
"grade": "moderate",
"fallback": false,
"artifactRevision": "f22241c2062b3b25272bf117397e03d73381037a"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"maximumWordErrorRate": 0.25,
"speakerCount": 1,
"expectedSpeakerCount": 1,
"speakerContent": {
"0": " And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country."
},
"plainTranscript": "And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country.",
"permutationInvariantSpeakerWer": null,
"maximumSpeakerWer": null,
"nativeStructure": true,
"timestampsOrderedAndInRange": true,
"maximumConsecutiveRepeatedWords": 1,
"completeGeneration": true,
"scope": "English authored/retained audio only; speaker labels not identities; timestamps checked against duration/order, not broad alignment accuracy"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 2408.265499994741,
"placement": {
"mps:0/torch.float16": 901
},
"checkpointLoading": {
"missing_keys": [],
"unexpected_keys": [],
"mismatched_keys": [],
"error_msgs": []
},
"acceleration": "partial",
"provider": "Metal/MPS",
"durationMs": 3852.8387089972966,
"inferenceMs": 3813.9363749942277,
"audioSeconds": 11,
"realTimeFactor": 0.3467214886358389,
"tokensPerSecond": 14.158600115630332,
"outputTokens": 54,
"outputLimitReached": false,
"eosReached": true,
"activationPlacement": {
"audioProjector": [
{
"device": "mps:0",
"dtype": "torch.float16"
}
],
"decoderLayer": [
{
"device": "mps:0",
"dtype": "torch.float16"
}
],
"logits": {
"device": "mps:0",
"dtype": "torch.float16"
}
},
"totalMs": 6261.1042089920375,
"firstChunkMs": 1112.3816249964875,
"sampledPeakHelperRssBytes": 24588025856,
"memoryScope": "250ms helper process RSS sampling; not peak or total Metal allocations",
"grade": "moderate",
"fallback": false,
"artifactRevision": "f22241c2062b3b25272bf117397e03d73381037a"
}
}
]Release validation still required
- Complete public backup and forced model/runtime source recovery pending.