Qwen3-TTS
Generate local speech with nine preset voices using Qwen3-TTS 0.6B CustomVoice. Supply text and choose a voice; returns complete 24 kHz mono WAV audio.
Desktop app
Run Qwen3-TTS on your computer
Download the desktop app to load this model, provide input and view results locally.
Public desktop downloads are coming soon.Device requirements
If your device falls short
MemoryAdvisory only: close other apps, reduce image/tile/token/batch settings or choose a smaller verified model/precision. CPU can need more RAM than GPU.
GPUIf the selected GPU runtime fails, existing CPU fallback may recover when enabled, but CPU has its own memory/runtime needs.
PlatformUse a compatible OS/architecture/runtime or another model. These estimates do not add hardware detection or block runs.
Benchmark evidence
Saved desktop measurements apply to the recorded hardware and runtime. Fallback cases report the device that actually ran; they do not prove GPU support on the requested device.
Qwen3-TTS Desktop — 0.6B CustomVoice
42 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | Task metric | |
|---|---|---|---|---|---|---|
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 6744 ms / 0 ms | 2.107 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 7471 ms / 0 ms | 2.224 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 7516 ms / 0 ms | 2.237 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 7524 ms / 0 ms | 2.239 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 7563 ms / 0 ms | 2.251 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 7670 ms / 0 ms | 2.283 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 7811 ms / 0 ms | 2.325 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 12886 ms / 993 ms | 2.334 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 8609 ms / 0 ms | 2.339 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 13695 ms / 1058 ms | 2.481 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 13717 ms / 1063 ms | 2.485 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 4717 ms / 0 ms | 1.404 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 3950 ms / 0 ms | 1.411 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 3962 ms / 0 ms | 1.415 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 3981 ms / 0 ms | 1.422 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 4015 ms / 0 ms | 1.434 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 4018 ms / 0 ms | 1.435 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 4036 ms / 0 ms | 1.441 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 4040 ms / 0 ms | 1.443 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 4065 ms / 628 ms | 1.452 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 4301 ms / 0 ms | 1.453 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 4088 ms / 621 ms | 1.460 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 4321 ms / 0 ms | 1.460 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 4925 ms / 0 ms | 1.466 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 4136 ms / 0 ms | 1.477 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 4168 ms / 651 ms | 1.489 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 4438 ms / 944 ms | 1.585 RTF |
FP16 | mps | Failed | — | Not recorded | — / — | — |
FP32 | mps | Failed | — | macos · aarch64 | 7210 ms / 1158 ms | 2.504 RTF |
BF16 | mps | Failed | — | macos · aarch64 | 12040 ms / 0 ms | 2.062 RTF |
BF16 | mps | Failed | — | macos · aarch64 | 11604 ms / 0 ms | 1.987 RTF |
BF16 | mps | Failed | — | macos · aarch64 | 11943 ms / 0 ms | 2.045 RTF |
BF16 | mps | Failed | — | macos · aarch64 | 11773 ms / 0 ms | 2.016 RTF |
BF16 | mps | Failed | — | macos · aarch64 | 12315 ms / 0 ms | 2.109 RTF |
BF16 | mps | Failed | — | macos · aarch64 | 11970 ms / 0 ms | 2.050 RTF |
BF16 | mps | Failed | — | macos · aarch64 | 22091 ms / 1044 ms | 2.321 RTF |
FP32 | mps | Failed | — | macos · aarch64 | 12091 ms / 1154 ms | 3.977 RTF |
FP32 | mps | Failed | — | macos · aarch64 | 12926 ms / 0 ms | 3.438 RTF |
FP32 | mps | Failed | — | macos · aarch64 | 19258 ms / 0 ms | 3.439 RTF |
FP32 | mps | Failed | — | macos · aarch64 | 7018 ms / 0 ms | 3.509 RTF |
FP32 | mps | Failed | — | macos · aarch64 | 6735 ms / 0 ms | 3.367 RTF |
FP32 | mps | Failed | — | macos · aarch64 | 7607 ms / 0 ms | 2.113 RTF |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-04T11:42:29.466947+00:00
- Environment
- { "hardware": "Apple M3 Max", "os": "macOS-27.2-arm64-arm-64bit-Mach-O", "runtime": "qwen-tts0.1.1/Transformers4.57.3/Torch2.10.0", "interpreterScope": "Existing app-owned pinned private Qwen probe venv; normal per-model provisioning tested separately" }
- Runtime
- qwen-tts 0.1.1
- Artifact revision
- 85e237c12c027371202489a0ec509ded67b5e4b5
- Fixture
{ "file": "model-samples/desktop/qwen3-tts/morning.txt", "size": 39, "sha1": "e1c3512efd9e8669d9d3027e2b20cdea5e875d6e", "sha256": "1335d38acab68b7e9bb398ef2ceaf8249a48ec10a0788514a7d91e14f821d0d5", "url": "https://assets.localai.tools/samples/v1/files/1335d38acab68b7e9bb398ef2ceaf8249a48ec10a0788514a7d91e14f821d0d5/morning.txt" }- Rerun when
- artifact revision/checksum; qwen-tts/Transformers/Torch versions, preprocessing or sampling; OS/device/backend behavior; fixture text, voice/language selections or independent ASR/quality/performance gates
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"transcript": "Good morning. The train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:46:09.519176+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 944.2220409982838,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 4443.348375003552,
"inferenceMs": 4438.48004200845,
"realTimeFactor": 1.5851714435744466,
"audioSeconds": 2.8,
"seed": 42,
"eosReached": true,
"generatedTokens": 36,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.08865950256586075,
"peak": 0.5263413786888123,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 5387.570416001836,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.6308465901612967,
"peakSampledHelperRssBytes": 5713543168,
"resourceMeasurementScope": "200ms sampled helper RSS, process lifetime high-water sample; not a validated minimum or GPU peak."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"transcript": "Please save this file before closing the window.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:46:09.608071+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 4925.707500020508,
"inferenceMs": 4924.814541009255,
"realTimeFactor": 1.4657186133956115,
"audioSeconds": 3.36,
"seed": 42,
"eosReached": true,
"generatedTokens": 43,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.17639891803264618,
"peak": 0.7897162437438965,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 4925.707500020508,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.6822591941322986,
"peakSampledHelperRssBytes": 5826412544,
"resourceMeasurementScope": "200ms sampled helper RSS, process lifetime high-water sample; not a validated minimum or GPU peak."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"transcript": "The blue box contains three small books.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:46:09.690041+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 4302.010625018738,
"inferenceMs": 4301.217416999862,
"realTimeFactor": 1.453113992229683,
"audioSeconds": 2.96,
"seed": 42,
"eosReached": true,
"generatedTokens": 38,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 8,
"maxNewTokens": 512,
"rms": 0.10349362343549728,
"peak": 0.7155910134315491,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 4302.010625018738,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.6881772561184845,
"peakSampledHelperRssBytes": 5831639040,
"resourceMeasurementScope": "200ms sampled helper RSS, process lifetime high-water sample; not a validated minimum or GPU peak."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"transcript": "Good morning. The train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:46:09.519176+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 4136.532708012965,
"inferenceMs": 4135.797833994729,
"realTimeFactor": 1.4770706549981178,
"audioSeconds": 2.8,
"seed": 42,
"eosReached": true,
"generatedTokens": 36,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.08865950256586075,
"peak": 0.5263413786888123,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 4136.532708012965,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.677015684128715,
"peakSampledHelperRssBytes": 5831639040,
"resourceMeasurementScope": "200ms sampled helper RSS, process lifetime high-water sample; not a validated minimum or GPU peak."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"transcript": "Good morning. The train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:46:09.519176+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 4036.3774170109536,
"inferenceMs": 4035.6358750141226,
"realTimeFactor": 1.441298526790758,
"audioSeconds": 2.8,
"seed": 42,
"eosReached": true,
"generatedTokens": 36,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.08865950256586075,
"peak": 0.5263413786888123,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 4036.3774170109536,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.6938187900785775,
"peakSampledHelperRssBytes": 5831639040,
"resourceMeasurementScope": "200ms sampled helper RSS, process lifetime high-water sample; not a validated minimum or GPU peak."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"transcript": "Good morning. The train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:46:09.519176+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 650.8436670119409,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 4172.994749998907,
"inferenceMs": 4168.060249998234,
"realTimeFactor": 1.4885929464279408,
"audioSeconds": 2.8,
"seed": 42,
"eosReached": true,
"generatedTokens": 36,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.08865950256586075,
"peak": 0.5263413786888123,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 4823.8384170108475,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.6717753180274172,
"peakSampledHelperRssBytes": 5464571904,
"resourceMeasurementScope": "200ms sampled helper RSS, process lifetime high-water sample; not a validated minimum or GPU peak."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"transcript": "Please save this file before closing the window.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:46:09.608071+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 4718.140917015262,
"inferenceMs": 4717.260334000457,
"realTimeFactor": 1.4039465279763266,
"audioSeconds": 3.36,
"seed": 42,
"eosReached": true,
"generatedTokens": 43,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.17639891803264618,
"peak": 0.7897162437438965,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 4718.140917015262,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.7122778396990787,
"peakSampledHelperRssBytes": 5517508608,
"resourceMeasurementScope": "200ms sampled helper RSS, process lifetime high-water sample; not a validated minimum or GPU peak."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"transcript": "The blue box contains three small books.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:46:09.690041+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 4322.318291990086,
"inferenceMs": 4321.41854200745,
"realTimeFactor": 1.4599386966241386,
"audioSeconds": 2.96,
"seed": 42,
"eosReached": true,
"generatedTokens": 38,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 8,
"maxNewTokens": 512,
"rms": 0.10349362343549728,
"peak": 0.7155910134315491,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 4322.318291990086,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.6849602673813161,
"peakSampledHelperRssBytes": 5835358208,
"resourceMeasurementScope": "200ms sampled helper RSS, process lifetime high-water sample; not a validated minimum or GPU peak."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"transcript": "Good morning. The train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:46:09.519176+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 3950.286041974323,
"inferenceMs": 3949.5460839825682,
"realTimeFactor": 1.4105521728509174,
"audioSeconds": 2.8,
"seed": 42,
"eosReached": true,
"generatedTokens": 36,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.08865950256586075,
"peak": 0.5263413786888123,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 3950.286041974323,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.7089422279070077,
"peakSampledHelperRssBytes": 5835456512,
"resourceMeasurementScope": "200ms sampled helper RSS, process lifetime high-water sample; not a validated minimum or GPU peak."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"transcript": "Good morning. The train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:46:09.519176+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 3962.3884170141537,
"inferenceMs": 3961.624874995323,
"realTimeFactor": 1.414866026784044,
"audioSeconds": 2.8,
"seed": 42,
"eosReached": true,
"generatedTokens": 36,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.08865950256586075,
"peak": 0.5263413786888123,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 3962.3884170141537,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.7067806994228107,
"peakSampledHelperRssBytes": 5835456512,
"resourceMeasurementScope": "200ms sampled helper RSS, process lifetime high-water sample; not a validated minimum or GPU peak."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"transcript": "Good morning. The train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:46:09.519176+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 628.0749580182601,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 4070.3229999926407,
"inferenceMs": 4065.393708006013,
"realTimeFactor": 1.451926324287862,
"audioSeconds": 2.8,
"seed": 42,
"eosReached": true,
"generatedTokens": 36,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.08865950256586075,
"peak": 0.5263413786888123,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 4698.397958010901,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.688740181421036,
"peakSampledHelperRssBytes": 5455855616,
"resourceMeasurementScope": "200ms sampled helper RSS, process lifetime high-water sample; not a validated minimum or GPU peak."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"transcript": "Good morning. The train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:46:09.519176+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 4015.81158299814,
"inferenceMs": 4014.9961249844637,
"realTimeFactor": 1.4339271874944515,
"audioSeconds": 2.8,
"seed": 42,
"eosReached": true,
"generatedTokens": 36,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.08865950256586075,
"peak": 0.5263413786888123,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 4015.81158299814,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.6973854800447247,
"peakSampledHelperRssBytes": 5465243648,
"resourceMeasurementScope": "200ms sampled helper RSS, process lifetime high-water sample; not a validated minimum or GPU peak."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"transcript": "Good morning. The train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:46:09.519176+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 4041.0304999968503,
"inferenceMs": 4040.260458015837,
"realTimeFactor": 1.4429501635770847,
"audioSeconds": 2.8,
"seed": 42,
"eosReached": true,
"generatedTokens": 36,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.08865950256586075,
"peak": 0.5263413786888123,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 4041.0304999968503,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.6930246277674568,
"peakSampledHelperRssBytes": 5474025472,
"resourceMeasurementScope": "200ms sampled helper RSS, process lifetime high-water sample; not a validated minimum or GPU peak."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"transcript": "Good morning. The train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:46:09.519176+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 621.1847079975996,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 4092.62562499498,
"inferenceMs": 4087.531333992956,
"realTimeFactor": 1.4598326192831987,
"audioSeconds": 2.8,
"seed": 42,
"eosReached": true,
"generatedTokens": 36,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.08865950256586075,
"peak": 0.5263413786888123,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 4713.81033299258,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.6850100393639759,
"peakSampledHelperRssBytes": 4972150784,
"resourceMeasurementScope": "200ms sampled helper RSS, process lifetime high-water sample; not a validated minimum or GPU peak."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"transcript": "Good morning. The train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:46:09.519176+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 4019.125457998598,
"inferenceMs": 4018.3474159857724,
"realTimeFactor": 1.435124077137776,
"audioSeconds": 2.8,
"seed": 42,
"eosReached": true,
"generatedTokens": 36,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.08865950256586075,
"peak": 0.5263413786888123,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 4019.125457998598,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.6968038624189267,
"peakSampledHelperRssBytes": 5470502912,
"resourceMeasurementScope": "200ms sampled helper RSS, process lifetime high-water sample; not a validated minimum or GPU peak."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"transcript": "Good morning. The train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:46:09.519176+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 3981.704042002093,
"inferenceMs": 3980.974874983076,
"realTimeFactor": 1.4217767410653843,
"audioSeconds": 2.8,
"seed": 42,
"eosReached": true,
"generatedTokens": 36,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.08865950256586075,
"peak": 0.5263413786888123,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 3981.704042002093,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.7033453081041873,
"peakSampledHelperRssBytes": 5477777408,
"resourceMeasurementScope": "200ms sampled helper RSS, process lifetime high-water sample; not a validated minimum or GPU peak."
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"transcript": "Good morning. The train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:46:09.788412+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 993.3586249826476,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.bfloat16"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.bfloat16",
"attentionImplementation": "sdpa",
"durationMs": 12891.231083020102,
"inferenceMs": 12886.217416991713,
"realTimeFactor": 2.3344596769912522,
"audioSeconds": 5.52,
"seed": 42,
"eosReached": true,
"generatedTokens": 70,
"generationPlacement": "mps:0",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.16219741106033325,
"peak": 0.8671875,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"driverAllocatedMemoryBytes": 3190013952,
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 13884.58970800275,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.42836464894045256,
"peakSampledHelperRssBytes": 1429176320,
"resourceMeasurementScope": "200ms sampled helper RSS, process lifetime high-water sample; not a validated minimum or GPU peak."
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"transcript": "Please save this file before closing the window.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:46:09.875722+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.bfloat16"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.bfloat16",
"attentionImplementation": "sdpa",
"durationMs": 6744.507124996744,
"inferenceMs": 6743.6192499881145,
"realTimeFactor": 2.1073810156212858,
"audioSeconds": 3.2,
"seed": 42,
"eosReached": true,
"generatedTokens": 41,
"generationPlacement": "mps:0",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.14652307331562042,
"peak": 0.80859375,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"driverAllocatedMemoryBytes": 3191144448,
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 6744.507124996744,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.47452263856765636,
"peakSampledHelperRssBytes": 1503707136,
"resourceMeasurementScope": "200ms sampled helper RSS, process lifetime high-water sample; not a validated minimum or GPU peak."
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"transcript": "The blue box contains three small books.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:46:09.957953+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.bfloat16"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.bfloat16",
"attentionImplementation": "sdpa",
"durationMs": 8609.927791025257,
"inferenceMs": 8609.022667020326,
"realTimeFactor": 2.3394083334294367,
"audioSeconds": 3.68,
"seed": 42,
"eosReached": true,
"generatedTokens": 47,
"generationPlacement": "mps:0",
"speaker": "Aiden",
"language": "English",
"promptTokens": 8,
"maxNewTokens": 512,
"rms": 0.12260419875383377,
"peak": 0.79296875,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"driverAllocatedMemoryBytes": 3192586240,
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 8609.927791025257,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.42745850979083166,
"peakSampledHelperRssBytes": 1600159744,
"resourceMeasurementScope": "200ms sampled helper RSS, process lifetime high-water sample; not a validated minimum or GPU peak."
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"transcript": "Good morning. The train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:46:10.043362+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.bfloat16"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.bfloat16",
"attentionImplementation": "sdpa",
"durationMs": 7670.93483399367,
"inferenceMs": 7670.082042022841,
"realTimeFactor": 2.282762512506798,
"audioSeconds": 3.36,
"seed": 42,
"eosReached": true,
"generatedTokens": 43,
"generationPlacement": "mps:0",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.13558422029018402,
"peak": 0.796875,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"driverAllocatedMemoryBytes": 3193733120,
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 7670.93483399367,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.4380657184096903,
"peakSampledHelperRssBytes": 1674969088,
"resourceMeasurementScope": "200ms sampled helper RSS, process lifetime high-water sample; not a validated minimum or GPU peak."
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"transcript": "Good morning. The train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:46:10.043362+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.bfloat16"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.bfloat16",
"attentionImplementation": "sdpa",
"durationMs": 7811.599208012922,
"inferenceMs": 7810.7803750026505,
"realTimeFactor": 2.3246370163698367,
"audioSeconds": 3.36,
"seed": 42,
"eosReached": true,
"generatedTokens": 43,
"generationPlacement": "mps:0",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.13558422029018402,
"peak": 0.796875,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"driverAllocatedMemoryBytes": 3193733120,
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 7811.599208012922,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.4301746866104732,
"peakSampledHelperRssBytes": 1688010752,
"resourceMeasurementScope": "200ms sampled helper RSS, process lifetime high-water sample; not a validated minimum or GPU peak."
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"transcript": "Good morning. The train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:46:09.788412+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 1057.6847080083098,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.bfloat16"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.bfloat16",
"attentionImplementation": "sdpa",
"durationMs": 13700.652041006833,
"inferenceMs": 13695.359959005145,
"realTimeFactor": 2.4810434708342655,
"audioSeconds": 5.52,
"seed": 42,
"eosReached": true,
"generatedTokens": 70,
"generationPlacement": "mps:0",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.16219741106033325,
"peak": 0.8671875,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"driverAllocatedMemoryBytes": 3190013952,
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 14758.336749015143,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.40305621878674464,
"peakSampledHelperRssBytes": 1434189824,
"resourceMeasurementScope": "200ms sampled helper RSS, process lifetime high-water sample; not a validated minimum or GPU peak."
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"transcript": "Good morning. The train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:46:10.043362+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.bfloat16"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.bfloat16",
"attentionImplementation": "sdpa",
"durationMs": 7472.342249995563,
"inferenceMs": 7471.438000007765,
"realTimeFactor": 2.2236422619070733,
"audioSeconds": 3.36,
"seed": 42,
"eosReached": true,
"generatedTokens": 43,
"generationPlacement": "mps:0",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.13558422029018402,
"peak": 0.796875,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"driverAllocatedMemoryBytes": 3191160832,
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 7472.342249995563,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.44971262560119046,
"peakSampledHelperRssBytes": 1510096896,
"resourceMeasurementScope": "200ms sampled helper RSS, process lifetime high-water sample; not a validated minimum or GPU peak."
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"transcript": "Good morning. The train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:46:10.043362+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.bfloat16"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.bfloat16",
"attentionImplementation": "sdpa",
"durationMs": 7516.54079198488,
"inferenceMs": 7515.75429100194,
"realTimeFactor": 2.236831634226768,
"audioSeconds": 3.36,
"seed": 42,
"eosReached": true,
"generatedTokens": 43,
"generationPlacement": "mps:0",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.13558422029018402,
"peak": 0.796875,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"driverAllocatedMemoryBytes": 3191160832,
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 7516.54079198488,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.44706091629720796,
"peakSampledHelperRssBytes": 1523662848,
"resourceMeasurementScope": "200ms sampled helper RSS, process lifetime high-water sample; not a validated minimum or GPU peak."
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"transcript": "Good morning. The train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:46:09.788412+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 1063.2034169975668,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.bfloat16"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.bfloat16",
"attentionImplementation": "sdpa",
"durationMs": 13721.999542001868,
"inferenceMs": 13716.71870801947,
"realTimeFactor": 2.4849128094238173,
"audioSeconds": 5.52,
"seed": 42,
"eosReached": true,
"generatedTokens": 70,
"generationPlacement": "mps:0",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.16219741106033325,
"peak": 0.8671875,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"driverAllocatedMemoryBytes": 3190013952,
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 14785.202958999434,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.40242860683384396,
"peakSampledHelperRssBytes": 1428832256,
"resourceMeasurementScope": "200ms sampled helper RSS, process lifetime high-water sample; not a validated minimum or GPU peak."
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"transcript": "Good morning. The train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:46:10.043362+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.bfloat16"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.bfloat16",
"attentionImplementation": "sdpa",
"durationMs": 7563.585791998776,
"inferenceMs": 7562.734624982113,
"realTimeFactor": 2.2508138764827716,
"audioSeconds": 3.36,
"seed": 42,
"eosReached": true,
"generatedTokens": 43,
"generationPlacement": "mps:0",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.13558422029018402,
"peak": 0.796875,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"driverAllocatedMemoryBytes": 3191160832,
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 7563.585791998776,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.4442837368510662,
"peakSampledHelperRssBytes": 1487126528,
"resourceMeasurementScope": "200ms sampled helper RSS, process lifetime high-water sample; not a validated minimum or GPU peak."
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"wordErrorRate": 0,
"transcript": "Good morning. The train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:46:10.043362+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.bfloat16"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.bfloat16",
"attentionImplementation": "sdpa",
"durationMs": 7524.681457987754,
"inferenceMs": 7523.823499999708,
"realTimeFactor": 2.2392331845237226,
"audioSeconds": 3.36,
"seed": 42,
"eosReached": true,
"generatedTokens": 43,
"generationPlacement": "mps:0",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.13558422029018402,
"peak": 0.796875,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"driverAllocatedMemoryBytes": 3191160832,
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 7524.681457987754,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.4465814489135917,
"peakSampledHelperRssBytes": 1519681536,
"resourceMeasurementScope": "200ms sampled helper RSS, process lifetime high-water sample; not a validated minimum or GPU peak."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "fail",
"quality": "Not embedded",
"runtime": "Not embedded",
"failure": "{'ok': False, 'error': 'probability tensor contains either `inf`, `nan` or element < 0', 'code': 'runtime_error', 'traceback': 'Traceback (most recent call last):\\n File \"/Users/pengzhang/Desktop/hubters-webai/webai-js/localai-desktop/crates/desktopai/src/models/qwen_tts/runtime.py\", line 53, in <module>\\n with contextlib.redirect_stdout(sys.stderr):data=init(r) if r[\\'type\\']==\\'init\\' else {\\'alive\\':True,\\'pid\\':os.getpid()} if r[\\'type\\']==\\'health\\' else generate(r)\\n ~~~~~~~~^^^\\n File \"/Users/pengzhang/Desktop/hubters-webai/webai-js/localai-desktop/crates/desktopai/src/models/qwen_tts/runtime.py\", line 42, in generate\\n with torch.inference_mode():wavs,sr=model.generate_custom_voice(text=text,language=language,speaker=speaker,non_streaming_mode=True,max_new_tokens=maximum,do_sample=True,subtalker_dosample=True,temperature=0.9,top_k=50,top_p=1.0,repetition_penalty=1.05)\\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\\n File \"/Users/pengzhang/Desktop/hubters-webai/webai-js/localai-desktop/.cache/qwen-voice-probe/venv/lib/python3.13/site-packages/torch/utils/_contextlib.py\", line 124, in decorate_context\\n return func(*args, **kwargs)\\n File \"/Users/pengzhang/Desktop/hubters-webai/webai-js/localai-desktop/.cache/qwen-voice-probe/venv/lib/python3.13/site-packages/qwen_tts/inference/qwen3_tts_model.py\", line 829, in generate_custom_voice\\n talker_codes_list, _ = self.model.generate(\\n ~~~~~~~~~~~~~~~~~~~^\\n input_ids=input_ids,\\n ^^^^^^^^^^^^^^^^^^^^\\n ...<4 lines>...\\n **gen_kwargs,\\n ^^^^^^^^^^^^^\\n )\\n ^\\n File \"/Users/pengzhang/Desktop/hubters-webai/webai-js/localai-desktop/.cache/qwen-voice-probe/venv/lib/python3.13/site-packages/torch/utils/_contextlib.py\", line 124, in decorate_context\\n return func(*args, **kwargs)\\n File \"/Users/pengzhang/Desktop/hubters-webai/webai-js/localai-desktop/.cache/qwen-voice-probe/venv/lib/python3.13/site-packages/qwen_tts/core/models/modeling_qwen3_tts.py\", line 2272, in generate\\n talker_result = self.talker.generate(\\n inputs_embeds=talker_input_embeds,\\n ...<3 lines>...\\n **talker_kwargs,\\n )\\n File \"/Users/pengzhang/Desktop/hubters-webai/webai-js/localai-desktop/crates/desktopai/src/models/qwen_tts/runtime.py\", line 25, in observed\\n result=original(*args,**kwargs);sequences=result.sequences\\n File \"/Users/pengzhang/Desktop/hubters-webai/webai-js/localai-desktop/.cache/qwen-voice-probe/venv/lib/python3.13/site-packages/torch/utils/_contextlib.py\", line 124, in decorate_context\\n return func(*args, **kwargs)\\n File \"/Users/pengzhang/Desktop/hubters-webai/webai-js/localai-desktop/.cache/qwen-voice-probe/venv/lib/python3.13/site-packages/transformers/generation/utils.py\", line 2564, in generate\\n result = decoding_method(\\n self,\\n ...<5 lines>...\\n **model_kwargs,\\n )\\n File \"/Users/pengzhang/Desktop/hubters-webai/webai-js/localai-desktop/.cache/qwen-voice-probe/venv/lib/python3.13/site-packages/transformers/generation/utils.py\", line 2787, in _sample\\n outputs = model_forward(**model_inputs, return_dict=True)\\n File \"/Users/pengzhang/Desktop/hubters-webai/webai-js/localai-desktop/.cache/qwen-voice-probe/venv/lib/python3.13/site-packages/torch/nn/modules/module.py\", line 1776, in _wrapped_call_impl\\n return self._call_impl(*args, **kwargs)\\n ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^\\n File \"/Users/pengzhang/Desktop/hubters-webai/webai-js/localai-desktop/.cache/qwen-voice-probe/venv/lib/python3.13/site-packages/torch/nn/modules/module.py\", line 1787, in _call_impl\\n return forward_call(*args, **kwargs)\\n File \"/Users/pengzhang/Desktop/hubters-webai/webai-js/localai-desktop/.cache/qwen-voice-probe/venv/lib/python3.13/site-packages/transformers/utils/generic.py\", line 918, in wrapper\\n output = func(self, *args, **kwargs)\\n File \"/Users/pengzhang/Desktop/hubters-webai/webai-js/localai-desktop/.cache/qwen-voice-probe/venv/lib/python3.13/site-packages/qwen_tts/core/models/modeling_qwen3_tts.py\", line 1671, in forward\\n predictor_result = self.code_predictor.generate(\\n inputs_embeds=torch.cat((past_hidden, last_id_hidden), dim=1),\\n ...<6 lines>...\\n return_dict_in_generate=True,\\n )\\n File \"/Users/pengzhang/Desktop/hubters-webai/webai-js/localai-desktop/.cache/qwen-voice-probe/venv/lib/python3.13/site-packages/torch/utils/_contextlib.py\", line 124, in decorate_context\\n return func(*args, **kwargs)\\n File \"/Users/pengzhang/Desktop/hubters-webai/webai-js/localai-desktop/.cache/qwen-voice-probe/venv/lib/python3.13/site-packages/transformers/generation/utils.py\", line 2564, in generate\\n result = decoding_method(\\n self,\\n ...<5 lines>...\\n **model_kwargs,\\n )\\n File \"/Users/pengzhang/Desktop/hubters-webai/webai-js/localai-desktop/.cache/qwen-voice-probe/venv/lib/python3.13/site-packages/transformers/generation/utils.py\", line 2829, in _sample\\n next_tokens = torch.multinomial(probs, num_samples=1).squeeze(1)\\n ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^\\nRuntimeError: probability tensor contains either `inf`, `nan` or element < 0\\n'}"
},
{
"precision": "fp32",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"wordErrorRate": 0.2857142857142857,
"transcript": "The train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:28:35.895740+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 1158.2479999924544,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.float32"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"durationMs": 7214.914374984801,
"inferenceMs": 7210.173625004245,
"realTimeFactor": 2.5035325086820297,
"audioSeconds": 2.88,
"seed": 42,
"eosReached": true,
"generatedTokens": 37,
"generationPlacement": "mps:0",
"speaker": "Ryan",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.06079854816198349,
"peak": 0.41497504711151123,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"driverAllocatedMemoryBytes": 5727633408,
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 8373.162374977255,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.3994355961155241,
"peakSampledHelperRssBytes": 3560357888
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"wordErrorRate": 0.2857142857142857,
"transcript": "Uh huh. Good morning. The train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:33:51.166387+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.bfloat16"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.bfloat16",
"durationMs": 12041.508540976793,
"inferenceMs": 12040.462000004482,
"realTimeFactor": 2.061722945206247,
"audioSeconds": 5.84,
"seed": 42,
"eosReached": true,
"generatedTokens": 74,
"generationPlacement": "mps:0",
"speaker": "Ryan",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.08339852839708328,
"peak": 0.546875,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"driverAllocatedMemoryBytes": 3185917952,
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 12041.508540976793,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.4850312222236842,
"peakSampledHelperRssBytes": 1698496512
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"wordErrorRate": 0.2857142857142857,
"transcript": "Uh huh. Good morning. The train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:33:51.166387+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.bfloat16"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.bfloat16",
"durationMs": 11604.526707989862,
"inferenceMs": 11603.5159160092,
"realTimeFactor": 1.986903410275548,
"audioSeconds": 5.84,
"seed": 42,
"eosReached": true,
"generatedTokens": 74,
"generationPlacement": "mps:0",
"speaker": "Ryan",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.08339852839708328,
"peak": 0.546875,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"driverAllocatedMemoryBytes": 3185917952,
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 11604.526707989862,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.5032957288353126,
"peakSampledHelperRssBytes": 1728823296
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"wordErrorRate": 0.2857142857142857,
"transcript": "Uh huh. Good morning. The train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:33:51.166387+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.bfloat16"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.bfloat16",
"durationMs": 11944.045832991833,
"inferenceMs": 11942.955917009385,
"realTimeFactor": 2.0450266981180456,
"audioSeconds": 5.84,
"seed": 42,
"eosReached": true,
"generatedTokens": 74,
"generationPlacement": "mps:0",
"speaker": "Ryan",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.08339852839708328,
"peak": 0.546875,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"driverAllocatedMemoryBytes": 3183296512,
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 11944.045832991833,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.48899117107872436,
"peakSampledHelperRssBytes": 1526218752
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"wordErrorRate": 0.2857142857142857,
"transcript": "Uh huh. Good morning. The train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:33:51.166387+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.bfloat16"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.bfloat16",
"durationMs": 11773.998374992516,
"inferenceMs": 11773.008999996819,
"realTimeFactor": 2.015926198629592,
"audioSeconds": 5.84,
"seed": 42,
"eosReached": true,
"generatedTokens": 74,
"generationPlacement": "mps:0",
"speaker": "Ryan",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.08339852839708328,
"peak": 0.546875,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"driverAllocatedMemoryBytes": 3183296512,
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 11773.998374992516,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.4960499053386928,
"peakSampledHelperRssBytes": 1569865728
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"wordErrorRate": 0.2857142857142857,
"transcript": "Uh huh. Good morning. The train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:33:51.166387+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.bfloat16"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.bfloat16",
"durationMs": 12315.832499996759,
"inferenceMs": 12314.80029100203,
"realTimeFactor": 2.108698679966101,
"audioSeconds": 5.84,
"seed": 42,
"eosReached": true,
"generatedTokens": 74,
"generationPlacement": "mps:0",
"speaker": "Ryan",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.08339852839708328,
"peak": 0.546875,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"driverAllocatedMemoryBytes": 3183296512,
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 12315.832499996759,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.4742261232012891,
"peakSampledHelperRssBytes": 1535098880
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"wordErrorRate": 0.2857142857142857,
"transcript": "Uh huh. Good morning. The train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:33:51.166387+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.bfloat16"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.bfloat16",
"durationMs": 11971.489499992458,
"inferenceMs": 11970.487959013553,
"realTimeFactor": 2.049741088872184,
"audioSeconds": 5.84,
"seed": 42,
"eosReached": true,
"generatedTokens": 74,
"generationPlacement": "mps:0",
"speaker": "Ryan",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.08339852839708328,
"peak": 0.546875,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"driverAllocatedMemoryBytes": 3183296512,
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 11971.489499992458,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.4878664946655403,
"peakSampledHelperRssBytes": 1556889600
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"wordErrorRate": 0.2857142857142857,
"transcript": "D but good morning, the train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:36:41.292676+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 1043.6494160094298,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.bfloat16"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.bfloat16",
"durationMs": 22097.087709000334,
"inferenceMs": 22091.48187498795,
"realTimeFactor": 2.320533810397894,
"audioSeconds": 9.52,
"seed": 0,
"eosReached": true,
"generatedTokens": 120,
"generationPlacement": "mps:0",
"speaker": "Ryan",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.036851946264505386,
"peak": 0.34765625,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"driverAllocatedMemoryBytes": 3269312512,
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 23140.737125009764,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.43093532855206856,
"peakSampledHelperRssBytes": 1756823552
}
},
{
"precision": "fp32",
"device": "mps",
"status": "fail",
"quality": {
"passed": true,
"wordErrorRate": 0,
"transcript": "Good morning. The train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:38:21.225979+00:00",
"structuralPassed": true,
"performancePassed": false
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 1153.8627919799183,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.float32"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "eager",
"durationMs": 12096.490583993727,
"inferenceMs": 12091.324209002778,
"realTimeFactor": 3.9774092792772295,
"audioSeconds": 3.04,
"seed": 42,
"eosReached": true,
"generatedTokens": 39,
"generationPlacement": "mps:0",
"speaker": "Ryan",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.08722323179244995,
"peak": 0.6686762571334839,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"driverAllocatedMemoryBytes": 5736005632,
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 13250.353375973646,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "slow",
"audioSecondsPerSecond": 0.2514199394088302,
"peakSampledHelperRssBytes": 3055747072
}
},
{
"precision": "fp32",
"device": "mps",
"status": "fail",
"quality": {
"passed": true,
"wordErrorRate": 0.25,
"transcript": "One bait. Please save this file before closing the window.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:38:21.317741+00:00",
"structuralPassed": true,
"performancePassed": false
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.float32"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "eager",
"durationMs": 12926.560332998633,
"inferenceMs": 12925.634916988201,
"realTimeFactor": 3.4376688609011175,
"audioSeconds": 3.76,
"seed": 42,
"eosReached": true,
"generatedTokens": 48,
"generationPlacement": "mps:0",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.08904044330120087,
"peak": 0.5335569381713867,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"driverAllocatedMemoryBytes": 5746065408,
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 12926.560332998633,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "slow",
"audioSecondsPerSecond": 0.2908948012339588,
"peakSampledHelperRssBytes": 3055747072
}
},
{
"precision": "fp32",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"wordErrorRate": 0.5714285714285714,
"transcript": "When Brom's the Blue Box contains three small books, uh.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:38:21.417511+00:00",
"structuralPassed": true,
"performancePassed": false
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.float32"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "eager",
"durationMs": 19259.12524998421,
"inferenceMs": 19258.111083006952,
"realTimeFactor": 3.438948407679813,
"audioSeconds": 5.6,
"seed": 42,
"eosReached": true,
"generatedTokens": 71,
"generationPlacement": "mps:0",
"speaker": "Ryan",
"language": "English",
"promptTokens": 8,
"maxNewTokens": 512,
"rms": 0.06005222350358963,
"peak": 0.4930441975593567,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"driverAllocatedMemoryBytes": 5757321216,
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 19259.12524998421,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "slow",
"audioSecondsPerSecond": 0.290786566546568,
"peakSampledHelperRssBytes": 3055747072
}
},
{
"precision": "fp32",
"device": "mps",
"status": "fail",
"quality": {
"passed": true,
"wordErrorRate": 0.14285714285714285,
"transcript": "Morning, the train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:38:21.493349+00:00",
"structuralPassed": true,
"performancePassed": false
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.float32"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "eager",
"durationMs": 7018.734832992777,
"inferenceMs": 7017.955791001441,
"realTimeFactor": 3.5089778955007205,
"audioSeconds": 2,
"seed": 42,
"eosReached": true,
"generatedTokens": 26,
"generationPlacement": "mps:0",
"speaker": "Ryan",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.04995442181825638,
"peak": 0.389910489320755,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"driverAllocatedMemoryBytes": 5758353408,
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 7018.734832992777,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "slow",
"audioSecondsPerSecond": 0.28498327141992524,
"peakSampledHelperRssBytes": 3055747072
}
},
{
"precision": "fp32",
"device": "mps",
"status": "fail",
"quality": {
"passed": true,
"wordErrorRate": 0.14285714285714285,
"transcript": "Morning, the train leaves at nine.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:38:21.493349+00:00",
"structuralPassed": true,
"performancePassed": false
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.float32"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "eager",
"durationMs": 6735.496207984397,
"inferenceMs": 6734.785541018937,
"realTimeFactor": 3.3673927705094684,
"audioSeconds": 2,
"seed": 42,
"eosReached": true,
"generatedTokens": 26,
"generationPlacement": "mps:0",
"speaker": "Ryan",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.04995442181825638,
"peak": 0.389910489320755,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"driverAllocatedMemoryBytes": 5758353408,
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 6735.496207984397,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "slow",
"audioSecondsPerSecond": 0.2969656550782181,
"peakSampledHelperRssBytes": 3055747072
}
},
{
"precision": "fp32",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"wordErrorRate": 0.5714285714285714,
"transcript": "Once runes known to the blue box contains three small books.",
"repetitive": false,
"maximumWordErrorRate": 0.25,
"evaluator": "Independent Qwen3-ASR0.6B MLXQ4, revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3 / mlx0.32.4",
"evaluatedAt": "2026-11-04T11:39:51.874648+00:00",
"structuralPassed": true,
"performancePassed": true
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 0,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.float32"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 7607.696624996606,
"inferenceMs": 7606.8151670042425,
"realTimeFactor": 2.113004213056734,
"audioSeconds": 3.6,
"seed": 42,
"eosReached": true,
"generatedTokens": 46,
"generationPlacement": "mps:0",
"speaker": "Aiden",
"language": "English",
"promptTokens": 8,
"maxNewTokens": 512,
"rms": 0.12580694258213043,
"peak": 0.8521938323974609,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"driverAllocatedMemoryBytes": 5679300608,
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 7607.696624996606,
"artifactRevision": "85e237c12c027371202489a0ec509ded67b5e4b5",
"fallback": false,
"grade": "moderate",
"audioSecondsPerSecond": 0.47325982306176784,
"peakSampledHelperRssBytes": 2958917632
}
}
]Qwen3-TTS 1.7B · CustomVoice — Controlled Speech
40 configurations testedQuality criteria
{ "maximumWordErrorRate": 0.25, "minimumSpeakerCosine": null, "minimumRms": 0.005, "maximumClippedFraction": 0.02, "eosRequired": true, "maximumRealTimeFactor": 3, "maximumRequestMs": 45000 }
| Precision | Device | Result | Platform | Inference / load | Task metric | |
|---|---|---|---|---|---|---|
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 1197 ms / 310 ms | 0.468 RTF |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 1124 ms / 310 ms | 0.468 RTF |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 1425 ms / 312 ms | 0.469 RTF |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 1204 ms / 312 ms | 0.470 RTF |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 1129 ms / 310 ms | 0.470 RTF |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 1430 ms / 310 ms | 0.471 RTF |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 1243 ms / 312 ms | 0.471 RTF |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 1135 ms / 312 ms | 0.473 RTF |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 1248 ms / 310 ms | 0.473 RTF |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 1135 ms / 312 ms | 0.473 RTF |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 1148 ms / 312 ms | 0.478 RTF |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 1163 ms / 310 ms | 0.485 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1573 ms / 332 ms | 0.517 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1245 ms / 332 ms | 0.519 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1247 ms / 332 ms | 0.520 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1453 ms / 332 ms | 0.550 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1540 ms / 332 ms | 0.601 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1859 ms / 332 ms | 0.774 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 6469 ms / 974 ms | 2.128 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 4944 ms / 925 ms | 2.131 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 6482 ms / 925 ms | 2.132 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 4955 ms / 914 ms | 2.136 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 6497 ms / 914 ms | 2.137 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 5301 ms / 974 ms | 2.137 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 4971 ms / 914 ms | 2.142 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 4971 ms / 974 ms | 2.142 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 4973 ms / 925 ms | 2.144 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 4981 ms / 914 ms | 2.147 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 4641 ms / 925 ms | 2.148 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 4659 ms / 914 ms | 2.157 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 5363 ms / 914 ms | 2.163 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 5373 ms / 925 ms | 2.167 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 5073 ms / 974 ms | 2.187 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 5097 ms / 974 ms | 2.197 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 4750 ms / 974 ms | 2.199 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 5114 ms / 925 ms | 2.204 RTF |
BF16 | mps | Failed | — | macos · aarch64 | 16008 ms / 1627 ms | 5.885 RTF |
FP32 | mps | Failed | — | macos · aarch64 | 9529 ms / 1574 ms | 4.254 RTF |
FP16 | mps | Failed | — | macos · aarch64 | 10812 ms / 1412 ms | 4.360 RTF |
FP32 | mps | Failed | — | macos · aarch64 | 14303 ms / 1623 ms | 4.832 RTF |
Test setup & sample details
- Quality gate
- { "maximumWordErrorRate": 0.25, "minimumSpeakerCosine": null, "minimumRms": 0.005, "maximumClippedFraction": 0.02, "eosRequired": true, "maximumRealTimeFactor": 3, "maximumRequestMs": 45000 }
- Tested
- 2026-11-06T14:34:13.031046+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2", "runtime": "App-owned CPython3.13.18; CPU qwen-tts0.1.1/Transformers4.57.3/Torch2.10.0; Metal mlx-audio0.6.3/MLX0.32.4/Transformers5.22.1; original identical checkpoint files" }
- Runtime
- mlx-audio 0.6.3
- Artifact revision
- 0c0e3051f131929182e2c023b9537f8b1c68adfe
- Fixture
{ "file": "model-samples/desktop/qwen3-tts-family/customvoice-morning-input.json", "url": "https://assets.localai.tools/samples/v1/files/44decdf28f478b082c7088eebe2beec02d39eaa0bc76e8dd10d23112e0a439fc/customvoice-morning-input.json", "size": 234, "sha1": "4e3c80615fb763a1a3fac3c165dbe529e820a05c", "sha256": "44decdf28f478b082c7088eebe2beec02d39eaa0bc76e8dd10d23112e0a439fc" }- Rerun when
- artifact revision/checksum; qwen-tts/Transformers/Torch versions, preprocessing or sampling; OS/device/backend behavior; fixture text, voice/language selections or independent ASR/quality/performance gates
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.11956550180912018,
"audioSeconds": 2.32,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:37.448742+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.41802436113357544
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 973.9282919999823,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 5101.3998329999595,
"inferenceMs": 5096.5938750014175,
"realTimeFactor": 2.1968077047419903,
"audioSeconds": 2.32,
"seed": 42,
"eosReached": true,
"generatedTokens": 30,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.11956549435853958,
"peak": 0.6288480758666992,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "custom_voice",
"instruct": "",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "fp32",
"cpuThreads": 4,
"fallback": false,
"totalMs": 6083.301667000342,
"audioSecondsPerSecond": 0.4552059781297278,
"grade": "moderate",
"residentMemoryBytes": 9379840000,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Please save this file before closing the window.",
"wordErrorRate": 0,
"rms": 0.10222581028938293,
"audioSeconds": 2.48,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:37.558922+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.3600354790687561
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 973.9282919999823,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 5301.646542000526,
"inferenceMs": 5300.851415999205,
"realTimeFactor": 2.1374400870964534,
"audioSeconds": 2.48,
"seed": 42,
"eosReached": true,
"generatedTokens": 32,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.10222581773996353,
"peak": 0.5643832087516785,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "custom_voice",
"instruct": "",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "fp32",
"cpuThreads": 4,
"fallback": false,
"totalMs": 6283.967458999541,
"audioSecondsPerSecond": 0.4678493708604588,
"grade": "moderate",
"residentMemoryBytes": 9675259904,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "The blue box contains three small books.",
"wordErrorRate": 0,
"rms": 0.12879793345928192,
"audioSeconds": 2.32,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:37.665807+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.4279877543449402
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 973.9282919999823,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 5073.718000001463,
"inferenceMs": 5072.972957999809,
"realTimeFactor": 2.186626274999918,
"audioSeconds": 2.32,
"seed": 42,
"eosReached": true,
"generatedTokens": 30,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 8,
"maxNewTokens": 512,
"rms": 0.12879793345928192,
"peak": 0.7126675844192505,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "custom_voice",
"instruct": "",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "fp32",
"cpuThreads": 4,
"fallback": false,
"totalMs": 6055.527874999825,
"audioSecondsPerSecond": 0.4573255207957462,
"grade": "moderate",
"residentMemoryBytes": 9675161600,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Please save this file before closing the window.",
"wordErrorRate": 0,
"rms": 0.1544686257839203,
"audioSeconds": 3.04,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:37.779011+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.38538694381713867
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 973.9282919999823,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 6469.744042000457,
"inferenceMs": 6468.830499999967,
"realTimeFactor": 2.127904769736831,
"audioSeconds": 3.04,
"seed": 42,
"eosReached": true,
"generatedTokens": 39,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.1544685661792755,
"peak": 0.6981966495513916,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "custom_voice",
"instruct": "Speak slowly and clearly.",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "fp32",
"cpuThreads": 4,
"fallback": false,
"totalMs": 7453.931417001513,
"audioSecondsPerSecond": 0.4699458426063282,
"grade": "moderate",
"residentMemoryBytes": 9986244608,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Please save this file before closing the window.",
"wordErrorRate": 0,
"rms": 0.1282392144203186,
"audioSeconds": 2.16,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:37.888374+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.38728201389312744
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 973.9282919999823,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 4750.806417001513,
"inferenceMs": 4750.051792001614,
"realTimeFactor": 2.199098051852599,
"audioSeconds": 2.16,
"seed": 42,
"eosReached": true,
"generatedTokens": 28,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.1282392293214798,
"peak": 0.8515697717666626,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "custom_voice",
"instruct": "Speak quickly and clearly.",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "fp32",
"cpuThreads": 4,
"fallback": false,
"totalMs": 5732.169332999547,
"audioSecondsPerSecond": 0.4547318839001127,
"grade": "moderate",
"residentMemoryBytes": 9986981888,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.11956550180912018,
"audioSeconds": 2.32,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:37.999289+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.41802436113357544
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 973.9282919999823,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 4971.3132079996285,
"inferenceMs": 4970.540416999938,
"realTimeFactor": 2.1424743176723875,
"audioSeconds": 2.32,
"seed": 42,
"eosReached": true,
"generatedTokens": 30,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.11956549435853958,
"peak": 0.6288480758666992,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "custom_voice",
"instruct": "",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "fp32",
"cpuThreads": 4,
"fallback": false,
"totalMs": 5952.824374999182,
"audioSecondsPerSecond": 0.4667500523816842,
"grade": "moderate",
"residentMemoryBytes": 9987162112,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.11956550180912018,
"audioSeconds": 2.32,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:38.108187+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.41802436113357544
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 925.3592499990191,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 4948.787875000562,
"inferenceMs": 4943.862791998981,
"realTimeFactor": 2.1309753413788712,
"audioSeconds": 2.32,
"seed": 42,
"eosReached": true,
"generatedTokens": 30,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.11956549435853958,
"peak": 0.6288480758666992,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "custom_voice",
"instruct": "",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "fp32",
"cpuThreads": 4,
"fallback": false,
"totalMs": 5881.7525829999795,
"audioSecondsPerSecond": 0.4692686867755771,
"grade": "moderate",
"residentMemoryBytes": 9383198720,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Please save this file before closing the window.",
"wordErrorRate": 0,
"rms": 0.10222581028938293,
"audioSeconds": 2.48,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:38.216653+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.3600354790687561
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 925.3592499990191,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 5374.14762499975,
"inferenceMs": 5373.307290999946,
"realTimeFactor": 2.166656165725785,
"audioSeconds": 2.48,
"seed": 42,
"eosReached": true,
"generatedTokens": 32,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.10222581773996353,
"peak": 0.5643832087516785,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "custom_voice",
"instruct": "",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "fp32",
"cpuThreads": 4,
"fallback": false,
"totalMs": 6307.126666999466,
"audioSecondsPerSecond": 0.46154069843611795,
"grade": "moderate",
"residentMemoryBytes": 9678225408,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "The blue box contains three small books.",
"wordErrorRate": 0,
"rms": 0.12879793345928192,
"audioSeconds": 2.32,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:38.326336+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.4279877543449402
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 925.3592499990191,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 4973.953500000789,
"inferenceMs": 4973.1902920011635,
"realTimeFactor": 2.1436165051729152,
"audioSeconds": 2.32,
"seed": 42,
"eosReached": true,
"generatedTokens": 30,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 8,
"maxNewTokens": 512,
"rms": 0.12879793345928192,
"peak": 0.7126675844192505,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "custom_voice",
"instruct": "",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "fp32",
"cpuThreads": 4,
"fallback": false,
"totalMs": 5907.225582999672,
"audioSecondsPerSecond": 0.4665013530110577,
"grade": "moderate",
"residentMemoryBytes": 9686499328,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Please save this file before closing the window.",
"wordErrorRate": 0,
"rms": 0.1544686257839203,
"audioSeconds": 3.04,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:38.440991+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.38538694381713867
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 925.3592499990191,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 6483.118916999956,
"inferenceMs": 6482.2800000001735,
"realTimeFactor": 2.132328947368478,
"audioSeconds": 3.04,
"seed": 42,
"eosReached": true,
"generatedTokens": 39,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.1544685661792755,
"peak": 0.6981966495513916,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "custom_voice",
"instruct": "Speak slowly and clearly.",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "fp32",
"cpuThreads": 4,
"fallback": false,
"totalMs": 7418.684416998076,
"audioSecondsPerSecond": 0.4689707942267102,
"grade": "moderate",
"residentMemoryBytes": 9981935616,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Please save this file before closing the window.",
"wordErrorRate": 0,
"rms": 0.1282392144203186,
"audioSeconds": 2.16,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:38.550631+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.38728201389312744
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 925.3592499990191,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 4641.575083000134,
"inferenceMs": 4640.665417000491,
"realTimeFactor": 2.1484562115743016,
"audioSeconds": 2.16,
"seed": 42,
"eosReached": true,
"generatedTokens": 28,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.1282392293214798,
"peak": 0.8515697717666626,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "custom_voice",
"instruct": "Speak quickly and clearly.",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "fp32",
"cpuThreads": 4,
"fallback": false,
"totalMs": 5574.149332998786,
"audioSecondsPerSecond": 0.4654504916659393,
"grade": "moderate",
"residentMemoryBytes": 9982869504,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.11956550180912018,
"audioSeconds": 2.32,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:38.657722+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.41802436113357544
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 925.3592499990191,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 5114.679457999955,
"inferenceMs": 5113.961332999679,
"realTimeFactor": 2.204293678017103,
"audioSeconds": 2.32,
"seed": 42,
"eosReached": true,
"generatedTokens": 30,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.11956549435853958,
"peak": 0.6288480758666992,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "custom_voice",
"instruct": "",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "fp32",
"cpuThreads": 4,
"fallback": false,
"totalMs": 6047.808999999688,
"audioSecondsPerSecond": 0.4536600589897627,
"grade": "moderate",
"residentMemoryBytes": 9982869504,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.11956550180912018,
"audioSeconds": 2.32,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:38.769815+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.41802436113357544
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 913.5980409992044,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 4975.732834000155,
"inferenceMs": 4970.532625000487,
"realTimeFactor": 2.142470959051934,
"audioSeconds": 2.32,
"seed": 42,
"eosReached": true,
"generatedTokens": 30,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.11956549435853958,
"peak": 0.6288480758666992,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "custom_voice",
"instruct": "",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "fp32",
"cpuThreads": 4,
"fallback": false,
"totalMs": 5897.185373998582,
"audioSecondsPerSecond": 0.46675078407714354,
"grade": "moderate",
"residentMemoryBytes": 9382969344,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Please save this file before closing the window.",
"wordErrorRate": 0,
"rms": 0.10222581028938293,
"audioSeconds": 2.48,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:38.878033+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.3600354790687561
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 913.5980409992044,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 5364.141542000652,
"inferenceMs": 5363.327791999836,
"realTimeFactor": 2.162632174193482,
"audioSeconds": 2.48,
"seed": 42,
"eosReached": true,
"generatedTokens": 32,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.10222581773996353,
"peak": 0.5643832087516785,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "custom_voice",
"instruct": "",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "fp32",
"cpuThreads": 4,
"fallback": false,
"totalMs": 6285.884748998797,
"audioSecondsPerSecond": 0.46239948333929387,
"grade": "moderate",
"residentMemoryBytes": 9684058112,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "The blue box contains three small books.",
"wordErrorRate": 0,
"rms": 0.12879793345928192,
"audioSeconds": 2.32,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:38.981939+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.4279877543449402
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 913.5980409992044,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 4956.122416999278,
"inferenceMs": 4955.36416700088,
"realTimeFactor": 2.135932830603828,
"audioSeconds": 2.32,
"seed": 42,
"eosReached": true,
"generatedTokens": 30,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 8,
"maxNewTokens": 512,
"rms": 0.12879793345928192,
"peak": 0.7126675844192505,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "custom_voice",
"instruct": "",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "fp32",
"cpuThreads": 4,
"fallback": false,
"totalMs": 5877.000290998694,
"audioSecondsPerSecond": 0.4681795165428026,
"grade": "moderate",
"residentMemoryBytes": 9684975616,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Please save this file before closing the window.",
"wordErrorRate": 0,
"rms": 0.1544686257839203,
"audioSeconds": 3.04,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:39.095017+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.38538694381713867
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 913.5980409992044,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 6498.445458000788,
"inferenceMs": 6497.485375000906,
"realTimeFactor": 2.1373307154608243,
"audioSeconds": 3.04,
"seed": 42,
"eosReached": true,
"generatedTokens": 39,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.1544685661792755,
"peak": 0.6981966495513916,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "custom_voice",
"instruct": "Speak slowly and clearly.",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "fp32",
"cpuThreads": 4,
"fallback": false,
"totalMs": 7422.254415998395,
"audioSecondsPerSecond": 0.46787331168091717,
"grade": "moderate",
"residentMemoryBytes": 9979428864,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Please save this file before closing the window.",
"wordErrorRate": 0,
"rms": 0.1282392144203186,
"audioSeconds": 2.16,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:39.202797+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.38728201389312744
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 913.5980409992044,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 4660.242541998741,
"inferenceMs": 4659.402291999868,
"realTimeFactor": 2.1571306907406793,
"audioSeconds": 2.16,
"seed": 42,
"eosReached": true,
"generatedTokens": 28,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.1282392293214798,
"peak": 0.8515697717666626,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "custom_voice",
"instruct": "Speak quickly and clearly.",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "fp32",
"cpuThreads": 4,
"fallback": false,
"totalMs": 5581.313874999978,
"audioSecondsPerSecond": 0.46357877354970867,
"grade": "moderate",
"residentMemoryBytes": 9979953152,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.11956550180912018,
"audioSeconds": 2.32,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:39.309547+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.41802436113357544
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 913.5980409992044,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 4981.399374999455,
"inferenceMs": 4980.639249999513,
"realTimeFactor": 2.1468272629308247,
"audioSeconds": 2.32,
"seed": 42,
"eosReached": true,
"generatedTokens": 30,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.11956549435853958,
"peak": 0.6288480758666992,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "custom_voice",
"instruct": "",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "fp32",
"cpuThreads": 4,
"fallback": false,
"totalMs": 5902.578831999563,
"audioSecondsPerSecond": 0.46580366164849757,
"grade": "moderate",
"residentMemoryBytes": 9979985920,
"supportsStreaming": false
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.1649700105190277,
"audioSeconds": 2.4,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:39.415478+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.3360149562358856
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 332.1784170002502,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 1859.382333999747,
"inferenceMs": 1858.7289170009171,
"realTimeFactor": 0.7744703820837155,
"audioSeconds": 2.4,
"seed": 42,
"eosReached": true,
"generatedTokens": 31,
"maxNewTokens": 512,
"promptTokens": 9,
"rms": 0.16496999561786652,
"peak": 0.8428813219070435,
"clippedFraction": 0,
"chunks": 5,
"firstAudioMs": 469.610707999891,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583183708,
"peakMemoryBytes": 5237400109,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "custom_voice",
"instruct": "",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 2203.878166999857,
"audioSecondsPerSecond": 1.2912049616532735,
"grade": "moderate",
"residentMemoryBytes": 4893425664
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Please save this file before closing the window.",
"wordErrorRate": 0,
"rms": 0.09189192205667496,
"audioSeconds": 2.56,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:39.525977+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.3798825144767761
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 332.1784170002502,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 1540.2527499991265,
"inferenceMs": 1539.71808300048,
"realTimeFactor": 0.6014523761720625,
"audioSeconds": 2.56,
"seed": 42,
"eosReached": true,
"generatedTokens": 33,
"maxNewTokens": 512,
"promptTokens": 9,
"rms": 0.09189192205667496,
"peak": 0.525258481502533,
"clippedFraction": 0,
"chunks": 6,
"firstAudioMs": 332.0858329989278,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583200100,
"peakMemoryBytes": 5237400109,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "custom_voice",
"instruct": "",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 1881.582250000065,
"audioSecondsPerSecond": 1.6626420305535907,
"grade": "moderate",
"residentMemoryBytes": 4895375360
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "The blue box contains three small books.",
"wordErrorRate": 0,
"rms": 0.10940630733966827,
"audioSeconds": 2.64,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:39.741611+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.46773505210876465
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 332.1784170002502,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 1452.8746249998221,
"inferenceMs": 1452.6106250013981,
"realTimeFactor": 0.5502312973490144,
"audioSeconds": 2.64,
"seed": 42,
"eosReached": true,
"generatedTokens": 34,
"maxNewTokens": 512,
"promptTokens": 8,
"rms": 0.10940626263618469,
"peak": 0.7453674674034119,
"clippedFraction": 0,
"chunks": 6,
"firstAudioMs": 302.7335000006133,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583216488,
"peakMemoryBytes": 5237400109,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "custom_voice",
"instruct": "",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 1794.3940830009524,
"audioSecondsPerSecond": 1.8174175202645644,
"grade": "moderate",
"residentMemoryBytes": 4895571968
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Please save this file before closing the window.",
"wordErrorRate": 0,
"rms": 0.10392040014266968,
"audioSeconds": 3.04,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:39.851080+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.3559342622756958
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 332.1784170002502,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 1573.1981249991804,
"inferenceMs": 1572.9596660003153,
"realTimeFactor": 0.5174209427632617,
"audioSeconds": 3.04,
"seed": 42,
"eosReached": true,
"generatedTokens": 39,
"maxNewTokens": 512,
"promptTokens": 14,
"rms": 0.10392042994499207,
"peak": 0.6027777194976807,
"clippedFraction": 0,
"chunks": 7,
"firstAudioMs": 304.58741599977657,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583297404,
"peakMemoryBytes": 5237400109,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "custom_voice",
"instruct": "Speak slowly and clearly.",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 1916.3867919996846,
"audioSecondsPerSecond": 1.9326623979685633,
"grade": "moderate",
"residentMemoryBytes": 4895768576
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Please save this file before closing the window.",
"wordErrorRate": 0,
"rms": 0.10883376747369766,
"audioSeconds": 2.4,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:39.957360+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.37157905101776123
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 332.1784170002502,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 1244.8362079994695,
"inferenceMs": 1244.5335419997718,
"realTimeFactor": 0.5185556424999049,
"audioSeconds": 2.4,
"seed": 42,
"eosReached": true,
"generatedTokens": 31,
"maxNewTokens": 512,
"promptTokens": 14,
"rms": 0.10883378237485886,
"peak": 0.6034771800041199,
"clippedFraction": 0,
"chunks": 5,
"firstAudioMs": 267.50766699842643,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583232860,
"peakMemoryBytes": 5237400109,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "custom_voice",
"instruct": "Speak quickly and clearly.",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 1585.9787080007663,
"audioSecondsPerSecond": 1.9284333599748331,
"grade": "moderate",
"residentMemoryBytes": 4895457280
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.1649700105190277,
"audioSeconds": 2.4,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:40.068681+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.3360149562358856
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 332.1784170002502,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 1247.24229200001,
"inferenceMs": 1247.0442920002824,
"realTimeFactor": 0.519601788333451,
"audioSeconds": 2.4,
"seed": 42,
"eosReached": true,
"generatedTokens": 31,
"maxNewTokens": 512,
"promptTokens": 9,
"rms": 0.16496999561786652,
"peak": 0.8428813219070435,
"clippedFraction": 0,
"chunks": 5,
"firstAudioMs": 275.8472499990603,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583183708,
"peakMemoryBytes": 5237400109,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "custom_voice",
"instruct": "",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 1588.2446670002537,
"audioSecondsPerSecond": 1.9245507279860563,
"grade": "moderate",
"residentMemoryBytes": 4896718848
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.1649700105190277,
"audioSeconds": 2.4,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:40.176323+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.3360149562358856
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 309.9599999986822,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 1163.4656250007538,
"inferenceMs": 1162.8724170004716,
"realTimeFactor": 0.4845301737501965,
"audioSeconds": 2.4,
"seed": 42,
"eosReached": true,
"generatedTokens": 31,
"maxNewTokens": 512,
"promptTokens": 9,
"rms": 0.16496999561786652,
"peak": 0.8428813219070435,
"clippedFraction": 0,
"chunks": 5,
"firstAudioMs": 265.27337500010617,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583183708,
"peakMemoryBytes": 5237259287,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "custom_voice",
"instruct": "",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 1484.5112499988318,
"audioSecondsPerSecond": 2.0638549551210366,
"grade": "fast",
"residentMemoryBytes": 4893425664
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Please save this file before closing the window.",
"wordErrorRate": 0,
"rms": 0.09189192205667496,
"audioSeconds": 2.56,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:40.283484+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.3798825144767761
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 309.9599999986822,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 1197.3001249989466,
"inferenceMs": 1196.9590839999,
"realTimeFactor": 0.46756214218746095,
"audioSeconds": 2.56,
"seed": 42,
"eosReached": true,
"generatedTokens": 33,
"maxNewTokens": 512,
"promptTokens": 9,
"rms": 0.09189192205667496,
"peak": 0.525258481502533,
"clippedFraction": 0,
"chunks": 6,
"firstAudioMs": 231.13362500043877,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583200100,
"peakMemoryBytes": 5237259287,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "custom_voice",
"instruct": "",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 1516.6909999989002,
"audioSecondsPerSecond": 2.138753140537771,
"grade": "fast",
"residentMemoryBytes": 4894457856
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "The blue box contains three small books.",
"wordErrorRate": 0,
"rms": 0.10940630733966827,
"audioSeconds": 2.64,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:40.394653+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.46773505210876465
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 309.9599999986822,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 1248.6763329998212,
"inferenceMs": 1248.4351660004904,
"realTimeFactor": 0.4728921083335191,
"audioSeconds": 2.64,
"seed": 42,
"eosReached": true,
"generatedTokens": 34,
"maxNewTokens": 512,
"promptTokens": 8,
"rms": 0.10940626263618469,
"peak": 0.7453674674034119,
"clippedFraction": 0,
"chunks": 6,
"firstAudioMs": 241.4589159998286,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583216488,
"peakMemoryBytes": 5237259287,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "custom_voice",
"instruct": "",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 1567.6933339982497,
"audioSecondsPerSecond": 2.1146472575404553,
"grade": "fast",
"residentMemoryBytes": 4895768576
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Please save this file before closing the window.",
"wordErrorRate": 0,
"rms": 0.10392040014266968,
"audioSeconds": 3.04,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:40.506835+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.3559342622756958
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 309.9599999986822,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 1430.6194170003437,
"inferenceMs": 1430.3737080008432,
"realTimeFactor": 0.4705176671055405,
"audioSeconds": 3.04,
"seed": 42,
"eosReached": true,
"generatedTokens": 39,
"maxNewTokens": 512,
"promptTokens": 14,
"rms": 0.10392042994499207,
"peak": 0.6027777194976807,
"clippedFraction": 0,
"chunks": 7,
"firstAudioMs": 241.00308299966855,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583281020,
"peakMemoryBytes": 5237259287,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "custom_voice",
"instruct": "Speak slowly and clearly.",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 1751.0307499996998,
"audioSecondsPerSecond": 2.1253187072690576,
"grade": "fast",
"residentMemoryBytes": 4896342016
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Please save this file before closing the window.",
"wordErrorRate": 0,
"rms": 0.10883376747369766,
"audioSeconds": 2.4,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:40.613470+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.37157905101776123
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 309.9599999986822,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 1129.1795420002018,
"inferenceMs": 1128.9415419996658,
"realTimeFactor": 0.4703923091665274,
"audioSeconds": 2.4,
"seed": 42,
"eosReached": true,
"generatedTokens": 31,
"maxNewTokens": 512,
"promptTokens": 14,
"rms": 0.10883378237485886,
"peak": 0.6034771800041199,
"clippedFraction": 0,
"chunks": 5,
"firstAudioMs": 239.95954199926928,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583232860,
"peakMemoryBytes": 5237259287,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "custom_voice",
"instruct": "Speak quickly and clearly.",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 1447.2084589979204,
"audioSecondsPerSecond": 2.1258850974240353,
"grade": "fast",
"residentMemoryBytes": 4896129024
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.1649700105190277,
"audioSeconds": 2.4,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:40.721913+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.3360149562358856
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 309.9599999986822,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 1124.5804159989348,
"inferenceMs": 1124.3720000002213,
"realTimeFactor": 0.46848833333342554,
"audioSeconds": 2.4,
"seed": 42,
"eosReached": true,
"generatedTokens": 31,
"maxNewTokens": 512,
"promptTokens": 9,
"rms": 0.16496999561786652,
"peak": 0.8428813219070435,
"clippedFraction": 0,
"chunks": 5,
"firstAudioMs": 231.59229200064146,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583183708,
"peakMemoryBytes": 5237377425,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "custom_voice",
"instruct": "",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 1442.943624997497,
"audioSecondsPerSecond": 2.1345248725506574,
"grade": "fast",
"residentMemoryBytes": 4896129024
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.1649700105190277,
"audioSeconds": 2.4,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:40.830665+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.3360149562358856
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 311.9535839996388,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 1148.6714579987165,
"inferenceMs": 1148.0654589995538,
"realTimeFactor": 0.4783606079164808,
"audioSeconds": 2.4,
"seed": 42,
"eosReached": true,
"generatedTokens": 31,
"maxNewTokens": 512,
"promptTokens": 9,
"rms": 0.16496999561786652,
"peak": 0.8428813219070435,
"clippedFraction": 0,
"chunks": 5,
"firstAudioMs": 261.75858399983554,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583183708,
"peakMemoryBytes": 5237325363,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "custom_voice",
"instruct": "",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 1470.9914590002882,
"audioSecondsPerSecond": 2.0904731356445527,
"grade": "fast",
"residentMemoryBytes": 4895670272
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Please save this file before closing the window.",
"wordErrorRate": 0,
"rms": 0.09189192205667496,
"audioSeconds": 2.56,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:40.939071+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.3798825144767761
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 311.9535839996388,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 1203.9577909999934,
"inferenceMs": 1203.647667000041,
"realTimeFactor": 0.47017486992189106,
"audioSeconds": 2.56,
"seed": 42,
"eosReached": true,
"generatedTokens": 33,
"maxNewTokens": 512,
"promptTokens": 9,
"rms": 0.09189192205667496,
"peak": 0.525258481502533,
"clippedFraction": 0,
"chunks": 6,
"firstAudioMs": 234.7428339999169,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583200100,
"peakMemoryBytes": 5237325363,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "custom_voice",
"instruct": "",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 1525.4904169996735,
"audioSecondsPerSecond": 2.1268682440772033,
"grade": "fast",
"residentMemoryBytes": 4897406976
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "The blue box contains three small books.",
"wordErrorRate": 0,
"rms": 0.10940630733966827,
"audioSeconds": 2.64,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:41.044019+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.46773505210876465
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 311.9535839996388,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 1243.3321669996076,
"inferenceMs": 1243.0634579995967,
"realTimeFactor": 0.4708573704543927,
"audioSeconds": 2.64,
"seed": 42,
"eosReached": true,
"generatedTokens": 34,
"maxNewTokens": 512,
"promptTokens": 8,
"rms": 0.10940626263618469,
"peak": 0.7453674674034119,
"clippedFraction": 0,
"chunks": 6,
"firstAudioMs": 229.58366599959845,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583216488,
"peakMemoryBytes": 5237325363,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "custom_voice",
"instruct": "",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 1564.4410839995544,
"audioSecondsPerSecond": 2.1237853811972136,
"grade": "fast",
"residentMemoryBytes": 4898455552
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Please save this file before closing the window.",
"wordErrorRate": 0,
"rms": 0.10392040014266968,
"audioSeconds": 3.04,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:41.157607+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.3559342622756958
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 311.9535839996388,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 1425.6837500015536,
"inferenceMs": 1425.4284170001483,
"realTimeFactor": 0.46889092664478565,
"audioSeconds": 3.04,
"seed": 42,
"eosReached": true,
"generatedTokens": 39,
"maxNewTokens": 512,
"promptTokens": 14,
"rms": 0.10392042994499207,
"peak": 0.6027777194976807,
"clippedFraction": 0,
"chunks": 7,
"firstAudioMs": 240.68145899946103,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583297404,
"peakMemoryBytes": 5237325363,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "custom_voice",
"instruct": "Speak slowly and clearly.",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 1748.2165839992376,
"audioSecondsPerSecond": 2.1326921532810186,
"grade": "fast",
"residentMemoryBytes": 4898340864
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Please save this file before closing the window.",
"wordErrorRate": 0,
"rms": 0.10883376747369766,
"audioSeconds": 2.4,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:41.266771+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.37157905101776123
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 311.9535839996388,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 1135.5378329990344,
"inferenceMs": 1135.2879579990258,
"realTimeFactor": 0.47303664916626076,
"audioSeconds": 2.4,
"seed": 42,
"eosReached": true,
"generatedTokens": 31,
"maxNewTokens": 512,
"promptTokens": 14,
"rms": 0.10883378237485886,
"peak": 0.6034771800041199,
"clippedFraction": 0,
"chunks": 5,
"firstAudioMs": 236.07204099971568,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583232860,
"peakMemoryBytes": 5237325363,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "custom_voice",
"instruct": "Speak quickly and clearly.",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 1455.4647500008286,
"audioSecondsPerSecond": 2.114001107023157,
"grade": "fast",
"residentMemoryBytes": 4898357248
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.1649700105190277,
"audioSeconds": 2.4,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:41.373601+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.3360149562358856
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 311.9535839996388,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 1134.85770799889,
"inferenceMs": 1134.6415420011908,
"realTimeFactor": 0.47276730916716286,
"audioSeconds": 2.4,
"seed": 42,
"eosReached": true,
"generatedTokens": 31,
"maxNewTokens": 512,
"promptTokens": 9,
"rms": 0.16496999561786652,
"peak": 0.8428813219070435,
"clippedFraction": 0,
"chunks": 5,
"firstAudioMs": 242.55087500023365,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583183708,
"peakMemoryBytes": 5237325363,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "custom_voice",
"instruct": "",
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 1454.9524590001965,
"audioSecondsPerSecond": 2.1152054734106334,
"grade": "fast",
"residentMemoryBytes": 4898979840
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.14905516803264618,
"audioSeconds": 2.72,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:28:57.674754+00:00",
"structuralPassed": true,
"performancePassed": false,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.33941158652305603
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 1626.769709000655,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.bfloat16"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.bfloat16"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.bfloat16",
"attentionImplementation": "sdpa",
"durationMs": 16012.851916999352,
"inferenceMs": 16007.618292000188,
"realTimeFactor": 5.885153783823598,
"audioSeconds": 2.72,
"seed": 42,
"eosReached": true,
"generatedTokens": 35,
"generationPlacement": "mps:0",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.14905516803264618,
"peak": 0.83203125,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "custom_voice",
"instruct": "",
"driverAllocatedMemoryBytes": 5175934976,
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 17648.293876001844,
"audioSecondsPerSecond": 0.16991909417026274,
"grade": "slow",
"residentMemoryBytes": 1181237248,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.12418601661920547,
"audioSeconds": 2.24,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:28:57.910852+00:00",
"structuralPassed": true,
"performancePassed": false,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.3386712074279785
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 1574.0860829992016,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.float32"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 9536.391583000295,
"inferenceMs": 9528.927832998306,
"realTimeFactor": 4.253985639731386,
"audioSeconds": 2.24,
"seed": 42,
"eosReached": true,
"generatedTokens": 29,
"generationPlacement": "mps:0",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.12418593466281891,
"peak": 0.8332152366638184,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "custom_voice",
"instruct": "",
"driverAllocatedMemoryBytes": 9745317888,
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "fp32",
"cpuThreads": 4,
"fallback": false,
"totalMs": 11121.686000000409,
"audioSecondsPerSecond": 0.23507366613093317,
"grade": "slow",
"residentMemoryBytes": 1192116224,
"supportsStreaming": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.13684940338134766,
"audioSeconds": 2.48,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:28:58.115460+00:00",
"structuralPassed": true,
"performancePassed": false,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.38086578249931335
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 1412.1661669996683,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.float16"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.float16"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float16",
"attentionImplementation": "sdpa",
"durationMs": 10818.091000001004,
"inferenceMs": 10811.847584000134,
"realTimeFactor": 4.359615961290377,
"audioSeconds": 2.48,
"seed": 42,
"eosReached": true,
"generatedTokens": 32,
"generationPlacement": "mps:0",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.13684940338134766,
"peak": 0.83349609375,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "custom_voice",
"instruct": "",
"driverAllocatedMemoryBytes": 5175754752,
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "fp16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 12239.815082999485,
"audioSecondsPerSecond": 0.22937800230092192,
"grade": "slow",
"residentMemoryBytes": 1178779648,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.10639488697052002,
"audioSeconds": 2.96,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:29:03.288988+00:00",
"structuralPassed": true,
"performancePassed": false,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.40386563539505005
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 1622.8039170000557,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.float32"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "eager",
"durationMs": 14309.05412500033,
"inferenceMs": 14303.209292000247,
"realTimeFactor": 4.832165301351435,
"audioSeconds": 2.96,
"seed": 42,
"eosReached": true,
"generatedTokens": 38,
"generationPlacement": "mps:0",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.10639490932226181,
"peak": 0.8179582953453064,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "custom_voice",
"instruct": "",
"driverAllocatedMemoryBytes": 9779249152,
"artifactRevision": "0c0e3051f131929182e2c023b9537f8b1c68adfe",
"precision": "fp32",
"cpuThreads": 4,
"fallback": false,
"totalMs": 15942.233916999612,
"audioSecondsPerSecond": 0.20694656280080592,
"grade": "slow",
"residentMemoryBytes": 1334312960,
"supportsStreaming": false
}
}
]Qwen3-TTS 1.7B · VoiceDesign — Described Voices
28 configurations testedQuality criteria
{ "maximumWordErrorRate": 0.25, "minimumSpeakerCosine": null, "minimumRms": 0.005, "maximumClippedFraction": 0.02, "eosRequired": true, "maximumRealTimeFactor": 3, "maximumRequestMs": 45000 }
| Precision | Device | Result | Platform | Inference / load | Task metric | |
|---|---|---|---|---|---|---|
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1962 ms / 324 ms | 0.584 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 2115 ms / 321 ms | 0.630 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 2121 ms / 324 ms | 0.631 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 2142 ms / 321 ms | 0.637 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1741 ms / 321 ms | 0.680 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1745 ms / 400 ms | 0.682 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 2312 ms / 400 ms | 0.688 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1933 ms / 324 ms | 0.755 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1768 ms / 321 ms | 0.762 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 2689 ms / 400 ms | 0.800 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1903 ms / 400 ms | 0.820 RTF |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 2197 ms / 324 ms | 0.947 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 5500 ms / 1136 ms | 2.148 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 5524 ms / 1526 ms | 2.158 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 6910 ms / 1526 ms | 2.159 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 5573 ms / 2875 ms | 2.177 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 6982 ms / 2875 ms | 2.182 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 6994 ms / 1136 ms | 2.186 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 6998 ms / 1526 ms | 2.187 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 7034 ms / 1136 ms | 2.198 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 7139 ms / 2875 ms | 2.231 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 5402 ms / 1526 ms | 2.251 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 5412 ms / 2875 ms | 2.255 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 5553 ms / 1136 ms | 2.314 RTF |
FP32 | cpu | Failed | — | macos · aarch64 | 7990 ms / 945 ms | 3.329 RTF |
FP32 | cpu | Failed | — | macos · aarch64 | 10018 ms / 945 ms | 3.130 RTF |
FP16 | mps | Failed | — | macos · aarch64 | 8884 ms / 1359 ms | 3.001 RTF |
FP32 | mps | Failed | — | macos · aarch64 | 15231 ms / 1619 ms | 4.231 RTF |
Test setup & sample details
- Quality gate
- { "maximumWordErrorRate": 0.25, "minimumSpeakerCosine": null, "minimumRms": 0.005, "maximumClippedFraction": 0.02, "eosRequired": true, "maximumRealTimeFactor": 3, "maximumRequestMs": 45000 }
- Tested
- 2026-11-06T14:34:13.031046+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2", "runtime": "App-owned CPython3.13.18; CPU qwen-tts0.1.1/Transformers4.57.3/Torch2.10.0; Metal mlx-audio0.6.3/MLX0.32.4/Transformers5.22.1; original identical checkpoint files" }
- Runtime
- mlx-audio 0.6.3
- Artifact revision
- 5ecdb67327fd37bb2e042aab12ff7391903235d3
- Fixture
{ "file": "model-samples/desktop/qwen3-tts-family/voicedesign-low-input.json", "url": "https://assets.localai.tools/samples/v1/files/eadae4dc62232f2fad097aa8ea3eb8b591aca626a0b4bd1495f907feb345e2f7/voicedesign-low-input.json", "size": 319, "sha1": "3b42d13dc8eb04fa9a98afe920da4b5339816fcd", "sha256": "eadae4dc62232f2fad097aa8ea3eb8b591aca626a0b4bd1495f907feb345e2f7" }- Rerun when
- artifact revision/checksum; qwen-tts/Transformers/Torch versions, preprocessing or sampling; OS/device/backend behavior; fixture text, voice/language selections or independent ASR/quality/performance gates
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": false,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.1320529729127884,
"audioSeconds": 2.4,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:44.457236+00:00",
"structuralPassed": true,
"performancePassed": false,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.4736924171447754
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 945.3067080012261,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 7991.344707999815,
"inferenceMs": 7990.082292000807,
"realTimeFactor": 3.3292009550003363,
"audioSeconds": 2.4,
"seed": 42,
"eosReached": true,
"generatedTokens": 31,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.13205301761627197,
"peak": 0.7125951647758484,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "voice_design",
"instruct": "An adult female speaker with a bright, high-pitched voice, speaking clearly at a natural pace.",
"artifactRevision": "5ecdb67327fd37bb2e042aab12ff7391903235d3",
"precision": "fp32",
"cpuThreads": 4,
"fallback": false,
"totalMs": 8950.60954100154,
"audioSecondsPerSecond": 0.3003723756891386,
"grade": "slow",
"residentMemoryBytes": 9637494784,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": false,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.0951128676533699,
"audioSeconds": 3.2,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:44.772603+00:00",
"structuralPassed": true,
"performancePassed": false,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.4283345639705658
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 945.3067080012261,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 10019.167708000168,
"inferenceMs": 10017.56770799875,
"realTimeFactor": 3.1304899087496096,
"audioSeconds": 3.2,
"seed": 42,
"eosReached": true,
"generatedTokens": 41,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.0951128825545311,
"peak": 0.6588311195373535,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "voice_design",
"instruct": "An adult male speaker with a deep, low-pitched voice, speaking clearly at a natural pace.",
"artifactRevision": "5ecdb67327fd37bb2e042aab12ff7391903235d3",
"precision": "fp32",
"cpuThreads": 4,
"fallback": false,
"totalMs": 10997.836249000102,
"audioSecondsPerSecond": 0.3194388192100652,
"grade": "slow",
"residentMemoryBytes": 9670164480,
"supportsStreaming": false
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.12029409408569336,
"audioSeconds": 3.36,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:46.340434+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.5384488105773926
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 399.66295900012483,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 2690.5820419997326,
"inferenceMs": 2688.8882080002077,
"realTimeFactor": 0.8002643476191095,
"audioSeconds": 3.36,
"seed": 42,
"eosReached": true,
"generatedTokens": 43,
"maxNewTokens": 512,
"promptTokens": 29,
"rms": 0.12029409408569336,
"peak": 0.7411079406738281,
"clippedFraction": 0,
"chunks": 7,
"firstAudioMs": 451.3404580011411,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583232908,
"peakMemoryBytes": 5237470947,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "voice_design",
"instruct": "An adult male speaker with a deep, low-pitched voice, speaking clearly at a natural pace.",
"artifactRevision": "5ecdb67327fd37bb2e042aab12ff7391903235d3",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 3109.20945900034,
"audioSecondsPerSecond": 1.2495870932837756,
"grade": "moderate",
"residentMemoryBytes": 4893720576
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.10172471404075623,
"audioSeconds": 2.32,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:46.512416+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.5870595574378967
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 399.66295900012483,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 1903.4165000011853,
"inferenceMs": 1902.9447920001985,
"realTimeFactor": 0.8202348241380166,
"audioSeconds": 2.32,
"seed": 42,
"eosReached": true,
"generatedTokens": 30,
"maxNewTokens": 512,
"promptTokens": 29,
"rms": 0.10172471404075623,
"peak": 0.6221702694892883,
"clippedFraction": 0,
"chunks": 5,
"firstAudioMs": 562.5405840000894,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583134552,
"peakMemoryBytes": 5237470947,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "voice_design",
"instruct": "An adult female speaker with a bright, high-pitched voice, speaking clearly at a natural pace.",
"artifactRevision": "5ecdb67327fd37bb2e042aab12ff7391903235d3",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 2317.4331250011164,
"audioSecondsPerSecond": 1.2191630622985292,
"grade": "moderate",
"residentMemoryBytes": 4895719424
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Please save this file before closing the window.",
"wordErrorRate": 0,
"rms": 0.11006369441747665,
"audioSeconds": 2.56,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:46.664741+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.5889490842819214
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 399.66295900012483,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 1745.6557500008785,
"inferenceMs": 1745.2004169990687,
"realTimeFactor": 0.6817189128902612,
"audioSeconds": 2.56,
"seed": 42,
"eosReached": true,
"generatedTokens": 33,
"maxNewTokens": 512,
"promptTokens": 29,
"rms": 0.11006369441747665,
"peak": 0.560906708240509,
"clippedFraction": 0,
"chunks": 6,
"firstAudioMs": 321.07941699905496,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583150948,
"peakMemoryBytes": 5237470947,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "voice_design",
"instruct": "An adult female speaker with a bright, high-pitched voice, speaking clearly at a natural pace.",
"artifactRevision": "5ecdb67327fd37bb2e042aab12ff7391903235d3",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 2160.7656680007494,
"audioSecondsPerSecond": 1.4668802362550468,
"grade": "moderate",
"residentMemoryBytes": 4896014336
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.12029409408569336,
"audioSeconds": 3.36,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:46.825960+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.5384488105773926
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 399.66295900012483,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 2312.5957920001383,
"inferenceMs": 2312.056292001216,
"realTimeFactor": 0.6881119916670286,
"audioSeconds": 3.36,
"seed": 42,
"eosReached": true,
"generatedTokens": 43,
"maxNewTokens": 512,
"promptTokens": 29,
"rms": 0.12029409408569336,
"peak": 0.7411079406738281,
"clippedFraction": 0,
"chunks": 7,
"firstAudioMs": 342.13933300088684,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583232908,
"peakMemoryBytes": 5237470947,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "voice_design",
"instruct": "An adult male speaker with a deep, low-pitched voice, speaking clearly at a natural pace.",
"artifactRevision": "5ecdb67327fd37bb2e042aab12ff7391903235d3",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 2730.5923759995494,
"audioSecondsPerSecond": 1.4532518138179624,
"grade": "moderate",
"residentMemoryBytes": 4895899648
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.12029409408569336,
"audioSeconds": 3.36,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:47.002470+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.5384488105773926
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 320.93075000011595,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 2115.962167001271,
"inferenceMs": 2115.226750000147,
"realTimeFactor": 0.6295317708333772,
"audioSeconds": 3.36,
"seed": 42,
"eosReached": true,
"generatedTokens": 43,
"maxNewTokens": 512,
"promptTokens": 29,
"rms": 0.12029409408569336,
"peak": 0.7411079406738281,
"clippedFraction": 0,
"chunks": 7,
"firstAudioMs": 274.1260000002512,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583282060,
"peakMemoryBytes": 5237782271,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "voice_design",
"instruct": "An adult male speaker with a deep, low-pitched voice, speaking clearly at a natural pace.",
"artifactRevision": "5ecdb67327fd37bb2e042aab12ff7391903235d3",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 2457.0431249994726,
"audioSecondsPerSecond": 1.588482180456429,
"grade": "moderate",
"residentMemoryBytes": 4894162944
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.10172471404075623,
"audioSeconds": 2.32,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:47.169490+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.5870595574378967
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 320.93075000011595,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 1768.2426669998677,
"inferenceMs": 1767.605791999813,
"realTimeFactor": 0.7618990482757815,
"audioSeconds": 2.32,
"seed": 42,
"eosReached": true,
"generatedTokens": 30,
"maxNewTokens": 512,
"promptTokens": 29,
"rms": 0.10172471404075623,
"peak": 0.6221702694892883,
"clippedFraction": 0,
"chunks": 5,
"firstAudioMs": 492.14095800016366,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583183704,
"peakMemoryBytes": 5237782271,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "voice_design",
"instruct": "An adult female speaker with a bright, high-pitched voice, speaking clearly at a natural pace.",
"artifactRevision": "5ecdb67327fd37bb2e042aab12ff7391903235d3",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 2102.360625001893,
"audioSecondsPerSecond": 1.312509842692485,
"grade": "moderate",
"residentMemoryBytes": 4896423936
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Please save this file before closing the window.",
"wordErrorRate": 0,
"rms": 0.11006369441747665,
"audioSeconds": 2.56,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:47.319802+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.5889490842819214
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 320.93075000011595,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 1741.5440420008963,
"inferenceMs": 1741.0897499994462,
"realTimeFactor": 0.6801131835935337,
"audioSeconds": 2.56,
"seed": 42,
"eosReached": true,
"generatedTokens": 33,
"maxNewTokens": 512,
"promptTokens": 29,
"rms": 0.11006369441747665,
"peak": 0.560906708240509,
"clippedFraction": 0,
"chunks": 6,
"firstAudioMs": 332.33279200067045,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583200100,
"peakMemoryBytes": 5237782271,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "voice_design",
"instruct": "An adult female speaker with a bright, high-pitched voice, speaking clearly at a natural pace.",
"artifactRevision": "5ecdb67327fd37bb2e042aab12ff7391903235d3",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 2075.6133750001027,
"audioSecondsPerSecond": 1.470343501821669,
"grade": "moderate",
"residentMemoryBytes": 4896653312
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.12029409408569336,
"audioSeconds": 3.36,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:47.540015+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.5384488105773926
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 320.93075000011595,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 2142.1050829994783,
"inferenceMs": 2141.7555839998386,
"realTimeFactor": 0.6374272571428091,
"audioSeconds": 3.36,
"seed": 42,
"eosReached": true,
"generatedTokens": 43,
"maxNewTokens": 512,
"promptTokens": 29,
"rms": 0.12029409408569336,
"peak": 0.7411079406738281,
"clippedFraction": 0,
"chunks": 7,
"firstAudioMs": 304.4497089995275,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583282060,
"peakMemoryBytes": 5237782271,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "voice_design",
"instruct": "An adult male speaker with a deep, low-pitched voice, speaking clearly at a natural pace.",
"artifactRevision": "5ecdb67327fd37bb2e042aab12ff7391903235d3",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 2478.1757079999807,
"audioSecondsPerSecond": 1.5688064619049702,
"grade": "moderate",
"residentMemoryBytes": 4898177024
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.12029409408569336,
"audioSeconds": 3.36,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:47.726086+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.5384488105773926
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 324.4161669990717,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 1962.460791999547,
"inferenceMs": 1961.6782080011035,
"realTimeFactor": 0.5838328000003284,
"audioSeconds": 3.36,
"seed": 42,
"eosReached": true,
"generatedTokens": 43,
"maxNewTokens": 512,
"promptTokens": 29,
"rms": 0.12029409408569336,
"peak": 0.7411079406738281,
"clippedFraction": 0,
"chunks": 7,
"firstAudioMs": 268.0612080002902,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583282060,
"peakMemoryBytes": 5237665105,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "voice_design",
"instruct": "An adult male speaker with a deep, low-pitched voice, speaking clearly at a natural pace.",
"artifactRevision": "5ecdb67327fd37bb2e042aab12ff7391903235d3",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 2324.9722509990534,
"audioSecondsPerSecond": 1.71281914959118,
"grade": "moderate",
"residentMemoryBytes": 4894081024
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.10172471404075623,
"audioSeconds": 2.32,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:47.903799+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.5870595574378967
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 324.4161669990717,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 2198.0918339995696,
"inferenceMs": 2197.0570419998694,
"realTimeFactor": 0.947007345689599,
"audioSeconds": 2.32,
"seed": 42,
"eosReached": true,
"generatedTokens": 30,
"maxNewTokens": 512,
"promptTokens": 29,
"rms": 0.10172471404075623,
"peak": 0.6221702694892883,
"clippedFraction": 0,
"chunks": 5,
"firstAudioMs": 317.72470800024166,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583183704,
"peakMemoryBytes": 5237665105,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "voice_design",
"instruct": "An adult female speaker with a bright, high-pitched voice, speaking clearly at a natural pace.",
"artifactRevision": "5ecdb67327fd37bb2e042aab12ff7391903235d3",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 2555.3507919994445,
"audioSecondsPerSecond": 1.0559580182261548,
"grade": "moderate",
"residentMemoryBytes": 4896161792
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Please save this file before closing the window.",
"wordErrorRate": 0,
"rms": 0.11006369441747665,
"audioSeconds": 2.56,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:48.055471+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.5889490842819214
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 324.4161669990717,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 1934.399540999948,
"inferenceMs": 1932.7620410003874,
"realTimeFactor": 0.7549851722657763,
"audioSeconds": 2.56,
"seed": 42,
"eosReached": true,
"generatedTokens": 33,
"maxNewTokens": 512,
"promptTokens": 29,
"rms": 0.11006369441747665,
"peak": 0.560906708240509,
"clippedFraction": 0,
"chunks": 6,
"firstAudioMs": 368.61995800063596,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583200100,
"peakMemoryBytes": 5237665105,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "voice_design",
"instruct": "An adult female speaker with a bright, high-pitched voice, speaking clearly at a natural pace.",
"artifactRevision": "5ecdb67327fd37bb2e042aab12ff7391903235d3",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 2283.8211669986777,
"audioSecondsPerSecond": 1.3245293241970737,
"grade": "moderate",
"residentMemoryBytes": 4897488896
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.12029409408569336,
"audioSeconds": 3.36,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:48.219368+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.5384488105773926
},
"runtime": {
"engine": "mlx-audio",
"version": "0.6.3",
"mlxVersion": "0.32.4",
"transformersVersion": "5.22.1",
"loadMs": 324.4161669990717,
"device": "mps",
"provider": "MLX Metal",
"acceleration": "partial",
"parameterDtype": "Original mixed BF16 talker / FP32 codec",
"parameterDtypes": {
"mlx.core.bfloat16": 404,
"mlx.core.float32": 464
},
"placement": {
"talker": "MLX default GPU",
"speechTokenizer": "MLX default GPU",
"speakerEncoder": "not applicable"
},
"accelerationDetails": "Neural tensor evaluation on explicit MLX GPU; CPU tokenizer, WAV/resampling and serialization; no CPU device fallback",
"supportsStreaming": true,
"strictCodecLoading": true,
"restoredOriginalCodebookStateFlags": 32,
"durationMs": 2121.943416999784,
"inferenceMs": 2121.306167000512,
"realTimeFactor": 0.6313411211311047,
"audioSeconds": 3.36,
"seed": 42,
"eosReached": true,
"generatedTokens": 43,
"maxNewTokens": 512,
"promptTokens": 29,
"rms": 0.12029409408569336,
"peak": 0.7411079406738281,
"clippedFraction": 0,
"chunks": 7,
"firstAudioMs": 337.9477920007048,
"generationPlacement": "MLX GPU",
"activeMemoryBytes": 4583282060,
"peakMemoryBytes": 5237665105,
"sampling": "Seeded .9/50/1.0/1.05; main/subtalker defaults from pinned MLX implementation",
"taskMode": "voice_design",
"instruct": "An adult male speaker with a deep, low-pitched voice, speaking clearly at a natural pace.",
"artifactRevision": "5ecdb67327fd37bb2e042aab12ff7391903235d3",
"precision": "bf16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 2467.543458998989,
"audioSecondsPerSecond": 1.5839297750927572,
"grade": "moderate",
"residentMemoryBytes": 4896980992
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.0951128676533699,
"audioSeconds": 3.2,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:48.385325+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.4283345639705658
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 1135.9806670006947,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 6999.244874999931,
"inferenceMs": 6994.063833000837,
"realTimeFactor": 2.1856449478127615,
"audioSeconds": 3.2,
"seed": 42,
"eosReached": true,
"generatedTokens": 41,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.0951128825545311,
"peak": 0.6588311195373535,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "voice_design",
"instruct": "An adult male speaker with a deep, low-pitched voice, speaking clearly at a natural pace.",
"artifactRevision": "5ecdb67327fd37bb2e042aab12ff7391903235d3",
"precision": "fp32",
"cpuThreads": 2,
"fallback": false,
"totalMs": 8146.110666999448,
"audioSecondsPerSecond": 0.4575308542225621,
"grade": "moderate",
"residentMemoryBytes": 9603350528,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.1320529729127884,
"audioSeconds": 2.4,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:48.551474+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.4736924171447754
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 1135.9806670006947,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 5553.613957999914,
"inferenceMs": 5552.744041999176,
"realTimeFactor": 2.31364335083299,
"audioSeconds": 2.4,
"seed": 42,
"eosReached": true,
"generatedTokens": 31,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.13205301761627197,
"peak": 0.7125951647758484,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "voice_design",
"instruct": "An adult female speaker with a bright, high-pitched voice, speaking clearly at a natural pace.",
"artifactRevision": "5ecdb67327fd37bb2e042aab12ff7391903235d3",
"precision": "fp32",
"cpuThreads": 2,
"fallback": false,
"totalMs": 6697.437250000803,
"audioSecondsPerSecond": 0.4322187339893878,
"grade": "moderate",
"residentMemoryBytes": 9621192704,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Please save this file before closing the window.",
"wordErrorRate": 0,
"rms": 0.09285616129636765,
"audioSeconds": 2.56,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:48.698776+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.49859219789505005
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 1135.9806670006947,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 5500.997332999759,
"inferenceMs": 5500.123750000057,
"realTimeFactor": 2.1484858398437723,
"audioSeconds": 2.56,
"seed": 42,
"eosReached": true,
"generatedTokens": 33,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.09285619109869003,
"peak": 0.5594902634620667,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "voice_design",
"instruct": "An adult female speaker with a bright, high-pitched voice, speaking clearly at a natural pace.",
"artifactRevision": "5ecdb67327fd37bb2e042aab12ff7391903235d3",
"precision": "fp32",
"cpuThreads": 2,
"fallback": false,
"totalMs": 6645.552042000418,
"audioSecondsPerSecond": 0.46544407296289897,
"grade": "moderate",
"residentMemoryBytes": 9654960128,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.0951128676533699,
"audioSeconds": 3.2,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:48.863012+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.4283345639705658
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 1135.9806670006947,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 7034.582166999826,
"inferenceMs": 7033.7211250007385,
"realTimeFactor": 2.198037851562731,
"audioSeconds": 3.2,
"seed": 42,
"eosReached": true,
"generatedTokens": 41,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.0951128825545311,
"peak": 0.6588311195373535,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "voice_design",
"instruct": "An adult male speaker with a deep, low-pitched voice, speaking clearly at a natural pace.",
"artifactRevision": "5ecdb67327fd37bb2e042aab12ff7391903235d3",
"precision": "fp32",
"cpuThreads": 2,
"fallback": false,
"totalMs": 8180.908626001838,
"audioSecondsPerSecond": 0.4549512190106434,
"grade": "moderate",
"residentMemoryBytes": 9658941440,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.0951128676533699,
"audioSeconds": 3.2,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:49.027861+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.4283345639705658
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 1525.945083998522,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 6915.3922500008775,
"inferenceMs": 6909.774749999997,
"realTimeFactor": 2.159304609374999,
"audioSeconds": 3.2,
"seed": 42,
"eosReached": true,
"generatedTokens": 41,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.0951128825545311,
"peak": 0.6588311195373535,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "voice_design",
"instruct": "An adult male speaker with a deep, low-pitched voice, speaking clearly at a natural pace.",
"artifactRevision": "5ecdb67327fd37bb2e042aab12ff7391903235d3",
"precision": "fp32",
"cpuThreads": 2,
"fallback": false,
"totalMs": 8451.378624997233,
"audioSecondsPerSecond": 0.4631120573069334,
"grade": "moderate",
"residentMemoryBytes": 9621962752,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.1320529729127884,
"audioSeconds": 2.4,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:49.182432+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.4736924171447754
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 1525.945083998522,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 5403.293041999859,
"inferenceMs": 5402.368625000236,
"realTimeFactor": 2.2509869270834315,
"audioSeconds": 2.4,
"seed": 42,
"eosReached": true,
"generatedTokens": 31,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.13205301761627197,
"peak": 0.7125951647758484,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "voice_design",
"instruct": "An adult female speaker with a bright, high-pitched voice, speaking clearly at a natural pace.",
"artifactRevision": "5ecdb67327fd37bb2e042aab12ff7391903235d3",
"precision": "fp32",
"cpuThreads": 2,
"fallback": false,
"totalMs": 6937.3372929985635,
"audioSecondsPerSecond": 0.44424958135837966,
"grade": "moderate",
"residentMemoryBytes": 9636134912,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Please save this file before closing the window.",
"wordErrorRate": 0,
"rms": 0.09285616129636765,
"audioSeconds": 2.56,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:49.327084+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.49859219789505005
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 1525.945083998522,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 5524.569958999564,
"inferenceMs": 5523.696042000665,
"realTimeFactor": 2.15769376640651,
"audioSeconds": 2.56,
"seed": 42,
"eosReached": true,
"generatedTokens": 33,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.09285619109869003,
"peak": 0.5594902634620667,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "voice_design",
"instruct": "An adult female speaker with a bright, high-pitched voice, speaking clearly at a natural pace.",
"artifactRevision": "5ecdb67327fd37bb2e042aab12ff7391903235d3",
"precision": "fp32",
"cpuThreads": 2,
"fallback": false,
"totalMs": 7058.979792998798,
"audioSecondsPerSecond": 0.4634577971949333,
"grade": "moderate",
"residentMemoryBytes": 9668591616,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.0951128676533699,
"audioSeconds": 3.2,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:49.482042+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.4283345639705658
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 1525.945083998522,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 6998.531291999825,
"inferenceMs": 6997.636000000057,
"realTimeFactor": 2.1867612500000178,
"audioSeconds": 3.2,
"seed": 42,
"eosReached": true,
"generatedTokens": 41,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.0951128825545311,
"peak": 0.6588311195373535,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "voice_design",
"instruct": "An adult male speaker with a deep, low-pitched voice, speaking clearly at a natural pace.",
"artifactRevision": "5ecdb67327fd37bb2e042aab12ff7391903235d3",
"precision": "fp32",
"cpuThreads": 2,
"fallback": false,
"totalMs": 8534.717791997537,
"audioSecondsPerSecond": 0.4572972929715084,
"grade": "moderate",
"residentMemoryBytes": 9671770112,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.0951128676533699,
"audioSeconds": 3.2,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:49.643961+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.4283345639705658
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 2875.0193340001715,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 6986.804041000141,
"inferenceMs": 6981.575957999667,
"realTimeFactor": 2.181742486874896,
"audioSeconds": 3.2,
"seed": 42,
"eosReached": true,
"generatedTokens": 41,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.0951128825545311,
"peak": 0.6588311195373535,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "voice_design",
"instruct": "An adult male speaker with a deep, low-pitched voice, speaking clearly at a natural pace.",
"artifactRevision": "5ecdb67327fd37bb2e042aab12ff7391903235d3",
"precision": "fp32",
"cpuThreads": 2,
"fallback": false,
"totalMs": 9871.83354300032,
"audioSecondsPerSecond": 0.45834923507970415,
"grade": "moderate",
"residentMemoryBytes": 9616031744,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.1320529729127884,
"audioSeconds": 2.4,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:49.797642+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.4736924171447754
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 2875.0193340001715,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 5413.329917000738,
"inferenceMs": 5412.345291999372,
"realTimeFactor": 2.255143871666405,
"audioSeconds": 2.4,
"seed": 42,
"eosReached": true,
"generatedTokens": 31,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.13205301761627197,
"peak": 0.7125951647758484,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "voice_design",
"instruct": "An adult female speaker with a bright, high-pitched voice, speaking clearly at a natural pace.",
"artifactRevision": "5ecdb67327fd37bb2e042aab12ff7391903235d3",
"precision": "fp32",
"cpuThreads": 2,
"fallback": false,
"totalMs": 8296.505917000104,
"audioSecondsPerSecond": 0.44343068864207974,
"grade": "moderate",
"residentMemoryBytes": 9623191552,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Please save this file before closing the window.",
"wordErrorRate": 0,
"rms": 0.09285616129636765,
"audioSeconds": 2.56,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:49.943168+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.49859219789505005
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 2875.0193340001715,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 5573.498708999978,
"inferenceMs": 5572.665957999561,
"realTimeFactor": 2.1768226398435786,
"audioSeconds": 2.56,
"seed": 42,
"eosReached": true,
"generatedTokens": 33,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.09285619109869003,
"peak": 0.5594902634620667,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "voice_design",
"instruct": "An adult female speaker with a bright, high-pitched voice, speaking clearly at a natural pace.",
"artifactRevision": "5ecdb67327fd37bb2e042aab12ff7391903235d3",
"precision": "fp32",
"cpuThreads": 2,
"fallback": false,
"totalMs": 8457.178084001498,
"audioSecondsPerSecond": 0.4593851523300298,
"grade": "moderate",
"residentMemoryBytes": 9658433536,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.0951128676533699,
"audioSeconds": 3.2,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:43:50.126428+00:00",
"structuralPassed": true,
"performancePassed": true,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.4283345639705658
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 2875.0193340001715,
"placement": {
"talker": {
"device": "cpu",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "cpu",
"dtype": "torch.float32"
}
},
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "sdpa",
"durationMs": 7140.439374999914,
"inferenceMs": 7139.469958001428,
"realTimeFactor": 2.231084361875446,
"audioSeconds": 3.2,
"seed": 42,
"eosReached": true,
"generatedTokens": 41,
"generationPlacement": "cpu",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.0951128825545311,
"peak": 0.6588311195373535,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "voice_design",
"instruct": "An adult male speaker with a deep, low-pitched voice, speaking clearly at a natural pace.",
"artifactRevision": "5ecdb67327fd37bb2e042aab12ff7391903235d3",
"precision": "fp32",
"cpuThreads": 2,
"fallback": false,
"totalMs": 10025.867124999422,
"audioSecondsPerSecond": 0.4482125450242507,
"grade": "moderate",
"residentMemoryBytes": 9658679296,
"supportsStreaming": false
}
},
{
"precision": "fp16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.12078907340765,
"audioSeconds": 2.96,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:29:03.049148+00:00",
"structuralPassed": true,
"performancePassed": false,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.46483319997787476
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 1359.3380419988534,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.float16"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.float16"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float16",
"attentionImplementation": "sdpa",
"durationMs": 8889.750374999494,
"inferenceMs": 8884.267333000025,
"realTimeFactor": 3.0014416665540624,
"audioSeconds": 2.96,
"seed": 42,
"eosReached": true,
"generatedTokens": 38,
"generationPlacement": "mps:0",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.12078910320997238,
"peak": 0.6474609375,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "voice_design",
"instruct": "An adult male speaker with a deep, low-pitched voice, speaking clearly at a natural pace.",
"driverAllocatedMemoryBytes": 5184684032,
"artifactRevision": "5ecdb67327fd37bb2e042aab12ff7391903235d3",
"precision": "fp16",
"cpuThreads": 4,
"fallback": false,
"totalMs": 10259.047875997567,
"audioSecondsPerSecond": 0.333173225101554,
"grade": "slow",
"residentMemoryBytes": 1217282048,
"supportsStreaming": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"transcript": "Good morning. The train leaves at nine.",
"wordErrorRate": 0,
"rms": 0.07820147275924683,
"audioSeconds": 3.6,
"repetitive": false,
"gates": {
"maximumWordErrorRate": 0.25,
"minimumRms": 0.005,
"maximumClippedFraction": 0.02,
"durationSeconds": [
0.5,
20
],
"eosRequired": true,
"repetitionRejected": true,
"scope": "Short-English intelligibility; no identity/similarity gate for preset or described voices"
},
"evaluator": "Pinned independent Qwen3-ASR0.6B MLXQ4; optional unrelated-reference cosine is diagnostic only, not a gate for this task",
"evaluatedAt": "2026-11-06T14:29:03.662831+00:00",
"structuralPassed": true,
"performancePassed": false,
"speakerSimilarityApplied": false,
"diagnosticReferenceCosine": 0.4044334888458252
},
"runtime": {
"engine": "qwen-tts",
"version": "0.1.1",
"transformersVersion": "4.57.3",
"torchVersion": "2.10.0",
"loadMs": 1619.334417000573,
"placement": {
"talker": {
"device": "mps:0",
"dtype": "torch.float32"
},
"speechTokenizer": {
"device": "mps:0",
"dtype": "torch.float32"
}
},
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"accelerationDetails": "Talker and speech tokenizer parameters on selected device; CPU text preparation and WAV encoding remain",
"parameterDtype": "torch.float32",
"attentionImplementation": "eager",
"durationMs": 15236.920375000409,
"inferenceMs": 15230.602292000185,
"realTimeFactor": 4.2307228588889405,
"audioSeconds": 3.6,
"seed": 42,
"eosReached": true,
"generatedTokens": 46,
"generationPlacement": "mps:0",
"speaker": "Aiden",
"language": "English",
"promptTokens": 9,
"maxNewTokens": 512,
"rms": 0.0782015472650528,
"peak": 0.5488303303718567,
"clippedFraction": 0,
"sampling": "Official seeded main/subtalker sampling: temperature0.9, topK50, topP1.0, repetitionPenalty1.05",
"taskMode": "voice_design",
"instruct": "An adult male speaker with a deep, low-pitched voice, speaking clearly at a natural pace.",
"driverAllocatedMemoryBytes": 9913958400,
"artifactRevision": "5ecdb67327fd37bb2e042aab12ff7391903235d3",
"precision": "fp32",
"cpuThreads": 4,
"fallback": false,
"totalMs": 16870.36716700095,
"audioSecondsPerSecond": 0.23636622708550967,
"grade": "slow",
"residentMemoryBytes": 1395851264,
"supportsStreaming": false
}
}
]