Chatterbox Multilingual V3 Desktop
Generate multilingual speech locally with Chatterbox Multilingual V3. Returns mono 24 kHz WAV with Perth watermark.
Desktop app
Run Chatterbox Multilingual V3 Desktop on your computer
Download the desktop app to load this model, provide input and view results locally.
Public desktop downloads are coming soon.Device requirements
If your device falls short
MemoryAdvisory only: close other apps, reduce image/tile/token/batch settings or choose a smaller verified model/precision. CPU can need more RAM than GPU.
GPUIf the selected GPU runtime fails, existing CPU fallback may recover when enabled, but CPU has its own memory/runtime needs.
PlatformUse a compatible OS/architecture/runtime or another model. These estimates do not add hardware detection or block runs.
Benchmark evidence
Saved desktop measurements apply to the recorded hardware and runtime. Fallback cases report the device that actually ran; they do not prove GPU support on the requested device.
Chatterbox Multilingual V3 Desktop
28 configurations testedQuality criteria
{ "version": 1, "scope": "Authored short-text intelligibility pilot; ASR agreement, not subjective speech or speaker identity quality.", "cases": [ { "id": "standard", "text": "Hello. This is a local speech synthesis test." }, { "id": "numbers", "text": "The meeting starts at nine thirty. Please bring three notebooks." }, { "id": "spanish", "text": "Hola. Esta es una prueba de voz. La reunión empieza a las nueve." } ], "seed": 42, "maxWordErrorRate": 0.35, "minAudioSeconds": 1, "maxAudioSeconds": 30, "minRms": 0.005, "maxClippedFraction": 0.02, "requirements": [ "finite mono waveform", "24000 Hz output", "official Perth watermark retained", "no excessive repetition", "strict GPU tensor placement and no CPU framework fallback" ] }
| Precision | Device | Result | Platform | Inference / load | Task metric | |
|---|---|---|---|---|---|---|
FP32 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 7913 ms / 11202 ms | 2.569 RTF |
FP32 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 6954 ms / 11202 ms | 2.070 RTF |
FP32 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 7355 ms / 11202 ms | 2.020 RTF |
FP32 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 5576 ms / 11202 ms | 1.810 RTF |
FP32 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 7573 ms / 11210 ms | 2.459 RTF |
FP32 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 7002 ms / 11210 ms | 2.084 RTF |
FP32 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 7336 ms / 11210 ms | 2.015 RTF |
FP32 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 5448 ms / 11210 ms | 1.769 RTF |
FP32 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 8695 ms / 11357 ms | 2.823 RTF |
FP32 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 7531 ms / 11357 ms | 2.241 RTF |
FP32 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 9401 ms / 11357 ms | 2.583 RTF |
FP32 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 8718 ms / 11357 ms | 2.831 RTF |
FP32 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 7790 ms / 8414 ms | 2.434 RTF |
FP32 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 8179 ms / 8414 ms | 2.222 RTF |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 8551 ms / 12020 ms | 2.889 RTF |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 9454 ms / 12020 ms | 2.918 RTF |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 11531 ms / 12020 ms | 2.826 RTF |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 8331 ms / 12020 ms | 2.815 RTF |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 8811 ms / 10374 ms | 2.977 RTF |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 9543 ms / 10374 ms | 2.945 RTF |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 11753 ms / 10374 ms | 2.881 RTF |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 8623 ms / 10374 ms | 2.913 RTF |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 8618 ms / 10636 ms | 2.911 RTF |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 10048 ms / 10636 ms | 3.101 RTF |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 11868 ms / 10636 ms | 2.909 RTF |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 8554 ms / 10636 ms | 2.890 RTF |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 12333 ms / 7725 ms | 3.351 RTF |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 10554 ms / 7725 ms | 3.033 RTF |
Test setup & sample details
- Quality gate
- { "version": 1, "scope": "Authored short-text intelligibility pilot; ASR agreement, not subjective speech or speaker identity quality.", "cases": [ { "id": "standard", "text": "Hello. This is a local speech synthesis test." }, { "id": "numbers", "text": "The meeting starts at nine thirty. Please bring three notebooks." }, { "id": "spanish", "text": "Hola. Esta es una prueba de voz. La reunión empieza a las nueve." } ], "seed": 42, "maxWordErrorRate": 0.35, "minAudioSeconds": 1, "maxAudioSeconds": 30, "minRms": 0.005, "maxClippedFraction": 0.02, "requirements": [ "finite mono waveform", "24000 Hz output", "official Perth watermark retained", "no excessive repetition", "strict GPU tensor placement and no CPU framework fallback" ] }
- Tested
- 2026-11-03T21:37:07Z
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "runtimeProvisioning": "App-owned CPython3.13.18; model-owned pinned packages" }
- Runtime
- chatterbox-tts 0.1.7
- Artifact revision
- 5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18
- Fixture
{ "file": "model-samples/desktop/chatterbox-v3-text.json", "size": 573, "sha1": "3be4ce15076efd82176c52f26e6b64ff3ead0e66", "sha256": "c45a43a936530310d61490e638453bb45d50ccc0d2e9060d5b6a8387fcbc6dc0", "url": "https://assets.localai.tools/samples/v1/files/c45a43a936530310d61490e638453bb45d50ccc0d2e9060d5b6a8387fcbc6dc0/chatterbox-v3-text.json" }- Rerun when
- Model artifact/runtime/dependency build change; Input text/quality gate/reference evaluator change; Generation seed/configuration or device/backend behavior change
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"wordErrorRate": 0,
"repetitive": false,
"passed": true,
"structuralPassed": true
},
"runtime": {
"durationMs": 8550.518207979621,
"inferenceMs": 8550.518207979621,
"audioSeconds": 2.96,
"realTimeFactor": 2.888688583776899,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.10713756829500198,
"peak": 0.8316142559051514,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"loadMs": 12020.18983400194,
"totalMs": 20570.70804198156,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 8,
"artifactRevision": "5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18",
"engine": "chatterbox-tts",
"version": "0.1.7",
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false,
"acceleration": "cpu",
"provider": "PyTorch CPU",
"grade": "usable",
"audioSecondsPerSecond": 0.3461778488744263
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"wordErrorRate": 0,
"repetitive": false,
"passed": true,
"structuralPassed": true
},
"runtime": {
"durationMs": 9453.605334012536,
"inferenceMs": 9453.605334012536,
"audioSeconds": 3.24,
"realTimeFactor": 2.9177794240779433,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.11787256598472595,
"peak": 0.8881117105484009,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"loadMs": 12020.18983400194,
"totalMs": 21473.795168014476,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 8,
"artifactRevision": "5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18",
"engine": "chatterbox-tts",
"version": "0.1.7",
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false,
"acceleration": "cpu",
"provider": "PyTorch CPU",
"grade": "usable",
"audioSecondsPerSecond": 0.34272638697355035
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"wordErrorRate": 0,
"repetitive": false,
"passed": true,
"structuralPassed": true
},
"runtime": {
"durationMs": 11531.066041992744,
"inferenceMs": 11531.066041992744,
"audioSeconds": 4.08,
"realTimeFactor": 2.8262416769590057,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.14519210159778595,
"peak": 1.0463939905166626,
"clippedFraction": 0.00012254901960784314,
"watermark": "Resemble Perth applied by official runtime",
"loadMs": 12020.18983400194,
"totalMs": 23551.255875994684,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 8,
"artifactRevision": "5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18",
"engine": "chatterbox-tts",
"version": "0.1.7",
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false,
"acceleration": "cpu",
"provider": "PyTorch CPU",
"grade": "usable",
"audioSecondsPerSecond": 0.35382678280931207
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"wordErrorRate": 0,
"repetitive": false,
"passed": true,
"structuralPassed": true
},
"runtime": {
"durationMs": 8331.261792016448,
"inferenceMs": 8331.261792016448,
"audioSeconds": 2.96,
"realTimeFactor": 2.814615470275827,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.10713756829500198,
"peak": 0.8316142559051514,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"loadMs": 12020.18983400194,
"totalMs": 20351.45162601839,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 8,
"artifactRevision": "5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18",
"engine": "chatterbox-tts",
"version": "0.1.7",
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false,
"acceleration": "cpu",
"provider": "PyTorch CPU",
"grade": "usable",
"audioSecondsPerSecond": 0.3552883193319483
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"wordErrorRate": 0,
"repetitive": false,
"passed": true,
"structuralPassed": true
},
"runtime": {
"durationMs": 8811.493624991272,
"inferenceMs": 8811.493624991272,
"audioSeconds": 2.96,
"realTimeFactor": 2.976855954388943,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.10713756829500198,
"peak": 0.8316142559051514,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"loadMs": 10373.842667002464,
"totalMs": 19185.336291993735,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 8,
"artifactRevision": "5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18",
"engine": "chatterbox-tts",
"version": "0.1.7",
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false,
"acceleration": "cpu",
"provider": "PyTorch CPU",
"grade": "usable",
"audioSecondsPerSecond": 0.3359248869686304
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"wordErrorRate": 0,
"repetitive": false,
"passed": true,
"structuralPassed": true
},
"runtime": {
"durationMs": 9542.548458994133,
"inferenceMs": 9542.548458994133,
"audioSeconds": 3.24,
"realTimeFactor": 2.9452310058623867,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.11787256598472595,
"peak": 0.8881117105484009,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"loadMs": 10373.842667002464,
"totalMs": 19916.391125996597,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 8,
"artifactRevision": "5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18",
"engine": "chatterbox-tts",
"version": "0.1.7",
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false,
"acceleration": "cpu",
"provider": "PyTorch CPU",
"grade": "usable",
"audioSecondsPerSecond": 0.3395319409613482
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"wordErrorRate": 0,
"repetitive": false,
"passed": true,
"structuralPassed": true
},
"runtime": {
"durationMs": 11753.079957998125,
"inferenceMs": 11753.079957998125,
"audioSeconds": 4.08,
"realTimeFactor": 2.880656852450521,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.14519210159778595,
"peak": 1.0463939905166626,
"clippedFraction": 0.00012254901960784314,
"watermark": "Resemble Perth applied by official runtime",
"loadMs": 10373.842667002464,
"totalMs": 22126.92262500059,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 8,
"artifactRevision": "5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18",
"engine": "chatterbox-tts",
"version": "0.1.7",
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false,
"acceleration": "cpu",
"provider": "PyTorch CPU",
"grade": "usable",
"audioSecondsPerSecond": 0.347143047999389
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"wordErrorRate": 0,
"repetitive": false,
"passed": true,
"structuralPassed": true
},
"runtime": {
"durationMs": 8622.757875011303,
"inferenceMs": 8622.757875011303,
"audioSeconds": 2.96,
"realTimeFactor": 2.9130938766930075,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.10713756829500198,
"peak": 0.8316142559051514,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"loadMs": 10373.842667002464,
"totalMs": 18996.600542013766,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 8,
"artifactRevision": "5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18",
"engine": "chatterbox-tts",
"version": "0.1.7",
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false,
"acceleration": "cpu",
"provider": "PyTorch CPU",
"grade": "usable",
"audioSecondsPerSecond": 0.343277643058732
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"wordErrorRate": 0,
"repetitive": false,
"passed": true,
"structuralPassed": true
},
"runtime": {
"durationMs": 8617.95766701107,
"inferenceMs": 8617.95766701107,
"audioSeconds": 2.96,
"realTimeFactor": 2.911472184801037,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.10713756829500198,
"peak": 0.8316142559051514,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"loadMs": 10635.850457998458,
"totalMs": 19253.808125009527,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 8,
"artifactRevision": "5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18",
"engine": "chatterbox-tts",
"version": "0.1.7",
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false,
"acceleration": "cpu",
"provider": "PyTorch CPU",
"grade": "usable",
"audioSecondsPerSecond": 0.34346884892817126
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"wordErrorRate": 0,
"repetitive": false,
"passed": true,
"structuralPassed": true
},
"runtime": {
"durationMs": 10047.816624981351,
"inferenceMs": 10047.816624981351,
"audioSeconds": 3.24,
"realTimeFactor": 3.1011779706732563,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.11787256598472595,
"peak": 0.8881117105484009,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"loadMs": 10635.850457998458,
"totalMs": 20683.66708297981,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 8,
"artifactRevision": "5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18",
"engine": "chatterbox-tts",
"version": "0.1.7",
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false,
"acceleration": "cpu",
"provider": "PyTorch CPU",
"grade": "usable",
"audioSecondsPerSecond": 0.32245811412845266
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"wordErrorRate": 0,
"repetitive": false,
"passed": true,
"structuralPassed": true
},
"runtime": {
"durationMs": 11867.779416003032,
"inferenceMs": 11867.779416003032,
"audioSeconds": 4.08,
"realTimeFactor": 2.9087694647066256,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.14519210159778595,
"peak": 1.0463939905166626,
"clippedFraction": 0.00012254901960784314,
"watermark": "Resemble Perth applied by official runtime",
"loadMs": 10635.850457998458,
"totalMs": 22503.62987400149,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 8,
"artifactRevision": "5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18",
"engine": "chatterbox-tts",
"version": "0.1.7",
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false,
"acceleration": "cpu",
"provider": "PyTorch CPU",
"grade": "usable",
"audioSecondsPerSecond": 0.3437879873717867
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"wordErrorRate": 0,
"repetitive": false,
"passed": true,
"structuralPassed": true
},
"runtime": {
"durationMs": 8553.85991599178,
"inferenceMs": 8553.85991599178,
"audioSeconds": 2.96,
"realTimeFactor": 2.889817539186412,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.10713756829500198,
"peak": 0.8316142559051514,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"loadMs": 10635.850457998458,
"totalMs": 19189.710373990238,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 8,
"artifactRevision": "5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18",
"engine": "chatterbox-tts",
"version": "0.1.7",
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false,
"acceleration": "cpu",
"provider": "PyTorch CPU",
"grade": "usable",
"audioSecondsPerSecond": 0.3460426087252333
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"wordErrorRate": 0,
"repetitive": false,
"passed": true,
"structuralPassed": true
},
"runtime": {
"durationMs": 7913.37608298636,
"inferenceMs": 7913.37608298636,
"audioSeconds": 3.08,
"realTimeFactor": 2.5692779490215454,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.10076991468667984,
"peak": 0.6262713670730591,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 8665284608,
"memoryMeasurementScope": "Current MPS driver allocation after inference; not peak VRAM",
"loadMs": 11202.151083009085,
"totalMs": 19115.527165995445,
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"artifactRevision": "5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18",
"engine": "chatterbox-tts",
"version": "0.1.7",
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false,
"acceleration": "partial",
"provider": "Metal/MPS",
"grade": "usable",
"audioSecondsPerSecond": 0.3892144095896003
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"wordErrorRate": 0,
"repetitive": false,
"passed": true,
"structuralPassed": true
},
"runtime": {
"durationMs": 6954.399707989069,
"inferenceMs": 6954.399707989069,
"audioSeconds": 3.36,
"realTimeFactor": 2.0697618178538897,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.11064503341913223,
"peak": 0.7359250783920288,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 8709718016,
"memoryMeasurementScope": "Current MPS driver allocation after inference; not peak VRAM",
"loadMs": 11202.151083009085,
"totalMs": 18156.550790998153,
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"artifactRevision": "5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18",
"engine": "chatterbox-tts",
"version": "0.1.7",
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false,
"acceleration": "partial",
"provider": "Metal/MPS",
"grade": "usable",
"audioSecondsPerSecond": 0.4831473802318412
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"wordErrorRate": 0,
"repetitive": false,
"passed": true,
"structuralPassed": true
},
"runtime": {
"durationMs": 7354.503415990621,
"inferenceMs": 7354.503415990621,
"audioSeconds": 3.64,
"realTimeFactor": 2.0204679714259948,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.13401532173156738,
"peak": 0.9841045141220093,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 11254710272,
"memoryMeasurementScope": "Current MPS driver allocation after inference; not peak VRAM",
"loadMs": 11202.151083009085,
"totalMs": 18556.654498999706,
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"artifactRevision": "5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18",
"engine": "chatterbox-tts",
"version": "0.1.7",
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false,
"acceleration": "partial",
"provider": "Metal/MPS",
"grade": "usable",
"audioSecondsPerSecond": 0.494934843878879
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"wordErrorRate": 0,
"repetitive": false,
"passed": true,
"structuralPassed": true
},
"runtime": {
"durationMs": 5576.2675419973675,
"inferenceMs": 5576.2675419973675,
"audioSeconds": 3.08,
"realTimeFactor": 1.81047647467447,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.10076991468667984,
"peak": 0.6262713670730591,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 11254710272,
"memoryMeasurementScope": "Current MPS driver allocation after inference; not peak VRAM",
"loadMs": 11202.151083009085,
"totalMs": 16778.418625006452,
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"artifactRevision": "5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18",
"engine": "chatterbox-tts",
"version": "0.1.7",
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false,
"acceleration": "partial",
"provider": "Metal/MPS",
"grade": "usable",
"audioSecondsPerSecond": 0.552340786521296
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"wordErrorRate": 0,
"repetitive": false,
"passed": true,
"structuralPassed": true
},
"runtime": {
"durationMs": 7573.308291001013,
"inferenceMs": 7573.308291001013,
"audioSeconds": 3.08,
"realTimeFactor": 2.458866328247082,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.10076991468667984,
"peak": 0.6262713670730591,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 8665284608,
"memoryMeasurementScope": "Current MPS driver allocation after inference; not peak VRAM",
"loadMs": 11209.61512500071,
"totalMs": 18782.923416001722,
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"artifactRevision": "5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18",
"engine": "chatterbox-tts",
"version": "0.1.7",
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false,
"acceleration": "partial",
"provider": "Metal/MPS",
"grade": "usable",
"audioSecondsPerSecond": 0.4066914856298418
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"wordErrorRate": 0,
"repetitive": false,
"passed": true,
"structuralPassed": true
},
"runtime": {
"durationMs": 7001.629249978578,
"inferenceMs": 7001.629249978578,
"audioSeconds": 3.36,
"realTimeFactor": 2.083818229160291,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.11064503341913223,
"peak": 0.7359250783920288,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 8701329408,
"memoryMeasurementScope": "Current MPS driver allocation after inference; not peak VRAM",
"loadMs": 11209.61512500071,
"totalMs": 18211.244374979287,
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"artifactRevision": "5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18",
"engine": "chatterbox-tts",
"version": "0.1.7",
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false,
"acceleration": "partial",
"provider": "Metal/MPS",
"grade": "usable",
"audioSecondsPerSecond": 0.4798883059982475
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"wordErrorRate": 0,
"repetitive": false,
"passed": true,
"structuralPassed": true
},
"runtime": {
"durationMs": 7335.502791014733,
"inferenceMs": 7335.502791014733,
"audioSeconds": 3.64,
"realTimeFactor": 2.0152480195095417,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.13401532173156738,
"peak": 0.9841045141220093,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 11254710272,
"memoryMeasurementScope": "Current MPS driver allocation after inference; not peak VRAM",
"loadMs": 11209.61512500071,
"totalMs": 18545.117916015442,
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"artifactRevision": "5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18",
"engine": "chatterbox-tts",
"version": "0.1.7",
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false,
"acceleration": "partial",
"provider": "Metal/MPS",
"grade": "usable",
"audioSecondsPerSecond": 0.4962168379867078
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"wordErrorRate": 0,
"repetitive": false,
"passed": true,
"structuralPassed": true
},
"runtime": {
"durationMs": 5447.898084006738,
"inferenceMs": 5447.898084006738,
"audioSeconds": 3.08,
"realTimeFactor": 1.7687980792229667,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.10076991468667984,
"peak": 0.6262713670730591,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 11254710272,
"memoryMeasurementScope": "Current MPS driver allocation after inference; not peak VRAM",
"loadMs": 11209.61512500071,
"totalMs": 16657.513209007448,
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"artifactRevision": "5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18",
"engine": "chatterbox-tts",
"version": "0.1.7",
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false,
"acceleration": "partial",
"provider": "Metal/MPS",
"grade": "usable",
"audioSecondsPerSecond": 0.5653556568985535
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"wordErrorRate": 0,
"repetitive": false,
"passed": true,
"structuralPassed": true
},
"runtime": {
"durationMs": 8694.52041600016,
"inferenceMs": 8694.52041600016,
"audioSeconds": 3.08,
"realTimeFactor": 2.8228962389610905,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.10076991468667984,
"peak": 0.6262713670730591,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 8665284608,
"memoryMeasurementScope": "Current MPS driver allocation after inference; not peak VRAM",
"loadMs": 11356.933667004341,
"totalMs": 20051.4540830045,
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"artifactRevision": "5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18",
"engine": "chatterbox-tts",
"version": "0.1.7",
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false,
"acceleration": "partial",
"provider": "Metal/MPS",
"grade": "usable",
"audioSecondsPerSecond": 0.3542461058958475
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"wordErrorRate": 0,
"repetitive": false,
"passed": true,
"structuralPassed": true
},
"runtime": {
"durationMs": 7531.051583995577,
"inferenceMs": 7531.051583995577,
"audioSeconds": 3.36,
"realTimeFactor": 2.2413843999986836,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.11064503341913223,
"peak": 0.7359250783920288,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 8701329408,
"memoryMeasurementScope": "Current MPS driver allocation after inference; not peak VRAM",
"loadMs": 11356.933667004341,
"totalMs": 18887.98525099992,
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"artifactRevision": "5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18",
"engine": "chatterbox-tts",
"version": "0.1.7",
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false,
"acceleration": "partial",
"provider": "Metal/MPS",
"grade": "usable",
"audioSecondsPerSecond": 0.4461528330439827
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"wordErrorRate": 0,
"repetitive": false,
"passed": true,
"structuralPassed": true
},
"runtime": {
"durationMs": 9401.491207972867,
"inferenceMs": 9401.491207972867,
"audioSeconds": 3.64,
"realTimeFactor": 2.582827254937601,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.13401532173156738,
"peak": 0.9841045141220093,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 11254710272,
"memoryMeasurementScope": "Current MPS driver allocation after inference; not peak VRAM",
"loadMs": 11356.933667004341,
"totalMs": 20758.42487497721,
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"artifactRevision": "5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18",
"engine": "chatterbox-tts",
"version": "0.1.7",
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false,
"acceleration": "partial",
"provider": "Metal/MPS",
"grade": "usable",
"audioSecondsPerSecond": 0.3871726218190923
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"wordErrorRate": 0,
"repetitive": false,
"passed": true,
"structuralPassed": true
},
"runtime": {
"durationMs": 8718.075417011278,
"inferenceMs": 8718.075417011278,
"audioSeconds": 3.08,
"realTimeFactor": 2.8305439665621033,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.10076991468667984,
"peak": 0.6262713670730591,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 11252613120,
"memoryMeasurementScope": "Current MPS driver allocation after inference; not peak VRAM",
"loadMs": 11356.933667004341,
"totalMs": 20075.00908401562,
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"artifactRevision": "5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18",
"engine": "chatterbox-tts",
"version": "0.1.7",
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false,
"acceleration": "partial",
"provider": "Metal/MPS",
"grade": "usable",
"audioSecondsPerSecond": 0.3532889832531275
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"wordErrorRate": 0,
"repetitive": false,
"passed": true,
"structuralPassed": true
},
"runtime": {
"durationMs": 12332.586417003768,
"inferenceMs": 12332.586417003768,
"audioSeconds": 3.68,
"realTimeFactor": 3.351246308968415,
"seed": 42,
"voiceMode": "reference",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.031679388135671616,
"peak": 0.31671804189682007,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"loadMs": 7724.797708011465,
"totalMs": 20057.384125015233,
"engine": "chatterbox-tts",
"version": "0.1.7",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 8,
"artifactRevision": "5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18",
"fallback": false,
"acceleration": "cpu",
"grade": "usable",
"audioSecondsPerSecond": 0.2983964495011473
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"wordErrorRate": 0,
"repetitive": false,
"passed": true,
"structuralPassed": true
},
"runtime": {
"durationMs": 10553.959709010087,
"inferenceMs": 10553.959709010087,
"audioSeconds": 3.48,
"realTimeFactor": 3.0327470428189907,
"seed": 42,
"voiceMode": "reference",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.030483761802315712,
"peak": 0.27670249342918396,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"loadMs": 7724.797708011465,
"totalMs": 18278.757417021552,
"engine": "chatterbox-tts",
"version": "0.1.7",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 8,
"artifactRevision": "5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18",
"fallback": false,
"acceleration": "cpu",
"grade": "usable",
"audioSecondsPerSecond": 0.32973406152281093
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"wordErrorRate": 0,
"repetitive": false,
"passed": true,
"structuralPassed": true
},
"runtime": {
"durationMs": 7790.333708020626,
"inferenceMs": 7790.333708020626,
"audioSeconds": 3.2,
"realTimeFactor": 2.4344792837564455,
"seed": 42,
"voiceMode": "reference",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.03403399884700775,
"peak": 0.32558536529541016,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 8731672576,
"memoryMeasurementScope": "Current MPS driver allocation after inference; not peak VRAM",
"loadMs": 8413.774332992034,
"totalMs": 16204.10804101266,
"engine": "chatterbox-tts",
"version": "0.1.7",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"artifactRevision": "5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18",
"fallback": false,
"acceleration": "partial",
"grade": "usable",
"audioSecondsPerSecond": 0.41076545882821475
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"wordErrorRate": 0,
"repetitive": false,
"passed": true,
"structuralPassed": true
},
"runtime": {
"durationMs": 8178.637040982721,
"inferenceMs": 8178.637040982721,
"audioSeconds": 3.68,
"realTimeFactor": 2.222455717658348,
"seed": 42,
"voiceMode": "reference",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.03294481709599495,
"peak": 0.34141847491264343,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10248355840,
"memoryMeasurementScope": "Current MPS driver allocation after inference; not peak VRAM",
"loadMs": 8413.774332992034,
"totalMs": 16592.411373974755,
"engine": "chatterbox-tts",
"version": "0.1.7",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"artifactRevision": "5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18",
"fallback": false,
"acceleration": "partial",
"grade": "usable",
"audioSecondsPerSecond": 0.44995272214180837
}
}
]Chatterbox Multilingual V3 · Spanish (Spain) Desktop
19 configurations testedQuality criteria
{ "scope": "Frozen authored language/mode suite; complete waveform, independent ASR agreement, no repetition, RTF<=5 offline narration. Not subjective voice/locale identity certification.", "cases": [ { "id": "short", "label": "Short speech", "text": "Hola. Esta es una prueba de voz.", "language": "es", "reference": false, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": null, "referenceSha256": null }, { "id": "numbers", "label": "Numbers and instructions", "text": "La reunión empieza a las nueve. Por favor, trae tres cuadernos.", "language": "es", "reference": false, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": null, "referenceSha256": null }, { "id": "reference", "label": "Reference voice", "text": "Hola. Esta es una prueba de voz.", "language": "es", "reference": true, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": "model-samples/desktop/chatterbox-reference-input.wav", "referenceSha256": "a890ca57ba62dc2159b6139e77d3c821a790db3813cb8cb4f0bab9ecee5177fe" } ], "maximumWordErrorRate": 0.35, "maximumChineseCharacterErrorRate": 0.25, "maximumHindiCharacterErrorRate": 0.35, "maximumRealTimeFactor": 5 }
| Precision | Device | Result | Platform | Inference / load | Task metric | |
|---|---|---|---|---|---|---|
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 4512 ms / 11931 ms | 2.350 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 4627 ms / 11931 ms | 2.410 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 4641 ms / 11931 ms | 2.417 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 10561 ms / 11931 ms | 2.563 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 10599 ms / 15161 ms | 2.573 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 10601 ms / 14602 ms | 2.573 RTF |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 6040 ms / 11931 ms | 3.146 RTF |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 7749 ms / 15161 ms | 3.176 RTF |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 7917 ms / 14602 ms | 3.245 RTF |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 7997 ms / 11931 ms | 3.278 RTF |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 6466 ms / 15161 ms | 3.367 RTF |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 6525 ms / 11931 ms | 3.398 RTF |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 6852 ms / 14602 ms | 3.569 RTF |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 7422 ms / 11931 ms | 3.866 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 12853 ms / 13086 ms | 3.493 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 8766 ms / 13086 ms | 3.593 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 10096 ms / 11800 ms | 4.138 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 11900 ms / 13086 ms | 4.440 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 10880 ms / 13081 ms | 4.459 RTF |
Test setup & sample details
- Quality gate
- { "scope": "Frozen authored language/mode suite; complete waveform, independent ASR agreement, no repetition, RTF<=5 offline narration. Not subjective voice/locale identity certification.", "cases": [ { "id": "short", "label": "Short speech", "text": "Hola. Esta es una prueba de voz.", "language": "es", "reference": false, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": null, "referenceSha256": null }, { "id": "numbers", "label": "Numbers and instructions", "text": "La reunión empieza a las nueve. Por favor, trae tres cuadernos.", "language": "es", "reference": false, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": null, "referenceSha256": null }, { "id": "reference", "label": "Reference voice", "text": "Hola. Esta es una prueba de voz.", "language": "es", "reference": true, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": "model-samples/desktop/chatterbox-reference-input.wav", "referenceSha256": "a890ca57ba62dc2159b6139e77d3c821a790db3813cb8cb4f0bab9ecee5177fe" } ], "maximumWordErrorRate": 0.35, "maximumChineseCharacterErrorRate": 0.25, "maximumHindiCharacterErrorRate": 0.35, "maximumRealTimeFactor": 5 }
- Tested
- 2026-11-06T18:56:37.577434+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "runtimeProvisioning": "App-owned CPython3.13.18; model-owned pinned packages", "concurrentUploads": true, "concurrentActivity": "Background source/backup transfers active; synthesis/ASR/builds run sequentially", "measurementScope": "Apple M3 Max/macOS27.2 current-device scope; private seeded local-cache loads are not network-download benchmarks" }
- Runtime
- chatterbox-tts 0.1.7
- Artifact revision
- 7262b364e3222ba9446c9289e7485eadc47ed902
- Fixture
{ "file": "model-samples/desktop/chatterbox-multilingual-family/v3-es-es-short-input.json", "size": 197, "sha1": "9b968ac8a5d61eddb78fddcda6e338783ccee3b7", "sha256": "58c075f3c060021a869797c841668765d1dbb06bac00374556a7bd3e9deacf23", "url": "https://assets.localai.tools/samples/v1/files/58c075f3c060021a869797c841668765d1dbb06bac00374556a7bd3e9deacf23/v3-es-es-short-input.json" }- Rerun when
- artifact revision or required file integrity; runtime/helper/model/processor/decoder semantics; fixture/config/checksum or quality gate; relevant backend/OS behavior
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Hola, esta es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.44235366582870483,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:00:44.364771+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 10895.51387499887,
"inferenceMs": 10879.659958998673,
"audioSeconds": 2.44,
"realTimeFactor": 4.458877032376505,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.11487490683794022,
"peak": 0.7765765190124512,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"loadMs": 13080.647292001231,
"totalMs": 23976.1611670001,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "7262b364e3222ba9446c9289e7485eadc47ed902",
"peakHelperRssBytes": 6210928640,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.22427171521862246,
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Hola, esta es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.4423332214355469,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:00:44.586315+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 10110.310832998948,
"inferenceMs": 10095.91937499863,
"audioSeconds": 2.44,
"realTimeFactor": 4.137671874999438,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.11487486958503723,
"peak": 0.7763928174972534,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 11800.176541000837,
"totalMs": 21910.487373999786,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "7262b364e3222ba9446c9289e7485eadc47ed902",
"peakHelperRssBytes": 6978387968,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.24168180324838728,
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Hola, esta es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.48232123255729675,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:00:44.705869+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 7434.3212919993675,
"inferenceMs": 7422.178082997561,
"audioSeconds": 1.92,
"realTimeFactor": 3.86571775156123,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.12028747797012329,
"peak": 0.9026481509208679,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 6087294976,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 11930.873334000353,
"totalMs": 19365.19462599972,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "7262b364e3222ba9446c9289e7485eadc47ed902",
"peakHelperRssBytes": 4844535808,
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.2586841730998427,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "La reunión empieza a las nueve. Por favor, trae tres cuadernos.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.4739052653312683,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:00:44.896107+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 10580.811958003324,
"inferenceMs": 10561.374375000014,
"audioSeconds": 4.12,
"realTimeFactor": 2.5634403822815566,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.10861934721469879,
"peak": 0.8547649383544922,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 11908890624,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 11930.873334000353,
"totalMs": 22511.685292003676,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "7262b364e3222ba9446c9289e7485eadc47ed902",
"peakHelperRssBytes": 4844535808,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.3901007438721719,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Hola, este es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0.14285714285714285,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.7793951630592346,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:00:45.022413+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 8047.452749997319,
"inferenceMs": 7997.13016700116,
"audioSeconds": 2.44,
"realTimeFactor": 3.277512363525066,
"seed": 42,
"voiceMode": "reference",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.025842295959591866,
"peak": 0.2958381175994873,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 11955781632,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 11930.873334000353,
"totalMs": 19978.326083997672,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "7262b364e3222ba9446c9289e7485eadc47ed902",
"peakHelperRssBytes": 4844535808,
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.30510945164657416,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Hola, esta es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.48232123255729675,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:00:45.023042+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 4523.491083000408,
"inferenceMs": 4511.793707999459,
"audioSeconds": 1.92,
"realTimeFactor": 2.3498925562497184,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.12028747797012329,
"peak": 0.9026481509208679,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 11953684480,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 11930.873334000353,
"totalMs": 16454.36441700076,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "7262b364e3222ba9446c9289e7485eadc47ed902",
"peakHelperRssBytes": 4844535808,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.42555137141926924,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Hola, esta es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.48232123255729675,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:00:45.023390+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 4638.610665999295,
"inferenceMs": 4627.059875001578,
"audioSeconds": 1.92,
"realTimeFactor": 2.409927018229989,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.12028747797012329,
"peak": 0.9026481509208679,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 11953684480,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 11930.873334000353,
"totalMs": 16569.483999999647,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "7262b364e3222ba9446c9289e7485eadc47ed902",
"peakHelperRssBytes": 4844535808,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.41495032523203407,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Hola, esta es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.48232123255729675,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:00:45.023706+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 4652.120416998514,
"inferenceMs": 4640.699832998507,
"audioSeconds": 1.92,
"realTimeFactor": 2.417031163020056,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.12028747797012329,
"peak": 0.9026481509208679,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 11953684480,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 11930.873334000353,
"totalMs": 16582.993750998867,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "7262b364e3222ba9446c9289e7485eadc47ed902",
"peakHelperRssBytes": 4844535808,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.41373070206943885,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Hola, esta es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.48232123255729675,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:00:45.024014+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 6050.571832998685,
"inferenceMs": 6039.5243339989975,
"audioSeconds": 1.92,
"realTimeFactor": 3.145585590624478,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.12028747797012329,
"peak": 0.9026481509208679,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 11953684480,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 11930.873334000353,
"totalMs": 17981.445166999038,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "7262b364e3222ba9446c9289e7485eadc47ed902",
"peakHelperRssBytes": 4844535808,
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.31790583062833616,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Hola, esta es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.48232123255729675,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:00:45.024317+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 6536.68149999794,
"inferenceMs": 6524.919082999986,
"audioSeconds": 1.92,
"realTimeFactor": 3.3983953557291597,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.12028747797012329,
"peak": 0.9026481509208679,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 11953684480,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 11930.873334000353,
"totalMs": 18467.554833998292,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "7262b364e3222ba9446c9289e7485eadc47ed902",
"peakHelperRssBytes": 4844535808,
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.2942565226597775,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Hola, esta es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.48232123255729675,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:00:45.024618+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 6478.863625001395,
"inferenceMs": 6465.545582999766,
"audioSeconds": 1.92,
"realTimeFactor": 3.3674716578123784,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.12028747797012329,
"peak": 0.9026481509208679,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 6087294976,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 15160.597625002993,
"totalMs": 21639.461250004388,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "7262b364e3222ba9446c9289e7485eadc47ed902",
"peakHelperRssBytes": 4762091520,
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.29695869828036897,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "La reunión empieza a las nueve. Por favor, trae tres cuadernos.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.4739052653312683,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:00:45.024937+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 10620.059124998079,
"inferenceMs": 10599.391083000228,
"audioSeconds": 4.12,
"realTimeFactor": 2.5726677385922883,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.10861934721469879,
"peak": 0.8547649383544922,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 11908890624,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 15160.597625002993,
"totalMs": 25780.65675000107,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "7262b364e3222ba9446c9289e7485eadc47ed902",
"peakHelperRssBytes": 4762091520,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.388701574244943,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Hola, este es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0.14285714285714285,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.7793951630592346,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:00:45.025557+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 7801.53637499825,
"inferenceMs": 7748.622500002966,
"audioSeconds": 2.44,
"realTimeFactor": 3.175664959017609,
"seed": 42,
"voiceMode": "reference",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.025842295959591866,
"peak": 0.2958381175994873,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 11968364544,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 15160.597625002993,
"totalMs": 22962.134000001242,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "7262b364e3222ba9446c9289e7485eadc47ed902",
"peakHelperRssBytes": 4762091520,
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.3148946796671364,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Hola, esta es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.48232123255729675,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:00:45.026166+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 6863.223459000437,
"inferenceMs": 6851.678166000056,
"audioSeconds": 1.92,
"realTimeFactor": 3.568582378125029,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.12028747797012329,
"peak": 0.9026481509208679,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 6087294976,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14602.336292002292,
"totalMs": 21465.55975100273,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "7262b364e3222ba9446c9289e7485eadc47ed902",
"peakHelperRssBytes": 4617977856,
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.2802233195259487,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "La reunión empieza a las nueve. Por favor, trae tres cuadernos.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.4739052653312683,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:00:45.026596+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 10622.203124999942,
"inferenceMs": 10601.370167001733,
"audioSeconds": 4.12,
"realTimeFactor": 2.5731480987868283,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.10861934721469879,
"peak": 0.8547649383544922,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 11908890624,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14602.336292002292,
"totalMs": 25224.539417002234,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "7262b364e3222ba9446c9289e7485eadc47ed902",
"peakHelperRssBytes": 4617977856,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.38862901069373884,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Hola, este es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0.14285714285714285,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.7793951630592346,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:00:45.026930+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 7968.972292001126,
"inferenceMs": 7916.699332999997,
"audioSeconds": 2.44,
"realTimeFactor": 3.244548906967212,
"seed": 42,
"voiceMode": "reference",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.025842295959591866,
"peak": 0.2958381175994873,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 11968364544,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14602.336292002292,
"totalMs": 22571.30858400342,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "7262b364e3222ba9446c9289e7485eadc47ed902",
"peakHelperRssBytes": 4617977856,
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.30820925456005327,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Hola, esta es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.44235366582870483,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:00:45.027733+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 8781.514832997345,
"inferenceMs": 8766.418834002252,
"audioSeconds": 2.44,
"realTimeFactor": 3.5927946040992835,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.11487490683794022,
"peak": 0.7765765190124512,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"loadMs": 13086.11670800019,
"totalMs": 21867.631540997536,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "7262b364e3222ba9446c9289e7485eadc47ed902",
"peakHelperRssBytes": 6447579136,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.2783348646925228,
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "La reunión empieza a las nueve. Por favor, trae tres cuadernos.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.5147868394851685,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:00:45.226198+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 12872.616499997093,
"inferenceMs": 12852.881457998592,
"audioSeconds": 3.68,
"realTimeFactor": 3.492630830977878,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.1302807778120041,
"peak": 0.9278187155723572,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"loadMs": 13086.11670800019,
"totalMs": 25958.733207997284,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "7262b364e3222ba9446c9289e7485eadc47ed902",
"peakHelperRssBytes": 6447579136,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.2863171197856078,
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Hola, esta es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.7562891244888306,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:00:45.367881+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 11954.374541001016,
"inferenceMs": 11900.094541000726,
"audioSeconds": 2.68,
"realTimeFactor": 4.440333783955494,
"seed": 42,
"voiceMode": "reference",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.02860417775809765,
"peak": 0.2330596148967743,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"loadMs": 13086.11670800019,
"totalMs": 25040.491249001207,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "7262b364e3222ba9446c9289e7485eadc47ed902",
"peakHelperRssBytes": 6447579136,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.22520829483886004,
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false
}
}
]Chatterbox Multilingual V3 · Spanish (Latin America) Desktop
19 configurations testedQuality criteria
{ "scope": "Frozen authored language/mode suite; complete waveform, independent ASR agreement, no repetition, RTF<=5 offline narration. Not subjective voice/locale identity certification.", "cases": [ { "id": "short", "label": "Short speech", "text": "Hola. Esta es una prueba de voz.", "language": "es", "reference": false, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": null, "referenceSha256": null }, { "id": "numbers", "label": "Numbers and instructions", "text": "La reunión empieza a las nueve. Por favor, trae tres cuadernos.", "language": "es", "reference": false, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": null, "referenceSha256": null }, { "id": "reference", "label": "Reference voice", "text": "Hola. Esta es una prueba de voz.", "language": "es", "reference": true, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": "model-samples/desktop/chatterbox-reference-input.wav", "referenceSha256": "a890ca57ba62dc2159b6139e77d3c821a790db3813cb8cb4f0bab9ecee5177fe" } ], "maximumWordErrorRate": 0.35, "maximumChineseCharacterErrorRate": 0.25, "maximumHindiCharacterErrorRate": 0.35, "maximumRealTimeFactor": 5 }
| Precision | Device | Result | Platform | Inference / load | Task metric | |
|---|---|---|---|---|---|---|
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 5209 ms / 15161 ms | 2.245 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 8485 ms / 15161 ms | 2.383 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 8652 ms / 16592 ms | 2.430 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 9152 ms / 11684 ms | 2.571 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 6021 ms / 15161 ms | 2.595 RTF |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 7247 ms / 16592 ms | 3.124 RTF |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 7173 ms / 16592 ms | 3.146 RTF |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 7339 ms / 15161 ms | 3.163 RTF |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 7764 ms / 11684 ms | 3.346 RTF |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 8384 ms / 15161 ms | 3.677 RTF |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 8945 ms / 15161 ms | 3.855 RTF |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 9610 ms / 15161 ms | 4.142 RTF |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 9763 ms / 15161 ms | 4.208 RTF |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 10832 ms / 11684 ms | 4.751 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 11304 ms / 15989 ms | 3.489 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 8960 ms / 15989 ms | 3.733 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 9527 ms / 15989 ms | 4.037 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 10898 ms / 12672 ms | 4.618 RTF |
FP32 | cpu | Failed | — | macos · aarch64 | 15760 ms / 12149 ms | 6.678 RTF |
Test setup & sample details
- Quality gate
- { "scope": "Frozen authored language/mode suite; complete waveform, independent ASR agreement, no repetition, RTF<=5 offline narration. Not subjective voice/locale identity certification.", "cases": [ { "id": "short", "label": "Short speech", "text": "Hola. Esta es una prueba de voz.", "language": "es", "reference": false, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": null, "referenceSha256": null }, { "id": "numbers", "label": "Numbers and instructions", "text": "La reunión empieza a las nueve. Por favor, trae tres cuadernos.", "language": "es", "reference": false, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": null, "referenceSha256": null }, { "id": "reference", "label": "Reference voice", "text": "Hola. Esta es una prueba de voz.", "language": "es", "reference": true, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": "model-samples/desktop/chatterbox-reference-input.wav", "referenceSha256": "a890ca57ba62dc2159b6139e77d3c821a790db3813cb8cb4f0bab9ecee5177fe" } ], "maximumWordErrorRate": 0.35, "maximumChineseCharacterErrorRate": 0.25, "maximumHindiCharacterErrorRate": 0.35, "maximumRealTimeFactor": 5 }
- Tested
- 2026-11-06T19:00:45.858660+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "runtimeProvisioning": "App-owned CPython3.13.18; model-owned pinned packages", "concurrentUploads": true, "concurrentActivity": "Background source/backup transfers active; synthesis/ASR/builds run sequentially", "measurementScope": "Apple M3 Max/macOS27.2 current-device scope; private seeded local-cache loads are not network-download benchmarks" }
- Runtime
- chatterbox-tts 0.1.7
- Artifact revision
- 27e595bf2fe7be0533ca299d9afafcde08b7cca7
- Fixture
{ "file": "model-samples/desktop/chatterbox-multilingual-family/v3-es-mx-latam-short-input.json", "size": 197, "sha1": "9b968ac8a5d61eddb78fddcda6e338783ccee3b7", "sha256": "58c075f3c060021a869797c841668765d1dbb06bac00374556a7bd3e9deacf23", "url": "https://assets.localai.tools/samples/v1/files/58c075f3c060021a869797c841668765d1dbb06bac00374556a7bd3e9deacf23/v3-es-mx-latam-short-input.json" }- Rerun when
- artifact revision or required file integrity; runtime/helper/model/processor/decoder semantics; fixture/config/checksum or quality gate; relevant backend/OS behavior
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Hola, esta es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.5108644962310791,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:05:13.549538+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 10911.968625001464,
"inferenceMs": 10897.899000003235,
"audioSeconds": 2.36,
"realTimeFactor": 4.617753813560693,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.09858034551143646,
"peak": 0.8398245573043823,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"loadMs": 12672.28024999713,
"totalMs": 23584.248874998593,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "27e595bf2fe7be0533ca299d9afafcde08b7cca7",
"peakHelperRssBytes": 6432489472,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.21655550303772308,
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": false,
"transcript": "Hola, esta es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.5108644962310791,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:05:13.550164+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 15773.723083002551,
"inferenceMs": 15760.080999996717,
"audioSeconds": 2.36,
"realTimeFactor": 6.678000423727423,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.09858034551143646,
"peak": 0.8398245573043823,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 12148.662915999012,
"totalMs": 27922.385999001563,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "27e595bf2fe7be0533ca299d9afafcde08b7cca7",
"peakHelperRssBytes": 6780518400,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"performanceGrade": "slow",
"engine": "chatterbox-tts",
"grade": "slow",
"audioSecondsPerSecond": 0.14974542326276696,
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Hola, este es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0.14285714285714285,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.4659584164619446,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:05:13.657359+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 8961.303916999896,
"inferenceMs": 8944.50420800058,
"audioSeconds": 2.32,
"realTimeFactor": 3.855389744827837,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.1274014115333557,
"peak": 0.9653469324111938,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 6719373312,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 15161.436541002331,
"totalMs": 24122.740458002227,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "27e595bf2fe7be0533ca299d9afafcde08b7cca7",
"peakHelperRssBytes": 4966744064,
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.2593771489228919,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "La reunión empieza a las nueve. Por favor, trae tres cuadernos.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.5047017335891724,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:05:13.800778+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 8502.220583999588,
"inferenceMs": 8485.032375003357,
"audioSeconds": 3.56,
"realTimeFactor": 2.3834360603942013,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.14320425689220428,
"peak": 0.9993268251419067,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10805231616,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 15161.436541002331,
"totalMs": 23663.65712500192,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "27e595bf2fe7be0533ca299d9afafcde08b7cca7",
"peakHelperRssBytes": 4966744064,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.41956233549416383,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Hola, este es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0.14285714285714285,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.7962858080863953,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:05:13.899668+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 8435.28433299798,
"inferenceMs": 8384.34158399832,
"audioSeconds": 2.28,
"realTimeFactor": 3.677342799999264,
"seed": 42,
"voiceMode": "reference",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.035459909588098526,
"peak": 0.3396835923194885,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10863788032,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 15161.436541002331,
"totalMs": 23596.72087400031,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "27e595bf2fe7be0533ca299d9afafcde08b7cca7",
"peakHelperRssBytes": 4966744064,
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.2719354855903562,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Hola, este es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0.14285714285714285,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.4659584164619446,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:05:13.900221+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 6034.579790997668,
"inferenceMs": 6021.390499998233,
"audioSeconds": 2.32,
"realTimeFactor": 2.595426939654411,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.1274014115333557,
"peak": 0.9653469324111938,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10853302272,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 15161.436541002331,
"totalMs": 21196.016332,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "27e595bf2fe7be0533ca299d9afafcde08b7cca7",
"peakHelperRssBytes": 4966744064,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.38529306478307307,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Hola, este es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0.14285714285714285,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.4659584164619446,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:05:13.900524+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 5221.620915999665,
"inferenceMs": 5208.859833997849,
"audioSeconds": 2.32,
"realTimeFactor": 2.2451982043094176,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.1274014115333557,
"peak": 0.9653469324111938,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10853302272,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 15161.436541002331,
"totalMs": 20383.057457001996,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "27e595bf2fe7be0533ca299d9afafcde08b7cca7",
"peakHelperRssBytes": 4966744064,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.4453949758558541,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Hola, este es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0.14285714285714285,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.4659584164619446,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:05:13.900805+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 7361.3517499979935,
"inferenceMs": 7338.795542000298,
"audioSeconds": 2.32,
"realTimeFactor": 3.16327394051737,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.1274014115333557,
"peak": 0.9653469324111938,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10853302272,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 15161.436541002331,
"totalMs": 22522.788291000325,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "27e595bf2fe7be0533ca299d9afafcde08b7cca7",
"peakHelperRssBytes": 4966744064,
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.3161281693600159,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Hola, este es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0.14285714285714285,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.4659584164619446,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:05:13.901094+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 9786.643084000389,
"inferenceMs": 9762.966040998435,
"audioSeconds": 2.32,
"realTimeFactor": 4.208175017671739,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.1274014115333557,
"peak": 0.9653469324111938,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10844913664,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 15161.436541002331,
"totalMs": 24948.07962500272,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "27e595bf2fe7be0533ca299d9afafcde08b7cca7",
"peakHelperRssBytes": 4966744064,
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.23763270201467782,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Hola, este es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0.14285714285714285,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.4659584164619446,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:05:13.901383+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 9625.10004199794,
"inferenceMs": 9610.22479100211,
"audioSeconds": 2.32,
"realTimeFactor": 4.142338271983669,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.1274014115333557,
"peak": 0.9653469324111938,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10859593728,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 15161.436541002331,
"totalMs": 24786.53658300027,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "27e595bf2fe7be0533ca299d9afafcde08b7cca7",
"peakHelperRssBytes": 4966744064,
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.24140954560939887,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Hola, este es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0.14285714285714285,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.4659584164619446,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:05:13.901679+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 7262.2128749972035,
"inferenceMs": 7247.364499999094,
"audioSeconds": 2.32,
"realTimeFactor": 3.1238640086202993,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.1274014115333557,
"peak": 0.9653469324111938,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 6719373312,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 16591.76225000192,
"totalMs": 23853.975124999124,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "27e595bf2fe7be0533ca299d9afafcde08b7cca7",
"peakHelperRssBytes": 4617928704,
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.3201163678189899,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "La reunión empieza a las nueve. Por favor, trae tres cuadernos.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.5047017335891724,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:05:13.901979+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 8669.156957999803,
"inferenceMs": 8652.07616599946,
"audioSeconds": 3.56,
"realTimeFactor": 2.430358473595354,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.14320425689220428,
"peak": 0.9993268251419067,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10805231616,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 16591.76225000192,
"totalMs": 25260.919208001724,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "27e595bf2fe7be0533ca299d9afafcde08b7cca7",
"peakHelperRssBytes": 4617928704,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.411461934881009,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Hola, este es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0.14285714285714285,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.7962858080863953,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:05:13.902262+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 7222.451333000208,
"inferenceMs": 7172.676625003078,
"audioSeconds": 2.28,
"realTimeFactor": 3.1459108004399465,
"seed": 42,
"voiceMode": "reference",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.035459909588098526,
"peak": 0.3396835923194885,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10863788032,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 16591.76225000192,
"totalMs": 23814.21358300213,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "27e595bf2fe7be0533ca299d9afafcde08b7cca7",
"peakHelperRssBytes": 4617928704,
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.3178729669830921,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Hola, este es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0.14285714285714285,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.4659584164619446,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:05:13.902555+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 7777.138292000018,
"inferenceMs": 7763.800917000481,
"audioSeconds": 2.32,
"realTimeFactor": 3.3464659125002076,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.1274014115333557,
"peak": 0.9653469324111938,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 6719373312,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 11683.613917000912,
"totalMs": 19460.75220900093,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "27e595bf2fe7be0533ca299d9afafcde08b7cca7",
"peakHelperRssBytes": 4617732096,
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.29882270614640183,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "La reunión empieza a las nueve. Por favor, trae tres cuadernos.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.5047017335891724,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:05:13.902848+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 9169.853249997686,
"inferenceMs": 9152.15624999837,
"audioSeconds": 3.56,
"realTimeFactor": 2.570830407302913,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.14320425689220428,
"peak": 0.9993268251419067,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10805231616,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 11683.613917000912,
"totalMs": 20853.4671669986,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "27e595bf2fe7be0533ca299d9afafcde08b7cca7",
"peakHelperRssBytes": 4617732096,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.3889793730303319,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Hola, este es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0.14285714285714285,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.7962858080863953,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:05:13.903122+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 10890.014915999927,
"inferenceMs": 10831.540040999244,
"audioSeconds": 2.28,
"realTimeFactor": 4.750675456578616,
"seed": 42,
"voiceMode": "reference",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.035459909588098526,
"peak": 0.3396835923194885,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10861690880,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 11683.613917000912,
"totalMs": 22573.62883300084,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "27e595bf2fe7be0533ca299d9afafcde08b7cca7",
"peakHelperRssBytes": 4617732096,
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.210496382912292,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Hola, esta es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.5108644962310791,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:05:13.903872+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 9542.439917000593,
"inferenceMs": 9527.059166000981,
"audioSeconds": 2.36,
"realTimeFactor": 4.03688947711906,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.09858034551143646,
"peak": 0.8398245573043823,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"loadMs": 15988.941625000734,
"totalMs": 25531.381542001327,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "27e595bf2fe7be0533ca299d9afafcde08b7cca7",
"peakHelperRssBytes": 6974111744,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.2477154764003233,
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "La reunión empieza a las nueve. Por favor, preárea tres cuadernos.",
"metric": "wordErrorRate",
"errorRate": 0.09090909090909091,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.49162352085113525,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:05:14.058913+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 11320.24704200012,
"inferenceMs": 11304.07704100071,
"audioSeconds": 3.24,
"realTimeFactor": 3.4889126669755277,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.14098206162452698,
"peak": 0.9166361093521118,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"loadMs": 15988.941625000734,
"totalMs": 27309.188667000853,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "27e595bf2fe7be0533ca299d9afafcde08b7cca7",
"peakHelperRssBytes": 6974111744,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.286622250383493,
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Hola, esta es una prueba de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.6907283067703247,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:05:14.168243+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 9010.05145900126,
"inferenceMs": 8960.303832998761,
"audioSeconds": 2.4,
"realTimeFactor": 3.7334599304161507,
"seed": 42,
"voiceMode": "reference",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.024879781529307365,
"peak": 0.24075374007225037,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"loadMs": 15988.941625000734,
"totalMs": 24998.993084001995,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "27e595bf2fe7be0533ca299d9afafcde08b7cca7",
"peakHelperRssBytes": 6974111744,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.26784806014739654,
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false
}
}
]Chatterbox Multilingual V3 · Hindi Desktop
19 configurations testedQuality criteria
{ "scope": "Frozen authored language/mode suite; complete waveform, independent ASR agreement, no repetition, RTF<=5 offline narration. Not subjective voice/locale identity certification.", "cases": [ { "id": "short", "label": "Short speech", "text": "नमस्ते। यह आवाज़ का एक परीक्षण है।", "language": "hi", "reference": false, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": null, "referenceSha256": null }, { "id": "numbers", "label": "Numbers and instructions", "text": "बैठक सुबह नौ बजे शुरू होती है। कृपया तीन किताबें लाएँ।", "language": "hi", "reference": false, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": null, "referenceSha256": null }, { "id": "reference", "label": "Reference voice", "text": "नमस्ते। यह आवाज़ का एक परीक्षण है।", "language": "hi", "reference": true, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": "model-samples/desktop/chatterbox-reference-input.wav", "referenceSha256": "a890ca57ba62dc2159b6139e77d3c821a790db3813cb8cb4f0bab9ecee5177fe" } ], "maximumWordErrorRate": 0.35, "maximumChineseCharacterErrorRate": 0.25, "maximumHindiCharacterErrorRate": 0.35, "maximumRealTimeFactor": 5 }
| Precision | Device | Result | Platform | Inference / load | Task metric | |
|---|---|---|---|---|---|---|
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 5670 ms / 14450 ms | 2.181 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 5713 ms / 14450 ms | 2.197 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 5923 ms / 14450 ms | 2.278 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 6049 ms / 14450 ms | 2.326 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 6306 ms / 14450 ms | 2.426 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 8391 ms / 13166 ms | 2.439 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 9192 ms / 14948 ms | 2.672 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 9602 ms / 14450 ms | 2.824 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 9701 ms / 13166 ms | 2.853 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 9952 ms / 14948 ms | 2.927 RTF |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 7995 ms / 14948 ms | 3.075 RTF |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 8392 ms / 13166 ms | 3.228 RTF |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 8646 ms / 14450 ms | 3.326 RTF |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 14833 ms / 14450 ms | 4.312 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 9216 ms / 10880 ms | 3.245 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 9976 ms / 12946 ms | 3.513 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 10290 ms / 12231 ms | 3.623 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 12565 ms / 12946 ms | 3.739 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 14274 ms / 12946 ms | 4.198 RTF |
Test setup & sample details
- Quality gate
- { "scope": "Frozen authored language/mode suite; complete waveform, independent ASR agreement, no repetition, RTF<=5 offline narration. Not subjective voice/locale identity certification.", "cases": [ { "id": "short", "label": "Short speech", "text": "नमस्ते। यह आवाज़ का एक परीक्षण है।", "language": "hi", "reference": false, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": null, "referenceSha256": null }, { "id": "numbers", "label": "Numbers and instructions", "text": "बैठक सुबह नौ बजे शुरू होती है। कृपया तीन किताबें लाएँ।", "language": "hi", "reference": false, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": null, "referenceSha256": null }, { "id": "reference", "label": "Reference voice", "text": "नमस्ते। यह आवाज़ का एक परीक्षण है।", "language": "hi", "reference": true, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": "model-samples/desktop/chatterbox-reference-input.wav", "referenceSha256": "a890ca57ba62dc2159b6139e77d3c821a790db3813cb8cb4f0bab9ecee5177fe" } ], "maximumWordErrorRate": 0.35, "maximumChineseCharacterErrorRate": 0.25, "maximumHindiCharacterErrorRate": 0.35, "maximumRealTimeFactor": 5 }
- Tested
- 2026-11-06T19:13:36.570957+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "runtimeProvisioning": "App-owned CPython3.13.18; model-owned pinned packages", "concurrentUploads": true, "concurrentActivity": "Background source/backup transfers active; synthesis/ASR/builds run sequentially", "measurementScope": "Apple M3 Max/macOS27.2 current-device scope; private seeded local-cache loads are not network-download benchmarks" }
- Runtime
- chatterbox-tts 0.1.7
- Artifact revision
- 82ca71273cc2a9ab19efdf8315f865c1a5af0ee7
- Fixture
{ "file": "model-samples/desktop/chatterbox-multilingual-family/v3-hi-short-input.json", "size": 255, "sha1": "c5e6478aa904ecf192880176ddf9396c6ec4c699", "sha256": "32565624564ac68a9054cb96da4f71f612780bde9074623465553197911f2026", "url": "https://assets.localai.tools/samples/v1/files/32565624564ac68a9054cb96da4f71f612780bde9074623465553197911f2026/v3-hi-short-input.json" }- Rerun when
- artifact revision or required file integrity; runtime/helper/model/processor/decoder semantics; fixture/config/checksum or quality gate; relevant backend/OS behavior
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "नमस्ते, ये आवास का एक परीक्षण है।",
"metric": "characterErrorRate",
"errorRate": 0.11538461538461539,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.4620899558067322,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:17:58.066183+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 10304.90479099899,
"inferenceMs": 10290.156709001167,
"audioSeconds": 2.84,
"realTimeFactor": 3.6232946158454813,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.12397696077823639,
"peak": 0.9367218613624573,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"loadMs": 12231.461874998786,
"totalMs": 22536.366665997775,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "82ca71273cc2a9ab19efdf8315f865c1a5af0ee7",
"peakHelperRssBytes": 5902778368,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.27599190958051695,
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "नमस्ते, ये आवास का एक परीक्षण है।",
"metric": "characterErrorRate",
"errorRate": 0.11538461538461539,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.4620899558067322,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:17:58.067084+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 9230.671790999622,
"inferenceMs": 9215.629791997344,
"audioSeconds": 2.84,
"realTimeFactor": 3.244940067604699,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.12397696077823639,
"peak": 0.9367218613624573,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 10880.221999999776,
"totalMs": 20110.8937909994,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "82ca71273cc2a9ab19efdf8315f865c1a5af0ee7",
"peakHelperRssBytes": 6896156672,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.3081721015384315,
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "नमस्ते, यह आवाज़ का एक परीक्षण है।",
"metric": "characterErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.5159662365913391,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:17:58.245019+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 8662.879208997765,
"inferenceMs": 8646.317459002603,
"audioSeconds": 2.6,
"realTimeFactor": 3.325506715001001,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.11813491582870483,
"peak": 0.8603562712669373,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 8007450624,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14449.69716599735,
"totalMs": 23112.576374995115,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "82ca71273cc2a9ab19efdf8315f865c1a5af0ee7",
"peakHelperRssBytes": 5270355968,
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.30070605345318,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "बेटक सुबह 9 बजे शुरू होती है, कृपया तीन किताब लाएँ।",
"metric": "characterErrorRate",
"errorRate": 0.14285714285714285,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.4892005920410156,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:17:58.476873+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 14849.800125000911,
"inferenceMs": 14832.79287500045,
"audioSeconds": 3.44,
"realTimeFactor": 4.31185839389548,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.14454029500484467,
"peak": 0.924462616443634,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10797940736,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14449.69716599735,
"totalMs": 29299.49729099826,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "82ca71273cc2a9ab19efdf8315f865c1a5af0ee7",
"peakHelperRssBytes": 5270355968,
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.23191856240356862,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "नमस्ते, ये आवाज़ का एक परीक्षण है।",
"metric": "characterErrorRate",
"errorRate": 0.038461538461538464,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.8012412190437317,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:17:58.658281+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 9657.654957998602,
"inferenceMs": 9602.366584000265,
"audioSeconds": 3.4,
"realTimeFactor": 2.824225465882431,
"seed": 42,
"voiceMode": "reference",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.03505629301071167,
"peak": 0.4096205234527588,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10863263744,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14449.69716599735,
"totalMs": 24107.352123995952,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "82ca71273cc2a9ab19efdf8315f865c1a5af0ee7",
"peakHelperRssBytes": 5270355968,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.35407937931313477,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "नमस्ते, यह आवाज़ का एक परीक्षण है।",
"metric": "characterErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.5159662365913391,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:17:58.658971+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 5726.469249999354,
"inferenceMs": 5712.811291999969,
"audioSeconds": 2.6,
"realTimeFactor": 2.1972351123076805,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.11813491582870483,
"peak": 0.8603562712669373,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10863263744,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14449.69716599735,
"totalMs": 20176.166415996704,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "82ca71273cc2a9ab19efdf8315f865c1a5af0ee7",
"peakHelperRssBytes": 5270355968,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.45511743117455206,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "नमस्ते, यह आवाज़ का एक परीक्षण है।",
"metric": "characterErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.5159662365913391,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:17:58.659364+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 5683.318874998804,
"inferenceMs": 5669.541749997734,
"audioSeconds": 2.6,
"realTimeFactor": 2.1805929807683593,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.11813491582870483,
"peak": 0.8603562712669373,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10863263744,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14449.69716599735,
"totalMs": 20133.016040996154,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "82ca71273cc2a9ab19efdf8315f865c1a5af0ee7",
"peakHelperRssBytes": 5270355968,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.4585908552487578,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "नमस्ते, यह आवाज़ का एक परीक्षण है।",
"metric": "characterErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.5159662365913391,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:17:58.659715+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 5938.122125000518,
"inferenceMs": 5922.604333998606,
"audioSeconds": 2.6,
"realTimeFactor": 2.2779247438456176,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.11813491582870483,
"peak": 0.8603562712669373,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10863263744,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14449.69716599735,
"totalMs": 20387.819290997868,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "82ca71273cc2a9ab19efdf8315f865c1a5af0ee7",
"peakHelperRssBytes": 5270355968,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.4389960654765921,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "नमस्ते, यह आवाज़ का एक परीक्षण है।",
"metric": "characterErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.5159662365913391,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:17:58.660053+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 6320.369749999372,
"inferenceMs": 6306.4092909989995,
"audioSeconds": 2.6,
"realTimeFactor": 2.4255420349996153,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.11813491582870483,
"peak": 0.8603562712669373,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10863263744,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14449.69716599735,
"totalMs": 20770.066915996722,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "82ca71273cc2a9ab19efdf8315f865c1a5af0ee7",
"peakHelperRssBytes": 5270355968,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.4122789815927303,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "नमस्ते, यह आवाज़ का एक परीक्षण है।",
"metric": "characterErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.5159662365913391,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:17:58.660398+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 6062.066458998743,
"inferenceMs": 6048.6793749987555,
"audioSeconds": 2.6,
"realTimeFactor": 2.3264151442302907,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.11813491582870483,
"peak": 0.8603562712669373,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10863263744,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14449.69716599735,
"totalMs": 20511.763624996092,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "82ca71273cc2a9ab19efdf8315f865c1a5af0ee7",
"peakHelperRssBytes": 5270355968,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.42984589508028515,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "नमस्ते, यह आवाज़ का एक परीक्षण है।",
"metric": "characterErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.5159662365913391,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:17:58.660729+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 8010.215124999377,
"inferenceMs": 7994.9030419993505,
"audioSeconds": 2.6,
"realTimeFactor": 3.0749627084612885,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.11813491582870483,
"peak": 0.8603562712669373,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 8007450624,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14948.26587499847,
"totalMs": 22958.480999997846,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "82ca71273cc2a9ab19efdf8315f865c1a5af0ee7",
"peakHelperRssBytes": 4822499328,
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.3252071959273939,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "बेटक सुबह 9 बजे शुरू होती है, कृपया तीन किताब लाएँ।",
"metric": "characterErrorRate",
"errorRate": 0.14285714285714285,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.4892005920410156,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:17:58.661374+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 9209.613290997368,
"inferenceMs": 9191.667499999312,
"audioSeconds": 3.44,
"realTimeFactor": 2.67199636627887,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.14454029500484467,
"peak": 0.924462616443634,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10797940736,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14948.26587499847,
"totalMs": 24157.879165995837,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "82ca71273cc2a9ab19efdf8315f865c1a5af0ee7",
"peakHelperRssBytes": 4822499328,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.37425200596086156,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "नमस्ते, ये आवाज़ का एक परीक्षण है।",
"metric": "characterErrorRate",
"errorRate": 0.038461538461538464,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.8012412190437317,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:17:58.661863+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 10008.735375002288,
"inferenceMs": 9951.974499999778,
"audioSeconds": 3.4,
"realTimeFactor": 2.9270513235293465,
"seed": 42,
"voiceMode": "reference",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.03505629301071167,
"peak": 0.4096205234527588,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10863263744,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14948.26587499847,
"totalMs": 24957.001250000758,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "82ca71273cc2a9ab19efdf8315f865c1a5af0ee7",
"peakHelperRssBytes": 4822499328,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.34164074676840017,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "नमस्ते, यह आवाज़ का एक परीक्षण है।",
"metric": "characterErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.5159662365913391,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:17:58.662253+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 8406.373707999592,
"inferenceMs": 8391.772582999693,
"audioSeconds": 2.6,
"realTimeFactor": 3.227604839615266,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.11813491582870483,
"peak": 0.8603562712669373,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 8007450624,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 13165.56212499927,
"totalMs": 21571.93583299886,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "82ca71273cc2a9ab19efdf8315f865c1a5af0ee7",
"peakHelperRssBytes": 4600037376,
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.30982727120932224,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "बेटक सुबह 9 बजे शुरू होती है, कृपया तीन किताब लाएँ।",
"metric": "characterErrorRate",
"errorRate": 0.14285714285714285,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.4892005920410156,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:17:58.662716+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 8406.923875001667,
"inferenceMs": 8390.608917001373,
"audioSeconds": 3.44,
"realTimeFactor": 2.4391304991283063,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.14454029500484467,
"peak": 0.924462616443634,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10797940736,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 13165.56212499927,
"totalMs": 21572.486000000936,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "82ca71273cc2a9ab19efdf8315f865c1a5af0ee7",
"peakHelperRssBytes": 4600037376,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.4099821638724861,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "नमस्ते, ये आवाज़ का एक परीक्षण है।",
"metric": "characterErrorRate",
"errorRate": 0.038461538461538464,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.8012412190437317,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:17:58.663211+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 9755.16133400015,
"inferenceMs": 9700.898249997408,
"audioSeconds": 3.4,
"realTimeFactor": 2.8532053676462965,
"seed": 42,
"voiceMode": "reference",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.03505629301071167,
"peak": 0.4096205234527588,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10863263744,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 13165.56212499927,
"totalMs": 22920.72345899942,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "82ca71273cc2a9ab19efdf8315f865c1a5af0ee7",
"peakHelperRssBytes": 4600037376,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.35048300810710065,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "नमस्ते, ये आवास का एक परीक्षण है।",
"metric": "characterErrorRate",
"errorRate": 0.11538461538461539,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.4620899558067322,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:17:58.664188+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 9992.988625002909,
"inferenceMs": 9975.898541997594,
"audioSeconds": 2.84,
"realTimeFactor": 3.512640331689294,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.12397696077823639,
"peak": 0.9367218613624573,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"loadMs": 12946.275082998909,
"totalMs": 22939.263708001818,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "82ca71273cc2a9ab19efdf8315f865c1a5af0ee7",
"peakHelperRssBytes": 6670647296,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.2846861350928808,
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "बैठक सुबह 9 बजे शुरू होती है। कृपया तीन किताबेलाएँ?",
"metric": "characterErrorRate",
"errorRate": 0.07142857142857142,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.5109958648681641,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:17:58.900156+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 12581.605792001938,
"inferenceMs": 12564.526958001807,
"audioSeconds": 3.36,
"realTimeFactor": 3.7394425470243475,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.13482867181301117,
"peak": 1.0200042724609375,
"clippedFraction": 0.000037202380952380956,
"watermark": "Resemble Perth applied by official runtime",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"loadMs": 12946.275082998909,
"totalMs": 25527.880875000847,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "82ca71273cc2a9ab19efdf8315f865c1a5af0ee7",
"peakHelperRssBytes": 6670647296,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.26741953845386596,
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "नमस्ते, यह आवाज़ का एक परीक्षण है.",
"metric": "characterErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.741580605506897,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:17:59.131164+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 14328.288124997925,
"inferenceMs": 14274.108165998769,
"audioSeconds": 3.4,
"realTimeFactor": 4.198267107646696,
"seed": 42,
"voiceMode": "reference",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.03898347169160843,
"peak": 0.40957844257354736,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"loadMs": 12946.275082998909,
"totalMs": 27274.563207996835,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "82ca71273cc2a9ab19efdf8315f865c1a5af0ee7",
"peakHelperRssBytes": 6670647296,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.23819351517167797,
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false
}
}
]Chatterbox Multilingual V3 · Portuguese (Brazil) Desktop
19 configurations testedQuality criteria
{ "scope": "Frozen authored language/mode suite; complete waveform, independent ASR agreement, no repetition, RTF<=5 offline narration. Not subjective voice/locale identity certification.", "cases": [ { "id": "short", "label": "Short speech", "text": "Olá. Este é um teste de voz.", "language": "pt", "reference": false, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": null, "referenceSha256": null }, { "id": "numbers", "label": "Numbers and instructions", "text": "A reunião começa às nove. Por favor, traga três cadernos.", "language": "pt", "reference": false, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": null, "referenceSha256": null }, { "id": "reference", "label": "Reference voice", "text": "Olá. Este é um teste de voz.", "language": "pt", "reference": true, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": "model-samples/desktop/chatterbox-reference-input.wav", "referenceSha256": "a890ca57ba62dc2159b6139e77d3c821a790db3813cb8cb4f0bab9ecee5177fe" } ], "maximumWordErrorRate": 0.35, "maximumChineseCharacterErrorRate": 0.25, "maximumHindiCharacterErrorRate": 0.35, "maximumRealTimeFactor": 5 }
| Precision | Device | Result | Platform | Inference / load | Task metric | |
|---|---|---|---|---|---|---|
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 6278 ms / 14555 ms | 1.962 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 6424 ms / 14555 ms | 2.008 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 6623 ms / 14555 ms | 2.070 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 8485 ms / 14555 ms | 2.210 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 8728 ms / 14644 ms | 2.273 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 8874 ms / 14082 ms | 2.311 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 9499 ms / 14644 ms | 2.474 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 7920 ms / 14555 ms | 2.475 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 9517 ms / 14555 ms | 2.478 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 9541 ms / 14082 ms | 2.485 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 9149 ms / 14555 ms | 2.859 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 9168 ms / 14082 ms | 2.865 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 9179 ms / 14555 ms | 2.868 RTF |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 9614 ms / 14644 ms | 3.004 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 13250 ms / 12299 ms | 3.562 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 8796 ms / 12299 ms | 3.727 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 11733 ms / 12299 ms | 3.761 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 11281 ms / 12361 ms | 4.780 RTF |
FP32 | cpu | Failed | — | macos · aarch64 | 12073 ms / 11193 ms | 5.116 RTF |
Test setup & sample details
- Quality gate
- { "scope": "Frozen authored language/mode suite; complete waveform, independent ASR agreement, no repetition, RTF<=5 offline narration. Not subjective voice/locale identity certification.", "cases": [ { "id": "short", "label": "Short speech", "text": "Olá. Este é um teste de voz.", "language": "pt", "reference": false, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": null, "referenceSha256": null }, { "id": "numbers", "label": "Numbers and instructions", "text": "A reunião começa às nove. Por favor, traga três cadernos.", "language": "pt", "reference": false, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": null, "referenceSha256": null }, { "id": "reference", "label": "Reference voice", "text": "Olá. Este é um teste de voz.", "language": "pt", "reference": true, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": "model-samples/desktop/chatterbox-reference-input.wav", "referenceSha256": "a890ca57ba62dc2159b6139e77d3c821a790db3813cb8cb4f0bab9ecee5177fe" } ], "maximumWordErrorRate": 0.35, "maximumChineseCharacterErrorRate": 0.25, "maximumHindiCharacterErrorRate": 0.35, "maximumRealTimeFactor": 5 }
- Tested
- 2026-11-06T19:09:08.247273+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "runtimeProvisioning": "App-owned CPython3.13.18; model-owned pinned packages", "concurrentUploads": true, "concurrentActivity": "Background source/backup transfers active; synthesis/ASR/builds run sequentially", "measurementScope": "Apple M3 Max/macOS27.2 current-device scope; private seeded local-cache loads are not network-download benchmarks" }
- Runtime
- chatterbox-tts 0.1.7
- Artifact revision
- b3952f18bc2eaa72b9bd7c17d2c4653bcad4770d
- Fixture
{ "file": "model-samples/desktop/chatterbox-multilingual-family/v3-pt-br-short-input.json", "size": 195, "sha1": "17123cd3c7639f0725785a945a82d1a37af910cc", "sha256": "58945d3c0db818ca76be2cb1f073cbb69889feea59ec23f88c8b140559f5970e", "url": "https://assets.localai.tools/samples/v1/files/58945d3c0db818ca76be2cb1f073cbb69889feea59ec23f88c8b140559f5970e/v3-pt-br-short-input.json" }- Rerun when
- artifact revision or required file integrity; runtime/helper/model/processor/decoder semantics; fixture/config/checksum or quality gate; relevant backend/OS behavior
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Olá, este é um teste de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.4099554419517517,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:13:35.397500+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 11294.71304099934,
"inferenceMs": 11281.241708999005,
"audioSeconds": 2.36,
"realTimeFactor": 4.780187164830087,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.10369430482387543,
"peak": 0.691417396068573,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"loadMs": 12361.323082997842,
"totalMs": 23656.036123997183,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "b3952f18bc2eaa72b9bd7c17d2c4653bcad4770d",
"peakHelperRssBytes": 6984335360,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.2091968296466369,
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": false,
"transcript": "Olá, este é um teste de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.4099554419517517,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:13:35.398156+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 12094.68254100284,
"inferenceMs": 12072.611334002431,
"audioSeconds": 2.36,
"realTimeFactor": 5.115513277119675,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.10369430482387543,
"peak": 0.691417396068573,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 11193.180124999344,
"totalMs": 23287.862666002184,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "b3952f18bc2eaa72b9bd7c17d2c4653bcad4770d",
"peakHelperRssBytes": 5901139968,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"performanceGrade": "slow",
"engine": "chatterbox-tts",
"grade": "slow",
"audioSecondsPerSecond": 0.1954838050118515,
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Olá! Este é um teste de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.4347701370716095,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:13:35.531787+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 9196.516916999826,
"inferenceMs": 9178.702290999354,
"audioSeconds": 3.2,
"realTimeFactor": 2.868344465937298,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.11287302523851395,
"peak": 0.7694332003593445,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 8649179136,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14555.279416999838,
"totalMs": 23751.796333999664,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "b3952f18bc2eaa72b9bd7c17d2c4653bcad4770d",
"peakHelperRssBytes": 5201215488,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.34863316169845965,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "A reunião começa às nove. Por favor, traz três cadernos.",
"metric": "wordErrorRate",
"errorRate": 0.1,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.5398639440536499,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:13:35.687651+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 8502.86162500197,
"inferenceMs": 8484.657459001028,
"audioSeconds": 3.84,
"realTimeFactor": 2.209546213281518,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.1353425681591034,
"peak": 0.9538277983665466,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 11307368448,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14555.279416999838,
"totalMs": 23058.141042001807,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "b3952f18bc2eaa72b9bd7c17d2c4653bcad4770d",
"peakHelperRssBytes": 5201215488,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.45258161788562246,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Olá! Este é um teste de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.7255810499191284,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:13:35.816808+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 9572.79862499854,
"inferenceMs": 9517.418957999325,
"audioSeconds": 3.84,
"realTimeFactor": 2.4784945203123243,
"seed": 42,
"voiceMode": "reference",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.04144201800227165,
"peak": 0.3891407549381256,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 11376820224,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14555.279416999838,
"totalMs": 24128.07804199838,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "b3952f18bc2eaa72b9bd7c17d2c4653bcad4770d",
"peakHelperRssBytes": 5201215488,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.4034707326583019,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Olá! Este é um teste de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.4347701370716095,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:13:35.817355+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 6639.236916998925,
"inferenceMs": 6623.214041999745,
"audioSeconds": 3.2,
"realTimeFactor": 2.0697543881249203,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.11287302523851395,
"peak": 0.7694332003593445,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 11376820224,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14555.279416999838,
"totalMs": 21194.516333998763,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "b3952f18bc2eaa72b9bd7c17d2c4653bcad4770d",
"peakHelperRssBytes": 5201215488,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.4831491145700351,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Olá! Este é um teste de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.4347701370716095,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:13:35.817705+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 6293.708207998861,
"inferenceMs": 6277.80574999997,
"audioSeconds": 3.2,
"realTimeFactor": 1.9618142968749908,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.11287302523851395,
"peak": 0.7694332003593445,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 11376820224,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14555.279416999838,
"totalMs": 20848.9876249987,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "b3952f18bc2eaa72b9bd7c17d2c4653bcad4770d",
"peakHelperRssBytes": 5201215488,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.5097322420337735,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Olá! Este é um teste de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.4347701370716095,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:13:35.818053+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 6440.606625001237,
"inferenceMs": 6424.342000002071,
"audioSeconds": 3.2,
"realTimeFactor": 2.007606875000647,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.11287302523851395,
"peak": 0.7694332003593445,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 11376820224,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14555.279416999838,
"totalMs": 20995.886042001075,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "b3952f18bc2eaa72b9bd7c17d2c4653bcad4770d",
"peakHelperRssBytes": 5201215488,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.49810548691196216,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Olá! Este é um teste de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.4347701370716095,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:13:35.818365+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 7935.131749996799,
"inferenceMs": 7919.937458002096,
"audioSeconds": 3.2,
"realTimeFactor": 2.474980455625655,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.11287302523851395,
"peak": 0.7694332003593445,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 11372625920,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14555.279416999838,
"totalMs": 22490.411166996637,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "b3952f18bc2eaa72b9bd7c17d2c4653bcad4770d",
"peakHelperRssBytes": 5201215488,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.4040435946582892,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Olá! Este é um teste de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.4347701370716095,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:13:35.818671+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 9164.934542000992,
"inferenceMs": 9149.427415999526,
"audioSeconds": 3.2,
"realTimeFactor": 2.859196067499852,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.11287302523851395,
"peak": 0.7694332003593445,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 11372625920,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14555.279416999838,
"totalMs": 23720.21395900083,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "b3952f18bc2eaa72b9bd7c17d2c4653bcad4770d",
"peakHelperRssBytes": 5201215488,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.34974866234844243,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Olá! Este é um teste de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.4347701370716095,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:13:35.818979+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 9184.461667002324,
"inferenceMs": 9167.876792002062,
"audioSeconds": 3.2,
"realTimeFactor": 2.8649614975006443,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.11287302523851395,
"peak": 0.7694332003593445,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 8649179136,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14082.362917000864,
"totalMs": 23266.82458400319,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "b3952f18bc2eaa72b9bd7c17d2c4653bcad4770d",
"peakHelperRssBytes": 4811603968,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.3490448304008229,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "A reunião começa às nove. Por favor, traz três cadernos.",
"metric": "wordErrorRate",
"errorRate": 0.1,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.5398639440536499,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:13:35.819291+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 8892.074874998798,
"inferenceMs": 8873.691041997517,
"audioSeconds": 3.84,
"realTimeFactor": 2.3108570421868535,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.1353425681591034,
"peak": 0.9538277983665466,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 11307368448,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14082.362917000864,
"totalMs": 22974.437791999662,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "b3952f18bc2eaa72b9bd7c17d2c4653bcad4770d",
"peakHelperRssBytes": 4811603968,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.4327398803751448,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Olá! Este é um teste de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.7255810499191284,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:13:35.819600+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 9596.973042000172,
"inferenceMs": 9540.753124998446,
"audioSeconds": 3.84,
"realTimeFactor": 2.4845711263016788,
"seed": 42,
"voiceMode": "reference",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.04144201800227165,
"peak": 0.3891407549381256,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 11376820224,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14082.362917000864,
"totalMs": 23679.335959001037,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "b3952f18bc2eaa72b9bd7c17d2c4653bcad4770d",
"peakHelperRssBytes": 4811603968,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.4024839496096515,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Olá! Este é um teste de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.4347701370716095,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:13:35.819915+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 9633.579041998019,
"inferenceMs": 9614.019249998819,
"audioSeconds": 3.2,
"realTimeFactor": 3.004381015624631,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.11287302523851395,
"peak": 0.7694332003593445,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 8649179136,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14643.529209002736,
"totalMs": 24277.108251000755,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "b3952f18bc2eaa72b9bd7c17d2c4653bcad4770d",
"peakHelperRssBytes": 5268373504,
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.33284726364578404,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "A reunião começa às nove. Por favor, traz três cadernos.",
"metric": "wordErrorRate",
"errorRate": 0.1,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.5398639440536499,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:13:35.820428+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 8746.012790998066,
"inferenceMs": 8727.875791999395,
"audioSeconds": 3.84,
"realTimeFactor": 2.272884320833176,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.1353425681591034,
"peak": 0.9538277983665466,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 11307368448,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14643.529209002736,
"totalMs": 23389.5420000008,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "b3952f18bc2eaa72b9bd7c17d2c4653bcad4770d",
"peakHelperRssBytes": 5268373504,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.4399695975875393,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Olá! Este é um teste de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.7255810499191284,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:13:35.820811+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 9554.494249998243,
"inferenceMs": 9498.693042001833,
"audioSeconds": 3.84,
"realTimeFactor": 2.4736179796879774,
"seed": 42,
"voiceMode": "reference",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.04144201800227165,
"peak": 0.3891407549381256,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 11376820224,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14643.529209002736,
"totalMs": 24198.02345900098,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "b3952f18bc2eaa72b9bd7c17d2c4653bcad4770d",
"peakHelperRssBytes": 5268373504,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.40426614303884556,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Olá, este é um teste de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.4099554419517517,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:13:35.821741+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 8809.457999999722,
"inferenceMs": 8796.179167002265,
"audioSeconds": 2.36,
"realTimeFactor": 3.7271945622890956,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.10369430482387543,
"peak": 0.691417396068573,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"loadMs": 12298.722999999882,
"totalMs": 21108.180999999604,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "b3952f18bc2eaa72b9bd7c17d2c4653bcad4770d",
"peakHelperRssBytes": 6960070656,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.2682983094356737,
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "A reunião começa às nove. Por favor, traga três cadernos.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.5210901498794556,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:13:35.979566+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 13267.618624999159,
"inferenceMs": 13249.884499997279,
"audioSeconds": 3.72,
"realTimeFactor": 3.5617969086014187,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.12264944612979889,
"peak": 0.8815847635269165,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"loadMs": 12298.722999999882,
"totalMs": 25566.34162499904,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "b3952f18bc2eaa72b9bd7c17d2c4653bcad4770d",
"peakHelperRssBytes": 6960070656,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.280757164336094,
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "Olá! Este é um teste de voz.",
"metric": "wordErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.35,
"repetitive": false,
"speakerCosineDiagnostic": 0.6198692321777344,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:13:36.111349+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 11786.812207999901,
"inferenceMs": 11732.817874999455,
"audioSeconds": 3.12,
"realTimeFactor": 3.7605185496793125,
"seed": 42,
"voiceMode": "reference",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.034289754927158356,
"peak": 0.25975513458251953,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"loadMs": 12298.722999999882,
"totalMs": 24085.535207999783,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "b3952f18bc2eaa72b9bd7c17d2c4653bcad4770d",
"peakHelperRssBytes": 6960070656,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.2659207731032938,
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false
}
}
]Chatterbox Multilingual V3 · Mandarin Chinese Desktop
19 configurations testedQuality criteria
{ "scope": "Frozen authored language/mode suite; complete waveform, independent ASR agreement, no repetition, RTF<=5 offline narration. Not subjective voice/locale identity certification.", "cases": [ { "id": "short", "label": "Short speech", "text": "你好。这是一个语音测试。", "language": "zh", "reference": false, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": null, "referenceSha256": null }, { "id": "numbers", "label": "Numbers and instructions", "text": "会议早上九点开始。请带三本书。", "language": "zh", "reference": false, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": null, "referenceSha256": null }, { "id": "reference", "label": "Reference voice", "text": "你好。这是一个语音测试。", "language": "zh", "reference": true, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": "model-samples/desktop/chatterbox-reference-input.wav", "referenceSha256": "a890ca57ba62dc2159b6139e77d3c821a790db3813cb8cb4f0bab9ecee5177fe" } ], "maximumWordErrorRate": 0.35, "maximumChineseCharacterErrorRate": 0.25, "maximumHindiCharacterErrorRate": 0.35, "maximumRealTimeFactor": 5 }
| Precision | Device | Result | Platform | Inference / load | Task metric | |
|---|---|---|---|---|---|---|
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 5235 ms / 12621 ms | 2.218 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 5446 ms / 12621 ms | 2.308 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 5497 ms / 12621 ms | 2.329 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 7529 ms / 12621 ms | 2.413 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 7554 ms / 14537 ms | 2.421 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 7831 ms / 14023 ms | 2.510 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 6047 ms / 12621 ms | 2.562 RTF |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 6214 ms / 12621 ms | 2.633 RTF |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 7674 ms / 14537 ms | 3.197 RTF |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 7692 ms / 14023 ms | 3.205 RTF |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 7683 ms / 14537 ms | 3.255 RTF |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 8142 ms / 12621 ms | 3.393 RTF |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 8074 ms / 12621 ms | 3.421 RTF |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 8214 ms / 14023 ms | 3.480 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 9897 ms / 12687 ms | 3.389 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 8472 ms / 13618 ms | 3.851 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 8783 ms / 13341 ms | 3.992 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 8950 ms / 12687 ms | 3.995 RTF |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 9826 ms / 12687 ms | 4.466 RTF |
Test setup & sample details
- Quality gate
- { "scope": "Frozen authored language/mode suite; complete waveform, independent ASR agreement, no repetition, RTF<=5 offline narration. Not subjective voice/locale identity certification.", "cases": [ { "id": "short", "label": "Short speech", "text": "你好。这是一个语音测试。", "language": "zh", "reference": false, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": null, "referenceSha256": null }, { "id": "numbers", "label": "Numbers and instructions", "text": "会议早上九点开始。请带三本书。", "language": "zh", "reference": false, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": null, "referenceSha256": null }, { "id": "reference", "label": "Reference voice", "text": "你好。这是一个语音测试。", "language": "zh", "reference": true, "source": "Authored text for reproducible local synthesis; public-domain test text", "referenceFile": "model-samples/desktop/chatterbox-reference-input.wav", "referenceSha256": "a890ca57ba62dc2159b6139e77d3c821a790db3813cb8cb4f0bab9ecee5177fe" } ], "maximumWordErrorRate": 0.35, "maximumChineseCharacterErrorRate": 0.25, "maximumHindiCharacterErrorRate": 0.35, "maximumRealTimeFactor": 5 }
- Tested
- 2026-11-06T19:17:59.670287+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "runtimeProvisioning": "App-owned CPython3.13.18; model-owned pinned packages", "concurrentUploads": true, "concurrentActivity": "Background source/backup transfers active; synthesis/ASR/builds run sequentially", "measurementScope": "Apple M3 Max/macOS27.2 current-device scope; private seeded local-cache loads are not network-download benchmarks" }
- Runtime
- chatterbox-tts 0.1.7
- Artifact revision
- 2432cb141088d54f666894996d959294a74a13ce
- Fixture
{ "file": "model-samples/desktop/chatterbox-multilingual-family/v3-zh-cmn-short-input.json", "size": 201, "sha1": "5bcb328287f7ca911ce4652dd846a5af038d66ca", "sha256": "64030344c67f7af793baf9b27a07ff77f7776a8c986fef99bd98d0ef3f8220e4", "url": "https://assets.localai.tools/samples/v1/files/64030344c67f7af793baf9b27a07ff77f7776a8c986fef99bd98d0ef3f8220e4/v3-zh-cmn-short-input.json" }- Rerun when
- artifact revision or required file integrity; runtime/helper/model/processor/decoder semantics; fixture/config/checksum or quality gate; relevant backend/OS behavior
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "你好,这是一个语音测试。",
"metric": "characterErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.25,
"repetitive": false,
"speakerCosineDiagnostic": 0.5084903836250305,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:21:55.212557+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 8484.944666000956,
"inferenceMs": 8472.07937499843,
"audioSeconds": 2.2,
"realTimeFactor": 3.8509451704538313,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.11619202792644501,
"peak": 0.6960462331771851,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"loadMs": 13618.347583000286,
"totalMs": 22103.292249001242,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "2432cb141088d54f666894996d959294a74a13ce",
"peakHelperRssBytes": 6961135616,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.2596765094638184,
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "你好,这是一个语音测试。",
"metric": "characterErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.25,
"repetitive": false,
"speakerCosineDiagnostic": 0.5084800720214844,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:21:55.302540+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 8796.425084001385,
"inferenceMs": 8783.496791998914,
"audioSeconds": 2.2,
"realTimeFactor": 3.992498541817688,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.11619206517934799,
"peak": 0.6960647702217102,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 13340.774249998503,
"totalMs": 22137.199333999888,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "2432cb141088d54f666894996d959294a74a13ce",
"peakHelperRssBytes": 6248054784,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.25046972203644796,
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "你好,这是一个语音测试。",
"metric": "characterErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.25,
"repetitive": false,
"speakerCosineDiagnostic": 0.45659172534942627,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:21:55.396682+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 8087.3308750014985,
"inferenceMs": 8073.783583000477,
"audioSeconds": 2.36,
"realTimeFactor": 3.4210947385595243,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.11236477643251419,
"peak": 0.8405289649963379,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 8088387584,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 12621.286875000806,
"totalMs": 20708.617750002304,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "2432cb141088d54f666894996d959294a74a13ce",
"peakHelperRssBytes": 4773298176,
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.29230409457209505,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "会议早上九点开始,请带三本书。",
"metric": "characterErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.25,
"repetitive": false,
"speakerCosineDiagnostic": 0.4878726601600647,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:21:55.508899+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 7544.635583999479,
"inferenceMs": 7529.3664159980835,
"audioSeconds": 3.12,
"realTimeFactor": 2.4132584666660524,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.12915213406085968,
"peak": 0.9594025015830994,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10340024320,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 12621.286875000806,
"totalMs": 20165.922459000285,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "2432cb141088d54f666894996d959294a74a13ce",
"peakHelperRssBytes": 4773298176,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.41437749574396515,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "你好,这是一个语音测试。",
"metric": "characterErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.25,
"repetitive": false,
"speakerCosineDiagnostic": 0.7332454323768616,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:21:55.608565+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 8192.982415999722,
"inferenceMs": 8142.371417001414,
"audioSeconds": 2.4,
"realTimeFactor": 3.3926547570839225,
"seed": 42,
"voiceMode": "reference",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.023556621745228767,
"peak": 0.3188202381134033,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10399268864,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 12621.286875000806,
"totalMs": 20814.269291000528,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "2432cb141088d54f666894996d959294a74a13ce",
"peakHelperRssBytes": 4773298176,
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.2947544243669305,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "你好,这是一个语音测试。",
"metric": "characterErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.25,
"repetitive": false,
"speakerCosineDiagnostic": 0.45659172534942627,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:21:55.609176+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 5510.576749999018,
"inferenceMs": 5497.360958001082,
"audioSeconds": 2.36,
"realTimeFactor": 2.3293902364411365,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.11236477643251419,
"peak": 0.8405289649963379,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10392977408,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 12621.286875000806,
"totalMs": 18131.863624999824,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "2432cb141088d54f666894996d959294a74a13ce",
"peakHelperRssBytes": 4773298176,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.42929689682558686,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "你好,这是一个语音测试。",
"metric": "characterErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.25,
"repetitive": false,
"speakerCosineDiagnostic": 0.45659172534942627,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:21:55.609623+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 6227.0807919994695,
"inferenceMs": 6214.018415997998,
"audioSeconds": 2.36,
"realTimeFactor": 2.6330586508466096,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.11236477643251419,
"peak": 0.8405289649963379,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10397171712,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 12621.286875000806,
"totalMs": 18848.367667000275,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "2432cb141088d54f666894996d959294a74a13ce",
"peakHelperRssBytes": 4773298176,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.3797864508937046,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "你好,这是一个语音测试。",
"metric": "characterErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.25,
"repetitive": false,
"speakerCosineDiagnostic": 0.45659172534942627,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:21:55.609967+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 6059.1534999985015,
"inferenceMs": 6046.542083000531,
"audioSeconds": 2.36,
"realTimeFactor": 2.562094102966327,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.11236477643251419,
"peak": 0.8405289649963379,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10390880256,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 12621.286875000806,
"totalMs": 18680.440374999307,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "2432cb141088d54f666894996d959294a74a13ce",
"peakHelperRssBytes": 4773298176,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.39030572641427397,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "你好,这是一个语音测试。",
"metric": "characterErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.25,
"repetitive": false,
"speakerCosineDiagnostic": 0.45659172534942627,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:21:55.610295+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 5458.744667001156,
"inferenceMs": 5446.245541999815,
"audioSeconds": 2.36,
"realTimeFactor": 2.3077311618643286,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.11236477643251419,
"peak": 0.8405289649963379,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10390880256,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 12621.286875000806,
"totalMs": 18080.03154200196,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "2432cb141088d54f666894996d959294a74a13ce",
"peakHelperRssBytes": 4773298176,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.4333260375060923,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "你好,这是一个语音测试。",
"metric": "characterErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.25,
"repetitive": false,
"speakerCosineDiagnostic": 0.45659172534942627,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:21:55.610600+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 5248.026375000336,
"inferenceMs": 5235.16679200111,
"audioSeconds": 2.36,
"realTimeFactor": 2.218291013559792,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.11236477643251419,
"peak": 0.8405289649963379,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10390880256,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 12621.286875000806,
"totalMs": 17869.31325000114,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "2432cb141088d54f666894996d959294a74a13ce",
"peakHelperRssBytes": 4773298176,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.4507974805322114,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "你好,这是一个语音测试。",
"metric": "characterErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.25,
"repetitive": false,
"speakerCosineDiagnostic": 0.45659172534942627,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:21:55.610884+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 7697.9504999981145,
"inferenceMs": 7682.644959000754,
"audioSeconds": 2.36,
"realTimeFactor": 3.255358033474896,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.11236477643251419,
"peak": 0.8405289649963379,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 8088387584,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14536.669040997367,
"totalMs": 22234.61954099548,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "2432cb141088d54f666894996d959294a74a13ce",
"peakHelperRssBytes": 5270863872,
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.3071858731718554,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "会议早上九点开始,请带三本书。",
"metric": "characterErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.25,
"repetitive": false,
"speakerCosineDiagnostic": 0.4878726601600647,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:21:55.611180+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 7569.651500001783,
"inferenceMs": 7553.884417000518,
"audioSeconds": 3.12,
"realTimeFactor": 2.4211168003206787,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.12915213406085968,
"peak": 0.9594025015830994,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10340024320,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14536.669040997367,
"totalMs": 22106.32054099915,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "2432cb141088d54f666894996d959294a74a13ce",
"peakHelperRssBytes": 5270863872,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.4130325310482953,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "你好,这是一个语音测试。",
"metric": "characterErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.25,
"repetitive": false,
"speakerCosineDiagnostic": 0.7332454323768616,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:21:55.611505+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 7725.44304099938,
"inferenceMs": 7673.895082996751,
"audioSeconds": 2.4,
"realTimeFactor": 3.1974562845819796,
"seed": 42,
"voiceMode": "reference",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.023556621745228767,
"peak": 0.3188202381134033,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10399268864,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14536.669040997367,
"totalMs": 22262.112081996747,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "2432cb141088d54f666894996d959294a74a13ce",
"peakHelperRssBytes": 5270863872,
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.3127486073295089,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "你好,这是一个语音测试。",
"metric": "characterErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.25,
"repetitive": false,
"speakerCosineDiagnostic": 0.45659172534942627,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:21:55.611801+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 8227.070749999257,
"inferenceMs": 8213.601375002327,
"audioSeconds": 2.36,
"realTimeFactor": 3.480339565678952,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.11236477643251419,
"peak": 0.8405289649963379,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 8088387584,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14022.610083997279,
"totalMs": 22249.680833996536,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "2432cb141088d54f666894996d959294a74a13ce",
"peakHelperRssBytes": 4617601024,
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.2873282853953125,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "会议早上九点开始,请带三本书。",
"metric": "characterErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.25,
"repetitive": false,
"speakerCosineDiagnostic": 0.4878726601600647,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:21:55.612088+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 7846.229124999809,
"inferenceMs": 7830.6924589996925,
"audioSeconds": 3.12,
"realTimeFactor": 2.5098373266024656,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.12915213406085968,
"peak": 0.9594025015830994,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10340024320,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14022.610083997279,
"totalMs": 21868.839208997088,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "2432cb141088d54f666894996d959294a74a13ce",
"peakHelperRssBytes": 4617601024,
"performanceGrade": "good",
"engine": "chatterbox-tts",
"grade": "good",
"audioSecondsPerSecond": 0.3984321969399057,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"transcript": "你好,这是一个语音测试。",
"metric": "characterErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.25,
"repetitive": false,
"speakerCosineDiagnostic": 0.7332454323768616,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:21:55.612373+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 7743.358042000182,
"inferenceMs": 7692.400208998151,
"audioSeconds": 2.4,
"realTimeFactor": 3.20516675374923,
"seed": 42,
"voiceMode": "reference",
"placement": {
"t3": {
"mps:0": 292
},
"s3gen": {
"mps:0": 2122
},
"ve": {
"mps:0": 16
}
},
"rms": 0.023556621745228767,
"peak": 0.3188202381134033,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"driverAllocatedMemoryBytes": 10399268864,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"device": "mps",
"precision": "fp32",
"cpuThreads": 8,
"loadMs": 14022.610083997279,
"totalMs": 21765.96812599746,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "2432cb141088d54f666894996d959294a74a13ce",
"peakHelperRssBytes": 4617601024,
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.3119962475681661,
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "你好,这是一个语音测试。",
"metric": "characterErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.25,
"repetitive": false,
"speakerCosineDiagnostic": 0.5084903836250305,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:21:55.613265+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 9838.696833998256,
"inferenceMs": 9825.983957998687,
"audioSeconds": 2.2,
"realTimeFactor": 4.466356344544858,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.11619202792644501,
"peak": 0.6960462331771851,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"loadMs": 12686.717459000647,
"totalMs": 22525.414292998903,
"initReuse": false,
"version": "0.1.7",
"artifactRevision": "2432cb141088d54f666894996d959294a74a13ce",
"peakHelperRssBytes": 5950472192,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.22389615222291553,
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "会议早上九点开始,请带三本书。",
"metric": "characterErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.25,
"repetitive": false,
"speakerCosineDiagnostic": 0.45930296182632446,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:21:55.720881+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 9911.270417000196,
"inferenceMs": 9896.916375000728,
"audioSeconds": 2.92,
"realTimeFactor": 3.389354922945455,
"seed": 42,
"voiceMode": "built-in",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.13102643191814423,
"peak": 0.8562802076339722,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"loadMs": 12686.717459000647,
"totalMs": 22597.987876000843,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "2432cb141088d54f666894996d959294a74a13ce",
"peakHelperRssBytes": 5950472192,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.2950413936381053,
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"transcript": "你好,这是一个语音测试。",
"metric": "characterErrorRate",
"errorRate": 0,
"maximumErrorRate": 0.25,
"repetitive": false,
"speakerCosineDiagnostic": 0.7961423397064209,
"speakerIdentityCertified": false,
"evaluatedAt": "2026-11-06T19:21:55.809132+00:00",
"evaluator": "Qwen3-ASR0.6B MLX4bit revision313d850181767edf09f00a9c289becca70e58cd0 / mlx-audio0.6.3+mlx0.32.4, Resemblyzer0.1.4 CPU speaker diagnostic"
},
"runtime": {
"durationMs": 8998.648208002123,
"inferenceMs": 8949.791499999264,
"audioSeconds": 2.24,
"realTimeFactor": 3.9954426339282425,
"seed": 42,
"voiceMode": "reference",
"placement": {
"t3": {
"cpu": 292
},
"s3gen": {
"cpu": 2122
},
"ve": {
"cpu": 16
}
},
"rms": 0.02801312692463398,
"peak": 0.3246081471443176,
"clippedFraction": 0,
"watermark": "Resemble Perth applied by official runtime",
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"loadMs": 12686.717459000647,
"totalMs": 21685.36566700277,
"initReuse": true,
"version": "0.1.7",
"artifactRevision": "2432cb141088d54f666894996d959294a74a13ce",
"peakHelperRssBytes": 5950472192,
"memoryMeasurementScope": "ps sampled helper RSS every500ms; MPS allocation after inference is not peak VRAM",
"performanceGrade": "acceptable",
"engine": "chatterbox-tts",
"grade": "acceptable",
"audioSecondsPerSecond": 0.25028516027442477,
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"fallback": false
}
}
]