Sweep Next Edit Desktop
Suggest a complete next version of a code file using repository context and recent edits. Runs the original Q8_0 GGUF through native Transformers.
Desktop app
Run Sweep Next Edit Desktop on your computer
Download the desktop app to load this model, provide input and view results locally.
Public desktop downloads are coming soon.Device requirements
If your device falls short
MemoryAdvisory only: close other apps, reduce image/tile/token/batch settings or choose a smaller verified model/precision. CPU can need more RAM than GPU.
GPUIf the selected GPU runtime fails, existing CPU fallback may recover when enabled, but CPU has its own memory/runtime needs.
PlatformUse a compatible OS/architecture/runtime or another model. These estimates do not add hardware detection or block runs.
Benchmark evidence
Saved desktop measurements apply to the recorded hardware and runtime. Fallback cases report the device that actually ran; they do not prove GPU support on the requested device.
Sweep Next-Edit 1.5B Desktop
36 configurations testedQuality criteria
{ "file": "research/sweep-next-edit-family-expansion/quality-gates.json", "sha256": "13d2464d0ec31b1ab766319b0b38fe938271a1ec206a46c2436bf819fa27537b", "description": "Actual helper call inside greet; valid complete Python; static integer-addition/square semantics and prompt retention. No code executed. All3fixtures, <=15s/inference and>=5tokens/s." }
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 874 ms / 9417 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 876 ms / 9406 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 878 ms / 9436 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 262 ms / 9406 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 283 ms / 9406 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 263 ms / 9436 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 263 ms / 9417 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 284 ms / 9417 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 285 ms / 9436 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 1420 ms / 9754 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 1421 ms / 9700 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 444 ms / 9754 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 446 ms / 9700 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 414 ms / 9700 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 414 ms / 9754 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 422 ms / 9820 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 465 ms / 9820 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 1660 ms / 9820 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 947 ms / 10117 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 953 ms / 10105 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 889 ms / 10117 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 962 ms / 9895 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 897 ms / 10698 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 975 ms / 10188 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 911 ms / 9895 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 916 ms / 10188 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 986 ms / 9950 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 932 ms / 9950 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 1004 ms / 10698 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 3335 ms / 10188 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 3357 ms / 10105 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 951 ms / 10105 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 3441 ms / 10698 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 3477 ms / 9950 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 3481 ms / 10117 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 3493 ms / 9895 ms |
Test setup & sample details
- Quality gate
- { "file": "research/sweep-next-edit-family-expansion/quality-gates.json", "sha256": "13d2464d0ec31b1ab766319b0b38fe938271a1ec206a46c2436bf819fa27537b", "description": "Actual helper call inside greet; valid complete Python; static integer-addition/square semantics and prompt retention. No code executed. All3fixtures, <=15s/inference and>=5tokens/s." }
- Tested
- 2026-11-12T20:58:49.500905+00:00
- Environment
- { "os": "macos", "osVersion": "27.2", "architecture": "aarch64", "hardware": "Apple M3 Max;Mac15,11;96GiB RAM", "cpu": "Apple M3 Max", "gpu": "Apple M3 Max", "driver": "OS-managed Metal/MPS macOS27.2 build26B118", "runtime": "App-owned CPython3.13.18;Transformers5.22.1;Torch2.10.0;GGUF0.19.0", "timingScope": "Fresh helper/model process per cold repetition. GGUF import/dequantization and tokenizer/runtime startup in load; OS files/driver caches may be warm. Warm repetition after three task fixtures. Snapshot-only transfer, no Internet timing.", "concurrentUploadActivity": "none; user excluded uploads" }
- Runtime
- transformers 5.22.1
- Artifact revision
- 409016591c6c1a94f545f22328a85a3516118f34
- Fixture
{ "file": "model-samples/native/sweep-next-edit/quality-gates.json", "url": "https://assets.localai.tools/samples/v1/files/13d2464d0ec31b1ab766319b0b38fe938271a1ec206a46c2436bf819fa27537b/sweep-next-edit-quality-gates.json", "size": 3408, "sha1": "25e62103b91e31ae2320aab7379dd8a04af06b40", "sha256": "13d2464d0ec31b1ab766319b0b38fe938271a1ec206a46c2436bf819fa27537b" }- Rerun when
- artifact-revision; runtime-version; fixture-checksum; quality-gate; backend-behavior
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "greeting-context",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 3481.3466669875197,
"inferenceMs": 3480.6580410222523,
"tokensPerSecond": 13.215891781914326,
"charactersPerSecond": 44.24450726988709,
"inputTokens": 157,
"outputTokens": 46,
"outputLimitReached": false,
"placement": {
"cpu/torch.float32": 339
},
"streamChunks": 0,
"loadMs": 10117.09308301215,
"totalMs": 13597.751124034403,
"grade": "good",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "rename-parameter",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 947.6490419765469,
"inferenceMs": 947.4046250106767,
"tokensPerSecond": 14.777213062309283,
"charactersPerSecond": 50.66473049934611,
"inputTokens": 38,
"outputTokens": 14,
"outputLimitReached": false,
"placement": {
"cpu/torch.float32": 339
},
"streamChunks": 0,
"loadMs": 10117.09308301215,
"totalMs": 11064.497708022827,
"grade": "good",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "no-edit",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 889.0892080089543,
"inferenceMs": 888.8817499973811,
"tokensPerSecond": 14.625117458017673,
"charactersPerSecond": 39.37531623312451,
"inputTokens": 37,
"outputTokens": 13,
"outputLimitReached": false,
"placement": {
"cpu/torch.float32": 339
},
"streamChunks": 0,
"loadMs": 10117.09308301215,
"totalMs": 11005.974833009532,
"grade": "good",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "greeting-context",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 3357.5666669930797,
"inferenceMs": 3356.8639580043964,
"tokensPerSecond": 13.703266076754057,
"charactersPerSecond": 45.87615164826358,
"inputTokens": 157,
"outputTokens": 46,
"outputLimitReached": false,
"placement": {
"cpu/torch.float32": 339
},
"streamChunks": 0,
"loadMs": 10105.382750014542,
"totalMs": 13462.246708018938,
"grade": "good",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "rename-parameter",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 953.6057500226889,
"inferenceMs": 953.3635829866398,
"tokensPerSecond": 14.684848729108833,
"charactersPerSecond": 50.348052785515996,
"inputTokens": 38,
"outputTokens": 14,
"outputLimitReached": false,
"placement": {
"cpu/torch.float32": 339
},
"streamChunks": 0,
"loadMs": 10105.382750014542,
"totalMs": 11058.746333001181,
"grade": "good",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "no-edit",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 950.8677079866175,
"inferenceMs": 950.6412080081645,
"tokensPerSecond": 13.674980518926073,
"charactersPerSecond": 36.81725524326251,
"inputTokens": 37,
"outputTokens": 13,
"outputLimitReached": false,
"placement": {
"cpu/torch.float32": 339
},
"streamChunks": 0,
"loadMs": 10105.382750014542,
"totalMs": 11056.023958022706,
"grade": "good",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "greeting-context",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 3477.5677919969894,
"inferenceMs": 3476.822833006736,
"tokensPerSecond": 13.230469946097159,
"charactersPerSecond": 44.293312428238316,
"inputTokens": 157,
"outputTokens": 46,
"outputLimitReached": false,
"placement": {
"cpu/torch.float32": 339
},
"streamChunks": 0,
"loadMs": 9950.387083023088,
"totalMs": 13427.209916029824,
"grade": "good",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "rename-parameter",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 986.3551669986919,
"inferenceMs": 986.0953750030603,
"tokensPerSecond": 14.197409657211455,
"charactersPerSecond": 48.676833110439276,
"inputTokens": 38,
"outputTokens": 14,
"outputLimitReached": false,
"placement": {
"cpu/torch.float32": 339
},
"streamChunks": 0,
"loadMs": 9950.387083023088,
"totalMs": 10936.482458026148,
"grade": "good",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "no-edit",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 931.7344170121942,
"inferenceMs": 931.5242079901509,
"tokensPerSecond": 13.955622289246454,
"charactersPerSecond": 37.572829240278914,
"inputTokens": 37,
"outputTokens": 13,
"outputLimitReached": false,
"placement": {
"cpu/torch.float32": 339
},
"streamChunks": 0,
"loadMs": 9950.387083023088,
"totalMs": 10881.911291013239,
"grade": "good",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 1,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "greeting-context",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 3493.264749995433,
"inferenceMs": 3492.5227500207257,
"tokensPerSecond": 13.170995092222956,
"charactersPerSecond": 44.09420096092033,
"inputTokens": 157,
"outputTokens": 46,
"outputLimitReached": false,
"placement": {
"cpu/torch.float32": 339
},
"streamChunks": 0,
"loadMs": 9895.09375000489,
"totalMs": 13387.616500025615,
"grade": "good",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "rename-parameter",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 962.456708017271,
"inferenceMs": 962.2079170076177,
"tokensPerSecond": 14.549869890427397,
"charactersPerSecond": 49.885268195751074,
"inputTokens": 38,
"outputTokens": 14,
"outputLimitReached": false,
"placement": {
"cpu/torch.float32": 339
},
"streamChunks": 0,
"loadMs": 9895.09375000489,
"totalMs": 10857.301667012507,
"grade": "good",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "no-edit",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 910.990500007756,
"inferenceMs": 910.7842499797698,
"tokensPerSecond": 14.273413270254458,
"charactersPerSecond": 38.42842034299277,
"inputTokens": 37,
"outputTokens": 13,
"outputLimitReached": false,
"placement": {
"cpu/torch.float32": 339
},
"streamChunks": 0,
"loadMs": 9895.09375000489,
"totalMs": 10805.87799998466,
"grade": "good",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "greeting-context",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 3441.7680419865064,
"inferenceMs": 3441.045875020791,
"tokensPerSecond": 13.368028695555262,
"charactersPerSecond": 44.75383519816327,
"inputTokens": 157,
"outputTokens": 46,
"outputLimitReached": false,
"placement": {
"cpu/torch.float32": 339
},
"streamChunks": 0,
"loadMs": 10697.612792020664,
"totalMs": 14138.658667041454,
"grade": "good",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "rename-parameter",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 1004.4087499845773,
"inferenceMs": 1004.1581670229789,
"tokensPerSecond": 13.942026724241767,
"charactersPerSecond": 47.801234483114634,
"inputTokens": 38,
"outputTokens": 14,
"outputLimitReached": false,
"placement": {
"cpu/torch.float32": 339
},
"streamChunks": 0,
"loadMs": 10697.612792020664,
"totalMs": 11701.770959043643,
"grade": "good",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "no-edit",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 897.2117500088643,
"inferenceMs": 896.9800000195391,
"tokensPerSecond": 14.493076768397085,
"charactersPerSecond": 39.019822068761385,
"inputTokens": 37,
"outputTokens": 13,
"outputLimitReached": false,
"placement": {
"cpu/torch.float32": 339
},
"streamChunks": 0,
"loadMs": 10697.612792020664,
"totalMs": 11594.592792040203,
"grade": "good",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "greeting-context",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 3335.722916992381,
"inferenceMs": 3335.014540993143,
"tokensPerSecond": 13.793043308981057,
"charactersPerSecond": 46.17671020832788,
"inputTokens": 157,
"outputTokens": 46,
"outputLimitReached": false,
"placement": {
"cpu/torch.float32": 339
},
"streamChunks": 0,
"loadMs": 10188.248666992877,
"totalMs": 13523.26320798602,
"grade": "good",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "rename-parameter",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 975.3110409947112,
"inferenceMs": 975.069459003862,
"tokensPerSecond": 14.357951498452737,
"charactersPerSecond": 49.227262280409384,
"inputTokens": 38,
"outputTokens": 14,
"outputLimitReached": false,
"placement": {
"cpu/torch.float32": 339
},
"streamChunks": 0,
"loadMs": 10188.248666992877,
"totalMs": 11163.318125996739,
"grade": "good",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "no-edit",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 915.8662499976344,
"inferenceMs": 915.6535420042928,
"tokensPerSecond": 14.197509651460566,
"charactersPerSecond": 38.224064446239986,
"inputTokens": 37,
"outputTokens": 13,
"outputLimitReached": false,
"placement": {
"cpu/torch.float32": 339
},
"streamChunks": 0,
"loadMs": 10188.248666992877,
"totalMs": 11103.90220899717,
"grade": "good",
"requestedDevice": "cpu",
"device": "cpu",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "greeting-context",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 1661.933666007826,
"inferenceMs": 1659.9813330103643,
"tokensPerSecond": 27.711154990266866,
"charactersPerSecond": 92.7721275761108,
"inputTokens": 157,
"outputTokens": 46,
"outputLimitReached": false,
"placement": {
"mps:0/torch.float32": 339
},
"streamChunks": 0,
"driverAllocatedMemoryBytes": 6556860416,
"loadMs": 9820.342332997825,
"totalMs": 11480.32366600819,
"grade": "good",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "rename-parameter",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 465.95145799801685,
"inferenceMs": 464.808874996379,
"tokensPerSecond": 30.11990681139439,
"charactersPerSecond": 103.26825192478077,
"inputTokens": 38,
"outputTokens": 14,
"outputLimitReached": false,
"placement": {
"mps:0/torch.float32": 339
},
"streamChunks": 0,
"driverAllocatedMemoryBytes": 6557057024,
"loadMs": 9820.342332997825,
"totalMs": 10285.151207994204,
"grade": "good",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "no-edit",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 422.9563330009114,
"inferenceMs": 421.64291700464673,
"tokensPerSecond": 30.83177607334676,
"charactersPerSecond": 83.00862788977973,
"inputTokens": 37,
"outputTokens": 13,
"outputLimitReached": false,
"placement": {
"mps:0/torch.float32": 339
},
"streamChunks": 0,
"driverAllocatedMemoryBytes": 6557253632,
"loadMs": 9820.342332997825,
"totalMs": 10241.985250002472,
"grade": "good",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "greeting-context",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 1422.132916020928,
"inferenceMs": 1420.3504170000087,
"tokensPerSecond": 32.386374129532655,
"charactersPerSecond": 108.42394817278323,
"inputTokens": 157,
"outputTokens": 46,
"outputLimitReached": false,
"placement": {
"mps:0/torch.float32": 339
},
"streamChunks": 0,
"driverAllocatedMemoryBytes": 6556860416,
"loadMs": 9754.36108399299,
"totalMs": 11174.711500992998,
"grade": "good",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "rename-parameter",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 445.175915985601,
"inferenceMs": 444.1577499965206,
"tokensPerSecond": 31.520332584784736,
"charactersPerSecond": 108.06971171926195,
"inputTokens": 38,
"outputTokens": 14,
"outputLimitReached": false,
"placement": {
"mps:0/torch.float32": 339
},
"streamChunks": 0,
"driverAllocatedMemoryBytes": 6557057024,
"loadMs": 9754.36108399299,
"totalMs": 10198.51883398951,
"grade": "good",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "no-edit",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 415.0151670037303,
"inferenceMs": 414.0651659981813,
"tokensPerSecond": 31.39602426748716,
"charactersPerSecond": 84.52775764323466,
"inputTokens": 37,
"outputTokens": 13,
"outputLimitReached": false,
"placement": {
"mps:0/torch.float32": 339
},
"streamChunks": 0,
"driverAllocatedMemoryBytes": 6557253632,
"loadMs": 9754.36108399299,
"totalMs": 10168.426249991171,
"grade": "good",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "greeting-context",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 1422.8192919981666,
"inferenceMs": 1420.8101670083124,
"tokensPerSecond": 32.37589444961431,
"charactersPerSecond": 108.38886402696964,
"inputTokens": 157,
"outputTokens": 46,
"outputLimitReached": false,
"placement": {
"mps:0/torch.float32": 339
},
"streamChunks": 0,
"driverAllocatedMemoryBytes": 6556860416,
"loadMs": 9699.818583991146,
"totalMs": 11120.628750999458,
"grade": "good",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "rename-parameter",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 446.84770799358375,
"inferenceMs": 445.84033300634474,
"tokensPerSecond": 31.40137615095664,
"charactersPerSecond": 107.6618610889942,
"inputTokens": 38,
"outputTokens": 14,
"outputLimitReached": false,
"placement": {
"mps:0/torch.float32": 339
},
"streamChunks": 0,
"driverAllocatedMemoryBytes": 6557057024,
"loadMs": 9699.818583991146,
"totalMs": 10145.65891699749,
"grade": "good",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "no-edit",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 415.04595801234245,
"inferenceMs": 414.0313329990022,
"tokensPerSecond": 31.398589826126347,
"charactersPerSecond": 84.53466491649401,
"inputTokens": 37,
"outputTokens": 13,
"outputLimitReached": false,
"placement": {
"mps:0/torch.float32": 339
},
"streamChunks": 0,
"driverAllocatedMemoryBytes": 6557253632,
"loadMs": 9699.818583991146,
"totalMs": 10113.849916990148,
"grade": "good",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp32",
"precision": "fp32",
"cpuThreads": 4,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "greeting-context",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 880.0392920093145,
"inferenceMs": 878.4852079988923,
"tokensPerSecond": 52.36286232386738,
"charactersPerSecond": 175.30175647555603,
"inputTokens": 157,
"outputTokens": 46,
"outputLimitReached": false,
"placement": {
"mps:0/torch.float16": 339
},
"streamChunks": 0,
"driverAllocatedMemoryBytes": 3327246336,
"loadMs": 9436.231500003487,
"totalMs": 10314.71670800238,
"grade": "good",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "rename-parameter",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 286.43641600501724,
"inferenceMs": 285.06245798780583,
"tokensPerSecond": 49.11204407210605,
"charactersPerSecond": 168.3841511043636,
"inputTokens": 38,
"outputTokens": 14,
"outputLimitReached": false,
"placement": {
"mps:0/torch.float16": 339
},
"streamChunks": 0,
"driverAllocatedMemoryBytes": 3327442944,
"loadMs": 9436.231500003487,
"totalMs": 9721.293957991293,
"grade": "good",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "no-edit",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 263.98049999261275,
"inferenceMs": 263.03425000514835,
"tokensPerSecond": 49.42322149965471,
"charactersPerSecond": 133.0625194221473,
"inputTokens": 37,
"outputTokens": 13,
"outputLimitReached": false,
"placement": {
"mps:0/torch.float16": 339
},
"streamChunks": 0,
"driverAllocatedMemoryBytes": 3327639552,
"loadMs": 9436.231500003487,
"totalMs": 9699.265750008635,
"grade": "good",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "greeting-context",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 878.4792499791365,
"inferenceMs": 876.4932499907445,
"tokensPerSecond": 52.481864521473206,
"charactersPerSecond": 175.70015513710595,
"inputTokens": 157,
"outputTokens": 46,
"outputLimitReached": false,
"placement": {
"mps:0/torch.float16": 339
},
"streamChunks": 0,
"driverAllocatedMemoryBytes": 3327246336,
"loadMs": 9406.246041005943,
"totalMs": 10282.739290996687,
"grade": "good",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "rename-parameter",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 283.5205409792252,
"inferenceMs": 282.61237501283176,
"tokensPerSecond": 49.537816591946275,
"charactersPerSecond": 169.84394260095866,
"inputTokens": 38,
"outputTokens": 14,
"outputLimitReached": false,
"placement": {
"mps:0/torch.float16": 339
},
"streamChunks": 0,
"driverAllocatedMemoryBytes": 3327442944,
"loadMs": 9406.246041005943,
"totalMs": 9688.858416018775,
"grade": "good",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "no-edit",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 263.06504098465666,
"inferenceMs": 262.262499978533,
"tokensPerSecond": 49.56865736071337,
"charactersPerSecond": 133.45407750961292,
"inputTokens": 37,
"outputTokens": 13,
"outputLimitReached": false,
"placement": {
"mps:0/torch.float16": 339
},
"streamChunks": 0,
"driverAllocatedMemoryBytes": 3327639552,
"loadMs": 9406.246041005943,
"totalMs": 9668.508540984476,
"grade": "good",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "greeting-context",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 875.8251669933088,
"inferenceMs": 874.2318339936901,
"tokensPerSecond": 52.617621792450095,
"charactersPerSecond": 176.1546468703764,
"inputTokens": 157,
"outputTokens": 46,
"outputLimitReached": false,
"placement": {
"mps:0/torch.float16": 339
},
"streamChunks": 0,
"driverAllocatedMemoryBytes": 3327246336,
"loadMs": 9417.092125018826,
"totalMs": 10291.323959012516,
"grade": "good",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "rename-parameter",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 285.6733750086278,
"inferenceMs": 284.3458329734858,
"tokensPerSecond": 49.23581912067425,
"charactersPerSecond": 168.80852269945458,
"inputTokens": 38,
"outputTokens": 14,
"outputLimitReached": false,
"placement": {
"mps:0/torch.float16": 339
},
"streamChunks": 0,
"driverAllocatedMemoryBytes": 3327442944,
"loadMs": 9417.092125018826,
"totalMs": 9701.437957992312,
"grade": "good",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"nonempty": true,
"repeatedLineRatio": 0,
"fixtureId": "no-edit",
"pythonParses": true,
"expectedEdit": true,
"passed": true,
"speedPassed": true
},
"runtime": {
"durationMs": 264.51366700348444,
"inferenceMs": 263.4720829955768,
"tokensPerSecond": 49.34109091253606,
"charactersPerSecond": 132.841398610674,
"inputTokens": 37,
"outputTokens": 13,
"outputLimitReached": false,
"placement": {
"mps:0/torch.float16": 339
},
"streamChunks": 0,
"driverAllocatedMemoryBytes": 3327639552,
"loadMs": 9417.092125018826,
"totalMs": 9680.564208014403,
"grade": "good",
"requestedDevice": "mps",
"device": "mps",
"requestedPrecision": "fp16",
"precision": "fp16",
"cpuThreads": 4,
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false,
"artifactQuantization": "Q8_0 dequantized to runtime dtype"
}
}
]Release validation still required
- Public sample publication, complete dependency backup and forced model/runtime-source recovery excluded by user; benchmark completion is not release certification.