A model is still running

A model is still running or downloading. Leaving will stop the current operation. Stay on this page to let it finish.

Skip to content
LocalAI.toolsBeta

Join the waitlist

Get an email when the new apps and model runs launch.

No account needed.

Back to models

Sweep Next Edit Desktop

Suggest a complete next version of a code file using repository context and recent edits. Runs the original Q8_0 GGUF through native Transformers.

Desktop inferencePrivate · Runs on your computer
About this model

Suggest a complete next version of a code file using repository context and recent edits. Runs the original Q8_0 GGUF through native Transformers.

Created by
Sweep AI
License
Apache-2.0
Model released
Unknown
Source repository created
Jan 20, 2026
Model weights · FP16
1.5 GB

Best for

  • Local next-edit code suggestions requiring developer review.
  • Completing consistent small Python refactors and using related-file context.

Keep in mind

  • Predicts a likely next edit, not an instruction-following chat assistant, exact patch executor or no-edit/abstention API.
  • Three controlled short Python fixtures only; broad language/repository correctness, secure code and long-context quality unverified.
  • May rename additional symbols or propose equivalent rewrites; review every prediction before applying it.
  • Original GGUF Q8_0 is dequantized on CPU and cast to FP32/FP16 execution; this does not recover original unquantized training weights.
  • Current official1.5B v2 GGUF has a pruned43839token vocabulary. The existing browser artifact has a different vocabulary; bit-identical checkpoint/output equivalence is not claimed.
  • Text only; final messages content is treated as an already formatted Sweep prompt. No image/audio/video input.
  • Input plus output budget is at most8192tokens by adapter policy. Windows, Intel Mac and other macOS/hardware remain unverified.

Processed on your deviceYour input is processed locally in the desktop app. Model and runtime files download from external hosts.

Target platforms · verification required

  • macos · aarch64, x86_64
  • windows · x86_64

Recommended configuration

FP16 · mps (verified platform only)

Sources & technical details
Original model
sweepai/sweep-next-edit-1.5B
Release verification
Unknown
Release evidence
Exact original checkpoint public release day remains unknown after bounded original model/card/repository checks. v2 in the original GGUF filename is not treated as a new model generation. Repository creation/update and conversion dates are distinct.
Source updated
Jan 22, 2026
Native model files
sweepai/sweep-next-edit-1.5B
Files updated
Jan 22, 2026
Artifact revision
409016591c6c1a94f545f22328a85a3516118f34

Desktop app

Sample result

Sweep Next-Edit 1.5B Desktop

Run Sweep Next Edit Desktop on your computer

Download the desktop app to load this model, provide input and view results locally.

Public desktop downloads are coming soon.

Device requirements

macOS Apple SiliconCPU / FP32Tested16.0 GiBRAM estimate
macOS Apple SiliconMPS / FP16Tested16.0 GiBRAM estimate
macOS Apple SiliconMPS / FP32Tested16.0 GiBRAM estimate
Windows Intel / AMDCPU / FP32Unverified16.0 GiBRAM estimate
macOS Intel / AMDCPU / FP32Unverified16.0 GiBRAM estimate
If your device falls short

MemoryAdvisory only: close other apps, reduce image/tile/token/batch settings or choose a smaller verified model/precision. CPU can need more RAM than GPU.

GPUIf the selected GPU runtime fails, existing CPU fallback may recover when enabled, but CPU has its own memory/runtime needs.

PlatformUse a compatible OS/architecture/runtime or another model. These estimates do not add hardware detection or block runs.

Benchmark evidence

Saved desktop measurements apply to the recorded hardware and runtime. Fallback cases report the device that actually ran; they do not prove GPU support on the requested device.

Sweep Next-Edit 1.5B Desktop

36 configurations tested
Passed36
Failed0
TestedNov 12, 2026
Runtimetransformers 5.22.1

Quality criteria

{ "file": "research/sweep-next-edit-family-expansion/quality-gates.json", "sha256": "13d2464d0ec31b1ab766319b0b38fe938271a1ec206a46c2436bf819fa27537b", "description": "Actual helper call inside greet; valid complete Python; static integer-addition/square semantics and prompt retention. No code executed. All3fixtures, <=15s/inference and>=5tokens/s." }

PrecisionDeviceResultPlatformInference / load
FP16mpsAcceleratedPassedFastmacos · aarch64874 ms / 9417 ms
FP16mpsAcceleratedPassedFastmacos · aarch64876 ms / 9406 ms
FP16mpsAcceleratedPassedFastmacos · aarch64878 ms / 9436 ms
FP16mpsAcceleratedPassedFastmacos · aarch64262 ms / 9406 ms
FP16mpsAcceleratedPassedFastmacos · aarch64283 ms / 9406 ms
FP16mpsAcceleratedPassedFastmacos · aarch64263 ms / 9436 ms
FP16mpsAcceleratedPassedFastmacos · aarch64263 ms / 9417 ms
FP16mpsAcceleratedPassedFastmacos · aarch64284 ms / 9417 ms
FP16mpsAcceleratedPassedFastmacos · aarch64285 ms / 9436 ms
FP32mpsAcceleratedPassedFastmacos · aarch641420 ms / 9754 ms
FP32mpsAcceleratedPassedFastmacos · aarch641421 ms / 9700 ms
FP32mpsAcceleratedPassedFastmacos · aarch64444 ms / 9754 ms
FP32mpsAcceleratedPassedFastmacos · aarch64446 ms / 9700 ms
FP32mpsAcceleratedPassedFastmacos · aarch64414 ms / 9700 ms
FP32mpsAcceleratedPassedFastmacos · aarch64414 ms / 9754 ms
FP32mpsAcceleratedPassedFastmacos · aarch64422 ms / 9820 ms
FP32mpsAcceleratedPassedFastmacos · aarch64465 ms / 9820 ms
FP32mpsAcceleratedPassedFastmacos · aarch641660 ms / 9820 ms
FP32cpuPassedFastmacos · aarch64947 ms / 10117 ms
FP32cpuPassedFastmacos · aarch64953 ms / 10105 ms
FP32cpuPassedFastmacos · aarch64889 ms / 10117 ms
FP32cpuPassedFastmacos · aarch64962 ms / 9895 ms
FP32cpuPassedFastmacos · aarch64897 ms / 10698 ms
FP32cpuPassedFastmacos · aarch64975 ms / 10188 ms
FP32cpuPassedFastmacos · aarch64911 ms / 9895 ms
FP32cpuPassedFastmacos · aarch64916 ms / 10188 ms
FP32cpuPassedFastmacos · aarch64986 ms / 9950 ms
FP32cpuPassedFastmacos · aarch64932 ms / 9950 ms
FP32cpuPassedFastmacos · aarch641004 ms / 10698 ms
FP32cpuPassedFastmacos · aarch643335 ms / 10188 ms
FP32cpuPassedFastmacos · aarch643357 ms / 10105 ms
FP32cpuPassedFastmacos · aarch64951 ms / 10105 ms
FP32cpuPassedFastmacos · aarch643441 ms / 10698 ms
FP32cpuPassedFastmacos · aarch643477 ms / 9950 ms
FP32cpuPassedFastmacos · aarch643481 ms / 10117 ms
FP32cpuPassedFastmacos · aarch643493 ms / 9895 ms
Test setup & sample details
Quality gate
{ "file": "research/sweep-next-edit-family-expansion/quality-gates.json", "sha256": "13d2464d0ec31b1ab766319b0b38fe938271a1ec206a46c2436bf819fa27537b", "description": "Actual helper call inside greet; valid complete Python; static integer-addition/square semantics and prompt retention. No code executed. All3fixtures, <=15s/inference and>=5tokens/s." }
Tested
2026-11-12T20:58:49.500905+00:00
Environment
{ "os": "macos", "osVersion": "27.2", "architecture": "aarch64", "hardware": "Apple M3 Max;Mac15,11;96GiB RAM", "cpu": "Apple M3 Max", "gpu": "Apple M3 Max", "driver": "OS-managed Metal/MPS macOS27.2 build26B118", "runtime": "App-owned CPython3.13.18;Transformers5.22.1;Torch2.10.0;GGUF0.19.0", "timingScope": "Fresh helper/model process per cold repetition. GGUF import/dequantization and tokenizer/runtime startup in load; OS files/driver caches may be warm. Warm repetition after three task fixtures. Snapshot-only transfer, no Internet timing.", "concurrentUploadActivity": "none; user excluded uploads" }
Runtime
transformers 5.22.1
Artifact revision
409016591c6c1a94f545f22328a85a3516118f34
Fixture
{
  "file": "model-samples/native/sweep-next-edit/quality-gates.json",
  "url": "https://assets.localai.tools/samples/v1/files/13d2464d0ec31b1ab766319b0b38fe938271a1ec206a46c2436bf819fa27537b/sweep-next-edit-quality-gates.json",
  "size": 3408,
  "sha1": "25e62103b91e31ae2320aab7379dd8a04af06b40",
  "sha256": "13d2464d0ec31b1ab766319b0b38fe938271a1ec206a46c2436bf819fa27537b"
}
Rerun when
artifact-revision; runtime-version; fixture-checksum; quality-gate; backend-behavior
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "greeting-context",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 3481.3466669875197,
      "inferenceMs": 3480.6580410222523,
      "tokensPerSecond": 13.215891781914326,
      "charactersPerSecond": 44.24450726988709,
      "inputTokens": 157,
      "outputTokens": 46,
      "outputLimitReached": false,
      "placement": {
        "cpu/torch.float32": 339
      },
      "streamChunks": 0,
      "loadMs": 10117.09308301215,
      "totalMs": 13597.751124034403,
      "grade": "good",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "rename-parameter",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 947.6490419765469,
      "inferenceMs": 947.4046250106767,
      "tokensPerSecond": 14.777213062309283,
      "charactersPerSecond": 50.66473049934611,
      "inputTokens": 38,
      "outputTokens": 14,
      "outputLimitReached": false,
      "placement": {
        "cpu/torch.float32": 339
      },
      "streamChunks": 0,
      "loadMs": 10117.09308301215,
      "totalMs": 11064.497708022827,
      "grade": "good",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "no-edit",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 889.0892080089543,
      "inferenceMs": 888.8817499973811,
      "tokensPerSecond": 14.625117458017673,
      "charactersPerSecond": 39.37531623312451,
      "inputTokens": 37,
      "outputTokens": 13,
      "outputLimitReached": false,
      "placement": {
        "cpu/torch.float32": 339
      },
      "streamChunks": 0,
      "loadMs": 10117.09308301215,
      "totalMs": 11005.974833009532,
      "grade": "good",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "greeting-context",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 3357.5666669930797,
      "inferenceMs": 3356.8639580043964,
      "tokensPerSecond": 13.703266076754057,
      "charactersPerSecond": 45.87615164826358,
      "inputTokens": 157,
      "outputTokens": 46,
      "outputLimitReached": false,
      "placement": {
        "cpu/torch.float32": 339
      },
      "streamChunks": 0,
      "loadMs": 10105.382750014542,
      "totalMs": 13462.246708018938,
      "grade": "good",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "rename-parameter",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 953.6057500226889,
      "inferenceMs": 953.3635829866398,
      "tokensPerSecond": 14.684848729108833,
      "charactersPerSecond": 50.348052785515996,
      "inputTokens": 38,
      "outputTokens": 14,
      "outputLimitReached": false,
      "placement": {
        "cpu/torch.float32": 339
      },
      "streamChunks": 0,
      "loadMs": 10105.382750014542,
      "totalMs": 11058.746333001181,
      "grade": "good",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "no-edit",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 950.8677079866175,
      "inferenceMs": 950.6412080081645,
      "tokensPerSecond": 13.674980518926073,
      "charactersPerSecond": 36.81725524326251,
      "inputTokens": 37,
      "outputTokens": 13,
      "outputLimitReached": false,
      "placement": {
        "cpu/torch.float32": 339
      },
      "streamChunks": 0,
      "loadMs": 10105.382750014542,
      "totalMs": 11056.023958022706,
      "grade": "good",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "greeting-context",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 3477.5677919969894,
      "inferenceMs": 3476.822833006736,
      "tokensPerSecond": 13.230469946097159,
      "charactersPerSecond": 44.293312428238316,
      "inputTokens": 157,
      "outputTokens": 46,
      "outputLimitReached": false,
      "placement": {
        "cpu/torch.float32": 339
      },
      "streamChunks": 0,
      "loadMs": 9950.387083023088,
      "totalMs": 13427.209916029824,
      "grade": "good",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "rename-parameter",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 986.3551669986919,
      "inferenceMs": 986.0953750030603,
      "tokensPerSecond": 14.197409657211455,
      "charactersPerSecond": 48.676833110439276,
      "inputTokens": 38,
      "outputTokens": 14,
      "outputLimitReached": false,
      "placement": {
        "cpu/torch.float32": 339
      },
      "streamChunks": 0,
      "loadMs": 9950.387083023088,
      "totalMs": 10936.482458026148,
      "grade": "good",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "no-edit",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 931.7344170121942,
      "inferenceMs": 931.5242079901509,
      "tokensPerSecond": 13.955622289246454,
      "charactersPerSecond": 37.572829240278914,
      "inputTokens": 37,
      "outputTokens": 13,
      "outputLimitReached": false,
      "placement": {
        "cpu/torch.float32": 339
      },
      "streamChunks": 0,
      "loadMs": 9950.387083023088,
      "totalMs": 10881.911291013239,
      "grade": "good",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 1,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "greeting-context",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 3493.264749995433,
      "inferenceMs": 3492.5227500207257,
      "tokensPerSecond": 13.170995092222956,
      "charactersPerSecond": 44.09420096092033,
      "inputTokens": 157,
      "outputTokens": 46,
      "outputLimitReached": false,
      "placement": {
        "cpu/torch.float32": 339
      },
      "streamChunks": 0,
      "loadMs": 9895.09375000489,
      "totalMs": 13387.616500025615,
      "grade": "good",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "rename-parameter",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 962.456708017271,
      "inferenceMs": 962.2079170076177,
      "tokensPerSecond": 14.549869890427397,
      "charactersPerSecond": 49.885268195751074,
      "inputTokens": 38,
      "outputTokens": 14,
      "outputLimitReached": false,
      "placement": {
        "cpu/torch.float32": 339
      },
      "streamChunks": 0,
      "loadMs": 9895.09375000489,
      "totalMs": 10857.301667012507,
      "grade": "good",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "no-edit",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 910.990500007756,
      "inferenceMs": 910.7842499797698,
      "tokensPerSecond": 14.273413270254458,
      "charactersPerSecond": 38.42842034299277,
      "inputTokens": 37,
      "outputTokens": 13,
      "outputLimitReached": false,
      "placement": {
        "cpu/torch.float32": 339
      },
      "streamChunks": 0,
      "loadMs": 9895.09375000489,
      "totalMs": 10805.87799998466,
      "grade": "good",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "greeting-context",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 3441.7680419865064,
      "inferenceMs": 3441.045875020791,
      "tokensPerSecond": 13.368028695555262,
      "charactersPerSecond": 44.75383519816327,
      "inputTokens": 157,
      "outputTokens": 46,
      "outputLimitReached": false,
      "placement": {
        "cpu/torch.float32": 339
      },
      "streamChunks": 0,
      "loadMs": 10697.612792020664,
      "totalMs": 14138.658667041454,
      "grade": "good",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "rename-parameter",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 1004.4087499845773,
      "inferenceMs": 1004.1581670229789,
      "tokensPerSecond": 13.942026724241767,
      "charactersPerSecond": 47.801234483114634,
      "inputTokens": 38,
      "outputTokens": 14,
      "outputLimitReached": false,
      "placement": {
        "cpu/torch.float32": 339
      },
      "streamChunks": 0,
      "loadMs": 10697.612792020664,
      "totalMs": 11701.770959043643,
      "grade": "good",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "no-edit",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 897.2117500088643,
      "inferenceMs": 896.9800000195391,
      "tokensPerSecond": 14.493076768397085,
      "charactersPerSecond": 39.019822068761385,
      "inputTokens": 37,
      "outputTokens": 13,
      "outputLimitReached": false,
      "placement": {
        "cpu/torch.float32": 339
      },
      "streamChunks": 0,
      "loadMs": 10697.612792020664,
      "totalMs": 11594.592792040203,
      "grade": "good",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "greeting-context",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 3335.722916992381,
      "inferenceMs": 3335.014540993143,
      "tokensPerSecond": 13.793043308981057,
      "charactersPerSecond": 46.17671020832788,
      "inputTokens": 157,
      "outputTokens": 46,
      "outputLimitReached": false,
      "placement": {
        "cpu/torch.float32": 339
      },
      "streamChunks": 0,
      "loadMs": 10188.248666992877,
      "totalMs": 13523.26320798602,
      "grade": "good",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "rename-parameter",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 975.3110409947112,
      "inferenceMs": 975.069459003862,
      "tokensPerSecond": 14.357951498452737,
      "charactersPerSecond": 49.227262280409384,
      "inputTokens": 38,
      "outputTokens": 14,
      "outputLimitReached": false,
      "placement": {
        "cpu/torch.float32": 339
      },
      "streamChunks": 0,
      "loadMs": 10188.248666992877,
      "totalMs": 11163.318125996739,
      "grade": "good",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "no-edit",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 915.8662499976344,
      "inferenceMs": 915.6535420042928,
      "tokensPerSecond": 14.197509651460566,
      "charactersPerSecond": 38.224064446239986,
      "inputTokens": 37,
      "outputTokens": 13,
      "outputLimitReached": false,
      "placement": {
        "cpu/torch.float32": 339
      },
      "streamChunks": 0,
      "loadMs": 10188.248666992877,
      "totalMs": 11103.90220899717,
      "grade": "good",
      "requestedDevice": "cpu",
      "device": "cpu",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "greeting-context",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 1661.933666007826,
      "inferenceMs": 1659.9813330103643,
      "tokensPerSecond": 27.711154990266866,
      "charactersPerSecond": 92.7721275761108,
      "inputTokens": 157,
      "outputTokens": 46,
      "outputLimitReached": false,
      "placement": {
        "mps:0/torch.float32": 339
      },
      "streamChunks": 0,
      "driverAllocatedMemoryBytes": 6556860416,
      "loadMs": 9820.342332997825,
      "totalMs": 11480.32366600819,
      "grade": "good",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "rename-parameter",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 465.95145799801685,
      "inferenceMs": 464.808874996379,
      "tokensPerSecond": 30.11990681139439,
      "charactersPerSecond": 103.26825192478077,
      "inputTokens": 38,
      "outputTokens": 14,
      "outputLimitReached": false,
      "placement": {
        "mps:0/torch.float32": 339
      },
      "streamChunks": 0,
      "driverAllocatedMemoryBytes": 6557057024,
      "loadMs": 9820.342332997825,
      "totalMs": 10285.151207994204,
      "grade": "good",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "no-edit",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 422.9563330009114,
      "inferenceMs": 421.64291700464673,
      "tokensPerSecond": 30.83177607334676,
      "charactersPerSecond": 83.00862788977973,
      "inputTokens": 37,
      "outputTokens": 13,
      "outputLimitReached": false,
      "placement": {
        "mps:0/torch.float32": 339
      },
      "streamChunks": 0,
      "driverAllocatedMemoryBytes": 6557253632,
      "loadMs": 9820.342332997825,
      "totalMs": 10241.985250002472,
      "grade": "good",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "greeting-context",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 1422.132916020928,
      "inferenceMs": 1420.3504170000087,
      "tokensPerSecond": 32.386374129532655,
      "charactersPerSecond": 108.42394817278323,
      "inputTokens": 157,
      "outputTokens": 46,
      "outputLimitReached": false,
      "placement": {
        "mps:0/torch.float32": 339
      },
      "streamChunks": 0,
      "driverAllocatedMemoryBytes": 6556860416,
      "loadMs": 9754.36108399299,
      "totalMs": 11174.711500992998,
      "grade": "good",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "rename-parameter",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 445.175915985601,
      "inferenceMs": 444.1577499965206,
      "tokensPerSecond": 31.520332584784736,
      "charactersPerSecond": 108.06971171926195,
      "inputTokens": 38,
      "outputTokens": 14,
      "outputLimitReached": false,
      "placement": {
        "mps:0/torch.float32": 339
      },
      "streamChunks": 0,
      "driverAllocatedMemoryBytes": 6557057024,
      "loadMs": 9754.36108399299,
      "totalMs": 10198.51883398951,
      "grade": "good",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "no-edit",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 415.0151670037303,
      "inferenceMs": 414.0651659981813,
      "tokensPerSecond": 31.39602426748716,
      "charactersPerSecond": 84.52775764323466,
      "inputTokens": 37,
      "outputTokens": 13,
      "outputLimitReached": false,
      "placement": {
        "mps:0/torch.float32": 339
      },
      "streamChunks": 0,
      "driverAllocatedMemoryBytes": 6557253632,
      "loadMs": 9754.36108399299,
      "totalMs": 10168.426249991171,
      "grade": "good",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "greeting-context",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 1422.8192919981666,
      "inferenceMs": 1420.8101670083124,
      "tokensPerSecond": 32.37589444961431,
      "charactersPerSecond": 108.38886402696964,
      "inputTokens": 157,
      "outputTokens": 46,
      "outputLimitReached": false,
      "placement": {
        "mps:0/torch.float32": 339
      },
      "streamChunks": 0,
      "driverAllocatedMemoryBytes": 6556860416,
      "loadMs": 9699.818583991146,
      "totalMs": 11120.628750999458,
      "grade": "good",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "rename-parameter",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 446.84770799358375,
      "inferenceMs": 445.84033300634474,
      "tokensPerSecond": 31.40137615095664,
      "charactersPerSecond": 107.6618610889942,
      "inputTokens": 38,
      "outputTokens": 14,
      "outputLimitReached": false,
      "placement": {
        "mps:0/torch.float32": 339
      },
      "streamChunks": 0,
      "driverAllocatedMemoryBytes": 6557057024,
      "loadMs": 9699.818583991146,
      "totalMs": 10145.65891699749,
      "grade": "good",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "no-edit",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 415.04595801234245,
      "inferenceMs": 414.0313329990022,
      "tokensPerSecond": 31.398589826126347,
      "charactersPerSecond": 84.53466491649401,
      "inputTokens": 37,
      "outputTokens": 13,
      "outputLimitReached": false,
      "placement": {
        "mps:0/torch.float32": 339
      },
      "streamChunks": 0,
      "driverAllocatedMemoryBytes": 6557253632,
      "loadMs": 9699.818583991146,
      "totalMs": 10113.849916990148,
      "grade": "good",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp32",
      "precision": "fp32",
      "cpuThreads": 4,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "greeting-context",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 880.0392920093145,
      "inferenceMs": 878.4852079988923,
      "tokensPerSecond": 52.36286232386738,
      "charactersPerSecond": 175.30175647555603,
      "inputTokens": 157,
      "outputTokens": 46,
      "outputLimitReached": false,
      "placement": {
        "mps:0/torch.float16": 339
      },
      "streamChunks": 0,
      "driverAllocatedMemoryBytes": 3327246336,
      "loadMs": 9436.231500003487,
      "totalMs": 10314.71670800238,
      "grade": "good",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "rename-parameter",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 286.43641600501724,
      "inferenceMs": 285.06245798780583,
      "tokensPerSecond": 49.11204407210605,
      "charactersPerSecond": 168.3841511043636,
      "inputTokens": 38,
      "outputTokens": 14,
      "outputLimitReached": false,
      "placement": {
        "mps:0/torch.float16": 339
      },
      "streamChunks": 0,
      "driverAllocatedMemoryBytes": 3327442944,
      "loadMs": 9436.231500003487,
      "totalMs": 9721.293957991293,
      "grade": "good",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "no-edit",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 263.98049999261275,
      "inferenceMs": 263.03425000514835,
      "tokensPerSecond": 49.42322149965471,
      "charactersPerSecond": 133.0625194221473,
      "inputTokens": 37,
      "outputTokens": 13,
      "outputLimitReached": false,
      "placement": {
        "mps:0/torch.float16": 339
      },
      "streamChunks": 0,
      "driverAllocatedMemoryBytes": 3327639552,
      "loadMs": 9436.231500003487,
      "totalMs": 9699.265750008635,
      "grade": "good",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "greeting-context",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 878.4792499791365,
      "inferenceMs": 876.4932499907445,
      "tokensPerSecond": 52.481864521473206,
      "charactersPerSecond": 175.70015513710595,
      "inputTokens": 157,
      "outputTokens": 46,
      "outputLimitReached": false,
      "placement": {
        "mps:0/torch.float16": 339
      },
      "streamChunks": 0,
      "driverAllocatedMemoryBytes": 3327246336,
      "loadMs": 9406.246041005943,
      "totalMs": 10282.739290996687,
      "grade": "good",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "rename-parameter",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 283.5205409792252,
      "inferenceMs": 282.61237501283176,
      "tokensPerSecond": 49.537816591946275,
      "charactersPerSecond": 169.84394260095866,
      "inputTokens": 38,
      "outputTokens": 14,
      "outputLimitReached": false,
      "placement": {
        "mps:0/torch.float16": 339
      },
      "streamChunks": 0,
      "driverAllocatedMemoryBytes": 3327442944,
      "loadMs": 9406.246041005943,
      "totalMs": 9688.858416018775,
      "grade": "good",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "no-edit",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 263.06504098465666,
      "inferenceMs": 262.262499978533,
      "tokensPerSecond": 49.56865736071337,
      "charactersPerSecond": 133.45407750961292,
      "inputTokens": 37,
      "outputTokens": 13,
      "outputLimitReached": false,
      "placement": {
        "mps:0/torch.float16": 339
      },
      "streamChunks": 0,
      "driverAllocatedMemoryBytes": 3327639552,
      "loadMs": 9406.246041005943,
      "totalMs": 9668.508540984476,
      "grade": "good",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "greeting-context",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 875.8251669933088,
      "inferenceMs": 874.2318339936901,
      "tokensPerSecond": 52.617621792450095,
      "charactersPerSecond": 176.1546468703764,
      "inputTokens": 157,
      "outputTokens": 46,
      "outputLimitReached": false,
      "placement": {
        "mps:0/torch.float16": 339
      },
      "streamChunks": 0,
      "driverAllocatedMemoryBytes": 3327246336,
      "loadMs": 9417.092125018826,
      "totalMs": 10291.323959012516,
      "grade": "good",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "rename-parameter",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 285.6733750086278,
      "inferenceMs": 284.3458329734858,
      "tokensPerSecond": 49.23581912067425,
      "charactersPerSecond": 168.80852269945458,
      "inputTokens": 38,
      "outputTokens": 14,
      "outputLimitReached": false,
      "placement": {
        "mps:0/torch.float16": 339
      },
      "streamChunks": 0,
      "driverAllocatedMemoryBytes": 3327442944,
      "loadMs": 9417.092125018826,
      "totalMs": 9701.437957992312,
      "grade": "good",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "nonempty": true,
      "repeatedLineRatio": 0,
      "fixtureId": "no-edit",
      "pythonParses": true,
      "expectedEdit": true,
      "passed": true,
      "speedPassed": true
    },
    "runtime": {
      "durationMs": 264.51366700348444,
      "inferenceMs": 263.4720829955768,
      "tokensPerSecond": 49.34109091253606,
      "charactersPerSecond": 132.841398610674,
      "inputTokens": 37,
      "outputTokens": 13,
      "outputLimitReached": false,
      "placement": {
        "mps:0/torch.float16": 339
      },
      "streamChunks": 0,
      "driverAllocatedMemoryBytes": 3327639552,
      "loadMs": 9417.092125018826,
      "totalMs": 9680.564208014403,
      "grade": "good",
      "requestedDevice": "mps",
      "device": "mps",
      "requestedPrecision": "fp16",
      "precision": "fp16",
      "cpuThreads": 4,
      "engine": "transformers",
      "version": "5.22.1",
      "artifactRevision": "409016591c6c1a94f545f22328a85a3516118f34",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false,
      "artifactQuantization": "Q8_0 dequantized to runtime dtype"
    }
  }
]
Release validation still required
  • Public sample publication, complete dependency backup and forced model/runtime-source recovery excluded by user; benchmark completion is not release certification.

Something not quite right?

Tell us what happened with Sweep Next Edit Desktop. Loading, results, or the page itself—we’re here for all of it.

Attachments Optional

Up to 3 files · 50 MB each

Browser, OS, CPU/GPU, memory, screen, storage, and network details are included automatically. Model inputs, results, and browsing history are not collected.