A model is still running

A model is still running or downloading. Leaving will stop the current operation. Stay on this page to let it finish.

Skip to content
LocalAI.toolsBeta

Join the waitlist

Get an email when the new apps and model runs launch.

No account needed.

Qwen3.5 Desktop

Run local text questions, still-image questions using Qwen3.5 2B. Answers can hallucinate or miss visual details.

Desktop inferencePrivate · Runs on your computer
About this model

Run local text questions, still-image questions using Qwen3.5 2B. Answers can hallucinate or miss visual details.

License
Apache-2.0
Model released
Mar 2, 2026
Source repository created
Feb 28, 2026
Model weights · FP16
4.5 GB

Best for

  • Local visual questions and descriptions
  • Exploring printed-text reading
  • Image-content assistance with human verification

Keep in mind

  • Can hallucinate objects, text, counts or relationships; limited3-fixture validation is not broad visual/OCR accuracy.
  • Large images/output budgets require substantial memory; CPU recovery can be slow and needs room for FP32 weights.
  • No content-safety classifier; outputs may be biased or unsafe. Do not use answers as medical, legal, identity or safety-critical determinations.
  • Pinned source weights are BF16; FP32/FP16 casting does not create additional source precision.
  • The app downloads its own pinned CPython and isolated packages on request; installed Python, user packages and Python/pip configuration are not used. Automatic model-package provisioning is verified on Apple Silicon Mac; other platform stacks remain unverified.
  • Validation covers three vision fixtures on Apple Silicon with MPS FP16 and CPU FP32.
  • Video and sampled video-frame inputs are not supported. Use a still image instead.

Processed on your deviceYour input is processed locally in the desktop app. Model and runtime files download from external hosts.

Target platforms · verification required

  • macos · aarch64, x86_64
  • windows · x86_64

Recommended configuration

FP16 · mps (verified platform only)

Sources & technical details
Original model
https://huggingface.co/Qwen/Qwen3.5-2B
Release verification
Verified
Release evidence
Qwen3.5 small-series public release on March 2, 2026.
Source updated
Mar 2, 2026
Native model files
https://huggingface.co/Qwen/Qwen3.5-2B
Files updated
Mar 2, 2026
Artifact revision
15852e8c16360a2fea060d615a32b45270f8a8fc

Desktop app

Sample result

Qwen3.5 2B — Text, Image Desktop

Run Qwen3.5 Desktop on your computer

Download the desktop app to load this model, provide input and view results locally.

Public desktop downloads are coming soon.

Device requirements

macOS Apple SiliconCPU / FP32Tested24.0 GiBRAM estimate
macOS Apple SiliconMPS / FP16Tested24.0 GiBRAM estimate
Windows Intel / AMDCPU / FP32Unverified24.0 GiBRAM estimate
macOS Intel / AMDCPU / FP32Unverified24.0 GiBRAM estimate
If your device falls short

MemoryAdvisory only: close other apps, reduce image/tile/token/batch settings or choose a smaller verified model/precision. CPU can need more RAM than GPU.

GPUIf the selected GPU runtime fails, existing CPU fallback may recover when enabled, but CPU has its own memory/runtime needs.

PlatformUse a compatible OS/architecture/runtime or another model. These estimates do not add hardware detection or block runs.

Benchmark evidence

Saved desktop measurements apply to the recorded hardware and runtime. Fallback cases report the device that actually ran; they do not prove GPU support on the requested device.

Qwen3.5 2B — Text, Image Desktop

30 configurations tested
Passed30
Failed0
TestedNov 3, 2026
Runtimetransformers 5.22.1

Quality criteria

No explicit quality gate recorded. Review per-case evidence; full certification is pending.

PrecisionDeviceResultPlatformInference / load
FP16mpsAcceleratedPassedFastmacos · aarch64409 ms / 4710 ms
FP16mpsAcceleratedPassedModeratemacos · aarch642818 ms / 4829 ms
FP16mpsAcceleratedPassedSlowmacos · aarch641686 ms / 4710 ms
FP16mpsAcceleratedPassedSlowmacos · aarch641859 ms / 4710 ms
FP16mpsAcceleratedPassedSlowmacos · aarch642418 ms / 4829 ms
FP16mpsAcceleratedPassedSlowmacos · aarch642433 ms / 3457 ms
FP16mpsAcceleratedPassedSlowmacos · aarch642448 ms / 2781 ms
FP16mpsAcceleratedPassedSlowmacos · aarch642505 ms / 2781 ms
FP16mpsAcceleratedPassedSlowmacos · aarch642539 ms / 4829 ms
FP16mpsAcceleratedPassedSlowmacos · aarch642542 ms / 3457 ms
FP16mpsAcceleratedPassedSlowmacos · aarch642621 ms / 4829 ms
FP16mpsAcceleratedPassedSlowmacos · aarch642729 ms / 3457 ms
FP16mpsAcceleratedPassedSlowmacos · aarch642819 ms / 2781 ms
FP16mpsAcceleratedPassedSlowmacos · aarch643262 ms / 4710 ms
FP16mpsAcceleratedPassedSlowmacos · aarch643501 ms / 4829 ms
FP16mpsAcceleratedPassedSlowmacos · aarch643630 ms / 4710 ms
FP16mpsAcceleratedPassedSlowmacos · aarch643990 ms / 4710 ms
FP32cpuPassedSlowmacos · aarch6410918 ms / 2248 ms
FP32cpuPassedSlowmacos · aarch6411008 ms / 2248 ms
FP32cpuPassedSlowmacos · aarch6411058 ms / 2248 ms
FP32cpuPassedSlowmacos · aarch6411372 ms / 2248 ms
FP32cpuPassedSlowmacos · aarch6411869 ms / 2515 ms
FP32cpuPassedSlowmacos · aarch6413672 ms / 2515 ms
FP32cpuPassedSlowmacos · aarch6421680 ms / 2248 ms
FP32cpuPassedVery slowmacos · aarch648118 ms / 4161 ms
FP32cpuPassedVery slowmacos · aarch6411686 ms / 4161 ms
FP32cpuPassedVery slowmacos · aarch6412184 ms / 4161 ms
FP32cpuPassedVery slowmacos · aarch6414365 ms / 4161 ms
FP32cpuPassedVery slowmacos · aarch6416174 ms / 4161 ms
FP32cpuPassedVery slowmacos · aarch6423331 ms / 4161 ms
Test setup & sample details
Quality gate
No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
Tested
2026-11-03T12:24:47Z
Environment
{ "platform": "macOS-27.2-arm64-arm-64bit-Mach-O", "hardware": "Apple M3 Max" }
Runtime
transformers 5.22.1
Artifact revision
15852e8c16360a2fea060d615a32b45270f8a8fc
Fixture
{
  "file": "model-samples/desktop/mobilenet-dog.jpg",
  "size": 83281,
  "sha1": "f1d042827c2f2154a6d8043b8e8f07c1be612941",
  "sha256": "d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1",
  "url": "https://assets.localai.tools/samples/v1/files/d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1/mobilenet-dog.jpg"
}
Rerun when
model revision/checksum; pinned runtime/processor/attention/backend/driver; fixture checksum/question/expected answers; visual pixel and token budgets; quality gates
Per-case quality & failure details
[
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "accuracy": 1,
      "repetitive": false,
      "expected": [
        "dog",
        "labrador",
        "puppy",
        "canine"
      ],
      "method": "contains an accepted animal label; nonempty, nonrepetitive answer required. Limited3-fixture compatibility, not broad VQA/OCR accuracy."
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4829.139582987409,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 617
      },
      "parameterDtype": "torch.float16",
      "attentionImplementation": "sdpa",
      "accelerationDetails": "Model parameters and input tensors on selected device; image decoding/processing and text tokenization/decoding on CPU",
      "durationMs": 3540.975958982017,
      "inferenceMs": 3501.2301669921726,
      "tokensPerSecond": 1.1424555968099377,
      "inputTokens": 830,
      "outputTokens": 4,
      "maxNewTokens": 32,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": 700,
      "inputHeight": 577,
      "visionGrid": [
        [
          1,
          52,
          62
        ]
      ],
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 8224980992,
      "memoryMeasurementScope": "Post-inference MPS driver allocation, not peak",
      "totalMs": 8395.510583010036,
      "requestMs": 3566.3710000226274,
      "precision": "fp16",
      "cpuThreads": 4,
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "fallback": false,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "accuracy": 1,
      "repetitive": false,
      "expected": [
        "localai2026"
      ],
      "method": "normalized exact printed text is contained in answer; nonempty, nonrepetitive answer required. Limited3-fixture compatibility, not broad VQA/OCR accuracy."
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4829.139582987409,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 617
      },
      "parameterDtype": "torch.float16",
      "attentionImplementation": "sdpa",
      "accelerationDetails": "Model parameters and input tensors on selected device; image decoding/processing and text tokenization/decoding on CPU",
      "durationMs": 2825.2386249951087,
      "inferenceMs": 2817.8865000081714,
      "tokensPerSecond": 3.5487589723613784,
      "inputTokens": 795,
      "outputTokens": 10,
      "maxNewTokens": 32,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": 768,
      "inputHeight": 256,
      "visionGrid": [
        [
          1,
          32,
          96
        ]
      ],
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 9507192832,
      "memoryMeasurementScope": "Post-inference MPS driver allocation, not peak",
      "totalMs": 7657.8495409630705,
      "requestMs": 2828.7099579756614,
      "precision": "fp16",
      "cpuThreads": 4,
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "fallback": false,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "accuracy": 1,
      "repetitive": false,
      "expected": [
        "3",
        "three"
      ],
      "method": "accepted count, with word boundaries; nonempty, nonrepetitive answer required. Limited3-fixture compatibility, not broad VQA/OCR accuracy."
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4829.139582987409,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 617
      },
      "parameterDtype": "torch.float16",
      "attentionImplementation": "sdpa",
      "accelerationDetails": "Model parameters and input tensors on selected device; image decoding/processing and text tokenization/decoding on CPU",
      "durationMs": 2629.1852919966914,
      "inferenceMs": 2621.339125005761,
      "tokensPerSecond": 1.5259376254841537,
      "inputTokens": 828,
      "outputTokens": 4,
      "maxNewTokens": 32,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": 640,
      "inputHeight": 320,
      "visionGrid": [
        [
          1,
          40,
          80
        ]
      ],
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 10896261120,
      "memoryMeasurementScope": "Post-inference MPS driver allocation, not peak",
      "totalMs": 7460.689415980596,
      "requestMs": 2631.549832993187,
      "precision": "fp16",
      "cpuThreads": 4,
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "fallback": false,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "accuracy": 1,
      "repetitive": false,
      "expected": [
        "dog",
        "labrador",
        "puppy",
        "canine"
      ],
      "method": "contains an accepted animal label; nonempty, nonrepetitive answer required. Limited3-fixture compatibility, not broad VQA/OCR accuracy."
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4829.139582987409,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 617
      },
      "parameterDtype": "torch.float16",
      "attentionImplementation": "sdpa",
      "accelerationDetails": "Model parameters and input tensors on selected device; image decoding/processing and text tokenization/decoding on CPU",
      "durationMs": 2547.124790988164,
      "inferenceMs": 2538.6683749966323,
      "tokensPerSecond": 1.5756291918219945,
      "inputTokens": 830,
      "outputTokens": 4,
      "maxNewTokens": 32,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": 700,
      "inputHeight": 577,
      "visionGrid": [
        [
          1,
          52,
          62
        ]
      ],
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 10896261120,
      "memoryMeasurementScope": "Post-inference MPS driver allocation, not peak",
      "totalMs": 7385.758457996417,
      "requestMs": 2556.6188750090078,
      "precision": "fp16",
      "cpuThreads": 4,
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "fallback": false,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "accuracy": 1,
      "repetitive": false,
      "expected": [
        "dog",
        "labrador",
        "puppy",
        "canine"
      ],
      "method": "contains an accepted animal label; nonempty, nonrepetitive answer required. Limited3-fixture compatibility, not broad VQA/OCR accuracy."
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 4829.139582987409,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 617
      },
      "parameterDtype": "torch.float16",
      "attentionImplementation": "sdpa",
      "accelerationDetails": "Model parameters and input tensors on selected device; image decoding/processing and text tokenization/decoding on CPU",
      "durationMs": 2426.6185000014957,
      "inferenceMs": 2417.6854169927537,
      "tokensPerSecond": 1.6544749667950653,
      "inputTokens": 830,
      "outputTokens": 4,
      "maxNewTokens": 32,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": 700,
      "inputHeight": 577,
      "visionGrid": [
        [
          1,
          52,
          62
        ]
      ],
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 10896261120,
      "memoryMeasurementScope": "Post-inference MPS driver allocation, not peak",
      "totalMs": 7265.1612909685355,
      "requestMs": 2436.0217079811264,
      "precision": "fp16",
      "cpuThreads": 4,
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "fallback": false,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "accuracy": 1,
      "repetitive": false,
      "expected": [
        "dog",
        "labrador",
        "puppy",
        "canine"
      ],
      "method": "contains an accepted animal label; nonempty, nonrepetitive answer required. Limited3-fixture compatibility, not broad VQA/OCR accuracy."
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3456.591624999419,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 617
      },
      "parameterDtype": "torch.float16",
      "attentionImplementation": "sdpa",
      "accelerationDetails": "Model parameters and input tensors on selected device; image decoding/processing and text tokenization/decoding on CPU",
      "durationMs": 2765.446167002665,
      "inferenceMs": 2729.4583750190213,
      "tokensPerSecond": 1.4654922150890557,
      "inputTokens": 830,
      "outputTokens": 4,
      "maxNewTokens": 32,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": 700,
      "inputHeight": 577,
      "visionGrid": [
        [
          1,
          52,
          62
        ]
      ],
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 8224980992,
      "memoryMeasurementScope": "Post-inference MPS driver allocation, not peak",
      "totalMs": 6236.358290974749,
      "requestMs": 2779.7666659753304,
      "precision": "fp16",
      "cpuThreads": 4,
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "fallback": false,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "accuracy": 1,
      "repetitive": false,
      "expected": [
        "dog",
        "labrador",
        "puppy",
        "canine"
      ],
      "method": "contains an accepted animal label; nonempty, nonrepetitive answer required. Limited3-fixture compatibility, not broad VQA/OCR accuracy."
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3456.591624999419,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 617
      },
      "parameterDtype": "torch.float16",
      "attentionImplementation": "sdpa",
      "accelerationDetails": "Model parameters and input tensors on selected device; image decoding/processing and text tokenization/decoding on CPU",
      "durationMs": 2442.4569999973755,
      "inferenceMs": 2433.4362499939743,
      "tokensPerSecond": 1.6437660941435819,
      "inputTokens": 830,
      "outputTokens": 4,
      "maxNewTokens": 32,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": 700,
      "inputHeight": 577,
      "visionGrid": [
        [
          1,
          52,
          62
        ]
      ],
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 8224980992,
      "memoryMeasurementScope": "Post-inference MPS driver allocation, not peak",
      "totalMs": 5908.416999998735,
      "requestMs": 2451.8253749993164,
      "precision": "fp16",
      "cpuThreads": 4,
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "fallback": false,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "accuracy": 1,
      "repetitive": false,
      "expected": [
        "dog",
        "labrador",
        "puppy",
        "canine"
      ],
      "method": "contains an accepted animal label; nonempty, nonrepetitive answer required. Limited3-fixture compatibility, not broad VQA/OCR accuracy."
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 3456.591624999419,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 617
      },
      "parameterDtype": "torch.float16",
      "attentionImplementation": "sdpa",
      "accelerationDetails": "Model parameters and input tensors on selected device; image decoding/processing and text tokenization/decoding on CPU",
      "durationMs": 2550.692542019533,
      "inferenceMs": 2541.820583981462,
      "tokensPerSecond": 1.5736751937599278,
      "inputTokens": 830,
      "outputTokens": 4,
      "maxNewTokens": 32,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": 700,
      "inputHeight": 577,
      "visionGrid": [
        [
          1,
          52,
          62
        ]
      ],
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 8224980992,
      "memoryMeasurementScope": "Post-inference MPS driver allocation, not peak",
      "totalMs": 6017.409708991181,
      "requestMs": 2560.8180839917623,
      "precision": "fp16",
      "cpuThreads": 4,
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "fallback": false,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "accuracy": 1,
      "repetitive": false,
      "expected": [
        "dog",
        "labrador",
        "puppy",
        "canine"
      ],
      "method": "contains an accepted animal label; nonempty, nonrepetitive answer required. Limited3-fixture compatibility, not broad VQA/OCR accuracy."
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 2780.9376250079367,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 617
      },
      "parameterDtype": "torch.float16",
      "attentionImplementation": "sdpa",
      "accelerationDetails": "Model parameters and input tensors on selected device; image decoding/processing and text tokenization/decoding on CPU",
      "durationMs": 2855.775083997287,
      "inferenceMs": 2819.052875012858,
      "tokensPerSecond": 1.418916273424547,
      "inputTokens": 830,
      "outputTokens": 4,
      "maxNewTokens": 32,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": 700,
      "inputHeight": 577,
      "visionGrid": [
        [
          1,
          52,
          62
        ]
      ],
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 8224980992,
      "memoryMeasurementScope": "Post-inference MPS driver allocation, not peak",
      "totalMs": 5654.870041995309,
      "requestMs": 2873.9324169873726,
      "precision": "fp16",
      "cpuThreads": 4,
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "fallback": false,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "accuracy": 1,
      "repetitive": false,
      "expected": [
        "dog",
        "labrador",
        "puppy",
        "canine"
      ],
      "method": "contains an accepted animal label; nonempty, nonrepetitive answer required. Limited3-fixture compatibility, not broad VQA/OCR accuracy."
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 2780.9376250079367,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 617
      },
      "parameterDtype": "torch.float16",
      "attentionImplementation": "sdpa",
      "accelerationDetails": "Model parameters and input tensors on selected device; image decoding/processing and text tokenization/decoding on CPU",
      "durationMs": 2456.1110000067856,
      "inferenceMs": 2447.9581670020707,
      "tokensPerSecond": 1.634014851200934,
      "inputTokens": 830,
      "outputTokens": 4,
      "maxNewTokens": 32,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": 700,
      "inputHeight": 577,
      "visionGrid": [
        [
          1,
          52,
          62
        ]
      ],
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 8224980992,
      "memoryMeasurementScope": "Post-inference MPS driver allocation, not peak",
      "totalMs": 5248.155583016342,
      "requestMs": 2467.217958008405,
      "precision": "fp16",
      "cpuThreads": 4,
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "fallback": false,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "accuracy": 1,
      "repetitive": false,
      "expected": [
        "dog",
        "labrador",
        "puppy",
        "canine"
      ],
      "method": "contains an accepted animal label; nonempty, nonrepetitive answer required. Limited3-fixture compatibility, not broad VQA/OCR accuracy."
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 2780.9376250079367,
      "device": "mps",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "placement": {
        "mps:0/torch.float16": 617
      },
      "parameterDtype": "torch.float16",
      "attentionImplementation": "sdpa",
      "accelerationDetails": "Model parameters and input tensors on selected device; image decoding/processing and text tokenization/decoding on CPU",
      "durationMs": 2514.555166999344,
      "inferenceMs": 2505.1045839791186,
      "tokensPerSecond": 1.5967397231960605,
      "inputTokens": 830,
      "outputTokens": 4,
      "maxNewTokens": 32,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": 700,
      "inputHeight": 577,
      "visionGrid": [
        [
          1,
          52,
          62
        ]
      ],
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 8224980992,
      "memoryMeasurementScope": "Post-inference MPS driver allocation, not peak",
      "totalMs": 5304.842166020535,
      "requestMs": 2523.904541012598,
      "precision": "fp16",
      "cpuThreads": 4,
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "fallback": false,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "accuracy": 1,
      "repetitive": false,
      "expected": [
        "dog",
        "labrador",
        "puppy",
        "canine"
      ],
      "method": "contains an accepted animal label; nonempty, nonrepetitive answer required. Limited3-fixture compatibility, not broad VQA/OCR accuracy."
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 2248.368542001117,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 617
      },
      "parameterDtype": "torch.float32",
      "attentionImplementation": "sdpa",
      "accelerationDetails": "Model parameters and input tensors on selected device; image decoding/processing and text tokenization/decoding on CPU",
      "durationMs": 11404.363916022703,
      "inferenceMs": 11371.931041008793,
      "tokensPerSecond": 0.35174325148257,
      "inputTokens": 830,
      "outputTokens": 4,
      "maxNewTokens": 32,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": 700,
      "inputHeight": 577,
      "visionGrid": [
        [
          1,
          52,
          62
        ]
      ],
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 13666.94083402399,
      "requestMs": 11418.572292022873,
      "precision": "fp32",
      "cpuThreads": 4,
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "fallback": false,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "accuracy": 1,
      "repetitive": false,
      "expected": [
        "localai2026"
      ],
      "method": "normalized exact printed text is contained in answer; nonempty, nonrepetitive answer required. Limited3-fixture compatibility, not broad VQA/OCR accuracy."
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 2248.368542001117,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 617
      },
      "parameterDtype": "torch.float32",
      "attentionImplementation": "sdpa",
      "accelerationDetails": "Model parameters and input tensors on selected device; image decoding/processing and text tokenization/decoding on CPU",
      "durationMs": 21685.598125011893,
      "inferenceMs": 21680.47308298992,
      "tokensPerSecond": 0.46124454764992223,
      "inputTokens": 795,
      "outputTokens": 10,
      "maxNewTokens": 32,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": 768,
      "inputHeight": 256,
      "visionGrid": [
        [
          1,
          32,
          96
        ]
      ],
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 23936.53741700109,
      "requestMs": 21688.168874999974,
      "precision": "fp32",
      "cpuThreads": 4,
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "fallback": false,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "accuracy": 1,
      "repetitive": false,
      "expected": [
        "3",
        "three"
      ],
      "method": "accepted count, with word boundaries; nonempty, nonrepetitive answer required. Limited3-fixture compatibility, not broad VQA/OCR accuracy."
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 2248.368542001117,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 617
      },
      "parameterDtype": "torch.float32",
      "attentionImplementation": "sdpa",
      "accelerationDetails": "Model parameters and input tensors on selected device; image decoding/processing and text tokenization/decoding on CPU",
      "durationMs": 10922.929207998095,
      "inferenceMs": 10917.805709003005,
      "tokensPerSecond": 0.36637398636811547,
      "inputTokens": 828,
      "outputTokens": 4,
      "maxNewTokens": 32,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": 640,
      "inputHeight": 320,
      "visionGrid": [
        [
          1,
          40,
          80
        ]
      ],
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 13173.251250991598,
      "requestMs": 10924.88270899048,
      "precision": "fp32",
      "cpuThreads": 4,
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "fallback": false,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "accuracy": 1,
      "repetitive": false,
      "expected": [
        "dog",
        "labrador",
        "puppy",
        "canine"
      ],
      "method": "contains an accepted animal label; nonempty, nonrepetitive answer required. Limited3-fixture compatibility, not broad VQA/OCR accuracy."
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 2248.368542001117,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 617
      },
      "parameterDtype": "torch.float32",
      "attentionImplementation": "sdpa",
      "accelerationDetails": "Model parameters and input tensors on selected device; image decoding/processing and text tokenization/decoding on CPU",
      "durationMs": 11013.895374984713,
      "inferenceMs": 11008.425042004092,
      "tokensPerSecond": 0.36335806300515056,
      "inputTokens": 830,
      "outputTokens": 4,
      "maxNewTokens": 32,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": 700,
      "inputHeight": 577,
      "visionGrid": [
        [
          1,
          52,
          62
        ]
      ],
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 13271.170833992073,
      "requestMs": 11022.802291990956,
      "precision": "fp32",
      "cpuThreads": 4,
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "fallback": false,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "accuracy": 1,
      "repetitive": false,
      "expected": [
        "dog",
        "labrador",
        "puppy",
        "canine"
      ],
      "method": "contains an accepted animal label; nonempty, nonrepetitive answer required. Limited3-fixture compatibility, not broad VQA/OCR accuracy."
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 2248.368542001117,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 617
      },
      "parameterDtype": "torch.float32",
      "attentionImplementation": "sdpa",
      "accelerationDetails": "Model parameters and input tensors on selected device; image decoding/processing and text tokenization/decoding on CPU",
      "durationMs": 11063.421124999877,
      "inferenceMs": 11057.724208018044,
      "tokensPerSecond": 0.3617380868569292,
      "inputTokens": 830,
      "outputTokens": 4,
      "maxNewTokens": 32,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": 700,
      "inputHeight": 577,
      "visionGrid": [
        [
          1,
          52,
          62
        ]
      ],
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 13320.362375001423,
      "requestMs": 11071.993833000306,
      "precision": "fp32",
      "cpuThreads": 4,
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "fallback": false,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "accuracy": 1,
      "repetitive": false,
      "expected": [
        "dog",
        "labrador",
        "puppy",
        "canine"
      ],
      "method": "contains an accepted animal label; nonempty, nonrepetitive answer required. Limited3-fixture compatibility, not broad VQA/OCR accuracy."
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 2514.729874994373,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 617
      },
      "parameterDtype": "torch.float32",
      "attentionImplementation": "sdpa",
      "accelerationDetails": "Model parameters and input tensors on selected device; image decoding/processing and text tokenization/decoding on CPU",
      "durationMs": 11904.162708990043,
      "inferenceMs": 11869.36270800652,
      "tokensPerSecond": 0.3370020866665222,
      "inputTokens": 830,
      "outputTokens": 4,
      "maxNewTokens": 32,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": 700,
      "inputHeight": 577,
      "visionGrid": [
        [
          1,
          52,
          62
        ]
      ],
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 14432.75312500191,
      "requestMs": 11918.023250007536,
      "precision": "fp32",
      "cpuThreads": 4,
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "fallback": false,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "accuracy": 1,
      "repetitive": false,
      "expected": [
        "dog",
        "labrador",
        "puppy",
        "canine"
      ],
      "method": "contains an accepted animal label; nonempty, nonrepetitive answer required. Limited3-fixture compatibility, not broad VQA/OCR accuracy."
    },
    "runtime": {
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "loadMs": 2514.729874994373,
      "device": "cpu",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "placement": {
        "cpu/torch.float32": 617
      },
      "parameterDtype": "torch.float32",
      "attentionImplementation": "sdpa",
      "accelerationDetails": "Model parameters and input tensors on selected device; image decoding/processing and text tokenization/decoding on CPU",
      "durationMs": 13678.311458003009,
      "inferenceMs": 13672.118499991484,
      "tokensPerSecond": 0.2925662178836799,
      "inputTokens": 830,
      "outputTokens": 4,
      "maxNewTokens": 32,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": 700,
      "inputHeight": 577,
      "visionGrid": [
        [
          1,
          52,
          62
        ]
      ],
      "decoding": "greedy",
      "outputLimitReached": false,
      "totalMs": 16202.244459011126,
      "requestMs": 13687.514584016753,
      "precision": "fp32",
      "cpuThreads": 4,
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "fallback": false,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Predeclared fixture answer check",
      "nonEmpty": true,
      "repetitive": false,
      "grade": "fixture-pass",
      "score": 1,
      "gibberish": false
    },
    "runtime": {
      "durationMs": 12227.530415984802,
      "inferenceMs": 12184.311417047866,
      "tokensPerSecond": 0.3282910181040956,
      "inputTokens": 830,
      "outputTokens": 4,
      "maxNewTokens": 64,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": 700,
      "inputHeight": 577,
      "visionGrid": [
        [
          1,
          52,
          62
        ]
      ],
      "inputModality": "image",
      "videoFrameCount": 0,
      "videoFps": null,
      "placement": {
        "cpu/torch.float32": 617
      },
      "decoding": "greedy",
      "outputLimitReached": false,
      "loadMs": 4161.168667022139,
      "totalMs": 16396.11612504814,
      "wallMs": 12234.045417048037,
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "device": "cpu",
      "precision": "fp32",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false,
      "parameterDtype": "torch.float32",
      "grade": "very-slow"
    },
    "failure": null
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Predeclared fixture answer check",
      "nonEmpty": true,
      "repetitive": false,
      "grade": "fixture-pass",
      "score": 1,
      "gibberish": false
    },
    "runtime": {
      "durationMs": 23337.12162496522,
      "inferenceMs": 23330.601792025845,
      "tokensPerSecond": 0.4286216056123291,
      "inputTokens": 795,
      "outputTokens": 10,
      "maxNewTokens": 64,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": 768,
      "inputHeight": 256,
      "visionGrid": [
        [
          1,
          32,
          96
        ]
      ],
      "inputModality": "image",
      "videoFrameCount": 0,
      "videoFps": null,
      "placement": {
        "cpu/torch.float32": 617
      },
      "decoding": "greedy",
      "outputLimitReached": false,
      "loadMs": 4161.168667022139,
      "totalMs": 39735.038042010274,
      "wallMs": 23338.824875012506,
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "device": "cpu",
      "precision": "fp32",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false,
      "parameterDtype": "torch.float32",
      "grade": "very-slow"
    },
    "failure": null
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Predeclared fixture answer check",
      "nonEmpty": true,
      "repetitive": false,
      "grade": "fixture-pass",
      "score": 1,
      "gibberish": false
    },
    "runtime": {
      "durationMs": 11692.36191699747,
      "inferenceMs": 11685.772624972742,
      "tokensPerSecond": 0.34229657964180443,
      "inputTokens": 828,
      "outputTokens": 4,
      "maxNewTokens": 64,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": 640,
      "inputHeight": 320,
      "visionGrid": [
        [
          1,
          40,
          80
        ]
      ],
      "inputModality": "image",
      "videoFrameCount": 0,
      "videoFps": null,
      "placement": {
        "cpu/torch.float32": 617
      },
      "decoding": "greedy",
      "outputLimitReached": false,
      "loadMs": 4161.168667022139,
      "totalMs": 51428.82162501337,
      "wallMs": 11693.634708004538,
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "device": "cpu",
      "precision": "fp32",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false,
      "parameterDtype": "torch.float32",
      "grade": "very-slow"
    },
    "failure": null
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Predeclared fixture answer check",
      "nonEmpty": true,
      "repetitive": false,
      "grade": "fixture-pass",
      "score": 1,
      "gibberish": false
    },
    "runtime": {
      "durationMs": 14373.424959019758,
      "inferenceMs": 14365.009874978568,
      "tokensPerSecond": 0.4176815785174635,
      "inputTokens": 430,
      "outputTokens": 6,
      "maxNewTokens": 64,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": 320,
      "inputHeight": 240,
      "visionGrid": [
        [
          2,
          24,
          32
        ]
      ],
      "inputModality": "video",
      "videoFrameCount": 4,
      "videoFps": 1,
      "placement": {
        "cpu/torch.float32": 617
      },
      "decoding": "greedy",
      "outputLimitReached": false,
      "loadMs": 4161.168667022139,
      "totalMs": 65803.58358303783,
      "wallMs": 14374.706832983065,
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "device": "cpu",
      "precision": "fp32",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false,
      "parameterDtype": "torch.float32",
      "grade": "very-slow"
    },
    "failure": null
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Predeclared fixture answer check",
      "nonEmpty": true,
      "repetitive": false,
      "grade": "fixture-pass",
      "score": 1,
      "gibberish": false
    },
    "runtime": {
      "durationMs": 16181.570166023448,
      "inferenceMs": 16174.156375054736,
      "tokensPerSecond": 0.37096216092319656,
      "inputTokens": 430,
      "outputTokens": 6,
      "maxNewTokens": 64,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": 320,
      "inputHeight": 240,
      "visionGrid": [
        [
          2,
          24,
          32
        ]
      ],
      "inputModality": "video",
      "videoFrameCount": 4,
      "videoFps": 1,
      "placement": {
        "cpu/torch.float32": 617
      },
      "decoding": "greedy",
      "outputLimitReached": false,
      "loadMs": 4161.168667022139,
      "totalMs": 81986.44641705323,
      "wallMs": 16182.79829196399,
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "device": "cpu",
      "precision": "fp32",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false,
      "parameterDtype": "torch.float32",
      "grade": "very-slow"
    },
    "failure": null
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Predeclared fixture answer check",
      "nonEmpty": true,
      "repetitive": false,
      "grade": "fixture-pass",
      "score": 1,
      "gibberish": false
    },
    "runtime": {
      "durationMs": 8118.9361669821665,
      "inferenceMs": 8118.294834042899,
      "tokensPerSecond": 0.49271430537686023,
      "inputTokens": 25,
      "outputTokens": 4,
      "maxNewTokens": 64,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": null,
      "inputHeight": null,
      "visionGrid": null,
      "inputModality": "text",
      "videoFrameCount": 0,
      "videoFps": null,
      "placement": {
        "cpu/torch.float32": 617
      },
      "decoding": "greedy",
      "outputLimitReached": false,
      "loadMs": 4161.168667022139,
      "totalMs": 90106.46545805503,
      "wallMs": 8119.945833983365,
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "device": "cpu",
      "precision": "fp32",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "provider": "PyTorch CPU",
      "acceleration": "cpu",
      "fallback": false,
      "parameterDtype": "torch.float32",
      "grade": "very-slow"
    },
    "failure": null
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Predeclared fixture answer check",
      "nonEmpty": true,
      "repetitive": false,
      "grade": "fixture-pass",
      "score": 1,
      "gibberish": false
    },
    "runtime": {
      "durationMs": 3677.52179200761,
      "inferenceMs": 3630.2081670146435,
      "tokensPerSecond": 1.1018651867805862,
      "inputTokens": 830,
      "outputTokens": 4,
      "maxNewTokens": 64,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": 700,
      "inputHeight": 577,
      "visionGrid": [
        [
          1,
          52,
          62
        ]
      ],
      "inputModality": "image",
      "videoFrameCount": 0,
      "videoFps": null,
      "placement": {
        "mps:0/torch.float16": 617
      },
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 8224980992,
      "memoryMeasurementScope": "Post-inference MPS driver allocation, not peak",
      "loadMs": 4710.468957957346,
      "totalMs": 8395.532499998808,
      "wallMs": 3684.2780000297353,
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "device": "mps",
      "precision": "fp16",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false,
      "parameterDtype": "torch.float16",
      "grade": "slow"
    },
    "failure": null
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Predeclared fixture answer check",
      "nonEmpty": true,
      "repetitive": false,
      "grade": "fixture-pass",
      "score": 1,
      "gibberish": false
    },
    "runtime": {
      "durationMs": 4003.968542034272,
      "inferenceMs": 3990.380709001329,
      "tokensPerSecond": 2.506026549657889,
      "inputTokens": 795,
      "outputTokens": 10,
      "maxNewTokens": 64,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": 768,
      "inputHeight": 256,
      "visionGrid": [
        [
          1,
          32,
          96
        ]
      ],
      "inputModality": "image",
      "videoFrameCount": 0,
      "videoFps": null,
      "placement": {
        "mps:0/torch.float16": 617
      },
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 9507192832,
      "memoryMeasurementScope": "Post-inference MPS driver allocation, not peak",
      "loadMs": 4710.468957957346,
      "totalMs": 12403.36599998409,
      "wallMs": 4007.735874969512,
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "device": "mps",
      "precision": "fp16",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false,
      "parameterDtype": "torch.float16",
      "grade": "slow"
    },
    "failure": null
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Predeclared fixture answer check",
      "nonEmpty": true,
      "repetitive": false,
      "grade": "fixture-pass",
      "score": 1,
      "gibberish": false
    },
    "runtime": {
      "durationMs": 3292.6492909900844,
      "inferenceMs": 3261.818917002529,
      "tokensPerSecond": 1.2263096455629816,
      "inputTokens": 828,
      "outputTokens": 4,
      "maxNewTokens": 64,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": 640,
      "inputHeight": 320,
      "visionGrid": [
        [
          1,
          40,
          80
        ]
      ],
      "inputModality": "image",
      "videoFrameCount": 0,
      "videoFps": null,
      "placement": {
        "mps:0/torch.float16": 617
      },
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 10896261120,
      "memoryMeasurementScope": "Post-inference MPS driver allocation, not peak",
      "loadMs": 4710.468957957346,
      "totalMs": 15698.154749989044,
      "wallMs": 3294.620875036344,
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "device": "mps",
      "precision": "fp16",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false,
      "parameterDtype": "torch.float16",
      "grade": "slow"
    },
    "failure": null
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Predeclared fixture answer check",
      "nonEmpty": true,
      "repetitive": false,
      "grade": "fixture-pass",
      "score": 1,
      "gibberish": false
    },
    "runtime": {
      "durationMs": 1900.186083978042,
      "inferenceMs": 1859.4473749981262,
      "tokensPerSecond": 3.2267651565057314,
      "inputTokens": 430,
      "outputTokens": 6,
      "maxNewTokens": 64,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": 320,
      "inputHeight": 240,
      "visionGrid": [
        [
          2,
          24,
          32
        ]
      ],
      "inputModality": "video",
      "videoFrameCount": 4,
      "videoFps": 1,
      "placement": {
        "mps:0/torch.float16": 617
      },
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 11039358976,
      "memoryMeasurementScope": "Post-inference MPS driver allocation, not peak",
      "loadMs": 4710.468957957346,
      "totalMs": 17601.346749986988,
      "wallMs": 1903.1107079936191,
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "device": "mps",
      "precision": "fp16",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false,
      "parameterDtype": "torch.float16",
      "grade": "slow"
    },
    "failure": null
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Predeclared fixture answer check",
      "nonEmpty": true,
      "repetitive": false,
      "grade": "fixture-pass",
      "score": 1,
      "gibberish": false
    },
    "runtime": {
      "durationMs": 1726.7867919872515,
      "inferenceMs": 1685.939832997974,
      "tokensPerSecond": 3.5588458630404816,
      "inputTokens": 430,
      "outputTokens": 6,
      "maxNewTokens": 64,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": 320,
      "inputHeight": 240,
      "visionGrid": [
        [
          2,
          24,
          32
        ]
      ],
      "inputModality": "video",
      "videoFrameCount": 4,
      "videoFps": 1,
      "placement": {
        "mps:0/torch.float16": 617
      },
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 11039358976,
      "memoryMeasurementScope": "Post-inference MPS driver allocation, not peak",
      "loadMs": 4710.468957957346,
      "totalMs": 19331.72083296813,
      "wallMs": 1730.173374991864,
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "device": "mps",
      "precision": "fp16",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false,
      "parameterDtype": "torch.float16",
      "grade": "slow"
    },
    "failure": null
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Predeclared fixture answer check",
      "nonEmpty": true,
      "repetitive": false,
      "grade": "fixture-pass",
      "score": 1,
      "gibberish": false
    },
    "runtime": {
      "durationMs": 411.3183749723248,
      "inferenceMs": 408.82558398880064,
      "tokensPerSecond": 9.784123490934892,
      "inputTokens": 25,
      "outputTokens": 4,
      "maxNewTokens": 64,
      "minPixels": 786432,
      "maxPixels": 1048576,
      "inputWidth": null,
      "inputHeight": null,
      "visionGrid": null,
      "inputModality": "text",
      "videoFrameCount": 0,
      "videoFps": null,
      "placement": {
        "mps:0/torch.float16": 617
      },
      "decoding": "greedy",
      "outputLimitReached": false,
      "driverAllocatedMemoryBytes": 10372792320,
      "memoryMeasurementScope": "Post-inference MPS driver allocation, not peak",
      "loadMs": 4710.468957957346,
      "totalMs": 19744.47350000264,
      "wallMs": 412.6771250157617,
      "engine": "transformers",
      "version": "5.22.1",
      "torchVersion": "2.10.0",
      "artifactRevision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
      "device": "mps",
      "precision": "fp16",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "provider": "Metal/MPS",
      "acceleration": "partial",
      "fallback": false,
      "parameterDtype": "torch.float16",
      "grade": "good"
    },
    "failure": null
  }
]

Something not quite right?

Tell us what happened with Qwen3.5 Desktop. Loading, results, or the page itself—we’re here for all of it.

Attachments Optional

Up to 3 files · 50 MB each

Browser, OS, CPU/GPU, memory, screen, storage, and network details are included automatically. Model inputs, results, and browsing history are not collected.