A model is still running

A model is still running or downloading. Leaving will stop the current operation. Stay on this page to let it finish.

Skip to content
LocalAI.toolsBeta

Join the waitlist

Get an email when the new apps and model runs launch.

No account needed.

EoMT Desktop

Separate common objects and background regions in still images into pixel masks with class labels.

Desktop inferencePrivate · Runs on your computer
About this model

Separate common objects and background regions in still images into pixel masks with class labels.

License
MIT
Model released
Unknown
Source repository created
Jul 1, 2025
Model weights · FP16
95.9 MB

Best for

  • Inspect and label objects and background in photos.

Keep in mind

  • Uses the COCO class vocabulary.
  • Small, occluded and unfamiliar objects may be missed.
  • Low light and blur can reduce accuracy. Review masks before relying on them.
  • Inference returns the complete result in one operation.

Processed on your deviceYour input is processed locally in the desktop app. Model and runtime files download from external hosts.

Target platforms · verification required

  • macos · aarch64

Recommended configuration

FP16 · mps (verified platform only)

Sources & technical details
Original model
https://huggingface.co/tue-mps/coco_panoptic_eomt_small_640_2x
Release verification
Unknown
Release evidence
Exact first public checkpoint date unverified.
Source updated
Nov 11, 2025
Native model files
https://huggingface.co/tue-mps/coco_panoptic_eomt_small_640_2x
Files updated
Nov 11, 2025
Artifact revision
10f5326794a8ce7e0e975791033ae1c13aec3767

Desktop app

Sample result

EoMT COCO Panoptic Small 640 2x Desktop

Run EoMT Desktop on your computer

Download the desktop app to load this model, provide input and view results locally.

Public desktop downloads are coming soon.

Device requirements

macOS Apple SiliconMPS / FP32TestedNot establishedRAM estimate
macOS Apple SiliconCPU / FP32TestedNot establishedRAM estimate
macOS Apple SiliconMPS / FP16TestedNot establishedRAM estimate
macOS Apple SiliconMPS / BF16TestedNot establishedRAM estimate
If your device falls short

MemoryAdvisory only: close other apps, reduce image/tile/token/batch settings or choose a smaller verified model/precision. CPU can need more RAM than GPU.

GPUIf the selected GPU runtime fails, existing CPU fallback may recover when enabled, but CPU has its own memory/runtime needs.

PlatformUse a compatible OS/architecture/runtime or another model. These estimates do not add hardware detection or block runs.

Benchmark evidence

Saved desktop measurements apply to the recorded hardware and runtime. Fallback cases report the device that actually ran; they do not prove GPU support on the requested device.

EoMT COCO Panoptic Small 640 2x Desktop

26 configurations tested
Passed26
Failed0
TestedNov 14, 2026
Runtimetransformers 4.57.6

Quality criteria

No explicit quality gate recorded. Review per-case evidence; full certification is pending.

PrecisionDeviceResultPlatformInference / load
FP32mpsAcceleratedPassedFastmacos · aarch64226 ms / 122 ms
BF16mpsAcceleratedPassedFastmacos · aarch64233 ms / 116 ms
FP16mpsAcceleratedPassedFastmacos · aarch64240 ms / 117 ms
FP16mpsAcceleratedPassedFastmacos · aarch64251 ms / 117 ms
FP32mpsAcceleratedPassedFastmacos · aarch64251 ms / 122 ms
FP16mpsAcceleratedPassedFastmacos · aarch64257 ms / 117 ms
BF16mpsAcceleratedPassedFastmacos · aarch64261 ms / 116 ms
FP32mpsAcceleratedPassedFastmacos · aarch64264 ms / 122 ms
FP32mpsAcceleratedPassedFastmacos · aarch64267 ms / 122 ms
FP16mpsAcceleratedPassedFastmacos · aarch64268 ms / 117 ms
BF16mpsAcceleratedPassedFastmacos · aarch64270 ms / 116 ms
BF16mpsAcceleratedPassedFastmacos · aarch64276 ms / 116 ms
FP16mpsAcceleratedPassedFastmacos · aarch64352 ms / 117 ms
BF16mpsAcceleratedPassedFastmacos · aarch64355 ms / 116 ms
FP32mpsAcceleratedPassedFastmacos · aarch64366 ms / 122 ms
FP16mpsAcceleratedPassedFastNot recorded436 ms / 3118 ms
BF16mpsAcceleratedPassedFastmacos · aarch64451 ms / 116 ms
FP16mpsAcceleratedPassedFastmacos · aarch64452 ms / 117 ms
FP32mpsAcceleratedPassedFastmacos · aarch64502 ms / 122 ms
FP16mpsAcceleratedPassedModerateNot recorded1779 ms / 3118 ms
FP32cpuPassedFastmacos · aarch64396 ms / 70 ms
FP32cpuPassedFastmacos · aarch64430 ms / 70 ms
FP32cpuPassedFastmacos · aarch64437 ms / 70 ms
FP32cpuPassedFastmacos · aarch64437 ms / 70 ms
FP32cpuPassedFastmacos · aarch64511 ms / 70 ms
FP32cpuPassedFastmacos · aarch64905 ms / 70 ms
Test setup & sample details
Quality gate
No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
Tested
2026-11-14T21:10:51.081407+00:00
Environment
{ "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
Runtime
transformers 4.57.6
Artifact revision
10f5326794a8ce7e0e975791033ae1c13aec3767
Fixture
{
  "file": "model-samples/desktop/mobilenet-dog.jpg",
  "size": 83281,
  "sha1": "f1d042827c2f2154a6d8043b8e8f07c1be612941",
  "sha256": "d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1",
  "url": "https://assets.localai.tools/samples/v1/files/d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1/mobilenet-dog.jpg"
}
Rerun when
Checkpoint or runtime revision; Preprocessing or postprocessing; Fixture or quality gate
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 70.15149999642745,
      "inferenceMs": 904.7755420324393,
      "totalMs": 974.9270420288667,
      "imagesPerSecond": 1.105246498765488,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 70.15149999642745,
      "inferenceMs": 511.2899169907905,
      "totalMs": 581.441416987218,
      "imagesPerSecond": 1.9558375136469046,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 70.15149999642745,
      "inferenceMs": 395.8383330027573,
      "totalMs": 465.9898329991847,
      "imagesPerSecond": 2.526283880629202,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 70.15149999642745,
      "inferenceMs": 436.6907919757068,
      "totalMs": 506.84229197213426,
      "imagesPerSecond": 2.2899498188998457,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 70.15149999642745,
      "inferenceMs": 437.0860829949379,
      "totalMs": 507.23758299136534,
      "imagesPerSecond": 2.287878838758592,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 70.15149999642745,
      "inferenceMs": 429.8469999921508,
      "totalMs": 499.99849998857826,
      "imagesPerSecond": 2.326409164233461,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 116.48004199378192,
      "inferenceMs": 451.4445000095293,
      "totalMs": 567.9245420033112,
      "imagesPerSecond": 2.2151117135747396,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 116.48004199378192,
      "inferenceMs": 354.55058398656547,
      "totalMs": 471.0306259803474,
      "imagesPerSecond": 2.8204720148984204,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 116.48004199378192,
      "inferenceMs": 232.7362920041196,
      "totalMs": 349.21633399790153,
      "imagesPerSecond": 4.296708482329431,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 116.48004199378192,
      "inferenceMs": 261.2045829882845,
      "totalMs": 377.68462498206645,
      "imagesPerSecond": 3.828416747361786,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 116.48004199378192,
      "inferenceMs": 276.177000021562,
      "totalMs": 392.6570420153439,
      "imagesPerSecond": 3.6208663281950595,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 116.48004199378192,
      "inferenceMs": 270.07074997527525,
      "totalMs": 386.5507919690572,
      "imagesPerSecond": 3.702733450740405,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 116.75829102750868,
      "inferenceMs": 452.4747079703957,
      "totalMs": 569.2329989979044,
      "imagesPerSecond": 2.2100682809113557,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 116.75829102750868,
      "inferenceMs": 352.3067500209436,
      "totalMs": 469.06504104845226,
      "imagesPerSecond": 2.838435539314966,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 116.75829102750868,
      "inferenceMs": 240.00308301765472,
      "totalMs": 356.7613740451634,
      "imagesPerSecond": 4.166613142742169,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 116.75829102750868,
      "inferenceMs": 251.07966701034456,
      "totalMs": 367.83795803785324,
      "imagesPerSecond": 3.98279961060646,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 116.75829102750868,
      "inferenceMs": 256.86374999349937,
      "totalMs": 373.62204102100804,
      "imagesPerSecond": 3.8931145403946945,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 116.75829102750868,
      "inferenceMs": 268.16470798803493,
      "totalMs": 384.9229990155436,
      "imagesPerSecond": 3.729051475500715,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 122.09537497255951,
      "inferenceMs": 502.0645420299843,
      "totalMs": 624.1599170025438,
      "imagesPerSecond": 1.9917757903331041,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 122.09537497255951,
      "inferenceMs": 366.10283399932086,
      "totalMs": 488.1982089718804,
      "imagesPerSecond": 2.7314729828118596,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 122.09537497255951,
      "inferenceMs": 226.47554200375453,
      "totalMs": 348.57091697631404,
      "imagesPerSecond": 4.415487832162565,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 122.09537497255951,
      "inferenceMs": 251.20533403242007,
      "totalMs": 373.3007090049796,
      "imagesPerSecond": 3.980807190467309,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 122.09537497255951,
      "inferenceMs": 267.2842090250924,
      "totalMs": 389.37958399765193,
      "imagesPerSecond": 3.7413358748257397,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 122.09537497255951,
      "inferenceMs": 263.88716604560614,
      "totalMs": 385.98254101816565,
      "imagesPerSecond": 3.789498424592485,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Native output for same fixture and configuration as spatial qualification"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
      "artifactRevision": "10f5326794a8ce7e0e975791033ae1c13aec3767",
      "cpuThreads": 4,
      "device": "mps",
      "durationMs": 1791.068292,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "family": {
        "catalogId": "eomt-native",
        "defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
        "displayName": "EoMT",
        "id": "eomt",
        "variantId": "coco-panoptic-small-640-2x",
        "variantName": "COCO Panoptic Small 640 2x"
      },
      "inferenceMs": 1778.7012500339188,
      "loadMs": 3118.425292,
      "modelId": "eomt-coco-panoptic-small-640-2x-native",
      "parameterDtype": "torch.float16",
      "placement": {
        "mps:0": 242
      },
      "precision": "fp16",
      "predictionDevices": [
        "mps:0",
        "mps:0"
      ],
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "torchVersion": "2.11.0",
      "version": "4.57.6",
      "totalMs": 4909.493584,
      "imagesPerSecond": 0.5622079593079111,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Native output for same fixture and configuration as spatial qualification"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
      "artifactRevision": "10f5326794a8ce7e0e975791033ae1c13aec3767",
      "cpuThreads": 4,
      "device": "mps",
      "durationMs": 448.009708,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "family": {
        "catalogId": "eomt-native",
        "defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
        "displayName": "EoMT",
        "id": "eomt",
        "variantId": "coco-panoptic-small-640-2x",
        "variantName": "COCO Panoptic Small 640 2x"
      },
      "inferenceMs": 436.3478330196813,
      "loadMs": 3118.425292,
      "modelId": "eomt-coco-panoptic-small-640-2x-native",
      "parameterDtype": "torch.float16",
      "placement": {
        "mps:0": 242
      },
      "precision": "fp16",
      "predictionDevices": [
        "mps:0",
        "mps:0"
      ],
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "torchVersion": "2.11.0",
      "version": "4.57.6",
      "totalMs": 3566.435,
      "imagesPerSecond": 2.291749664664647,
      "grade": "fast"
    }
  }
]

EoMT ADE20K Panoptic Giant 640 Desktop

26 configurations tested
Passed26
Failed0
TestedNov 14, 2026
Runtimetransformers 4.57.6

Quality criteria

No explicit quality gate recorded. Review per-case evidence; full certification is pending.

PrecisionDeviceResultPlatformInference / load
FP16mpsAcceleratedPassedModeratemacos · aarch641064 ms / 3052 ms
BF16mpsAcceleratedPassedModeratemacos · aarch641088 ms / 656 ms
FP16mpsAcceleratedPassedModeratemacos · aarch641118 ms / 3052 ms
FP32mpsAcceleratedPassedModeratemacos · aarch641138 ms / 845 ms
FP32mpsAcceleratedPassedModeratemacos · aarch641154 ms / 845 ms
FP16mpsAcceleratedPassedModeratemacos · aarch641161 ms / 3052 ms
FP16mpsAcceleratedPassedModeratemacos · aarch641162 ms / 3052 ms
BF16mpsAcceleratedPassedModeratemacos · aarch641171 ms / 656 ms
BF16mpsAcceleratedPassedModeratemacos · aarch641176 ms / 656 ms
FP16mpsAcceleratedPassedModeratemacos · aarch641183 ms / 3052 ms
FP32mpsAcceleratedPassedModeratemacos · aarch641186 ms / 845 ms
FP16mpsAcceleratedPassedModerateNot recorded1187 ms / 6672 ms
BF16mpsAcceleratedPassedModeratemacos · aarch641188 ms / 656 ms
FP32mpsAcceleratedPassedModeratemacos · aarch641200 ms / 845 ms
FP32mpsAcceleratedPassedModeratemacos · aarch641219 ms / 845 ms
BF16mpsAcceleratedPassedModeratemacos · aarch641263 ms / 656 ms
FP16mpsAcceleratedPassedModeratemacos · aarch641264 ms / 3052 ms
BF16mpsAcceleratedPassedModeratemacos · aarch641378 ms / 656 ms
FP32mpsAcceleratedPassedModeratemacos · aarch641407 ms / 845 ms
FP16mpsAcceleratedPassedModerateNot recorded1815 ms / 6672 ms
FP32cpuPassedSlowmacos · aarch643990 ms / 79 ms
FP32cpuPassedSlowmacos · aarch643998 ms / 79 ms
FP32cpuPassedSlowmacos · aarch644030 ms / 79 ms
FP32cpuPassedSlowmacos · aarch644081 ms / 79 ms
FP32cpuPassedSlowmacos · aarch644106 ms / 79 ms
FP32cpuPassedSlowmacos · aarch644534 ms / 79 ms
Test setup & sample details
Quality gate
No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
Tested
2026-11-14T22:51:40.911723+00:00
Environment
{ "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
Runtime
transformers 4.57.6
Artifact revision
bb8de35cf2dc3827ea4a6ff06024752eeb18a37c
Fixture
{
  "file": "model-samples/desktop/mobilenet-dog.jpg",
  "size": 83281,
  "sha1": "f1d042827c2f2154a6d8043b8e8f07c1be612941",
  "sha256": "d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1",
  "url": "https://assets.localai.tools/samples/v1/files/d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1/mobilenet-dog.jpg"
}
Rerun when
Checkpoint or runtime revision; Preprocessing or postprocessing; Fixture or quality gate
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "attentionImplementation": "eager",
      "loadMs": 78.9822080405429,
      "inferenceMs": 4534.354167000856,
      "totalMs": 4613.336375041399,
      "imagesPerSecond": 0.22053857355863027,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "attentionImplementation": "eager",
      "loadMs": 78.9822080405429,
      "inferenceMs": 3997.847874998115,
      "totalMs": 4076.830083038658,
      "imagesPerSecond": 0.2501345802209824,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "attentionImplementation": "eager",
      "loadMs": 78.9822080405429,
      "inferenceMs": 3989.517457957845,
      "totalMs": 4068.499665998388,
      "imagesPerSecond": 0.25065688032153144,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "attentionImplementation": "eager",
      "loadMs": 78.9822080405429,
      "inferenceMs": 4105.786125001032,
      "totalMs": 4184.768333041575,
      "imagesPerSecond": 0.24355871678526575,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "attentionImplementation": "eager",
      "loadMs": 78.9822080405429,
      "inferenceMs": 4081.0487920534797,
      "totalMs": 4160.031000094023,
      "imagesPerSecond": 0.2450350512709321,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "attentionImplementation": "eager",
      "loadMs": 78.9822080405429,
      "inferenceMs": 4029.6836660127155,
      "totalMs": 4108.665874053258,
      "imagesPerSecond": 0.24815843695976222,
      "grade": "slow"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "attentionImplementation": "eager",
      "loadMs": 655.7425830396824,
      "inferenceMs": 1377.691750007216,
      "totalMs": 2033.4343330468982,
      "imagesPerSecond": 0.7258517734426169,
      "grade": "moderate"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "attentionImplementation": "eager",
      "loadMs": 655.7425830396824,
      "inferenceMs": 1171.2372910114937,
      "totalMs": 1826.979874051176,
      "imagesPerSecond": 0.8537979516826935,
      "grade": "moderate"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "attentionImplementation": "eager",
      "loadMs": 655.7425830396824,
      "inferenceMs": 1088.1721670157276,
      "totalMs": 1743.91475005541,
      "imagesPerSecond": 0.9189722272923626,
      "grade": "moderate"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "attentionImplementation": "eager",
      "loadMs": 655.7425830396824,
      "inferenceMs": 1263.4776670020074,
      "totalMs": 1919.2202500416897,
      "imagesPerSecond": 0.7914663045630321,
      "grade": "moderate"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "attentionImplementation": "eager",
      "loadMs": 655.7425830396824,
      "inferenceMs": 1175.8374579949304,
      "totalMs": 1831.5800410346128,
      "imagesPerSecond": 0.8504576829056176,
      "grade": "moderate"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "attentionImplementation": "eager",
      "loadMs": 655.7425830396824,
      "inferenceMs": 1187.9299589782022,
      "totalMs": 1843.6725420178846,
      "imagesPerSecond": 0.8418004718562279,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "attentionImplementation": "eager",
      "loadMs": 3051.7537919804454,
      "inferenceMs": 1263.995999994222,
      "totalMs": 4315.7497919746675,
      "imagesPerSecond": 0.7911417441230598,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "attentionImplementation": "eager",
      "loadMs": 3051.7537919804454,
      "inferenceMs": 1118.3492499985732,
      "totalMs": 4170.103041979019,
      "imagesPerSecond": 0.8941750530983732,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "attentionImplementation": "eager",
      "loadMs": 3051.7537919804454,
      "inferenceMs": 1063.70912498096,
      "totalMs": 4115.462916961405,
      "imagesPerSecond": 0.940106629260983,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "attentionImplementation": "eager",
      "loadMs": 3051.7537919804454,
      "inferenceMs": 1161.6565000149421,
      "totalMs": 4213.4102919953875,
      "imagesPerSecond": 0.8608396716130261,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "attentionImplementation": "eager",
      "loadMs": 3051.7537919804454,
      "inferenceMs": 1160.8822079724632,
      "totalMs": 4212.635999952909,
      "imagesPerSecond": 0.8614138395199873,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "attentionImplementation": "eager",
      "loadMs": 3051.7537919804454,
      "inferenceMs": 1182.6251659658737,
      "totalMs": 4234.378957946319,
      "imagesPerSecond": 0.8455764588632612,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "attentionImplementation": "eager",
      "loadMs": 844.9827910517342,
      "inferenceMs": 1407.2576250182465,
      "totalMs": 2252.2404160699807,
      "imagesPerSecond": 0.7106019411243439,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "attentionImplementation": "eager",
      "loadMs": 844.9827910517342,
      "inferenceMs": 1153.8906670175493,
      "totalMs": 1998.8734580692835,
      "imagesPerSecond": 0.8666332336188236,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "attentionImplementation": "eager",
      "loadMs": 844.9827910517342,
      "inferenceMs": 1137.6059169997461,
      "totalMs": 1982.5887080514804,
      "imagesPerSecond": 0.8790390284162202,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "attentionImplementation": "eager",
      "loadMs": 844.9827910517342,
      "inferenceMs": 1199.9884170363657,
      "totalMs": 2044.9712080881,
      "imagesPerSecond": 0.8333413771357219,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "attentionImplementation": "eager",
      "loadMs": 844.9827910517342,
      "inferenceMs": 1185.9302920056507,
      "totalMs": 2030.913083057385,
      "imagesPerSecond": 0.8432198812535562,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "attentionImplementation": "eager",
      "loadMs": 844.9827910517342,
      "inferenceMs": 1219.370125036221,
      "totalMs": 2064.3529160879552,
      "imagesPerSecond": 0.8200955390557033,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Native output for same fixture and configuration as spatial qualification"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
      "artifactRevision": "bb8de35cf2dc3827ea4a6ff06024752eeb18a37c",
      "cpuThreads": 4,
      "device": "mps",
      "durationMs": 1828.207458,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "family": {
        "catalogId": "eomt-native",
        "defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
        "displayName": "EoMT Desktop",
        "id": "eomt",
        "variantId": "ade20k-panoptic-giant-640",
        "variantName": "ADE20K Panoptic Giant 640"
      },
      "inferenceMs": 1815.219874959439,
      "loadMs": 6671.750083,
      "modelId": "eomt-ade20k-panoptic-giant-640-native",
      "parameterDtype": "torch.float16",
      "placement": {
        "mps:0": 746
      },
      "precision": "fp16",
      "predictionDevices": [
        "mps:0",
        "mps:0"
      ],
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "torchVersion": "2.11.0",
      "version": "4.57.6",
      "totalMs": 8499.957541,
      "imagesPerSecond": 0.5508974498322662,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Native output for same fixture and configuration as spatial qualification"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
      "artifactRevision": "bb8de35cf2dc3827ea4a6ff06024752eeb18a37c",
      "cpuThreads": 4,
      "device": "mps",
      "durationMs": 1227.761208,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "family": {
        "catalogId": "eomt-native",
        "defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
        "displayName": "EoMT Desktop",
        "id": "eomt",
        "variantId": "ade20k-panoptic-giant-640",
        "variantName": "ADE20K Panoptic Giant 640"
      },
      "inferenceMs": 1186.7824159562588,
      "loadMs": 6671.750083,
      "modelId": "eomt-ade20k-panoptic-giant-640-native",
      "parameterDtype": "torch.float16",
      "placement": {
        "mps:0": 746
      },
      "precision": "fp16",
      "predictionDevices": [
        "mps:0",
        "mps:0"
      ],
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "torchVersion": "2.11.0",
      "version": "4.57.6",
      "totalMs": 7899.511291,
      "imagesPerSecond": 0.8426144393066716,
      "grade": "moderate"
    }
  }
]

EoMT ADE20K Panoptic Large 1280 Desktop

26 configurations tested
Passed20
Failed6
TestedNov 14, 2026
Runtimetransformers 4.57.6

Quality criteria

No explicit quality gate recorded. Review per-case evidence; full certification is pending.

PrecisionDeviceResultPlatformInference / load
FP16mpsAcceleratedPassedModerateNot recorded3799 ms / 5017 ms
FP16mpsAcceleratedPassedModerateNot recorded4138 ms / 5017 ms
FP32mpsAcceleratedPassedSlowmacos · aarch643295 ms / 377 ms
FP32mpsAcceleratedPassedSlowmacos · aarch643391 ms / 377 ms
FP32mpsAcceleratedPassedSlowmacos · aarch643436 ms / 377 ms
FP32mpsAcceleratedPassedSlowmacos · aarch643458 ms / 377 ms
FP32mpsAcceleratedPassedSlowmacos · aarch643465 ms / 377 ms
BF16mpsAcceleratedPassedSlowmacos · aarch643706 ms / 2886 ms
FP32mpsAcceleratedPassedSlowmacos · aarch643806 ms / 377 ms
BF16mpsAcceleratedPassedSlowmacos · aarch643821 ms / 2886 ms
FP16mpsAcceleratedPassedSlowmacos · aarch643821 ms / 296 ms
FP16mpsAcceleratedPassedSlowmacos · aarch643866 ms / 296 ms
BF16mpsAcceleratedPassedSlowmacos · aarch643875 ms / 2886 ms
FP16mpsAcceleratedPassedSlowmacos · aarch643875 ms / 296 ms
BF16mpsAcceleratedPassedSlowmacos · aarch643892 ms / 2886 ms
FP16mpsAcceleratedPassedSlowmacos · aarch643904 ms / 296 ms
BF16mpsAcceleratedPassedSlowmacos · aarch643978 ms / 2886 ms
FP16mpsAcceleratedPassedSlowmacos · aarch643998 ms / 296 ms
BF16mpsAcceleratedPassedSlowmacos · aarch644109 ms / 2886 ms
FP16mpsAcceleratedPassedSlowmacos · aarch644192 ms / 296 ms
FP32cpuFailed—macos · aarch6414292 ms / 73 ms
FP32cpuFailed—macos · aarch6413663 ms / 73 ms
FP32cpuFailed—macos · aarch6413533 ms / 73 ms
FP32cpuFailed—macos · aarch6413746 ms / 73 ms
FP32cpuFailed—macos · aarch6414472 ms / 73 ms
FP32cpuFailed—macos · aarch6413916 ms / 73 ms
Test setup & sample details
Quality gate
No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
Tested
2026-11-14T21:44:25.405282+00:00
Environment
{ "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
Runtime
transformers 4.57.6
Artifact revision
6ffa3eea77f099f26d0541a86c7920d24cc743e6
Fixture
{
  "file": "model-samples/desktop/mobilenet-dog.jpg",
  "size": 83281,
  "sha1": "f1d042827c2f2154a6d8043b8e8f07c1be612941",
  "sha256": "d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1",
  "url": "https://assets.localai.tools/samples/v1/files/d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1/mobilenet-dog.jpg"
}
Rerun when
Checkpoint or runtime revision; Preprocessing or postprocessing; Fixture or quality gate
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 72.94116698903963,
      "inferenceMs": 14291.612625005655,
      "totalMs": 14364.553791994695,
      "imagesPerSecond": 0.06997111006565673,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 72.94116698903963,
      "inferenceMs": 13663.021667045541,
      "totalMs": 13735.96283403458,
      "imagesPerSecond": 0.0731902520810565,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": false,
      "pointRecall": null
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 72.94116698903963,
      "inferenceMs": 13532.808499992825,
      "totalMs": 13605.749666981865,
      "imagesPerSecond": 0.07389449130241739,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 72.94116698903963,
      "inferenceMs": 13745.805832964834,
      "totalMs": 13818.746999953873,
      "imagesPerSecond": 0.07274946352012525,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 72.94116698903963,
      "inferenceMs": 14471.914499998093,
      "totalMs": 14544.855666987132,
      "imagesPerSecond": 0.06909935793223017,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 72.94116698903963,
      "inferenceMs": 13915.923041000497,
      "totalMs": 13988.864207989536,
      "imagesPerSecond": 0.07186012721209359,
      "grade": "slow"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 2885.7585419900715,
      "inferenceMs": 4108.745916979387,
      "totalMs": 6994.504458969459,
      "imagesPerSecond": 0.2433832658932501,
      "grade": "slow"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 2885.7585419900715,
      "inferenceMs": 3820.54991699988,
      "totalMs": 6706.308458989952,
      "imagesPerSecond": 0.26174242497144456,
      "grade": "slow"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 2885.7585419900715,
      "inferenceMs": 3706.245334004052,
      "totalMs": 6592.0038759941235,
      "imagesPerSecond": 0.26981484221381735,
      "grade": "slow"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 2885.7585419900715,
      "inferenceMs": 3891.557708964683,
      "totalMs": 6777.316250954755,
      "imagesPerSecond": 0.256966509245482,
      "grade": "slow"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 2885.7585419900715,
      "inferenceMs": 3875.1974159968086,
      "totalMs": 6760.95595798688,
      "imagesPerSecond": 0.25805136942752943,
      "grade": "slow"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 2885.7585419900715,
      "inferenceMs": 3978.3199580269866,
      "totalMs": 6864.078500017058,
      "imagesPerSecond": 0.25136238677392386,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 295.57708295760676,
      "inferenceMs": 4192.050500016194,
      "totalMs": 4487.627582973801,
      "imagesPerSecond": 0.23854674460532788,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 295.57708295760676,
      "inferenceMs": 3903.579832985997,
      "totalMs": 4199.156915943604,
      "imagesPerSecond": 0.2561751117653105,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 295.57708295760676,
      "inferenceMs": 3820.678042015061,
      "totalMs": 4116.2551249726675,
      "imagesPerSecond": 0.26173364753670547,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 295.57708295760676,
      "inferenceMs": 3875.2151670050807,
      "totalMs": 4170.7922499626875,
      "imagesPerSecond": 0.2580501873842632,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 295.57708295760676,
      "inferenceMs": 3866.1919579608366,
      "totalMs": 4161.769040918443,
      "imagesPerSecond": 0.2586524442845912,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 295.57708295760676,
      "inferenceMs": 3997.9276669910178,
      "totalMs": 4293.5047499486245,
      "imagesPerSecond": 0.2501295879504082,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 376.562167017255,
      "inferenceMs": 3806.1767920153216,
      "totalMs": 4182.7389590325765,
      "imagesPerSecond": 0.26273083323344865,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 376.562167017255,
      "inferenceMs": 3464.961250021588,
      "totalMs": 3841.523417038843,
      "imagesPerSecond": 0.28860351612698976,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 376.562167017255,
      "inferenceMs": 3295.2619590214454,
      "totalMs": 3671.8241260387003,
      "imagesPerSecond": 0.3034660104221147,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 376.562167017255,
      "inferenceMs": 3391.2080000154674,
      "totalMs": 3767.7701670327224,
      "imagesPerSecond": 0.29488017249176074,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 376.562167017255,
      "inferenceMs": 3436.339041974861,
      "totalMs": 3812.901208992116,
      "imagesPerSecond": 0.2910073737733692,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 376.562167017255,
      "inferenceMs": 3458.3000410348177,
      "totalMs": 3834.8622080520727,
      "imagesPerSecond": 0.28915941015365826,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Native output for same fixture and configuration as spatial qualification"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
      "artifactRevision": "6ffa3eea77f099f26d0541a86c7920d24cc743e6",
      "cpuThreads": 4,
      "device": "mps",
      "durationMs": 4151.007541,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "family": {
        "catalogId": "eomt-native",
        "defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
        "displayName": "EoMT Desktop",
        "id": "eomt",
        "variantId": "ade20k-panoptic-large-1280",
        "variantName": "ADE20K Panoptic Large 1280"
      },
      "inferenceMs": 4138.230875018053,
      "loadMs": 5017.085416,
      "modelId": "eomt-ade20k-panoptic-large-1280-native",
      "parameterDtype": "torch.float16",
      "placement": {
        "mps:0": 458
      },
      "precision": "fp16",
      "predictionDevices": [
        "mps:0",
        "mps:0"
      ],
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "torchVersion": "2.11.0",
      "version": "4.57.6",
      "totalMs": 9168.092957,
      "imagesPerSecond": 0.24164915641533355,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Native output for same fixture and configuration as spatial qualification"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
      "artifactRevision": "6ffa3eea77f099f26d0541a86c7920d24cc743e6",
      "cpuThreads": 4,
      "device": "mps",
      "durationMs": 3842.434875,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "family": {
        "catalogId": "eomt-native",
        "defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
        "displayName": "EoMT Desktop",
        "id": "eomt",
        "variantId": "ade20k-panoptic-large-1280",
        "variantName": "ADE20K Panoptic Large 1280"
      },
      "inferenceMs": 3798.851499974262,
      "loadMs": 5017.085416,
      "modelId": "eomt-ade20k-panoptic-large-1280-native",
      "parameterDtype": "torch.float16",
      "placement": {
        "mps:0": 458
      },
      "precision": "fp16",
      "predictionDevices": [
        "mps:0",
        "mps:0"
      ],
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "torchVersion": "2.11.0",
      "version": "4.57.6",
      "totalMs": 8859.520291,
      "imagesPerSecond": 0.26323745479568633,
      "grade": "moderate"
    }
  }
]

EoMT ADE20K Panoptic Large 640 Desktop

26 configurations tested
Passed26
Failed0
TestedNov 14, 2026
Runtimetransformers 4.57.6

Quality criteria

No explicit quality gate recorded. Review per-case evidence; full certification is pending.

PrecisionDeviceResultPlatformInference / load
FP32mpsAcceleratedPassedFastmacos · aarch64389 ms / 405 ms
BF16mpsAcceleratedPassedFastmacos · aarch64398 ms / 291 ms
FP16mpsAcceleratedPassedFastmacos · aarch64398 ms / 275 ms
BF16mpsAcceleratedPassedFastmacos · aarch64433 ms / 291 ms
FP16mpsAcceleratedPassedFastmacos · aarch64437 ms / 275 ms
FP32mpsAcceleratedPassedFastmacos · aarch64444 ms / 405 ms
FP16mpsAcceleratedPassedFastNot recorded446 ms / 4795 ms
FP32mpsAcceleratedPassedFastmacos · aarch64504 ms / 405 ms
BF16mpsAcceleratedPassedFastmacos · aarch64505 ms / 291 ms
FP32mpsAcceleratedPassedFastmacos · aarch64506 ms / 405 ms
FP32mpsAcceleratedPassedFastmacos · aarch64508 ms / 405 ms
BF16mpsAcceleratedPassedFastmacos · aarch64510 ms / 291 ms
FP16mpsAcceleratedPassedFastmacos · aarch64515 ms / 275 ms
FP16mpsAcceleratedPassedFastmacos · aarch64516 ms / 275 ms
BF16mpsAcceleratedPassedFastmacos · aarch64516 ms / 291 ms
FP16mpsAcceleratedPassedFastmacos · aarch64517 ms / 275 ms
FP16mpsAcceleratedPassedFastmacos · aarch64667 ms / 275 ms
BF16mpsAcceleratedPassedFastmacos · aarch64672 ms / 291 ms
FP16mpsAcceleratedPassedFastNot recorded723 ms / 4795 ms
FP32mpsAcceleratedPassedModeratemacos · aarch641062 ms / 405 ms
FP32cpuPassedModeratemacos · aarch641928 ms / 71 ms
FP32cpuPassedModeratemacos · aarch642011 ms / 71 ms
FP32cpuPassedModeratemacos · aarch642016 ms / 71 ms
FP32cpuPassedModeratemacos · aarch642020 ms / 71 ms
FP32cpuPassedModeratemacos · aarch642074 ms / 71 ms
FP32cpuPassedModeratemacos · aarch642843 ms / 71 ms
Test setup & sample details
Quality gate
No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
Tested
2026-11-14T21:36:13.706942+00:00
Environment
{ "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
Runtime
transformers 4.57.6
Artifact revision
4d0ace83675d88497d6c4a10e17a0f46047685a5
Fixture
{
  "file": "model-samples/desktop/mobilenet-dog.jpg",
  "size": 83281,
  "sha1": "f1d042827c2f2154a6d8043b8e8f07c1be612941",
  "sha256": "d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1",
  "url": "https://assets.localai.tools/samples/v1/files/d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1/mobilenet-dog.jpg"
}
Rerun when
Checkpoint or runtime revision; Preprocessing or postprocessing; Fixture or quality gate
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 70.96766703762114,
      "inferenceMs": 2843.483416014351,
      "totalMs": 2914.451083051972,
      "imagesPerSecond": 0.35168131959836724,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 70.96766703762114,
      "inferenceMs": 1927.7984999935143,
      "totalMs": 1998.7661670311354,
      "imagesPerSecond": 0.5187264125391551,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 70.96766703762114,
      "inferenceMs": 2074.122958991211,
      "totalMs": 2145.090626028832,
      "imagesPerSecond": 0.4821314935380538,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 70.96766703762114,
      "inferenceMs": 2019.82250000583,
      "totalMs": 2090.790167043451,
      "imagesPerSecond": 0.4950930094090513,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 70.96766703762114,
      "inferenceMs": 2016.0378749715164,
      "totalMs": 2087.0055420091376,
      "imagesPerSecond": 0.4960224271650296,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 70.96766703762114,
      "inferenceMs": 2010.5368749937043,
      "totalMs": 2081.5045420313254,
      "imagesPerSecond": 0.4973795867350761,
      "grade": "moderate"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 290.59691599104553,
      "inferenceMs": 672.252209042199,
      "totalMs": 962.8491250332445,
      "imagesPerSecond": 1.48753694900425,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 290.59691599104553,
      "inferenceMs": 433.39750001905486,
      "totalMs": 723.9944160101004,
      "imagesPerSecond": 2.3073506422072896,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 290.59691599104553,
      "inferenceMs": 397.66716602025554,
      "totalMs": 688.2640820113011,
      "imagesPerSecond": 2.5146657442396543,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 290.59691599104553,
      "inferenceMs": 510.4521670145914,
      "totalMs": 801.0490830056369,
      "imagesPerSecond": 1.9590474183870292,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 290.59691599104553,
      "inferenceMs": 505.2602499490604,
      "totalMs": 795.8571659401059,
      "imagesPerSecond": 1.9791780574482527,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 290.59691599104553,
      "inferenceMs": 515.7839999883436,
      "totalMs": 806.3809159793891,
      "imagesPerSecond": 1.9387960852267605,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 275.2712499932386,
      "inferenceMs": 667.0129169942811,
      "totalMs": 942.2841669875197,
      "imagesPerSecond": 1.4992213411791753,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 275.2712499932386,
      "inferenceMs": 436.6954590077512,
      "totalMs": 711.9667090009898,
      "imagesPerSecond": 2.289925345851261,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 275.2712499932386,
      "inferenceMs": 398.14145799027756,
      "totalMs": 673.4127079835162,
      "imagesPerSecond": 2.5116701110398294,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 275.2712499932386,
      "inferenceMs": 516.6843329789117,
      "totalMs": 791.9555829721503,
      "imagesPerSecond": 1.9354176934968428,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 275.2712499932386,
      "inferenceMs": 515.7011250266805,
      "totalMs": 790.9723750199191,
      "imagesPerSecond": 1.9391076564904983,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 275.2712499932386,
      "inferenceMs": 515.2240410097875,
      "totalMs": 790.4952910030261,
      "imagesPerSecond": 1.9409032195782252,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 405.02095804549754,
      "inferenceMs": 1062.4704170040786,
      "totalMs": 1467.4913750495762,
      "imagesPerSecond": 0.9412026763246446,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 405.02095804549754,
      "inferenceMs": 444.40691702766344,
      "totalMs": 849.427875073161,
      "imagesPerSecond": 2.2501899985903053,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 405.02095804549754,
      "inferenceMs": 389.44295898545533,
      "totalMs": 794.4639170309529,
      "imagesPerSecond": 2.567770136620566,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 405.02095804549754,
      "inferenceMs": 507.93833303032443,
      "totalMs": 912.959291075822,
      "imagesPerSecond": 1.9687429260045608,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 405.02095804549754,
      "inferenceMs": 506.41833298141137,
      "totalMs": 911.4392910269089,
      "imagesPerSecond": 1.974652051225614,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 405.02095804549754,
      "inferenceMs": 504.480124975089,
      "totalMs": 909.5010830205865,
      "imagesPerSecond": 1.9822386462685315,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Native output for same fixture and configuration as spatial qualification"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
      "artifactRevision": "4d0ace83675d88497d6c4a10e17a0f46047685a5",
      "cpuThreads": 4,
      "device": "mps",
      "durationMs": 736.433041,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "family": {
        "catalogId": "eomt-native",
        "defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
        "displayName": "EoMT Desktop",
        "id": "eomt",
        "variantId": "ade20k-panoptic-large-640",
        "variantName": "ADE20K Panoptic Large 640"
      },
      "inferenceMs": 723.2872089953162,
      "loadMs": 4795.383291,
      "modelId": "eomt-ade20k-panoptic-large-640-native",
      "parameterDtype": "torch.float16",
      "placement": {
        "mps:0": 458
      },
      "precision": "fp16",
      "predictionDevices": [
        "mps:0",
        "mps:0"
      ],
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "torchVersion": "2.11.0",
      "version": "4.57.6",
      "totalMs": 5531.816332,
      "imagesPerSecond": 1.3825766411506881,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Native output for same fixture and configuration as spatial qualification"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
      "artifactRevision": "4d0ace83675d88497d6c4a10e17a0f46047685a5",
      "cpuThreads": 4,
      "device": "mps",
      "durationMs": 486.724625,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "family": {
        "catalogId": "eomt-native",
        "defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
        "displayName": "EoMT Desktop",
        "id": "eomt",
        "variantId": "ade20k-panoptic-large-640",
        "variantName": "ADE20K Panoptic Large 640"
      },
      "inferenceMs": 446.42116699833423,
      "loadMs": 4795.383291,
      "modelId": "eomt-ade20k-panoptic-large-640-native",
      "parameterDtype": "torch.float16",
      "placement": {
        "mps:0": 458
      },
      "precision": "fp16",
      "predictionDevices": [
        "mps:0",
        "mps:0"
      ],
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "torchVersion": "2.11.0",
      "version": "4.57.6",
      "totalMs": 5282.107916,
      "imagesPerSecond": 2.2400371530853764,
      "grade": "fast"
    }
  }
]

EoMT ADE20K Semantic Large 512 Desktop

26 configurations tested
Passed26
Failed0
TestedNov 14, 2026
Runtimetransformers 4.57.6

Quality criteria

No explicit quality gate recorded. Review per-case evidence; full certification is pending.

PrecisionDeviceResultPlatformInference / load
BF16mpsAcceleratedPassedFastmacos · aarch64318 ms / 278 ms
FP16mpsAcceleratedPassedFastmacos · aarch64335 ms / 293 ms
FP32mpsAcceleratedPassedFastmacos · aarch64340 ms / 456 ms
FP16mpsAcceleratedPassedFastNot recorded377 ms / 3123 ms
BF16mpsAcceleratedPassedFastmacos · aarch64404 ms / 278 ms
FP16mpsAcceleratedPassedFastmacos · aarch64407 ms / 293 ms
BF16mpsAcceleratedPassedFastmacos · aarch64625 ms / 278 ms
BF16mpsAcceleratedPassedFastmacos · aarch64630 ms / 278 ms
FP32mpsAcceleratedPassedFastmacos · aarch64636 ms / 456 ms
BF16mpsAcceleratedPassedFastmacos · aarch64640 ms / 278 ms
FP16mpsAcceleratedPassedFastmacos · aarch64654 ms / 293 ms
FP16mpsAcceleratedPassedFastmacos · aarch64661 ms / 293 ms
FP16mpsAcceleratedPassedFastmacos · aarch64662 ms / 293 ms
FP32mpsAcceleratedPassedFastmacos · aarch64677 ms / 456 ms
FP32mpsAcceleratedPassedFastmacos · aarch64692 ms / 456 ms
FP16mpsAcceleratedPassedFastNot recorded754 ms / 3123 ms
BF16mpsAcceleratedPassedFastmacos · aarch64781 ms / 278 ms
FP16mpsAcceleratedPassedFastmacos · aarch64799 ms / 293 ms
FP32mpsAcceleratedPassedModeratemacos · aarch641205 ms / 456 ms
FP32mpsAcceleratedPassedModeratemacos · aarch642133 ms / 456 ms
FP32cpuPassedModeratemacos · aarch641544 ms / 47 ms
FP32cpuPassedModeratemacos · aarch641580 ms / 47 ms
FP32cpuPassedSlowmacos · aarch643411 ms / 47 ms
FP32cpuPassedSlowmacos · aarch643468 ms / 47 ms
FP32cpuPassedSlowmacos · aarch643613 ms / 47 ms
FP32cpuPassedSlowmacos · aarch643759 ms / 47 ms
Test setup & sample details
Quality gate
No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
Tested
2026-11-14T21:19:15.498340+00:00
Environment
{ "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
Runtime
transformers 4.57.6
Artifact revision
aeb262f5a0bc609236d27f1277c3173ce57aeb27
Fixture
{
  "file": "model-samples/desktop/mobilenet-dog.jpg",
  "size": 83281,
  "sha1": "f1d042827c2f2154a6d8043b8e8f07c1be612941",
  "sha256": "d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1",
  "url": "https://assets.localai.tools/samples/v1/files/d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1/mobilenet-dog.jpg"
}
Rerun when
Checkpoint or runtime revision; Preprocessing or postprocessing; Fixture or quality gate
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 46.91187501884997,
      "inferenceMs": 3467.647791025229,
      "totalMs": 3514.5596660440788,
      "imagesPerSecond": 0.2883799221443838,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 46.91187501884997,
      "inferenceMs": 1579.9649159889668,
      "totalMs": 1626.8767910078168,
      "imagesPerSecond": 0.6329254465590824,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 46.91187501884997,
      "inferenceMs": 1544.0384169924073,
      "totalMs": 1590.9502920112573,
      "imagesPerSecond": 0.6476522792404831,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 46.91187501884997,
      "inferenceMs": 3613.4377499693073,
      "totalMs": 3660.3496249881573,
      "imagesPerSecond": 0.2767447702699442,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 46.91187501884997,
      "inferenceMs": 3410.6579999788664,
      "totalMs": 3457.5698749977164,
      "imagesPerSecond": 0.2931985558230102,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 46.91187501884997,
      "inferenceMs": 3759.0924999676645,
      "totalMs": 3806.0043749865144,
      "imagesPerSecond": 0.2660216528347206,
      "grade": "slow"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 277.66599995084107,
      "inferenceMs": 781.2259580241516,
      "totalMs": 1058.8919579749927,
      "imagesPerSecond": 1.280039391585456,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 277.66599995084107,
      "inferenceMs": 404.2990840389393,
      "totalMs": 681.9650839897804,
      "imagesPerSecond": 2.4734164371831393,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 277.66599995084107,
      "inferenceMs": 318.1022500502877,
      "totalMs": 595.7682500011288,
      "imagesPerSecond": 3.1436432777256793,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 277.66599995084107,
      "inferenceMs": 625.0824580201879,
      "totalMs": 902.748457971029,
      "imagesPerSecond": 1.5997889353146806,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 277.66599995084107,
      "inferenceMs": 640.008290996775,
      "totalMs": 917.6742909476161,
      "imagesPerSecond": 1.562479758570876,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 277.66599995084107,
      "inferenceMs": 629.6731249894947,
      "totalMs": 907.3391249403358,
      "imagesPerSecond": 1.5881255850274438,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 293.33683400182053,
      "inferenceMs": 799.3859160342254,
      "totalMs": 1092.722750036046,
      "imagesPerSecond": 1.2509602432840279,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 293.33683400182053,
      "inferenceMs": 407.45595801854506,
      "totalMs": 700.7927920203656,
      "imagesPerSecond": 2.4542529819983288,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 293.33683400182053,
      "inferenceMs": 335.1195409777574,
      "totalMs": 628.456374979578,
      "imagesPerSecond": 2.9840098165638516,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 293.33683400182053,
      "inferenceMs": 662.3322919476777,
      "totalMs": 955.6691259494983,
      "imagesPerSecond": 1.5098161635141851,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 293.33683400182053,
      "inferenceMs": 653.8968330132775,
      "totalMs": 947.233667015098,
      "imagesPerSecond": 1.529293230236053,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 293.33683400182053,
      "inferenceMs": 660.9414169797674,
      "totalMs": 954.2782509815879,
      "imagesPerSecond": 1.5129933974626557,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 456.00425003794953,
      "inferenceMs": 2132.687708013691,
      "totalMs": 2588.6919580516405,
      "imagesPerSecond": 0.4688919039775234,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 456.00425003794953,
      "inferenceMs": 1204.5093339984305,
      "totalMs": 1660.51358403638,
      "imagesPerSecond": 0.83021357475118,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 456.00425003794953,
      "inferenceMs": 340.41054197587073,
      "totalMs": 796.4147920138203,
      "imagesPerSecond": 2.9376293524742922,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 456.00425003794953,
      "inferenceMs": 692.4639160279185,
      "totalMs": 1148.468166065868,
      "imagesPerSecond": 1.4441185697244077,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 456.00425003794953,
      "inferenceMs": 677.0300409989432,
      "totalMs": 1133.0342910368927,
      "imagesPerSecond": 1.4770393327370255,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 456.00425003794953,
      "inferenceMs": 635.7941669994034,
      "totalMs": 1091.798417037353,
      "imagesPerSecond": 1.5728360716479148,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Native output for same fixture and configuration as spatial qualification"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
      "artifactRevision": "aeb262f5a0bc609236d27f1277c3173ce57aeb27",
      "cpuThreads": 4,
      "device": "mps",
      "durationMs": 764.852875,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "family": {
        "catalogId": "eomt-native",
        "defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
        "displayName": "EoMT Desktop",
        "id": "eomt",
        "variantId": "ade20k-semantic-large-512",
        "variantName": "ADE20K Semantic Large 512"
      },
      "inferenceMs": 753.8305000052787,
      "loadMs": 3123.454625,
      "modelId": "eomt-ade20k-semantic-large-512-native",
      "parameterDtype": "torch.float16",
      "placement": {
        "mps:0": 458
      },
      "precision": "fp16",
      "predictionDevices": [
        "mps:0",
        "mps:0"
      ],
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "torchVersion": "2.11.0",
      "version": "4.57.6",
      "totalMs": 3888.3075,
      "imagesPerSecond": 1.3265581586218618,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Native output for same fixture and configuration as spatial qualification"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
      "artifactRevision": "aeb262f5a0bc609236d27f1277c3173ce57aeb27",
      "cpuThreads": 4,
      "device": "mps",
      "durationMs": 420.436417,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "family": {
        "catalogId": "eomt-native",
        "defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
        "displayName": "EoMT Desktop",
        "id": "eomt",
        "variantId": "ade20k-semantic-large-512",
        "variantName": "ADE20K Semantic Large 512"
      },
      "inferenceMs": 377.3945000139065,
      "loadMs": 3123.454625,
      "modelId": "eomt-ade20k-semantic-large-512-native",
      "parameterDtype": "torch.float16",
      "placement": {
        "mps:0": 458
      },
      "precision": "fp16",
      "predictionDevices": [
        "mps:0",
        "mps:0"
      ],
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "torchVersion": "2.11.0",
      "version": "4.57.6",
      "totalMs": 3543.8910419999997,
      "imagesPerSecond": 2.649747147780774,
      "grade": "fast"
    }
  }
]

EoMT CITYSCAPES Semantic Large 1024 Desktop

26 configurations tested
Passed20
Failed6
TestedNov 14, 2026
Runtimetransformers 4.57.6

Quality criteria

No explicit quality gate recorded. Review per-case evidence; full certification is pending.

PrecisionDeviceResultPlatformInference / load
FP32mpsAcceleratedPassedModeratemacos · aarch641508 ms / 339 ms
FP32mpsAcceleratedPassedModeratemacos · aarch641550 ms / 339 ms
FP16mpsAcceleratedPassedModeratemacos · aarch641683 ms / 279 ms
FP16mpsAcceleratedPassedModerateNot recorded1700 ms / 3339 ms
BF16mpsAcceleratedPassedModeratemacos · aarch641700 ms / 270 ms
BF16mpsAcceleratedPassedModeratemacos · aarch641708 ms / 270 ms
FP16mpsAcceleratedPassedModeratemacos · aarch641717 ms / 279 ms
FP16mpsAcceleratedPassedModerateNot recorded3748 ms / 3339 ms
FP32mpsAcceleratedPassedSlowmacos · aarch643149 ms / 339 ms
FP32mpsAcceleratedPassedSlowmacos · aarch643180 ms / 339 ms
FP32mpsAcceleratedPassedSlowmacos · aarch643195 ms / 339 ms
FP16mpsAcceleratedPassedSlowmacos · aarch643392 ms / 279 ms
FP32mpsAcceleratedPassedSlowmacos · aarch643400 ms / 339 ms
BF16mpsAcceleratedPassedSlowmacos · aarch643416 ms / 270 ms
FP16mpsAcceleratedPassedSlowmacos · aarch643416 ms / 279 ms
BF16mpsAcceleratedPassedSlowmacos · aarch643449 ms / 270 ms
FP16mpsAcceleratedPassedSlowmacos · aarch643460 ms / 279 ms
BF16mpsAcceleratedPassedSlowmacos · aarch643481 ms / 270 ms
BF16mpsAcceleratedPassedSlowmacos · aarch643738 ms / 270 ms
FP16mpsAcceleratedPassedSlowmacos · aarch643807 ms / 279 ms
FP32cpuFailed—macos · aarch6415971 ms / 75 ms
FP32cpuFailed—macos · aarch646419 ms / 75 ms
FP32cpuFailed—macos · aarch646420 ms / 75 ms
FP32cpuFailed—macos · aarch6415741 ms / 75 ms
FP32cpuFailed—macos · aarch6415567 ms / 75 ms
FP32cpuFailed—macos · aarch6415231 ms / 75 ms
Test setup & sample details
Quality gate
No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
Tested
2026-11-14T21:51:44.238813+00:00
Environment
{ "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
Runtime
transformers 4.57.6
Artifact revision
8d6b6d1a3f7b50d441afd7d247c2ed10db186e8f
Fixture
{
  "file": "model-samples/desktop/mobilenet-dog.jpg",
  "size": 83281,
  "sha1": "f1d042827c2f2154a6d8043b8e8f07c1be612941",
  "sha256": "d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1",
  "url": "https://assets.localai.tools/samples/v1/files/d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1/mobilenet-dog.jpg"
}
Rerun when
Checkpoint or runtime revision; Preprocessing or postprocessing; Fixture or quality gate
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 74.86766699003056,
      "inferenceMs": 15971.039334021043,
      "totalMs": 16045.907001011074,
      "imagesPerSecond": 0.06261333273845422,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 74.86766699003056,
      "inferenceMs": 6419.192541972734,
      "totalMs": 6494.060208962765,
      "imagesPerSecond": 0.1557828330372346,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": false,
      "pointRecall": null
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 74.86766699003056,
      "inferenceMs": 6419.963833002839,
      "totalMs": 6494.83149999287,
      "imagesPerSecond": 0.15576411737077736,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 74.86766699003056,
      "inferenceMs": 15741.11304199323,
      "totalMs": 15815.980708983261,
      "imagesPerSecond": 0.06352790919754263,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 74.86766699003056,
      "inferenceMs": 15566.559666011017,
      "totalMs": 15641.427333001047,
      "imagesPerSecond": 0.06424027026237926,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 74.86766699003056,
      "inferenceMs": 15231.452291016467,
      "totalMs": 15306.319958006497,
      "imagesPerSecond": 0.06565362126300993,
      "grade": "slow"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 269.6552079869434,
      "inferenceMs": 3738.159666012507,
      "totalMs": 4007.8148739994504,
      "imagesPerSecond": 0.2675113128773067,
      "grade": "slow"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 269.6552079869434,
      "inferenceMs": 1699.8922500060871,
      "totalMs": 1969.5474579930305,
      "imagesPerSecond": 0.5882725802158455,
      "grade": "moderate"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 269.6552079869434,
      "inferenceMs": 1708.4945410024375,
      "totalMs": 1978.149748989381,
      "imagesPerSecond": 0.5853106205497518,
      "grade": "moderate"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 269.6552079869434,
      "inferenceMs": 3481.155708024744,
      "totalMs": 3750.8109160116874,
      "imagesPerSecond": 0.28726092248468077,
      "grade": "slow"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 269.6552079869434,
      "inferenceMs": 3449.4774999911897,
      "totalMs": 3719.132707978133,
      "imagesPerSecond": 0.2898989774545722,
      "grade": "slow"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 269.6552079869434,
      "inferenceMs": 3415.9150830237195,
      "totalMs": 3685.570291010663,
      "imagesPerSecond": 0.29274732412692595,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 279.33120797388256,
      "inferenceMs": 3806.640749971848,
      "totalMs": 4085.9719579457305,
      "imagesPerSecond": 0.2626988112832543,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 279.33120797388256,
      "inferenceMs": 1717.1941670239903,
      "totalMs": 1996.5253749978729,
      "imagesPerSecond": 0.5823453277465211,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 279.33120797388256,
      "inferenceMs": 1682.9552080016583,
      "totalMs": 1962.2864159755409,
      "imagesPerSecond": 0.5941928788392415,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 279.33120797388256,
      "inferenceMs": 3459.8387500154786,
      "totalMs": 3739.169957989361,
      "imagesPerSecond": 0.28903081104445294,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 279.33120797388256,
      "inferenceMs": 3392.320708022453,
      "totalMs": 3671.6519159963354,
      "imagesPerSecond": 0.29478344946428964,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 279.33120797388256,
      "inferenceMs": 3416.326874983497,
      "totalMs": 3695.6580829573795,
      "imagesPerSecond": 0.2927120373997674,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 338.5887499898672,
      "inferenceMs": 3399.805249995552,
      "totalMs": 3738.393999985419,
      "imagesPerSecond": 0.2941344949100565,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 338.5887499898672,
      "inferenceMs": 1549.7517080511898,
      "totalMs": 1888.340458041057,
      "imagesPerSecond": 0.6452646542054781,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 338.5887499898672,
      "inferenceMs": 1508.0046249786392,
      "totalMs": 1846.5933749685064,
      "imagesPerSecond": 0.6631279396866339,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 338.5887499898672,
      "inferenceMs": 3194.946291972883,
      "totalMs": 3533.5350419627503,
      "imagesPerSecond": 0.31299430682526397,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 338.5887499898672,
      "inferenceMs": 3148.639541992452,
      "totalMs": 3487.228291982319,
      "imagesPerSecond": 0.3175974850926258,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 338.5887499898672,
      "inferenceMs": 3180.060707964003,
      "totalMs": 3518.6494579538703,
      "imagesPerSecond": 0.31445940560054225,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Native output for same fixture and configuration as spatial qualification"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
      "artifactRevision": "8d6b6d1a3f7b50d441afd7d247c2ed10db186e8f",
      "cpuThreads": 4,
      "device": "mps",
      "durationMs": 3759.5588749999997,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "family": {
        "catalogId": "eomt-native",
        "defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
        "displayName": "EoMT Desktop",
        "id": "eomt",
        "variantId": "cityscapes-semantic-large-1024",
        "variantName": "CITYSCAPES Semantic Large 1024"
      },
      "inferenceMs": 3747.7345000370406,
      "loadMs": 3339.24625,
      "modelId": "eomt-cityscapes-semantic-large-1024-native",
      "parameterDtype": "torch.float16",
      "placement": {
        "mps:0": 458
      },
      "precision": "fp16",
      "predictionDevices": [
        "mps:0",
        "mps:0"
      ],
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "torchVersion": "2.11.0",
      "version": "4.57.6",
      "totalMs": 7098.805125,
      "imagesPerSecond": 0.26682786627230837,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Native output for same fixture and configuration as spatial qualification"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
      "artifactRevision": "8d6b6d1a3f7b50d441afd7d247c2ed10db186e8f",
      "cpuThreads": 4,
      "device": "mps",
      "durationMs": 1739.9657080000002,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "family": {
        "catalogId": "eomt-native",
        "defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
        "displayName": "EoMT Desktop",
        "id": "eomt",
        "variantId": "cityscapes-semantic-large-1024",
        "variantName": "CITYSCAPES Semantic Large 1024"
      },
      "inferenceMs": 1699.535499967169,
      "loadMs": 3339.24625,
      "modelId": "eomt-cityscapes-semantic-large-1024-native",
      "parameterDtype": "torch.float16",
      "placement": {
        "mps:0": 458
      },
      "precision": "fp16",
      "predictionDevices": [
        "mps:0",
        "mps:0"
      ],
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "torchVersion": "2.11.0",
      "version": "4.57.6",
      "totalMs": 5079.211958,
      "imagesPerSecond": 0.5883960647008065,
      "grade": "moderate"
    }
  }
]

EoMT COCO Instance Large 1280 Desktop

26 configurations tested
Passed20
Failed6
TestedNov 14, 2026
Runtimetransformers 4.57.6

Quality criteria

No explicit quality gate recorded. Review per-case evidence; full certification is pending.

PrecisionDeviceResultPlatformInference / load
FP16mpsAcceleratedPassedModerateNot recorded3946 ms / 5072 ms
FP16mpsAcceleratedPassedModerateNot recorded4364 ms / 5072 ms
FP32mpsAcceleratedPassedSlowmacos · aarch643377 ms / 352 ms
FP32mpsAcceleratedPassedSlowmacos · aarch643379 ms / 352 ms
FP32mpsAcceleratedPassedSlowmacos · aarch643381 ms / 352 ms
FP32mpsAcceleratedPassedSlowmacos · aarch643398 ms / 352 ms
FP32mpsAcceleratedPassedSlowmacos · aarch643529 ms / 352 ms
FP32mpsAcceleratedPassedSlowmacos · aarch643736 ms / 352 ms
FP16mpsAcceleratedPassedSlowmacos · aarch643803 ms / 275 ms
FP16mpsAcceleratedPassedSlowmacos · aarch643825 ms / 275 ms
FP16mpsAcceleratedPassedSlowmacos · aarch643828 ms / 275 ms
BF16mpsAcceleratedPassedSlowmacos · aarch643864 ms / 269 ms
BF16mpsAcceleratedPassedSlowmacos · aarch643890 ms / 269 ms
BF16mpsAcceleratedPassedSlowmacos · aarch643900 ms / 269 ms
BF16mpsAcceleratedPassedSlowmacos · aarch643926 ms / 269 ms
FP16mpsAcceleratedPassedSlowmacos · aarch643971 ms / 275 ms
BF16mpsAcceleratedPassedSlowmacos · aarch644021 ms / 269 ms
FP16mpsAcceleratedPassedSlowmacos · aarch644031 ms / 275 ms
BF16mpsAcceleratedPassedSlowmacos · aarch644153 ms / 269 ms
FP16mpsAcceleratedPassedSlowmacos · aarch644158 ms / 275 ms
FP32cpuFailed—macos · aarch6413996 ms / 65 ms
FP32cpuFailed—macos · aarch6413407 ms / 65 ms
FP32cpuFailed—macos · aarch6413596 ms / 65 ms
FP32cpuFailed—macos · aarch6413492 ms / 65 ms
FP32cpuFailed—macos · aarch6413487 ms / 65 ms
FP32cpuFailed—macos · aarch6413694 ms / 65 ms
Test setup & sample details
Quality gate
No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
Tested
2026-11-14T21:32:39.471043+00:00
Environment
{ "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
Runtime
transformers 4.57.6
Artifact revision
057a34b54f9f1cf5395de72edaeaefca43e66494
Fixture
{
  "file": "model-samples/desktop/mobilenet-dog.jpg",
  "size": 83281,
  "sha1": "f1d042827c2f2154a6d8043b8e8f07c1be612941",
  "sha256": "d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1",
  "url": "https://assets.localai.tools/samples/v1/files/d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1/mobilenet-dog.jpg"
}
Rerun when
Checkpoint or runtime revision; Preprocessing or postprocessing; Fixture or quality gate
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 64.69133298378438,
      "inferenceMs": 13995.703165943269,
      "totalMs": 14060.394498927053,
      "imagesPerSecond": 0.07145050078179498,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 64.69133298378438,
      "inferenceMs": 13407.404666999355,
      "totalMs": 13472.09599998314,
      "imagesPerSecond": 0.07458565060405573,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": false,
      "pointRecall": null
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 64.69133298378438,
      "inferenceMs": 13595.744749996811,
      "totalMs": 13660.436082980596,
      "imagesPerSecond": 0.07355242529102604,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 64.69133298378438,
      "inferenceMs": 13491.55170796439,
      "totalMs": 13556.243040948175,
      "imagesPerSecond": 0.07412045861334658,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 64.69133298378438,
      "inferenceMs": 13486.85204202775,
      "totalMs": 13551.543375011533,
      "imagesPerSecond": 0.0741462868343034,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 64.69133298378438,
      "inferenceMs": 13693.713375017978,
      "totalMs": 13758.404708001763,
      "imagesPerSecond": 0.07302621083221607,
      "grade": "slow"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 269.2592079984024,
      "inferenceMs": 4153.020208002999,
      "totalMs": 4422.279416001402,
      "imagesPerSecond": 0.24078861886416272,
      "grade": "slow"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 269.2592079984024,
      "inferenceMs": 4020.541208970826,
      "totalMs": 4289.8004169692285,
      "imagesPerSecond": 0.24872273358839145,
      "grade": "slow"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 269.2592079984024,
      "inferenceMs": 3889.7433749516495,
      "totalMs": 4159.002582950052,
      "imagesPerSecond": 0.2570863688436593,
      "grade": "slow"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 269.2592079984024,
      "inferenceMs": 3925.9483750211075,
      "totalMs": 4195.20758301951,
      "imagesPerSecond": 0.2547155246264856,
      "grade": "slow"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 269.2592079984024,
      "inferenceMs": 3900.380084000062,
      "totalMs": 4169.639291998465,
      "imagesPerSecond": 0.25638526975925974,
      "grade": "slow"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 269.2592079984024,
      "inferenceMs": 3863.787041977048,
      "totalMs": 4133.04624997545,
      "imagesPerSecond": 0.25881343592071093,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 274.85849999357015,
      "inferenceMs": 4031.470707966946,
      "totalMs": 4306.329207960516,
      "imagesPerSecond": 0.24804843503484014,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 274.85849999357015,
      "inferenceMs": 3971.031333960127,
      "totalMs": 4245.889833953697,
      "imagesPerSecond": 0.25182374952522624,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 274.85849999357015,
      "inferenceMs": 3824.672083952464,
      "totalMs": 4099.530583946034,
      "imagesPerSecond": 0.2614603234080626,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 274.85849999357015,
      "inferenceMs": 3827.7053750352934,
      "totalMs": 4102.563875028864,
      "imagesPerSecond": 0.26125312740163015,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 274.85849999357015,
      "inferenceMs": 3803.3817499526776,
      "totalMs": 4078.240249946248,
      "imagesPerSecond": 0.26292390975805735,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 274.85849999357015,
      "inferenceMs": 4158.285583020188,
      "totalMs": 4433.144083013758,
      "imagesPerSecond": 0.24048372340836052,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 351.8199999816716,
      "inferenceMs": 3735.617583966814,
      "totalMs": 4087.4375839484856,
      "imagesPerSecond": 0.2676933539160907,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 351.8199999816716,
      "inferenceMs": 3529.098958999384,
      "totalMs": 3880.9189589810558,
      "imagesPerSecond": 0.28335844690610007,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 351.8199999816716,
      "inferenceMs": 3378.588374995161,
      "totalMs": 3730.4083749768324,
      "imagesPerSecond": 0.2959816020800203,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 351.8199999816716,
      "inferenceMs": 3397.867874999065,
      "totalMs": 3749.6878749807365,
      "imagesPerSecond": 0.2943022026718079,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 351.8199999816716,
      "inferenceMs": 3377.2522910148837,
      "totalMs": 3729.0722909965552,
      "imagesPerSecond": 0.2960986961680302,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 351.8199999816716,
      "inferenceMs": 3381.327167036943,
      "totalMs": 3733.1471670186147,
      "imagesPerSecond": 0.2957418642444765,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Native output for same fixture and configuration as spatial qualification"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
      "artifactRevision": "057a34b54f9f1cf5395de72edaeaefca43e66494",
      "cpuThreads": 4,
      "device": "mps",
      "durationMs": 4374.495083,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "family": {
        "catalogId": "eomt-native",
        "defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
        "displayName": "EoMT Desktop",
        "id": "eomt",
        "variantId": "coco-instance-large-1280",
        "variantName": "COCO Instance Large 1280"
      },
      "inferenceMs": 4363.544624997303,
      "loadMs": 5071.599959,
      "modelId": "eomt-coco-instance-large-1280-native",
      "parameterDtype": "torch.float16",
      "placement": {
        "mps:0": 458
      },
      "precision": "fp16",
      "predictionDevices": [
        "mps:0",
        "mps:0"
      ],
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "torchVersion": "2.11.0",
      "version": "4.57.6",
      "totalMs": 9446.095042,
      "imagesPerSecond": 0.2291714846391924,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Native output for same fixture and configuration as spatial qualification"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
      "artifactRevision": "057a34b54f9f1cf5395de72edaeaefca43e66494",
      "cpuThreads": 4,
      "device": "mps",
      "durationMs": 3957.007667,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "family": {
        "catalogId": "eomt-native",
        "defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
        "displayName": "EoMT Desktop",
        "id": "eomt",
        "variantId": "coco-instance-large-1280",
        "variantName": "COCO Instance Large 1280"
      },
      "inferenceMs": 3946.1205839761533,
      "loadMs": 5071.599959,
      "modelId": "eomt-coco-instance-large-1280-native",
      "parameterDtype": "torch.float16",
      "placement": {
        "mps:0": 458
      },
      "precision": "fp16",
      "predictionDevices": [
        "mps:0",
        "mps:0"
      ],
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "torchVersion": "2.11.0",
      "version": "4.57.6",
      "totalMs": 9028.607626,
      "imagesPerSecond": 0.25341344206780153,
      "grade": "moderate"
    }
  }
]

EoMT COCO Instance Large 640 Desktop

26 configurations tested
Passed26
Failed0
TestedNov 14, 2026
Runtimetransformers 4.57.6

Quality criteria

No explicit quality gate recorded. Review per-case evidence; full certification is pending.

PrecisionDeviceResultPlatformInference / load
FP16mpsAcceleratedPassedFastmacos · aarch64478 ms / 279 ms
FP16mpsAcceleratedPassedFastmacos · aarch64480 ms / 279 ms
FP16mpsAcceleratedPassedFastmacos · aarch64480 ms / 279 ms
BF16mpsAcceleratedPassedFastmacos · aarch64480 ms / 319 ms
FP16mpsAcceleratedPassedFastmacos · aarch64482 ms / 279 ms
BF16mpsAcceleratedPassedFastmacos · aarch64486 ms / 319 ms
BF16mpsAcceleratedPassedFastmacos · aarch64489 ms / 319 ms
BF16mpsAcceleratedPassedFastmacos · aarch64489 ms / 319 ms
FP32mpsAcceleratedPassedFastmacos · aarch64520 ms / 402 ms
FP32mpsAcceleratedPassedFastmacos · aarch64541 ms / 402 ms
FP32mpsAcceleratedPassedFastmacos · aarch64579 ms / 402 ms
FP16mpsAcceleratedPassedFastNot recorded590 ms / 5102 ms
FP16mpsAcceleratedPassedFastmacos · aarch64599 ms / 279 ms
BF16mpsAcceleratedPassedFastmacos · aarch64612 ms / 319 ms
FP32mpsAcceleratedPassedFastmacos · aarch64654 ms / 402 ms
FP16mpsAcceleratedPassedFastmacos · aarch64668 ms / 279 ms
BF16mpsAcceleratedPassedFastmacos · aarch64675 ms / 319 ms
FP32mpsAcceleratedPassedFastmacos · aarch64854 ms / 402 ms
FP32mpsAcceleratedPassedModeratemacos · aarch641080 ms / 402 ms
FP16mpsAcceleratedPassedModerateNot recorded1149 ms / 5102 ms
FP32cpuPassedModeratemacos · aarch641879 ms / 67 ms
FP32cpuPassedModeratemacos · aarch641879 ms / 67 ms
FP32cpuPassedModeratemacos · aarch641907 ms / 67 ms
FP32cpuPassedModeratemacos · aarch641934 ms / 67 ms
FP32cpuPassedModeratemacos · aarch642022 ms / 67 ms
FP32cpuPassedModeratemacos · aarch642159 ms / 67 ms
Test setup & sample details
Quality gate
No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
Tested
2026-11-14T21:14:05.010047+00:00
Environment
{ "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
Runtime
transformers 4.57.6
Artifact revision
a43666870d47a7010bfea2c4554f7ca829714558
Fixture
{
  "file": "model-samples/desktop/mobilenet-dog.jpg",
  "size": 83281,
  "sha1": "f1d042827c2f2154a6d8043b8e8f07c1be612941",
  "sha256": "d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1",
  "url": "https://assets.localai.tools/samples/v1/files/d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1/mobilenet-dog.jpg"
}
Rerun when
Checkpoint or runtime revision; Preprocessing or postprocessing; Fixture or quality gate
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 67.03858397668228,
      "inferenceMs": 2159.350500034634,
      "totalMs": 2226.3890840113163,
      "imagesPerSecond": 0.4631022152188637,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 67.03858397668228,
      "inferenceMs": 2021.743458986748,
      "totalMs": 2088.7820429634303,
      "imagesPerSecond": 0.4946225969249221,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 67.03858397668228,
      "inferenceMs": 1879.0390830254182,
      "totalMs": 1946.0776670021005,
      "imagesPerSecond": 0.5321869082094408,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 67.03858397668228,
      "inferenceMs": 1879.0461250464432,
      "totalMs": 1946.0847090231255,
      "imagesPerSecond": 0.5321849137552617,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 67.03858397668228,
      "inferenceMs": 1907.3512500035577,
      "totalMs": 1974.38983398024,
      "imagesPerSecond": 0.5242872805929871,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 67.03858397668228,
      "inferenceMs": 1934.0937500237487,
      "totalMs": 2001.132334000431,
      "imagesPerSecond": 0.5170380184454456,
      "grade": "moderate"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 318.8823329983279,
      "inferenceMs": 675.1152499928139,
      "totalMs": 993.9975829911418,
      "imagesPerSecond": 1.481228575433075,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 318.8823329983279,
      "inferenceMs": 612.3045000131242,
      "totalMs": 931.1868330114521,
      "imagesPerSecond": 1.6331743437759576,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 318.8823329983279,
      "inferenceMs": 480.3947500186041,
      "totalMs": 799.277083016932,
      "imagesPerSecond": 2.0816214164731677,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 318.8823329983279,
      "inferenceMs": 488.61841700272635,
      "totalMs": 807.5007500010543,
      "imagesPerSecond": 2.046586794935362,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 318.8823329983279,
      "inferenceMs": 486.34987499099225,
      "totalMs": 805.2322079893202,
      "imagesPerSecond": 2.0561329434258027,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 318.8823329983279,
      "inferenceMs": 488.843375002034,
      "totalMs": 807.7257080003619,
      "imagesPerSecond": 2.0456449880206296,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 278.6241670255549,
      "inferenceMs": 667.5229580141604,
      "totalMs": 946.1471250397153,
      "imagesPerSecond": 1.498075815961354,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 278.6241670255549,
      "inferenceMs": 598.7348750350066,
      "totalMs": 877.3590420605615,
      "imagesPerSecond": 1.6701883282505174,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 278.6241670255549,
      "inferenceMs": 480.26133299572393,
      "totalMs": 758.8855000212789,
      "imagesPerSecond": 2.082199692742917,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 278.6241670255549,
      "inferenceMs": 482.31491597834975,
      "totalMs": 760.9390830039047,
      "imagesPerSecond": 2.073334178296257,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 278.6241670255549,
      "inferenceMs": 478.00366597948596,
      "totalMs": 756.6278330050409,
      "imagesPerSecond": 2.092034164530688,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 278.6241670255549,
      "inferenceMs": 479.8966670059599,
      "totalMs": 758.5208340315148,
      "imagesPerSecond": 2.083781923802319,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 402.2555000265129,
      "inferenceMs": 1079.9683749792166,
      "totalMs": 1482.2238750057295,
      "imagesPerSecond": 0.9259530400779046,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 402.2555000265129,
      "inferenceMs": 854.4880410190672,
      "totalMs": 1256.74354104558,
      "imagesPerSecond": 1.1702913932035777,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 402.2555000265129,
      "inferenceMs": 654.0843340335414,
      "totalMs": 1056.3398340600543,
      "imagesPerSecond": 1.528854840221139,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 402.2555000265129,
      "inferenceMs": 579.4954590382986,
      "totalMs": 981.7509590648115,
      "imagesPerSecond": 1.7256390613647767,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 402.2555000265129,
      "inferenceMs": 541.144874994643,
      "totalMs": 943.4003750211559,
      "imagesPerSecond": 1.8479339751852946,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 402.2555000265129,
      "inferenceMs": 520.3491250285879,
      "totalMs": 922.6046250551008,
      "imagesPerSecond": 1.9217866465040374,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Native output for same fixture and configuration as spatial qualification"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
      "artifactRevision": "a43666870d47a7010bfea2c4554f7ca829714558",
      "cpuThreads": 4,
      "device": "mps",
      "durationMs": 1160.2174169999998,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "family": {
        "catalogId": "eomt-native",
        "defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
        "displayName": "EoMT Desktop",
        "id": "eomt",
        "variantId": "coco-instance-large-640",
        "variantName": "COCO Instance Large 640"
      },
      "inferenceMs": 1148.9204170065932,
      "loadMs": 5102.389333,
      "modelId": "eomt-coco-instance-large-640-native",
      "parameterDtype": "torch.float16",
      "placement": {
        "mps:0": 458
      },
      "precision": "fp16",
      "predictionDevices": [
        "mps:0",
        "mps:0"
      ],
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "torchVersion": "2.11.0",
      "version": "4.57.6",
      "totalMs": 6262.60675,
      "imagesPerSecond": 0.8703823042899771,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Native output for same fixture and configuration as spatial qualification"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
      "artifactRevision": "a43666870d47a7010bfea2c4554f7ca829714558",
      "cpuThreads": 4,
      "device": "mps",
      "durationMs": 601.5315,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "family": {
        "catalogId": "eomt-native",
        "defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
        "displayName": "EoMT Desktop",
        "id": "eomt",
        "variantId": "coco-instance-large-640",
        "variantName": "COCO Instance Large 640"
      },
      "inferenceMs": 590.0762910023332,
      "loadMs": 5102.389333,
      "modelId": "eomt-coco-instance-large-640-native",
      "parameterDtype": "torch.float16",
      "placement": {
        "mps:0": 458
      },
      "precision": "fp16",
      "predictionDevices": [
        "mps:0",
        "mps:0"
      ],
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "torchVersion": "2.11.0",
      "version": "4.57.6",
      "totalMs": 5703.920833,
      "imagesPerSecond": 1.694696118532995,
      "grade": "fast"
    }
  }
]

EoMT COCO Panoptic 7B 640 Desktop

32 configurations tested
Passed14
Failed18
TestedNov 15, 2026
Runtimetransformers 4.57.6

Quality criteria

No explicit quality gate recorded. Review per-case evidence; full certification is pending.

PrecisionDeviceResultPlatformInference / load
FP16mpsAcceleratedPassedModeratemacos · aarch643921 ms / 276451 ms
FP16mpsAcceleratedPassedModeratemacos · aarch644262 ms / 276451 ms
FP16mpsAcceleratedPassedSlowmacos · aarch643876 ms / 33594 ms
FP16mpsAcceleratedPassedSlowmacos · aarch643944 ms / 33594 ms
FP16mpsAcceleratedPassedSlowmacos · aarch644067 ms / 33594 ms
FP16mpsAcceleratedPassedSlowmacos · aarch644185 ms / 33594 ms
FP16mpsAcceleratedPassedSlowmacos · aarch644238 ms / 33594 ms
FP16mpsAcceleratedPassedSlowmacos · aarch644418 ms / 31635 ms
FP16mpsAcceleratedPassedSlowmacos · aarch644430 ms / 31635 ms
FP16mpsAcceleratedPassedSlowmacos · aarch644516 ms / 31635 ms
FP16mpsAcceleratedPassedSlowmacos · aarch644583 ms / 31635 ms
FP16mpsAcceleratedPassedSlowmacos · aarch644674 ms / 31635 ms
FP16mpsAcceleratedPassedSlowmacos · aarch644677 ms / 31635 ms
FP16mpsAcceleratedPassedSlowmacos · aarch644811 ms / 33594 ms
BF16mpsFailed—macos · aarch645727 ms / 26465 ms
BF16mpsFailed—macos · aarch644617 ms / 26465 ms
BF16mpsFailed—macos · aarch644081 ms / 26465 ms
BF16mpsFailed—macos · aarch644086 ms / 26465 ms
BF16mpsFailed—macos · aarch644110 ms / 26465 ms
BF16mpsFailed—macos · aarch644511 ms / 26465 ms
FP32cpuFailed—macos · aarch6495358 ms / 151 ms
FP32cpuFailed—macos · aarch6444109 ms / 151 ms
FP32cpuFailed—macos · aarch6450881 ms / 151 ms
FP32cpuFailed—macos · aarch6441249 ms / 151 ms
FP32cpuFailed—macos · aarch6442040 ms / 151 ms
FP32cpuFailed—macos · aarch6452235 ms / 151 ms
FP32mpsFailed—macos · aarch646598 ms / 19066 ms
FP32mpsFailed—macos · aarch644892 ms / 19066 ms
FP32mpsFailed—macos · aarch644718 ms / 19066 ms
FP32mpsFailed—macos · aarch644914 ms / 19066 ms
FP32mpsFailed—macos · aarch645084 ms / 19066 ms
FP32mpsFailed—macos · aarch645315 ms / 19066 ms
Test setup & sample details
Quality gate
No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
Tested
2026-11-15T06:53:24.543030+00:00
Environment
{ "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
Runtime
transformers 4.57.6
Artifact revision
a45f5da9787a3bad985155039050226d9fbecca5
Fixture
{
  "file": "model-samples/desktop/mobilenet-dog.jpg",
  "size": 83281,
  "sha1": "f1d042827c2f2154a6d8043b8e8f07c1be612941",
  "sha256": "d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1",
  "url": "https://assets.localai.tools/samples/v1/files/d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1/mobilenet-dog.jpg"
}
Rerun when
Checkpoint or runtime revision; Preprocessing or postprocessing; Fixture or quality gate
Per-case quality & failure details
[
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "Metal/MPS",
      "loadMs": 33593.50700001232,
      "inferenceMs": 4811.2799159716815,
      "totalMs": 38404.786915984005,
      "peakRssBytes": 22430482432,
      "imagesPerSecond": 0.20784490145342976,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "Metal/MPS",
      "loadMs": 33593.50700001232,
      "inferenceMs": 3944.016749970615,
      "totalMs": 37537.52374998294,
      "peakRssBytes": 22430482432,
      "imagesPerSecond": 0.2535486189320698,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "Metal/MPS",
      "loadMs": 33593.50700001232,
      "inferenceMs": 3875.861707958393,
      "totalMs": 37469.368707970716,
      "peakRssBytes": 22430482432,
      "imagesPerSecond": 0.25800714146912873,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "Metal/MPS",
      "loadMs": 33593.50700001232,
      "inferenceMs": 4238.054167013615,
      "totalMs": 37831.56116702594,
      "peakRssBytes": 22430482432,
      "imagesPerSecond": 0.23595734282571934,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "Metal/MPS",
      "loadMs": 33593.50700001232,
      "inferenceMs": 4185.032167006284,
      "totalMs": 37778.53916701861,
      "peakRssBytes": 22430482432,
      "imagesPerSecond": 0.2389467894377832,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "Metal/MPS",
      "loadMs": 33593.50700001232,
      "inferenceMs": 4067.2992920153774,
      "totalMs": 37660.8062920277,
      "peakRssBytes": 22430482432,
      "imagesPerSecond": 0.24586339194736084,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "Metal/MPS",
      "loadMs": 31635.103542008437,
      "inferenceMs": 4677.024625008926,
      "totalMs": 36312.12816701736,
      "peakRssBytes": 11318706176,
      "imagesPerSecond": 0.213811147081162,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "Metal/MPS",
      "loadMs": 31635.103542008437,
      "inferenceMs": 4430.008624971379,
      "totalMs": 36065.112166979816,
      "peakRssBytes": 11318706176,
      "imagesPerSecond": 0.22573319482114118,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "Metal/MPS",
      "loadMs": 31635.103542008437,
      "inferenceMs": 4417.87679202389,
      "totalMs": 36052.98033403233,
      "peakRssBytes": 11318706176,
      "imagesPerSecond": 0.22635307571397578,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "Metal/MPS",
      "loadMs": 31635.103542008437,
      "inferenceMs": 4516.275916015729,
      "totalMs": 36151.37945802417,
      "peakRssBytes": 11318706176,
      "imagesPerSecond": 0.2214213698622299,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "Metal/MPS",
      "loadMs": 31635.103542008437,
      "inferenceMs": 4674.305790977087,
      "totalMs": 36309.409332985524,
      "peakRssBytes": 11318706176,
      "imagesPerSecond": 0.21393551143579898,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "Metal/MPS",
      "loadMs": 31635.103542008437,
      "inferenceMs": 4583.446374977939,
      "totalMs": 36218.549916986376,
      "peakRssBytes": 11318706176,
      "imagesPerSecond": 0.21817643715855914,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "Metal/MPS",
      "loadMs": 26465.21195897367,
      "inferenceMs": 5726.736209006049,
      "totalMs": 32191.948167979717,
      "peakRssBytes": 17485021184,
      "imagesPerSecond": 0.1746195325755302,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "Metal/MPS",
      "loadMs": 26465.21195897367,
      "inferenceMs": 4616.539833019488,
      "totalMs": 31081.751791993156,
      "peakRssBytes": 17485021184,
      "imagesPerSecond": 0.21661244918706601,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "Metal/MPS",
      "loadMs": 26465.21195897367,
      "inferenceMs": 4081.294458999764,
      "totalMs": 30546.506417973433,
      "peakRssBytes": 17485021184,
      "imagesPerSecond": 0.24502030178069487,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "Metal/MPS",
      "loadMs": 26465.21195897367,
      "inferenceMs": 4086.222457990516,
      "totalMs": 30551.434416964184,
      "peakRssBytes": 17485021184,
      "imagesPerSecond": 0.24472480641491326,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "Metal/MPS",
      "loadMs": 26465.21195897367,
      "inferenceMs": 4109.718374966178,
      "totalMs": 30574.930333939847,
      "peakRssBytes": 17485021184,
      "imagesPerSecond": 0.24332567557216855,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "Metal/MPS",
      "loadMs": 26465.21195897367,
      "inferenceMs": 4511.223124980461,
      "totalMs": 30976.43508395413,
      "peakRssBytes": 17485021184,
      "imagesPerSecond": 0.22166937265031225,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "PyTorch CPU",
      "loadMs": 150.91558400308713,
      "inferenceMs": 95358.02529199282,
      "totalMs": 95508.9408759959,
      "peakRssBytes": 29611065344,
      "imagesPerSecond": 0.01048679434098946,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "PyTorch CPU",
      "loadMs": 150.91558400308713,
      "inferenceMs": 44109.261416015215,
      "totalMs": 44260.1770000183,
      "peakRssBytes": 29611065344,
      "imagesPerSecond": 0.022670975842658737,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "PyTorch CPU",
      "loadMs": 150.91558400308713,
      "inferenceMs": 50880.95858297311,
      "totalMs": 51031.874166976195,
      "peakRssBytes": 29611065344,
      "imagesPerSecond": 0.019653717772814164,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "PyTorch CPU",
      "loadMs": 150.91558400308713,
      "inferenceMs": 41248.72304103337,
      "totalMs": 41399.63862503646,
      "peakRssBytes": 29611065344,
      "imagesPerSecond": 0.024243174728226635,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "PyTorch CPU",
      "loadMs": 150.91558400308713,
      "inferenceMs": 42040.499666996766,
      "totalMs": 42191.41525099985,
      "peakRssBytes": 29611065344,
      "imagesPerSecond": 0.02378658693214901,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "PyTorch CPU",
      "loadMs": 150.91558400308713,
      "inferenceMs": 52234.89391698968,
      "totalMs": 52385.809500992764,
      "peakRssBytes": 29611065344,
      "imagesPerSecond": 0.019144290818110473,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "Metal/MPS",
      "loadMs": 19066.379999974743,
      "inferenceMs": 6598.004333034623,
      "totalMs": 25664.384333009366,
      "peakRssBytes": 1586331648,
      "imagesPerSecond": 0.15156097958184722,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "Metal/MPS",
      "loadMs": 19066.379999974743,
      "inferenceMs": 4891.629500023555,
      "totalMs": 23958.009499998298,
      "peakRssBytes": 1586331648,
      "imagesPerSecond": 0.20443085478881518,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "Metal/MPS",
      "loadMs": 19066.379999974743,
      "inferenceMs": 4717.508417030331,
      "totalMs": 23783.888417005073,
      "peakRssBytes": 1586331648,
      "imagesPerSecond": 0.21197630435379267,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "Metal/MPS",
      "loadMs": 19066.379999974743,
      "inferenceMs": 4913.599958003033,
      "totalMs": 23979.979957977775,
      "peakRssBytes": 1586331648,
      "imagesPerSecond": 0.2035167715213056,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "Metal/MPS",
      "loadMs": 19066.379999974743,
      "inferenceMs": 5083.821790991351,
      "totalMs": 24150.201790966094,
      "peakRssBytes": 1586331648,
      "imagesPerSecond": 0.1967024103346862,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "attentionImplementation": "sdpa",
      "provider": "Metal/MPS",
      "loadMs": 19066.379999974743,
      "inferenceMs": 5315.2464590384625,
      "totalMs": 24381.626459013205,
      "peakRssBytes": 1586331648,
      "imagesPerSecond": 0.18813803041992935,
      "grade": "slow",
      "engine": "transformers",
      "version": "4.57.6",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Native output for same fixture and configuration as spatial qualification"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5",
      "attentionImplementation": "sdpa",
      "cpuThreads": 4,
      "device": "mps",
      "durationMs": 4272.326999999999,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "family": {
        "catalogId": "eomt-native",
        "defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
        "displayName": "EoMT Desktop",
        "id": "eomt",
        "variantId": "coco-panoptic-7b-640",
        "variantName": "COCO Panoptic 7B 640"
      },
      "inferenceMs": 4261.654041998554,
      "loadMs": 276451.09150000004,
      "modelId": "eomt-coco-panoptic-7b-640-native",
      "parameterDtype": "torch.float16",
      "placement": {
        "mps:0": 602
      },
      "precision": "fp16",
      "predictionDevices": [
        "mps:0",
        "mps:0"
      ],
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "torchVersion": "2.11.0",
      "version": "4.57.6",
      "totalMs": 280723.4185,
      "imagesPerSecond": 0.23465067556986346,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Native output for same fixture and configuration as spatial qualification"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
      "artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5",
      "attentionImplementation": "sdpa",
      "cpuThreads": 4,
      "device": "mps",
      "durationMs": 3933.489583,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "family": {
        "catalogId": "eomt-native",
        "defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
        "displayName": "EoMT Desktop",
        "id": "eomt",
        "variantId": "coco-panoptic-7b-640",
        "variantName": "COCO Panoptic 7B 640"
      },
      "inferenceMs": 3921.0635410272516,
      "loadMs": 276451.09150000004,
      "modelId": "eomt-coco-panoptic-7b-640-native",
      "parameterDtype": "torch.float16",
      "placement": {
        "mps:0": 602
      },
      "precision": "fp16",
      "predictionDevices": [
        "mps:0",
        "mps:0"
      ],
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "torchVersion": "2.11.0",
      "version": "4.57.6",
      "totalMs": 280384.58108300006,
      "imagesPerSecond": 0.2550328474753605,
      "grade": "moderate"
    }
  }
]

EoMT COCO Panoptic Base 640 2x Desktop

26 configurations tested
Passed26
Failed0
TestedNov 14, 2026
Runtimetransformers 4.57.6

Quality criteria

No explicit quality gate recorded. Review per-case evidence; full certification is pending.

PrecisionDeviceResultPlatformInference / load
FP16mpsAcceleratedPassedFastmacos · aarch64318 ms / 138 ms
FP32mpsAcceleratedPassedFastmacos · aarch64325 ms / 158 ms
FP32mpsAcceleratedPassedFastmacos · aarch64328 ms / 158 ms
FP32mpsAcceleratedPassedFastmacos · aarch64330 ms / 158 ms
FP16mpsAcceleratedPassedFastmacos · aarch64331 ms / 138 ms
FP16mpsAcceleratedPassedFastmacos · aarch64334 ms / 138 ms
FP16mpsAcceleratedPassedFastmacos · aarch64339 ms / 138 ms
FP32mpsAcceleratedPassedFastmacos · aarch64346 ms / 158 ms
BF16mpsAcceleratedPassedFastmacos · aarch64407 ms / 241 ms
BF16mpsAcceleratedPassedFastmacos · aarch64415 ms / 241 ms
FP16mpsAcceleratedPassedFastNot recorded420 ms / 3109 ms
BF16mpsAcceleratedPassedFastmacos · aarch64424 ms / 241 ms
BF16mpsAcceleratedPassedFastmacos · aarch64434 ms / 241 ms
FP16mpsAcceleratedPassedFastmacos · aarch64460 ms / 138 ms
FP32mpsAcceleratedPassedFastmacos · aarch64471 ms / 158 ms
BF16mpsAcceleratedPassedFastmacos · aarch64572 ms / 241 ms
FP32mpsAcceleratedPassedFastmacos · aarch64588 ms / 158 ms
FP16mpsAcceleratedPassedFastmacos · aarch64588 ms / 138 ms
FP16mpsAcceleratedPassedFastNot recorded678 ms / 3109 ms
BF16mpsAcceleratedPassedFastmacos · aarch64701 ms / 241 ms
FP32cpuPassedFastmacos · aarch64806 ms / 53 ms
FP32cpuPassedFastmacos · aarch64835 ms / 53 ms
FP32cpuPassedFastmacos · aarch64838 ms / 53 ms
FP32cpuPassedFastmacos · aarch64838 ms / 53 ms
FP32cpuPassedFastmacos · aarch64923 ms / 53 ms
FP32cpuPassedModeratemacos · aarch642434 ms / 53 ms
Test setup & sample details
Quality gate
No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
Tested
2026-11-14T21:11:18.630023+00:00
Environment
{ "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
Runtime
transformers 4.57.6
Artifact revision
1ac497947a2410fdb05279c5907c5209af37b9f1
Fixture
{
  "file": "model-samples/desktop/mobilenet-dog.jpg",
  "size": 83281,
  "sha1": "f1d042827c2f2154a6d8043b8e8f07c1be612941",
  "sha256": "d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1",
  "url": "https://assets.localai.tools/samples/v1/files/d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1/mobilenet-dog.jpg"
}
Rerun when
Checkpoint or runtime revision; Preprocessing or postprocessing; Fixture or quality gate
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 52.826541010290384,
      "inferenceMs": 2433.665375050623,
      "totalMs": 2486.4919160609134,
      "imagesPerSecond": 0.41090283415779744,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 52.826541010290384,
      "inferenceMs": 922.5171250291169,
      "totalMs": 975.3436660394073,
      "imagesPerSecond": 1.0839907172112795,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 52.826541010290384,
      "inferenceMs": 806.1512910062447,
      "totalMs": 858.9778320165351,
      "imagesPerSecond": 1.240461946977461,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 52.826541010290384,
      "inferenceMs": 838.0493330187164,
      "totalMs": 890.8758740290068,
      "imagesPerSecond": 1.1932471760318992,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 52.826541010290384,
      "inferenceMs": 838.1419579964131,
      "totalMs": 890.9684990067035,
      "imagesPerSecond": 1.1931153075673604,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 52.826541010290384,
      "inferenceMs": 835.1196660078131,
      "totalMs": 887.9462070181035,
      "imagesPerSecond": 1.1974331831752654,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 241.47554201772436,
      "inferenceMs": 701.2341669760644,
      "totalMs": 942.7097089937888,
      "imagesPerSecond": 1.4260571533647668,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 241.47554201772436,
      "inferenceMs": 571.9608750077896,
      "totalMs": 813.436417025514,
      "imagesPerSecond": 1.7483713374387382,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 241.47554201772436,
      "inferenceMs": 424.3493750109337,
      "totalMs": 665.8249170286581,
      "imagesPerSecond": 2.356548775343982,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 241.47554201772436,
      "inferenceMs": 415.3332499554381,
      "totalMs": 656.8087919731624,
      "imagesPerSecond": 2.407705138240899,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 241.47554201772436,
      "inferenceMs": 434.4510420341976,
      "totalMs": 675.926584051922,
      "imagesPerSecond": 2.3017553262567283,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 241.47554201772436,
      "inferenceMs": 407.0659999852069,
      "totalMs": 648.5415420029312,
      "imagesPerSecond": 2.4566040888611202,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 138.0069589940831,
      "inferenceMs": 588.3609169977717,
      "totalMs": 726.3678759918548,
      "imagesPerSecond": 1.6996370273924692,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 138.0069589940831,
      "inferenceMs": 459.55174998380244,
      "totalMs": 597.5587089778855,
      "imagesPerSecond": 2.1760334935842294,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 138.0069589940831,
      "inferenceMs": 317.93570797890425,
      "totalMs": 455.94266697298735,
      "imagesPerSecond": 3.1452899907246414,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 138.0069589940831,
      "inferenceMs": 338.94779096590355,
      "totalMs": 476.95474995998666,
      "imagesPerSecond": 2.950306881039962,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 138.0069589940831,
      "inferenceMs": 333.6381250410341,
      "totalMs": 471.6450840351172,
      "imagesPerSecond": 2.9972593805849086,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 138.0069589940831,
      "inferenceMs": 330.9533340507187,
      "totalMs": 468.9602930448018,
      "imagesPerSecond": 3.0215740320861966,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 157.62016701046377,
      "inferenceMs": 587.7712920191698,
      "totalMs": 745.3914590296336,
      "imagesPerSecond": 1.7013420246584374,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 157.62016701046377,
      "inferenceMs": 471.26441705040634,
      "totalMs": 628.8845840608701,
      "imagesPerSecond": 2.1219509978260045,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 157.62016701046377,
      "inferenceMs": 327.58645905414596,
      "totalMs": 485.20662606460974,
      "imagesPerSecond": 3.0526292292036175,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 157.62016701046377,
      "inferenceMs": 345.6185840186663,
      "totalMs": 503.23875102913007,
      "imagesPerSecond": 2.8933629331285946,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 157.62016701046377,
      "inferenceMs": 330.1146670128219,
      "totalMs": 487.73483402328566,
      "imagesPerSecond": 3.0292504390941204,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 157.62016701046377,
      "inferenceMs": 324.7161660110578,
      "totalMs": 482.33633302152157,
      "imagesPerSecond": 3.079612611482812,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Native output for same fixture and configuration as spatial qualification"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
      "artifactRevision": "1ac497947a2410fdb05279c5907c5209af37b9f1",
      "cpuThreads": 4,
      "device": "mps",
      "durationMs": 689.8155830000001,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "family": {
        "catalogId": "eomt-native",
        "defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
        "displayName": "EoMT",
        "id": "eomt",
        "variantId": "coco-panoptic-base-640-2x",
        "variantName": "COCO Panoptic Base 640 2x"
      },
      "inferenceMs": 678.021790983621,
      "loadMs": 3109.4634170000004,
      "modelId": "eomt-coco-panoptic-base-640-2x-native",
      "parameterDtype": "torch.float16",
      "placement": {
        "mps:0": 242
      },
      "precision": "fp16",
      "predictionDevices": [
        "mps:0",
        "mps:0"
      ],
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "torchVersion": "2.11.0",
      "version": "4.57.6",
      "totalMs": 3799.2790000000005,
      "imagesPerSecond": 1.4748788509426491,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Native output for same fixture and configuration as spatial qualification"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
      "artifactRevision": "1ac497947a2410fdb05279c5907c5209af37b9f1",
      "cpuThreads": 4,
      "device": "mps",
      "durationMs": 431.868792,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "family": {
        "catalogId": "eomt-native",
        "defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
        "displayName": "EoMT",
        "id": "eomt",
        "variantId": "coco-panoptic-base-640-2x",
        "variantName": "COCO Panoptic Base 640 2x"
      },
      "inferenceMs": 419.69904105644673,
      "loadMs": 3109.4634170000004,
      "modelId": "eomt-coco-panoptic-base-640-2x-native",
      "parameterDtype": "torch.float16",
      "placement": {
        "mps:0": 242
      },
      "precision": "fp16",
      "predictionDevices": [
        "mps:0",
        "mps:0"
      ],
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "torchVersion": "2.11.0",
      "version": "4.57.6",
      "totalMs": 3541.332209,
      "imagesPerSecond": 2.3826597208391207,
      "grade": "fast"
    }
  }
]

EoMT COCO Panoptic Giant 640 Desktop

26 configurations tested
Passed26
Failed0
TestedNov 14, 2026
Runtimetransformers 4.57.6

Quality criteria

No explicit quality gate recorded. Review per-case evidence; full certification is pending.

PrecisionDeviceResultPlatformInference / load
BF16mpsAcceleratedPassedModeratemacos · aarch641076 ms / 699 ms
FP16mpsAcceleratedPassedModeratemacos · aarch641080 ms / 1055 ms
FP32mpsAcceleratedPassedModeratemacos · aarch641081 ms / 888 ms
FP16mpsAcceleratedPassedModeratemacos · aarch641125 ms / 1055 ms
FP16mpsAcceleratedPassedModeratemacos · aarch641128 ms / 1055 ms
BF16mpsAcceleratedPassedModeratemacos · aarch641132 ms / 699 ms
FP32mpsAcceleratedPassedModeratemacos · aarch641135 ms / 888 ms
FP32mpsAcceleratedPassedModeratemacos · aarch641135 ms / 888 ms
FP16mpsAcceleratedPassedModeratemacos · aarch641137 ms / 1055 ms
BF16mpsAcceleratedPassedModeratemacos · aarch641147 ms / 699 ms
BF16mpsAcceleratedPassedModeratemacos · aarch641151 ms / 699 ms
FP32mpsAcceleratedPassedModeratemacos · aarch641160 ms / 888 ms
FP32mpsAcceleratedPassedModeratemacos · aarch641195 ms / 888 ms
FP16mpsAcceleratedPassedModerateNot recorded1197 ms / 6759 ms
BF16mpsAcceleratedPassedModeratemacos · aarch641214 ms / 699 ms
FP16mpsAcceleratedPassedModeratemacos · aarch641231 ms / 1055 ms
FP32mpsAcceleratedPassedModeratemacos · aarch641301 ms / 888 ms
BF16mpsAcceleratedPassedModeratemacos · aarch641319 ms / 699 ms
FP16mpsAcceleratedPassedModerateNot recorded1355 ms / 6759 ms
FP16mpsAcceleratedPassedModeratemacos · aarch642935 ms / 1055 ms
FP32cpuPassedSlowmacos · aarch644149 ms / 90 ms
FP32cpuPassedSlowmacos · aarch644194 ms / 90 ms
FP32cpuPassedSlowmacos · aarch644275 ms / 90 ms
FP32cpuPassedSlowmacos · aarch644306 ms / 90 ms
FP32cpuPassedSlowmacos · aarch644307 ms / 90 ms
FP32cpuPassedSlowmacos · aarch644765 ms / 90 ms
Test setup & sample details
Quality gate
No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
Tested
2026-11-14T22:18:18.399974+00:00
Environment
{ "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
Runtime
transformers 4.57.6
Artifact revision
b141595af7ff6d9f3e78c847903f6761b8575298
Fixture
{
  "file": "model-samples/desktop/mobilenet-dog.jpg",
  "size": 83281,
  "sha1": "f1d042827c2f2154a6d8043b8e8f07c1be612941",
  "sha256": "d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1",
  "url": "https://assets.localai.tools/samples/v1/files/d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1/mobilenet-dog.jpg"
}
Rerun when
Checkpoint or runtime revision; Preprocessing or postprocessing; Fixture or quality gate
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 90.40616598213091,
      "inferenceMs": 4764.8558329674415,
      "totalMs": 4855.261998949572,
      "imagesPerSecond": 0.20986993836857037,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 90.40616598213091,
      "inferenceMs": 4305.557249987032,
      "totalMs": 4395.963415969163,
      "imagesPerSecond": 0.23225797311207785,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 90.40616598213091,
      "inferenceMs": 4148.856999992859,
      "totalMs": 4239.2631659749895,
      "imagesPerSecond": 0.24103024037746332,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 90.40616598213091,
      "inferenceMs": 4275.357208040077,
      "totalMs": 4365.763374022208,
      "imagesPerSecond": 0.23389858468888572,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 90.40616598213091,
      "inferenceMs": 4193.52383405203,
      "totalMs": 4283.930000034161,
      "imagesPerSecond": 0.23846293465172488,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 90.40616598213091,
      "inferenceMs": 4306.714584003203,
      "totalMs": 4397.1207499853335,
      "imagesPerSecond": 0.2321955589335744,
      "grade": "slow"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 699.3644169997424,
      "inferenceMs": 1319.4539999822155,
      "totalMs": 2018.8184169819579,
      "imagesPerSecond": 0.7578892481386079,
      "grade": "moderate"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 699.3644169997424,
      "inferenceMs": 1214.0130000188947,
      "totalMs": 1913.377417018637,
      "imagesPerSecond": 0.823714408317239,
      "grade": "moderate"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 699.3644169997424,
      "inferenceMs": 1075.6408749730326,
      "totalMs": 1775.005291972775,
      "imagesPerSecond": 0.9296783185420235,
      "grade": "moderate"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 699.3644169997424,
      "inferenceMs": 1146.9129999750294,
      "totalMs": 1846.2774169747718,
      "imagesPerSecond": 0.8719057156225206,
      "grade": "moderate"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 699.3644169997424,
      "inferenceMs": 1132.3877499671653,
      "totalMs": 1831.7521669669077,
      "imagesPerSecond": 0.8830897367346088,
      "grade": "moderate"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 699.3644169997424,
      "inferenceMs": 1151.298000011593,
      "totalMs": 1850.6624170113355,
      "imagesPerSecond": 0.8685848494394417,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 1054.9896669690497,
      "inferenceMs": 2935.386458004359,
      "totalMs": 3990.376124973409,
      "imagesPerSecond": 0.3406706456906721,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 1054.9896669690497,
      "inferenceMs": 1230.5514580220915,
      "totalMs": 2285.5411249911413,
      "imagesPerSecond": 0.812643789482266,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 1054.9896669690497,
      "inferenceMs": 1079.6370829921216,
      "totalMs": 2134.6267499611713,
      "imagesPerSecond": 0.9262371733551295,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 1054.9896669690497,
      "inferenceMs": 1128.2044170075096,
      "totalMs": 2183.1940839765593,
      "imagesPerSecond": 0.8863641951096383,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 1054.9896669690497,
      "inferenceMs": 1125.0781249837019,
      "totalMs": 2180.0677919527516,
      "imagesPerSecond": 0.8888271647930993,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 1054.9896669690497,
      "inferenceMs": 1137.2789589804597,
      "totalMs": 2192.2686259495094,
      "imagesPerSecond": 0.8792917446538124,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 888.4135420084931,
      "inferenceMs": 1300.6567499833182,
      "totalMs": 2189.0702919918112,
      "imagesPerSecond": 0.7688423559965577,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 888.4135420084931,
      "inferenceMs": 1194.7302499902435,
      "totalMs": 2083.1437919987366,
      "imagesPerSecond": 0.8370090235918662,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 888.4135420084931,
      "inferenceMs": 1080.7537920190953,
      "totalMs": 1969.1673340275884,
      "imagesPerSecond": 0.9252801215083144,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 888.4135420084931,
      "inferenceMs": 1134.982957970351,
      "totalMs": 2023.396499978844,
      "imagesPerSecond": 0.8810704979996034,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 888.4135420084931,
      "inferenceMs": 1159.804124967195,
      "totalMs": 2048.217666975688,
      "imagesPerSecond": 0.8622145571591969,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 888.4135420084931,
      "inferenceMs": 1135.193291993346,
      "totalMs": 2023.6068340018392,
      "imagesPerSecond": 0.8809072490589219,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Native output for same fixture and configuration as spatial qualification"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
      "artifactRevision": "b141595af7ff6d9f3e78c847903f6761b8575298",
      "cpuThreads": 4,
      "device": "mps",
      "durationMs": 1367.108792,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "family": {
        "catalogId": "eomt-native",
        "defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
        "displayName": "EoMT Desktop",
        "id": "eomt",
        "variantId": "coco-panoptic-giant-640",
        "variantName": "COCO Panoptic Giant 640"
      },
      "inferenceMs": 1355.2602909621783,
      "loadMs": 6759.449958,
      "modelId": "eomt-coco-panoptic-giant-640-native",
      "parameterDtype": "torch.float16",
      "placement": {
        "mps:0": 746
      },
      "precision": "fp16",
      "predictionDevices": [
        "mps:0",
        "mps:0"
      ],
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "torchVersion": "2.11.0",
      "version": "4.57.6",
      "totalMs": 8126.55875,
      "imagesPerSecond": 0.7378656385557063,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Native output for same fixture and configuration as spatial qualification"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
      "artifactRevision": "b141595af7ff6d9f3e78c847903f6761b8575298",
      "cpuThreads": 4,
      "device": "mps",
      "durationMs": 1207.791042,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "family": {
        "catalogId": "eomt-native",
        "defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
        "displayName": "EoMT Desktop",
        "id": "eomt",
        "variantId": "coco-panoptic-giant-640",
        "variantName": "COCO Panoptic Giant 640"
      },
      "inferenceMs": 1197.1005420200527,
      "loadMs": 6759.449958,
      "modelId": "eomt-coco-panoptic-giant-640-native",
      "parameterDtype": "torch.float16",
      "placement": {
        "mps:0": 746
      },
      "precision": "fp16",
      "predictionDevices": [
        "mps:0",
        "mps:0"
      ],
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "torchVersion": "2.11.0",
      "version": "4.57.6",
      "totalMs": 7967.241,
      "imagesPerSecond": 0.8353517226819942,
      "grade": "moderate"
    }
  }
]

EoMT COCO Panoptic Large 1280 Desktop

26 configurations tested
Passed20
Failed6
TestedNov 14, 2026
Runtimetransformers 4.57.6

Quality criteria

No explicit quality gate recorded. Review per-case evidence; full certification is pending.

PrecisionDeviceResultPlatformInference / load
FP16mpsAcceleratedPassedModerateNot recorded3807 ms / 4377 ms
FP16mpsAcceleratedPassedModerateNot recorded4119 ms / 4377 ms
FP32mpsAcceleratedPassedSlowmacos · aarch643351 ms / 385 ms
FP32mpsAcceleratedPassedSlowmacos · aarch643365 ms / 385 ms
FP32mpsAcceleratedPassedSlowmacos · aarch643438 ms / 385 ms
FP32mpsAcceleratedPassedSlowmacos · aarch643442 ms / 385 ms
FP32mpsAcceleratedPassedSlowmacos · aarch643513 ms / 385 ms
FP32mpsAcceleratedPassedSlowmacos · aarch643611 ms / 385 ms
FP16mpsAcceleratedPassedSlowmacos · aarch643778 ms / 268 ms
FP16mpsAcceleratedPassedSlowmacos · aarch643778 ms / 268 ms
FP16mpsAcceleratedPassedSlowmacos · aarch643790 ms / 268 ms
FP16mpsAcceleratedPassedSlowmacos · aarch643824 ms / 268 ms
BF16mpsAcceleratedPassedSlowmacos · aarch643878 ms / 266 ms
FP16mpsAcceleratedPassedSlowmacos · aarch643910 ms / 268 ms
BF16mpsAcceleratedPassedSlowmacos · aarch644023 ms / 266 ms
BF16mpsAcceleratedPassedSlowmacos · aarch644025 ms / 266 ms
BF16mpsAcceleratedPassedSlowmacos · aarch644058 ms / 266 ms
BF16mpsAcceleratedPassedSlowmacos · aarch644095 ms / 266 ms
BF16mpsAcceleratedPassedSlowmacos · aarch644245 ms / 266 ms
FP16mpsAcceleratedPassedSlowmacos · aarch644667 ms / 268 ms
FP32cpuFailed—macos · aarch6415172 ms / 65 ms
FP32cpuFailed—macos · aarch6413861 ms / 65 ms
FP32cpuFailed—macos · aarch6414248 ms / 65 ms
FP32cpuFailed—macos · aarch6414027 ms / 65 ms
FP32cpuFailed—macos · aarch6414685 ms / 65 ms
FP32cpuFailed—macos · aarch6413938 ms / 65 ms
Test setup & sample details
Quality gate
No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
Tested
2026-11-14T21:26:17.333945+00:00
Environment
{ "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
Runtime
transformers 4.57.6
Artifact revision
d1bb11420d0d054623ffb2b6899ff0430e4cc655
Fixture
{
  "file": "model-samples/desktop/mobilenet-dog.jpg",
  "size": 83281,
  "sha1": "f1d042827c2f2154a6d8043b8e8f07c1be612941",
  "sha256": "d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1",
  "url": "https://assets.localai.tools/samples/v1/files/d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1/mobilenet-dog.jpg"
}
Rerun when
Checkpoint or runtime revision; Preprocessing or postprocessing; Fixture or quality gate
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 65.29920798493549,
      "inferenceMs": 15171.545999997761,
      "totalMs": 15236.845207982697,
      "imagesPerSecond": 0.06591286082513592,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 65.29920798493549,
      "inferenceMs": 13860.936082957778,
      "totalMs": 13926.235290942714,
      "imagesPerSecond": 0.07214519957490566,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": false,
      "pointRecall": null
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 65.29920798493549,
      "inferenceMs": 14248.368541011587,
      "totalMs": 14313.667748996522,
      "imagesPerSecond": 0.07018347378660682,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 65.29920798493549,
      "inferenceMs": 14026.922458957415,
      "totalMs": 14092.22166694235,
      "imagesPerSecond": 0.07129147558389849,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 65.29920798493549,
      "inferenceMs": 14684.519000002183,
      "totalMs": 14749.818207987119,
      "imagesPerSecond": 0.06809892785727958,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 65.29920798493549,
      "inferenceMs": 13937.820542021655,
      "totalMs": 14003.11975000659,
      "imagesPerSecond": 0.0717472288429215,
      "grade": "slow"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 266.3343329913914,
      "inferenceMs": 4057.889040966984,
      "totalMs": 4324.223373958375,
      "imagesPerSecond": 0.2464335495387776,
      "grade": "slow"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 266.3343329913914,
      "inferenceMs": 4022.585916041862,
      "totalMs": 4288.920249033254,
      "imagesPerSecond": 0.248596306175103,
      "grade": "slow"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 266.3343329913914,
      "inferenceMs": 3877.560082997661,
      "totalMs": 4143.8944159890525,
      "imagesPerSecond": 0.25789413409344797,
      "grade": "slow"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 266.3343329913914,
      "inferenceMs": 4024.6608330053277,
      "totalMs": 4290.995165996719,
      "imagesPerSecond": 0.2484681421597635,
      "grade": "slow"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 266.3343329913914,
      "inferenceMs": 4094.9511250364594,
      "totalMs": 4361.285458027851,
      "imagesPerSecond": 0.24420315883284113,
      "grade": "slow"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 266.3343329913914,
      "inferenceMs": 4244.703958975151,
      "totalMs": 4511.038291966543,
      "imagesPerSecond": 0.2355876898989775,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 268.4909590170719,
      "inferenceMs": 4667.368875001557,
      "totalMs": 4935.859834018629,
      "imagesPerSecond": 0.21425347487660623,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 268.4909590170719,
      "inferenceMs": 3909.7971249721013,
      "totalMs": 4178.288083989173,
      "imagesPerSecond": 0.2557677465188518,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 268.4909590170719,
      "inferenceMs": 3777.9040840105154,
      "totalMs": 4046.3950430275872,
      "imagesPerSecond": 0.26469703247162074,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 268.4909590170719,
      "inferenceMs": 3778.4952499787323,
      "totalMs": 4046.986208995804,
      "imagesPerSecond": 0.26465561919275365,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 268.4909590170719,
      "inferenceMs": 3790.1447499752976,
      "totalMs": 4058.6357089923695,
      "imagesPerSecond": 0.2638421659242744,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 268.4909590170719,
      "inferenceMs": 3823.8740409724414,
      "totalMs": 4092.3649999895133,
      "imagesPerSecond": 0.2615148902095353,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 384.88262495957315,
      "inferenceMs": 3610.5999590363353,
      "totalMs": 3995.4825839959085,
      "imagesPerSecond": 0.2769622808800172,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 384.88262495957315,
      "inferenceMs": 3438.265999953728,
      "totalMs": 3823.1486249133013,
      "imagesPerSecond": 0.2908442802312148,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 384.88262495957315,
      "inferenceMs": 3350.5487089860253,
      "totalMs": 3735.4313339455985,
      "imagesPerSecond": 0.2984585770438268,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 384.88262495957315,
      "inferenceMs": 3365.4015000211075,
      "totalMs": 3750.2841249806806,
      "imagesPerSecond": 0.2971413663403098,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 384.88262495957315,
      "inferenceMs": 3441.6352080297656,
      "totalMs": 3826.517832989339,
      "imagesPerSecond": 0.2905595565930041,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 384.88262495957315,
      "inferenceMs": 3512.5767920399085,
      "totalMs": 3897.4594169994816,
      "imagesPerSecond": 0.2846912848328807,
      "grade": "slow"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Native output for same fixture and configuration as spatial qualification"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
      "artifactRevision": "d1bb11420d0d054623ffb2b6899ff0430e4cc655",
      "cpuThreads": 4,
      "device": "mps",
      "durationMs": 4129.924792,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "family": {
        "catalogId": "eomt-native",
        "defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
        "displayName": "EoMT Desktop",
        "id": "eomt",
        "variantId": "coco-panoptic-large-1280",
        "variantName": "COCO Panoptic Large 1280"
      },
      "inferenceMs": 4118.7245000037365,
      "loadMs": 4377.031083000001,
      "modelId": "eomt-coco-panoptic-large-1280-native",
      "parameterDtype": "torch.float16",
      "placement": {
        "mps:0": 458
      },
      "precision": "fp16",
      "predictionDevices": [
        "mps:0",
        "mps:0"
      ],
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "torchVersion": "2.11.0",
      "version": "4.57.6",
      "totalMs": 8506.955875,
      "imagesPerSecond": 0.2427936124397475,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Native output for same fixture and configuration as spatial qualification"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
      "artifactRevision": "d1bb11420d0d054623ffb2b6899ff0430e4cc655",
      "cpuThreads": 4,
      "device": "mps",
      "durationMs": 3817.920584,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "family": {
        "catalogId": "eomt-native",
        "defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
        "displayName": "EoMT Desktop",
        "id": "eomt",
        "variantId": "coco-panoptic-large-1280",
        "variantName": "COCO Panoptic Large 1280"
      },
      "inferenceMs": 3806.836832955014,
      "loadMs": 4377.031083000001,
      "modelId": "eomt-coco-panoptic-large-1280-native",
      "parameterDtype": "torch.float16",
      "placement": {
        "mps:0": 458
      },
      "precision": "fp16",
      "predictionDevices": [
        "mps:0",
        "mps:0"
      ],
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "torchVersion": "2.11.0",
      "version": "4.57.6",
      "totalMs": 8194.951667000001,
      "imagesPerSecond": 0.262685280163101,
      "grade": "moderate"
    }
  }
]

EoMT COCO Panoptic Large 640 Desktop

32 configurations tested
Passed31
Failed1
TestedNov 14, 2026
Runtimetransformers 4.57.6

Quality criteria

No explicit quality gate recorded. Review per-case evidence; full certification is pending.

PrecisionDeviceResultPlatformInference / load
FP32mpsAcceleratedPassedFastmacos · aarch64397 ms / 371 ms
BF16mpsAcceleratedPassedFastmacos · aarch64403 ms / 282 ms
FP16mpsAcceleratedPassedFastmacos · aarch64408 ms / 286 ms
FP32mpsAcceleratedPassedFastmacos · aarch64441 ms / 371 ms
BF16mpsAcceleratedPassedFastmacos · aarch64444 ms / 282 ms
FP32mpsAcceleratedPassedFastmacos · aarch64445 ms / 371 ms
BF16mpsAcceleratedPassedFastmacos · aarch64446 ms / 282 ms
FP32mpsAcceleratedPassedFastmacos · aarch64448 ms / 371 ms
FP16mpsAcceleratedPassedFastmacos · aarch64455 ms / 286 ms
BF16mpsAcceleratedPassedFastmacos · aarch64455 ms / 282 ms
FP16mpsAcceleratedPassedFastmacos · aarch64456 ms / 286 ms
FP16mpsAcceleratedPassedFastmacos · aarch64457 ms / 286 ms
BF16mpsAcceleratedPassedFastmacos · aarch64507 ms / 282 ms
FP16mpsAcceleratedPassedFastmacos · aarch64516 ms / 286 ms
FP32mpsAcceleratedPassedFastmacos · aarch64529 ms / 371 ms
FP32mpsAcceleratedPassedFastmacos · aarch64537 ms / 371 ms
BF16mpsAcceleratedPassedFastmacos · aarch64603 ms / 282 ms
FP16mpsAcceleratedPassedFastmacos · aarch64615 ms / 286 ms
FP16mpsAcceleratedPassedFastNot recorded964 ms / 3879 ms
FP16mpsAcceleratedPassedModerateNot recorded1294 ms / 3879 ms
FP32cpuPassedModeratemacos · aarch641777 ms / 78 ms
FP32cpuPassedModeratemacos · aarch641799 ms / 116 ms
FP32cpuPassedModeratemacos · aarch641815 ms / 78 ms
FP32cpuPassedModeratemacos · aarch641830 ms / 78 ms
FP32cpuPassedModeratemacos · aarch641830 ms / 78 ms
FP32cpuPassedModeratemacos · aarch641859 ms / 116 ms
FP32cpuPassedModeratemacos · aarch641874 ms / 116 ms
FP32cpuPassedModeratemacos · aarch641893 ms / 116 ms
FP32cpuPassedModeratemacos · aarch641925 ms / 78 ms
FP32cpuPassedModeratemacos · aarch641927 ms / 116 ms
FP32cpuPassedSlowmacos · aarch646040 ms / 116 ms
FP32cpuFailed—macos · aarch645296 ms / 78 ms
Test setup & sample details
Quality gate
No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
Tested
2026-11-14T21:19:31.165285+00:00
Environment
{ "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
Runtime
transformers 4.57.6
Artifact revision
dcd130bed9b1ebda7041fd660fddb16f905b9c3b
Fixture
{
  "file": "model-samples/desktop/mobilenet-dog.jpg",
  "size": 83281,
  "sha1": "f1d042827c2f2154a6d8043b8e8f07c1be612941",
  "sha256": "d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1",
  "url": "https://assets.localai.tools/samples/v1/files/d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1/mobilenet-dog.jpg"
}
Rerun when
Checkpoint or runtime revision; Preprocessing or postprocessing; Fixture or quality gate
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "fail",
    "quality": {
      "passed": false,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 77.70008296938613,
      "inferenceMs": 5296.401540981606,
      "totalMs": 5374.101623950992,
      "imagesPerSecond": 0.18880743694796703,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 77.70008296938613,
      "inferenceMs": 1924.6754579944536,
      "totalMs": 2002.3755409638397,
      "imagesPerSecond": 0.5195681151574604,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 77.70008296938613,
      "inferenceMs": 1776.6283339587972,
      "totalMs": 1854.3284169281833,
      "imagesPerSecond": 0.562863926509455,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 77.70008296938613,
      "inferenceMs": 1814.7650830214843,
      "totalMs": 1892.4651659908704,
      "imagesPerSecond": 0.5510355083177239,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 77.70008296938613,
      "inferenceMs": 1829.777541977819,
      "totalMs": 1907.4776249472052,
      "imagesPerSecond": 0.5465145226993513,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 77.70008296938613,
      "inferenceMs": 1830.203665944282,
      "totalMs": 1907.903748913668,
      "imagesPerSecond": 0.5463872784256808,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 115.972374973353,
      "inferenceMs": 6039.950290985871,
      "totalMs": 6155.922665959224,
      "imagesPerSecond": 0.16556427649618535,
      "grade": "slow"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 115.972374973353,
      "inferenceMs": 1926.5651250025257,
      "totalMs": 2042.5374999758787,
      "imagesPerSecond": 0.5190584979569216,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 115.972374973353,
      "inferenceMs": 1798.6600840231404,
      "totalMs": 1914.6324589964934,
      "imagesPerSecond": 0.5559694179476408,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 115.972374973353,
      "inferenceMs": 1873.5294159851037,
      "totalMs": 1989.5017909584567,
      "imagesPerSecond": 0.5337519611210368,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 115.972374973353,
      "inferenceMs": 1859.203708008863,
      "totalMs": 1975.176082982216,
      "imagesPerSecond": 0.5378646759859156,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "provider": "PyTorch CPU",
      "loadMs": 115.972374973353,
      "inferenceMs": 1892.8136669564992,
      "totalMs": 2008.7860419298522,
      "imagesPerSecond": 0.5283140213203997,
      "grade": "moderate"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 281.87954099848866,
      "inferenceMs": 603.4478329820558,
      "totalMs": 885.3273739805445,
      "imagesPerSecond": 1.657144073346496,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 281.87954099848866,
      "inferenceMs": 506.68249995214865,
      "totalMs": 788.5620409506373,
      "imagesPerSecond": 1.9736225350084926,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 281.87954099848866,
      "inferenceMs": 402.5952079682611,
      "totalMs": 684.4747489667498,
      "imagesPerSecond": 2.4838845078325815,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 281.87954099848866,
      "inferenceMs": 444.11112496163696,
      "totalMs": 725.9906659601256,
      "imagesPerSecond": 2.2516886963513505,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 281.87954099848866,
      "inferenceMs": 455.49562497762963,
      "totalMs": 737.3751659761183,
      "imagesPerSecond": 2.195410768323213,
      "grade": "fast"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 281.87954099848866,
      "inferenceMs": 445.60620799893513,
      "totalMs": 727.4857489974238,
      "imagesPerSecond": 2.2441339057879324,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 285.9305420424789,
      "inferenceMs": 615.2746669831686,
      "totalMs": 901.2052090256475,
      "imagesPerSecond": 1.6252903843794302,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 285.9305420424789,
      "inferenceMs": 516.4917499641888,
      "totalMs": 802.4222920066677,
      "imagesPerSecond": 1.9361393479553846,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 285.9305420424789,
      "inferenceMs": 407.85075002349913,
      "totalMs": 693.781292065978,
      "imagesPerSecond": 2.4518773103699893,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 285.9305420424789,
      "inferenceMs": 457.03470800071955,
      "totalMs": 742.9652500431985,
      "imagesPerSecond": 2.188017633003106,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 285.9305420424789,
      "inferenceMs": 455.0348339835182,
      "totalMs": 740.9653760259971,
      "imagesPerSecond": 2.197633950890496,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 285.9305420424789,
      "inferenceMs": 456.17958396906033,
      "totalMs": 742.1101260115393,
      "imagesPerSecond": 2.1921191459278973,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 371.2309999973513,
      "inferenceMs": 537.4406250193715,
      "totalMs": 908.6716250167228,
      "imagesPerSecond": 1.8606706554123182,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 371.2309999973513,
      "inferenceMs": 528.5210000001825,
      "totalMs": 899.7519999975339,
      "imagesPerSecond": 1.8920724058261726,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": null
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 371.2309999973513,
      "inferenceMs": 397.4365000030957,
      "totalMs": 768.667500000447,
      "imagesPerSecond": 2.516125217467975,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 371.2309999973513,
      "inferenceMs": 441.2139590131119,
      "totalMs": 812.4449590104632,
      "imagesPerSecond": 2.2664740758355792,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 371.2309999973513,
      "inferenceMs": 444.7900840314105,
      "totalMs": 816.0210840287618,
      "imagesPerSecond": 2.248251559334181,
      "grade": "fast"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "pointRecall": 1
    },
    "runtime": {
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "provider": "Metal/MPS",
      "loadMs": 371.2309999973513,
      "inferenceMs": 448.1362079968676,
      "totalMs": 819.3672079942189,
      "imagesPerSecond": 2.231464412281968,
      "grade": "fast"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Native output for same fixture and configuration as spatial qualification"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
      "artifactRevision": "dcd130bed9b1ebda7041fd660fddb16f905b9c3b",
      "cpuThreads": 4,
      "device": "mps",
      "durationMs": 1306.739292,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "family": {
        "catalogId": "eomt-native",
        "defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
        "displayName": "EoMT",
        "id": "eomt",
        "variantId": "coco-panoptic-large-640",
        "variantName": "COCO Panoptic Large 640"
      },
      "inferenceMs": 1293.7580840080045,
      "loadMs": 3879.1905,
      "modelId": "eomt-coco-panoptic-large-640-native",
      "parameterDtype": "torch.float16",
      "placement": {
        "mps:0": 458
      },
      "precision": "fp16",
      "predictionDevices": [
        "mps:0",
        "mps:0"
      ],
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "torchVersion": "2.11.0",
      "version": "4.57.6",
      "totalMs": 5185.929792,
      "imagesPerSecond": 0.7729420301684569,
      "grade": "moderate"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scope": "Native output for same fixture and configuration as spatial qualification"
    },
    "runtime": {
      "acceleration": "partial",
      "accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
      "artifactRevision": "dcd130bed9b1ebda7041fd660fddb16f905b9c3b",
      "cpuThreads": 4,
      "device": "mps",
      "durationMs": 974.563083,
      "engine": "transformers",
      "fallback": false,
      "fallbackReason": null,
      "family": {
        "catalogId": "eomt-native",
        "defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
        "displayName": "EoMT",
        "id": "eomt",
        "variantId": "coco-panoptic-large-640",
        "variantName": "COCO Panoptic Large 640"
      },
      "inferenceMs": 963.8849589973688,
      "loadMs": 3879.1905,
      "modelId": "eomt-coco-panoptic-large-640-native",
      "parameterDtype": "torch.float16",
      "placement": {
        "mps:0": 458
      },
      "precision": "fp16",
      "predictionDevices": [
        "mps:0",
        "mps:0"
      ],
      "provider": "Metal/MPS",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "reused": false,
      "selectionReason": "Explicit device selection; actual placement is reported after inference",
      "torchVersion": "2.11.0",
      "version": "4.57.6",
      "totalMs": 4853.753583,
      "imagesPerSecond": 1.037468206828539,
      "grade": "fast"
    }
  }
]
Release validation still required
  • Managed runtime, full precision lifecycle, backup and forced recovery require qualification.

Something not quite right?

Tell us what happened with EoMT Desktop. Loading, results, or the page itself—we’re here for all of it.

Attachments Optional

Up to 3 files · 50 MB each

Browser, OS, CPU/GPU, memory, screen, storage, and network details are included automatically. Model inputs, results, and browsing history are not collected.