EoMT Desktop
Separate common objects and background regions in still images into pixel masks with class labels.
Desktop app
Run EoMT Desktop on your computer
Download the desktop app to load this model, provide input and view results locally.
Public desktop downloads are coming soon.Device requirements
If your device falls short
MemoryAdvisory only: close other apps, reduce image/tile/token/batch settings or choose a smaller verified model/precision. CPU can need more RAM than GPU.
GPUIf the selected GPU runtime fails, existing CPU fallback may recover when enabled, but CPU has its own memory/runtime needs.
PlatformUse a compatible OS/architecture/runtime or another model. These estimates do not add hardware detection or block runs.
Benchmark evidence
Saved desktop measurements apply to the recorded hardware and runtime. Fallback cases report the device that actually ran; they do not prove GPU support on the requested device.
EoMT COCO Panoptic Small 640 2x Desktop
26 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 226 ms / 122 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 233 ms / 116 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 240 ms / 117 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 251 ms / 117 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 251 ms / 122 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 257 ms / 117 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 261 ms / 116 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 264 ms / 122 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 267 ms / 122 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 268 ms / 117 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 270 ms / 116 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 276 ms / 116 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 352 ms / 117 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 355 ms / 116 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 366 ms / 122 ms |
FP16 | mpsAccelerated | Passed | Fast | Not recorded | 436 ms / 3118 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 451 ms / 116 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 452 ms / 117 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 502 ms / 122 ms |
FP16 | mpsAccelerated | Passed | Moderate | Not recorded | 1779 ms / 3118 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 396 ms / 70 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 430 ms / 70 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 437 ms / 70 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 437 ms / 70 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 511 ms / 70 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 905 ms / 70 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-14T21:10:51.081407+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
- Runtime
- transformers 4.57.6
- Artifact revision
- 10f5326794a8ce7e0e975791033ae1c13aec3767
- Fixture
{ "file": "model-samples/desktop/mobilenet-dog.jpg", "size": 83281, "sha1": "f1d042827c2f2154a6d8043b8e8f07c1be612941", "sha256": "d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1", "url": "https://assets.localai.tools/samples/v1/files/d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1/mobilenet-dog.jpg" }- Rerun when
- Checkpoint or runtime revision; Preprocessing or postprocessing; Fixture or quality gate
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 70.15149999642745,
"inferenceMs": 904.7755420324393,
"totalMs": 974.9270420288667,
"imagesPerSecond": 1.105246498765488,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 70.15149999642745,
"inferenceMs": 511.2899169907905,
"totalMs": 581.441416987218,
"imagesPerSecond": 1.9558375136469046,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 70.15149999642745,
"inferenceMs": 395.8383330027573,
"totalMs": 465.9898329991847,
"imagesPerSecond": 2.526283880629202,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 70.15149999642745,
"inferenceMs": 436.6907919757068,
"totalMs": 506.84229197213426,
"imagesPerSecond": 2.2899498188998457,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 70.15149999642745,
"inferenceMs": 437.0860829949379,
"totalMs": 507.23758299136534,
"imagesPerSecond": 2.287878838758592,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 70.15149999642745,
"inferenceMs": 429.8469999921508,
"totalMs": 499.99849998857826,
"imagesPerSecond": 2.326409164233461,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 116.48004199378192,
"inferenceMs": 451.4445000095293,
"totalMs": 567.9245420033112,
"imagesPerSecond": 2.2151117135747396,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 116.48004199378192,
"inferenceMs": 354.55058398656547,
"totalMs": 471.0306259803474,
"imagesPerSecond": 2.8204720148984204,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 116.48004199378192,
"inferenceMs": 232.7362920041196,
"totalMs": 349.21633399790153,
"imagesPerSecond": 4.296708482329431,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 116.48004199378192,
"inferenceMs": 261.2045829882845,
"totalMs": 377.68462498206645,
"imagesPerSecond": 3.828416747361786,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 116.48004199378192,
"inferenceMs": 276.177000021562,
"totalMs": 392.6570420153439,
"imagesPerSecond": 3.6208663281950595,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 116.48004199378192,
"inferenceMs": 270.07074997527525,
"totalMs": 386.5507919690572,
"imagesPerSecond": 3.702733450740405,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 116.75829102750868,
"inferenceMs": 452.4747079703957,
"totalMs": 569.2329989979044,
"imagesPerSecond": 2.2100682809113557,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 116.75829102750868,
"inferenceMs": 352.3067500209436,
"totalMs": 469.06504104845226,
"imagesPerSecond": 2.838435539314966,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 116.75829102750868,
"inferenceMs": 240.00308301765472,
"totalMs": 356.7613740451634,
"imagesPerSecond": 4.166613142742169,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 116.75829102750868,
"inferenceMs": 251.07966701034456,
"totalMs": 367.83795803785324,
"imagesPerSecond": 3.98279961060646,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 116.75829102750868,
"inferenceMs": 256.86374999349937,
"totalMs": 373.62204102100804,
"imagesPerSecond": 3.8931145403946945,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 116.75829102750868,
"inferenceMs": 268.16470798803493,
"totalMs": 384.9229990155436,
"imagesPerSecond": 3.729051475500715,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 122.09537497255951,
"inferenceMs": 502.0645420299843,
"totalMs": 624.1599170025438,
"imagesPerSecond": 1.9917757903331041,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 122.09537497255951,
"inferenceMs": 366.10283399932086,
"totalMs": 488.1982089718804,
"imagesPerSecond": 2.7314729828118596,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 122.09537497255951,
"inferenceMs": 226.47554200375453,
"totalMs": 348.57091697631404,
"imagesPerSecond": 4.415487832162565,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 122.09537497255951,
"inferenceMs": 251.20533403242007,
"totalMs": 373.3007090049796,
"imagesPerSecond": 3.980807190467309,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 122.09537497255951,
"inferenceMs": 267.2842090250924,
"totalMs": 389.37958399765193,
"imagesPerSecond": 3.7413358748257397,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 122.09537497255951,
"inferenceMs": 263.88716604560614,
"totalMs": 385.98254101816565,
"imagesPerSecond": 3.789498424592485,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Native output for same fixture and configuration as spatial qualification"
},
"runtime": {
"acceleration": "partial",
"accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
"artifactRevision": "10f5326794a8ce7e0e975791033ae1c13aec3767",
"cpuThreads": 4,
"device": "mps",
"durationMs": 1791.068292,
"engine": "transformers",
"fallback": false,
"fallbackReason": null,
"family": {
"catalogId": "eomt-native",
"defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
"displayName": "EoMT",
"id": "eomt",
"variantId": "coco-panoptic-small-640-2x",
"variantName": "COCO Panoptic Small 640 2x"
},
"inferenceMs": 1778.7012500339188,
"loadMs": 3118.425292,
"modelId": "eomt-coco-panoptic-small-640-2x-native",
"parameterDtype": "torch.float16",
"placement": {
"mps:0": 242
},
"precision": "fp16",
"predictionDevices": [
"mps:0",
"mps:0"
],
"provider": "Metal/MPS",
"requestedDevice": "mps",
"requestedPrecision": "fp16",
"reused": false,
"selectionReason": "Explicit device selection; actual placement is reported after inference",
"torchVersion": "2.11.0",
"version": "4.57.6",
"totalMs": 4909.493584,
"imagesPerSecond": 0.5622079593079111,
"grade": "moderate"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Native output for same fixture and configuration as spatial qualification"
},
"runtime": {
"acceleration": "partial",
"accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
"artifactRevision": "10f5326794a8ce7e0e975791033ae1c13aec3767",
"cpuThreads": 4,
"device": "mps",
"durationMs": 448.009708,
"engine": "transformers",
"fallback": false,
"fallbackReason": null,
"family": {
"catalogId": "eomt-native",
"defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
"displayName": "EoMT",
"id": "eomt",
"variantId": "coco-panoptic-small-640-2x",
"variantName": "COCO Panoptic Small 640 2x"
},
"inferenceMs": 436.3478330196813,
"loadMs": 3118.425292,
"modelId": "eomt-coco-panoptic-small-640-2x-native",
"parameterDtype": "torch.float16",
"placement": {
"mps:0": 242
},
"precision": "fp16",
"predictionDevices": [
"mps:0",
"mps:0"
],
"provider": "Metal/MPS",
"requestedDevice": "mps",
"requestedPrecision": "fp16",
"reused": false,
"selectionReason": "Explicit device selection; actual placement is reported after inference",
"torchVersion": "2.11.0",
"version": "4.57.6",
"totalMs": 3566.435,
"imagesPerSecond": 2.291749664664647,
"grade": "fast"
}
}
]EoMT ADE20K Panoptic Giant 640 Desktop
26 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1064 ms / 3052 ms |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1088 ms / 656 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1118 ms / 3052 ms |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1138 ms / 845 ms |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1154 ms / 845 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1161 ms / 3052 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1162 ms / 3052 ms |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1171 ms / 656 ms |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1176 ms / 656 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1183 ms / 3052 ms |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1186 ms / 845 ms |
FP16 | mpsAccelerated | Passed | Moderate | Not recorded | 1187 ms / 6672 ms |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1188 ms / 656 ms |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1200 ms / 845 ms |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1219 ms / 845 ms |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1263 ms / 656 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1264 ms / 3052 ms |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1378 ms / 656 ms |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1407 ms / 845 ms |
FP16 | mpsAccelerated | Passed | Moderate | Not recorded | 1815 ms / 6672 ms |
FP32 | cpu | Passed | Slow | macos · aarch64 | 3990 ms / 79 ms |
FP32 | cpu | Passed | Slow | macos · aarch64 | 3998 ms / 79 ms |
FP32 | cpu | Passed | Slow | macos · aarch64 | 4030 ms / 79 ms |
FP32 | cpu | Passed | Slow | macos · aarch64 | 4081 ms / 79 ms |
FP32 | cpu | Passed | Slow | macos · aarch64 | 4106 ms / 79 ms |
FP32 | cpu | Passed | Slow | macos · aarch64 | 4534 ms / 79 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-14T22:51:40.911723+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
- Runtime
- transformers 4.57.6
- Artifact revision
- bb8de35cf2dc3827ea4a6ff06024752eeb18a37c
- Fixture
{ "file": "model-samples/desktop/mobilenet-dog.jpg", "size": 83281, "sha1": "f1d042827c2f2154a6d8043b8e8f07c1be612941", "sha256": "d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1", "url": "https://assets.localai.tools/samples/v1/files/d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1/mobilenet-dog.jpg" }- Rerun when
- Checkpoint or runtime revision; Preprocessing or postprocessing; Fixture or quality gate
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"attentionImplementation": "eager",
"loadMs": 78.9822080405429,
"inferenceMs": 4534.354167000856,
"totalMs": 4613.336375041399,
"imagesPerSecond": 0.22053857355863027,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"attentionImplementation": "eager",
"loadMs": 78.9822080405429,
"inferenceMs": 3997.847874998115,
"totalMs": 4076.830083038658,
"imagesPerSecond": 0.2501345802209824,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"attentionImplementation": "eager",
"loadMs": 78.9822080405429,
"inferenceMs": 3989.517457957845,
"totalMs": 4068.499665998388,
"imagesPerSecond": 0.25065688032153144,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"attentionImplementation": "eager",
"loadMs": 78.9822080405429,
"inferenceMs": 4105.786125001032,
"totalMs": 4184.768333041575,
"imagesPerSecond": 0.24355871678526575,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"attentionImplementation": "eager",
"loadMs": 78.9822080405429,
"inferenceMs": 4081.0487920534797,
"totalMs": 4160.031000094023,
"imagesPerSecond": 0.2450350512709321,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"attentionImplementation": "eager",
"loadMs": 78.9822080405429,
"inferenceMs": 4029.6836660127155,
"totalMs": 4108.665874053258,
"imagesPerSecond": 0.24815843695976222,
"grade": "slow"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"attentionImplementation": "eager",
"loadMs": 655.7425830396824,
"inferenceMs": 1377.691750007216,
"totalMs": 2033.4343330468982,
"imagesPerSecond": 0.7258517734426169,
"grade": "moderate"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"attentionImplementation": "eager",
"loadMs": 655.7425830396824,
"inferenceMs": 1171.2372910114937,
"totalMs": 1826.979874051176,
"imagesPerSecond": 0.8537979516826935,
"grade": "moderate"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"attentionImplementation": "eager",
"loadMs": 655.7425830396824,
"inferenceMs": 1088.1721670157276,
"totalMs": 1743.91475005541,
"imagesPerSecond": 0.9189722272923626,
"grade": "moderate"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"attentionImplementation": "eager",
"loadMs": 655.7425830396824,
"inferenceMs": 1263.4776670020074,
"totalMs": 1919.2202500416897,
"imagesPerSecond": 0.7914663045630321,
"grade": "moderate"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"attentionImplementation": "eager",
"loadMs": 655.7425830396824,
"inferenceMs": 1175.8374579949304,
"totalMs": 1831.5800410346128,
"imagesPerSecond": 0.8504576829056176,
"grade": "moderate"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"attentionImplementation": "eager",
"loadMs": 655.7425830396824,
"inferenceMs": 1187.9299589782022,
"totalMs": 1843.6725420178846,
"imagesPerSecond": 0.8418004718562279,
"grade": "moderate"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"attentionImplementation": "eager",
"loadMs": 3051.7537919804454,
"inferenceMs": 1263.995999994222,
"totalMs": 4315.7497919746675,
"imagesPerSecond": 0.7911417441230598,
"grade": "moderate"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"attentionImplementation": "eager",
"loadMs": 3051.7537919804454,
"inferenceMs": 1118.3492499985732,
"totalMs": 4170.103041979019,
"imagesPerSecond": 0.8941750530983732,
"grade": "moderate"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"attentionImplementation": "eager",
"loadMs": 3051.7537919804454,
"inferenceMs": 1063.70912498096,
"totalMs": 4115.462916961405,
"imagesPerSecond": 0.940106629260983,
"grade": "moderate"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"attentionImplementation": "eager",
"loadMs": 3051.7537919804454,
"inferenceMs": 1161.6565000149421,
"totalMs": 4213.4102919953875,
"imagesPerSecond": 0.8608396716130261,
"grade": "moderate"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"attentionImplementation": "eager",
"loadMs": 3051.7537919804454,
"inferenceMs": 1160.8822079724632,
"totalMs": 4212.635999952909,
"imagesPerSecond": 0.8614138395199873,
"grade": "moderate"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"attentionImplementation": "eager",
"loadMs": 3051.7537919804454,
"inferenceMs": 1182.6251659658737,
"totalMs": 4234.378957946319,
"imagesPerSecond": 0.8455764588632612,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"attentionImplementation": "eager",
"loadMs": 844.9827910517342,
"inferenceMs": 1407.2576250182465,
"totalMs": 2252.2404160699807,
"imagesPerSecond": 0.7106019411243439,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"attentionImplementation": "eager",
"loadMs": 844.9827910517342,
"inferenceMs": 1153.8906670175493,
"totalMs": 1998.8734580692835,
"imagesPerSecond": 0.8666332336188236,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"attentionImplementation": "eager",
"loadMs": 844.9827910517342,
"inferenceMs": 1137.6059169997461,
"totalMs": 1982.5887080514804,
"imagesPerSecond": 0.8790390284162202,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"attentionImplementation": "eager",
"loadMs": 844.9827910517342,
"inferenceMs": 1199.9884170363657,
"totalMs": 2044.9712080881,
"imagesPerSecond": 0.8333413771357219,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"attentionImplementation": "eager",
"loadMs": 844.9827910517342,
"inferenceMs": 1185.9302920056507,
"totalMs": 2030.913083057385,
"imagesPerSecond": 0.8432198812535562,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"attentionImplementation": "eager",
"loadMs": 844.9827910517342,
"inferenceMs": 1219.370125036221,
"totalMs": 2064.3529160879552,
"imagesPerSecond": 0.8200955390557033,
"grade": "moderate"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Native output for same fixture and configuration as spatial qualification"
},
"runtime": {
"acceleration": "partial",
"accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
"artifactRevision": "bb8de35cf2dc3827ea4a6ff06024752eeb18a37c",
"cpuThreads": 4,
"device": "mps",
"durationMs": 1828.207458,
"engine": "transformers",
"fallback": false,
"fallbackReason": null,
"family": {
"catalogId": "eomt-native",
"defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
"displayName": "EoMT Desktop",
"id": "eomt",
"variantId": "ade20k-panoptic-giant-640",
"variantName": "ADE20K Panoptic Giant 640"
},
"inferenceMs": 1815.219874959439,
"loadMs": 6671.750083,
"modelId": "eomt-ade20k-panoptic-giant-640-native",
"parameterDtype": "torch.float16",
"placement": {
"mps:0": 746
},
"precision": "fp16",
"predictionDevices": [
"mps:0",
"mps:0"
],
"provider": "Metal/MPS",
"requestedDevice": "mps",
"requestedPrecision": "fp16",
"reused": false,
"selectionReason": "Explicit device selection; actual placement is reported after inference",
"torchVersion": "2.11.0",
"version": "4.57.6",
"totalMs": 8499.957541,
"imagesPerSecond": 0.5508974498322662,
"grade": "moderate"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Native output for same fixture and configuration as spatial qualification"
},
"runtime": {
"acceleration": "partial",
"accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
"artifactRevision": "bb8de35cf2dc3827ea4a6ff06024752eeb18a37c",
"cpuThreads": 4,
"device": "mps",
"durationMs": 1227.761208,
"engine": "transformers",
"fallback": false,
"fallbackReason": null,
"family": {
"catalogId": "eomt-native",
"defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
"displayName": "EoMT Desktop",
"id": "eomt",
"variantId": "ade20k-panoptic-giant-640",
"variantName": "ADE20K Panoptic Giant 640"
},
"inferenceMs": 1186.7824159562588,
"loadMs": 6671.750083,
"modelId": "eomt-ade20k-panoptic-giant-640-native",
"parameterDtype": "torch.float16",
"placement": {
"mps:0": 746
},
"precision": "fp16",
"predictionDevices": [
"mps:0",
"mps:0"
],
"provider": "Metal/MPS",
"requestedDevice": "mps",
"requestedPrecision": "fp16",
"reused": false,
"selectionReason": "Explicit device selection; actual placement is reported after inference",
"torchVersion": "2.11.0",
"version": "4.57.6",
"totalMs": 7899.511291,
"imagesPerSecond": 0.8426144393066716,
"grade": "moderate"
}
}
]EoMT ADE20K Panoptic Large 1280 Desktop
26 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
FP16 | mpsAccelerated | Passed | Moderate | Not recorded | 3799 ms / 5017 ms |
FP16 | mpsAccelerated | Passed | Moderate | Not recorded | 4138 ms / 5017 ms |
FP32 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3295 ms / 377 ms |
FP32 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3391 ms / 377 ms |
FP32 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3436 ms / 377 ms |
FP32 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3458 ms / 377 ms |
FP32 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3465 ms / 377 ms |
BF16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3706 ms / 2886 ms |
FP32 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3806 ms / 377 ms |
BF16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3821 ms / 2886 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3821 ms / 296 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3866 ms / 296 ms |
BF16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3875 ms / 2886 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3875 ms / 296 ms |
BF16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3892 ms / 2886 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3904 ms / 296 ms |
BF16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3978 ms / 2886 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3998 ms / 296 ms |
BF16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 4109 ms / 2886 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 4192 ms / 296 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 14292 ms / 73 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 13663 ms / 73 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 13533 ms / 73 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 13746 ms / 73 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 14472 ms / 73 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 13916 ms / 73 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-14T21:44:25.405282+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
- Runtime
- transformers 4.57.6
- Artifact revision
- 6ffa3eea77f099f26d0541a86c7920d24cc743e6
- Fixture
{ "file": "model-samples/desktop/mobilenet-dog.jpg", "size": 83281, "sha1": "f1d042827c2f2154a6d8043b8e8f07c1be612941", "sha256": "d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1", "url": "https://assets.localai.tools/samples/v1/files/d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1/mobilenet-dog.jpg" }- Rerun when
- Checkpoint or runtime revision; Preprocessing or postprocessing; Fixture or quality gate
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 72.94116698903963,
"inferenceMs": 14291.612625005655,
"totalMs": 14364.553791994695,
"imagesPerSecond": 0.06997111006565673,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 72.94116698903963,
"inferenceMs": 13663.021667045541,
"totalMs": 13735.96283403458,
"imagesPerSecond": 0.0731902520810565,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": false,
"pointRecall": null
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 72.94116698903963,
"inferenceMs": 13532.808499992825,
"totalMs": 13605.749666981865,
"imagesPerSecond": 0.07389449130241739,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 72.94116698903963,
"inferenceMs": 13745.805832964834,
"totalMs": 13818.746999953873,
"imagesPerSecond": 0.07274946352012525,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 72.94116698903963,
"inferenceMs": 14471.914499998093,
"totalMs": 14544.855666987132,
"imagesPerSecond": 0.06909935793223017,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 72.94116698903963,
"inferenceMs": 13915.923041000497,
"totalMs": 13988.864207989536,
"imagesPerSecond": 0.07186012721209359,
"grade": "slow"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 2885.7585419900715,
"inferenceMs": 4108.745916979387,
"totalMs": 6994.504458969459,
"imagesPerSecond": 0.2433832658932501,
"grade": "slow"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 2885.7585419900715,
"inferenceMs": 3820.54991699988,
"totalMs": 6706.308458989952,
"imagesPerSecond": 0.26174242497144456,
"grade": "slow"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 2885.7585419900715,
"inferenceMs": 3706.245334004052,
"totalMs": 6592.0038759941235,
"imagesPerSecond": 0.26981484221381735,
"grade": "slow"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 2885.7585419900715,
"inferenceMs": 3891.557708964683,
"totalMs": 6777.316250954755,
"imagesPerSecond": 0.256966509245482,
"grade": "slow"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 2885.7585419900715,
"inferenceMs": 3875.1974159968086,
"totalMs": 6760.95595798688,
"imagesPerSecond": 0.25805136942752943,
"grade": "slow"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 2885.7585419900715,
"inferenceMs": 3978.3199580269866,
"totalMs": 6864.078500017058,
"imagesPerSecond": 0.25136238677392386,
"grade": "slow"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 295.57708295760676,
"inferenceMs": 4192.050500016194,
"totalMs": 4487.627582973801,
"imagesPerSecond": 0.23854674460532788,
"grade": "slow"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 295.57708295760676,
"inferenceMs": 3903.579832985997,
"totalMs": 4199.156915943604,
"imagesPerSecond": 0.2561751117653105,
"grade": "slow"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 295.57708295760676,
"inferenceMs": 3820.678042015061,
"totalMs": 4116.2551249726675,
"imagesPerSecond": 0.26173364753670547,
"grade": "slow"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 295.57708295760676,
"inferenceMs": 3875.2151670050807,
"totalMs": 4170.7922499626875,
"imagesPerSecond": 0.2580501873842632,
"grade": "slow"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 295.57708295760676,
"inferenceMs": 3866.1919579608366,
"totalMs": 4161.769040918443,
"imagesPerSecond": 0.2586524442845912,
"grade": "slow"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 295.57708295760676,
"inferenceMs": 3997.9276669910178,
"totalMs": 4293.5047499486245,
"imagesPerSecond": 0.2501295879504082,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 376.562167017255,
"inferenceMs": 3806.1767920153216,
"totalMs": 4182.7389590325765,
"imagesPerSecond": 0.26273083323344865,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 376.562167017255,
"inferenceMs": 3464.961250021588,
"totalMs": 3841.523417038843,
"imagesPerSecond": 0.28860351612698976,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 376.562167017255,
"inferenceMs": 3295.2619590214454,
"totalMs": 3671.8241260387003,
"imagesPerSecond": 0.3034660104221147,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 376.562167017255,
"inferenceMs": 3391.2080000154674,
"totalMs": 3767.7701670327224,
"imagesPerSecond": 0.29488017249176074,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 376.562167017255,
"inferenceMs": 3436.339041974861,
"totalMs": 3812.901208992116,
"imagesPerSecond": 0.2910073737733692,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 376.562167017255,
"inferenceMs": 3458.3000410348177,
"totalMs": 3834.8622080520727,
"imagesPerSecond": 0.28915941015365826,
"grade": "slow"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Native output for same fixture and configuration as spatial qualification"
},
"runtime": {
"acceleration": "partial",
"accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
"artifactRevision": "6ffa3eea77f099f26d0541a86c7920d24cc743e6",
"cpuThreads": 4,
"device": "mps",
"durationMs": 4151.007541,
"engine": "transformers",
"fallback": false,
"fallbackReason": null,
"family": {
"catalogId": "eomt-native",
"defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
"displayName": "EoMT Desktop",
"id": "eomt",
"variantId": "ade20k-panoptic-large-1280",
"variantName": "ADE20K Panoptic Large 1280"
},
"inferenceMs": 4138.230875018053,
"loadMs": 5017.085416,
"modelId": "eomt-ade20k-panoptic-large-1280-native",
"parameterDtype": "torch.float16",
"placement": {
"mps:0": 458
},
"precision": "fp16",
"predictionDevices": [
"mps:0",
"mps:0"
],
"provider": "Metal/MPS",
"requestedDevice": "mps",
"requestedPrecision": "fp16",
"reused": false,
"selectionReason": "Explicit device selection; actual placement is reported after inference",
"torchVersion": "2.11.0",
"version": "4.57.6",
"totalMs": 9168.092957,
"imagesPerSecond": 0.24164915641533355,
"grade": "moderate"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Native output for same fixture and configuration as spatial qualification"
},
"runtime": {
"acceleration": "partial",
"accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
"artifactRevision": "6ffa3eea77f099f26d0541a86c7920d24cc743e6",
"cpuThreads": 4,
"device": "mps",
"durationMs": 3842.434875,
"engine": "transformers",
"fallback": false,
"fallbackReason": null,
"family": {
"catalogId": "eomt-native",
"defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
"displayName": "EoMT Desktop",
"id": "eomt",
"variantId": "ade20k-panoptic-large-1280",
"variantName": "ADE20K Panoptic Large 1280"
},
"inferenceMs": 3798.851499974262,
"loadMs": 5017.085416,
"modelId": "eomt-ade20k-panoptic-large-1280-native",
"parameterDtype": "torch.float16",
"placement": {
"mps:0": 458
},
"precision": "fp16",
"predictionDevices": [
"mps:0",
"mps:0"
],
"provider": "Metal/MPS",
"requestedDevice": "mps",
"requestedPrecision": "fp16",
"reused": false,
"selectionReason": "Explicit device selection; actual placement is reported after inference",
"torchVersion": "2.11.0",
"version": "4.57.6",
"totalMs": 8859.520291,
"imagesPerSecond": 0.26323745479568633,
"grade": "moderate"
}
}
]EoMT ADE20K Panoptic Large 640 Desktop
26 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 389 ms / 405 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 398 ms / 291 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 398 ms / 275 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 433 ms / 291 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 437 ms / 275 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 444 ms / 405 ms |
FP16 | mpsAccelerated | Passed | Fast | Not recorded | 446 ms / 4795 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 504 ms / 405 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 505 ms / 291 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 506 ms / 405 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 508 ms / 405 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 510 ms / 291 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 515 ms / 275 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 516 ms / 275 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 516 ms / 291 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 517 ms / 275 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 667 ms / 275 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 672 ms / 291 ms |
FP16 | mpsAccelerated | Passed | Fast | Not recorded | 723 ms / 4795 ms |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1062 ms / 405 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1928 ms / 71 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 2011 ms / 71 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 2016 ms / 71 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 2020 ms / 71 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 2074 ms / 71 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 2843 ms / 71 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-14T21:36:13.706942+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
- Runtime
- transformers 4.57.6
- Artifact revision
- 4d0ace83675d88497d6c4a10e17a0f46047685a5
- Fixture
{ "file": "model-samples/desktop/mobilenet-dog.jpg", "size": 83281, "sha1": "f1d042827c2f2154a6d8043b8e8f07c1be612941", "sha256": "d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1", "url": "https://assets.localai.tools/samples/v1/files/d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1/mobilenet-dog.jpg" }- Rerun when
- Checkpoint or runtime revision; Preprocessing or postprocessing; Fixture or quality gate
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 70.96766703762114,
"inferenceMs": 2843.483416014351,
"totalMs": 2914.451083051972,
"imagesPerSecond": 0.35168131959836724,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 70.96766703762114,
"inferenceMs": 1927.7984999935143,
"totalMs": 1998.7661670311354,
"imagesPerSecond": 0.5187264125391551,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 70.96766703762114,
"inferenceMs": 2074.122958991211,
"totalMs": 2145.090626028832,
"imagesPerSecond": 0.4821314935380538,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 70.96766703762114,
"inferenceMs": 2019.82250000583,
"totalMs": 2090.790167043451,
"imagesPerSecond": 0.4950930094090513,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 70.96766703762114,
"inferenceMs": 2016.0378749715164,
"totalMs": 2087.0055420091376,
"imagesPerSecond": 0.4960224271650296,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 70.96766703762114,
"inferenceMs": 2010.5368749937043,
"totalMs": 2081.5045420313254,
"imagesPerSecond": 0.4973795867350761,
"grade": "moderate"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 290.59691599104553,
"inferenceMs": 672.252209042199,
"totalMs": 962.8491250332445,
"imagesPerSecond": 1.48753694900425,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 290.59691599104553,
"inferenceMs": 433.39750001905486,
"totalMs": 723.9944160101004,
"imagesPerSecond": 2.3073506422072896,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 290.59691599104553,
"inferenceMs": 397.66716602025554,
"totalMs": 688.2640820113011,
"imagesPerSecond": 2.5146657442396543,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 290.59691599104553,
"inferenceMs": 510.4521670145914,
"totalMs": 801.0490830056369,
"imagesPerSecond": 1.9590474183870292,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 290.59691599104553,
"inferenceMs": 505.2602499490604,
"totalMs": 795.8571659401059,
"imagesPerSecond": 1.9791780574482527,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 290.59691599104553,
"inferenceMs": 515.7839999883436,
"totalMs": 806.3809159793891,
"imagesPerSecond": 1.9387960852267605,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 275.2712499932386,
"inferenceMs": 667.0129169942811,
"totalMs": 942.2841669875197,
"imagesPerSecond": 1.4992213411791753,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 275.2712499932386,
"inferenceMs": 436.6954590077512,
"totalMs": 711.9667090009898,
"imagesPerSecond": 2.289925345851261,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 275.2712499932386,
"inferenceMs": 398.14145799027756,
"totalMs": 673.4127079835162,
"imagesPerSecond": 2.5116701110398294,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 275.2712499932386,
"inferenceMs": 516.6843329789117,
"totalMs": 791.9555829721503,
"imagesPerSecond": 1.9354176934968428,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 275.2712499932386,
"inferenceMs": 515.7011250266805,
"totalMs": 790.9723750199191,
"imagesPerSecond": 1.9391076564904983,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 275.2712499932386,
"inferenceMs": 515.2240410097875,
"totalMs": 790.4952910030261,
"imagesPerSecond": 1.9409032195782252,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 405.02095804549754,
"inferenceMs": 1062.4704170040786,
"totalMs": 1467.4913750495762,
"imagesPerSecond": 0.9412026763246446,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 405.02095804549754,
"inferenceMs": 444.40691702766344,
"totalMs": 849.427875073161,
"imagesPerSecond": 2.2501899985903053,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 405.02095804549754,
"inferenceMs": 389.44295898545533,
"totalMs": 794.4639170309529,
"imagesPerSecond": 2.567770136620566,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 405.02095804549754,
"inferenceMs": 507.93833303032443,
"totalMs": 912.959291075822,
"imagesPerSecond": 1.9687429260045608,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 405.02095804549754,
"inferenceMs": 506.41833298141137,
"totalMs": 911.4392910269089,
"imagesPerSecond": 1.974652051225614,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 405.02095804549754,
"inferenceMs": 504.480124975089,
"totalMs": 909.5010830205865,
"imagesPerSecond": 1.9822386462685315,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Native output for same fixture and configuration as spatial qualification"
},
"runtime": {
"acceleration": "partial",
"accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
"artifactRevision": "4d0ace83675d88497d6c4a10e17a0f46047685a5",
"cpuThreads": 4,
"device": "mps",
"durationMs": 736.433041,
"engine": "transformers",
"fallback": false,
"fallbackReason": null,
"family": {
"catalogId": "eomt-native",
"defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
"displayName": "EoMT Desktop",
"id": "eomt",
"variantId": "ade20k-panoptic-large-640",
"variantName": "ADE20K Panoptic Large 640"
},
"inferenceMs": 723.2872089953162,
"loadMs": 4795.383291,
"modelId": "eomt-ade20k-panoptic-large-640-native",
"parameterDtype": "torch.float16",
"placement": {
"mps:0": 458
},
"precision": "fp16",
"predictionDevices": [
"mps:0",
"mps:0"
],
"provider": "Metal/MPS",
"requestedDevice": "mps",
"requestedPrecision": "fp16",
"reused": false,
"selectionReason": "Explicit device selection; actual placement is reported after inference",
"torchVersion": "2.11.0",
"version": "4.57.6",
"totalMs": 5531.816332,
"imagesPerSecond": 1.3825766411506881,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Native output for same fixture and configuration as spatial qualification"
},
"runtime": {
"acceleration": "partial",
"accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
"artifactRevision": "4d0ace83675d88497d6c4a10e17a0f46047685a5",
"cpuThreads": 4,
"device": "mps",
"durationMs": 486.724625,
"engine": "transformers",
"fallback": false,
"fallbackReason": null,
"family": {
"catalogId": "eomt-native",
"defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
"displayName": "EoMT Desktop",
"id": "eomt",
"variantId": "ade20k-panoptic-large-640",
"variantName": "ADE20K Panoptic Large 640"
},
"inferenceMs": 446.42116699833423,
"loadMs": 4795.383291,
"modelId": "eomt-ade20k-panoptic-large-640-native",
"parameterDtype": "torch.float16",
"placement": {
"mps:0": 458
},
"precision": "fp16",
"predictionDevices": [
"mps:0",
"mps:0"
],
"provider": "Metal/MPS",
"requestedDevice": "mps",
"requestedPrecision": "fp16",
"reused": false,
"selectionReason": "Explicit device selection; actual placement is reported after inference",
"torchVersion": "2.11.0",
"version": "4.57.6",
"totalMs": 5282.107916,
"imagesPerSecond": 2.2400371530853764,
"grade": "fast"
}
}
]EoMT ADE20K Semantic Large 512 Desktop
26 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 318 ms / 278 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 335 ms / 293 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 340 ms / 456 ms |
FP16 | mpsAccelerated | Passed | Fast | Not recorded | 377 ms / 3123 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 404 ms / 278 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 407 ms / 293 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 625 ms / 278 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 630 ms / 278 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 636 ms / 456 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 640 ms / 278 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 654 ms / 293 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 661 ms / 293 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 662 ms / 293 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 677 ms / 456 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 692 ms / 456 ms |
FP16 | mpsAccelerated | Passed | Fast | Not recorded | 754 ms / 3123 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 781 ms / 278 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 799 ms / 293 ms |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1205 ms / 456 ms |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 2133 ms / 456 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1544 ms / 47 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1580 ms / 47 ms |
FP32 | cpu | Passed | Slow | macos · aarch64 | 3411 ms / 47 ms |
FP32 | cpu | Passed | Slow | macos · aarch64 | 3468 ms / 47 ms |
FP32 | cpu | Passed | Slow | macos · aarch64 | 3613 ms / 47 ms |
FP32 | cpu | Passed | Slow | macos · aarch64 | 3759 ms / 47 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-14T21:19:15.498340+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
- Runtime
- transformers 4.57.6
- Artifact revision
- aeb262f5a0bc609236d27f1277c3173ce57aeb27
- Fixture
{ "file": "model-samples/desktop/mobilenet-dog.jpg", "size": 83281, "sha1": "f1d042827c2f2154a6d8043b8e8f07c1be612941", "sha256": "d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1", "url": "https://assets.localai.tools/samples/v1/files/d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1/mobilenet-dog.jpg" }- Rerun when
- Checkpoint or runtime revision; Preprocessing or postprocessing; Fixture or quality gate
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 46.91187501884997,
"inferenceMs": 3467.647791025229,
"totalMs": 3514.5596660440788,
"imagesPerSecond": 0.2883799221443838,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 46.91187501884997,
"inferenceMs": 1579.9649159889668,
"totalMs": 1626.8767910078168,
"imagesPerSecond": 0.6329254465590824,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 46.91187501884997,
"inferenceMs": 1544.0384169924073,
"totalMs": 1590.9502920112573,
"imagesPerSecond": 0.6476522792404831,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 46.91187501884997,
"inferenceMs": 3613.4377499693073,
"totalMs": 3660.3496249881573,
"imagesPerSecond": 0.2767447702699442,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 46.91187501884997,
"inferenceMs": 3410.6579999788664,
"totalMs": 3457.5698749977164,
"imagesPerSecond": 0.2931985558230102,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 46.91187501884997,
"inferenceMs": 3759.0924999676645,
"totalMs": 3806.0043749865144,
"imagesPerSecond": 0.2660216528347206,
"grade": "slow"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 277.66599995084107,
"inferenceMs": 781.2259580241516,
"totalMs": 1058.8919579749927,
"imagesPerSecond": 1.280039391585456,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 277.66599995084107,
"inferenceMs": 404.2990840389393,
"totalMs": 681.9650839897804,
"imagesPerSecond": 2.4734164371831393,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 277.66599995084107,
"inferenceMs": 318.1022500502877,
"totalMs": 595.7682500011288,
"imagesPerSecond": 3.1436432777256793,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 277.66599995084107,
"inferenceMs": 625.0824580201879,
"totalMs": 902.748457971029,
"imagesPerSecond": 1.5997889353146806,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 277.66599995084107,
"inferenceMs": 640.008290996775,
"totalMs": 917.6742909476161,
"imagesPerSecond": 1.562479758570876,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 277.66599995084107,
"inferenceMs": 629.6731249894947,
"totalMs": 907.3391249403358,
"imagesPerSecond": 1.5881255850274438,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 293.33683400182053,
"inferenceMs": 799.3859160342254,
"totalMs": 1092.722750036046,
"imagesPerSecond": 1.2509602432840279,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 293.33683400182053,
"inferenceMs": 407.45595801854506,
"totalMs": 700.7927920203656,
"imagesPerSecond": 2.4542529819983288,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 293.33683400182053,
"inferenceMs": 335.1195409777574,
"totalMs": 628.456374979578,
"imagesPerSecond": 2.9840098165638516,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 293.33683400182053,
"inferenceMs": 662.3322919476777,
"totalMs": 955.6691259494983,
"imagesPerSecond": 1.5098161635141851,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 293.33683400182053,
"inferenceMs": 653.8968330132775,
"totalMs": 947.233667015098,
"imagesPerSecond": 1.529293230236053,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 293.33683400182053,
"inferenceMs": 660.9414169797674,
"totalMs": 954.2782509815879,
"imagesPerSecond": 1.5129933974626557,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 456.00425003794953,
"inferenceMs": 2132.687708013691,
"totalMs": 2588.6919580516405,
"imagesPerSecond": 0.4688919039775234,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 456.00425003794953,
"inferenceMs": 1204.5093339984305,
"totalMs": 1660.51358403638,
"imagesPerSecond": 0.83021357475118,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 456.00425003794953,
"inferenceMs": 340.41054197587073,
"totalMs": 796.4147920138203,
"imagesPerSecond": 2.9376293524742922,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 456.00425003794953,
"inferenceMs": 692.4639160279185,
"totalMs": 1148.468166065868,
"imagesPerSecond": 1.4441185697244077,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 456.00425003794953,
"inferenceMs": 677.0300409989432,
"totalMs": 1133.0342910368927,
"imagesPerSecond": 1.4770393327370255,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 456.00425003794953,
"inferenceMs": 635.7941669994034,
"totalMs": 1091.798417037353,
"imagesPerSecond": 1.5728360716479148,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Native output for same fixture and configuration as spatial qualification"
},
"runtime": {
"acceleration": "partial",
"accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
"artifactRevision": "aeb262f5a0bc609236d27f1277c3173ce57aeb27",
"cpuThreads": 4,
"device": "mps",
"durationMs": 764.852875,
"engine": "transformers",
"fallback": false,
"fallbackReason": null,
"family": {
"catalogId": "eomt-native",
"defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
"displayName": "EoMT Desktop",
"id": "eomt",
"variantId": "ade20k-semantic-large-512",
"variantName": "ADE20K Semantic Large 512"
},
"inferenceMs": 753.8305000052787,
"loadMs": 3123.454625,
"modelId": "eomt-ade20k-semantic-large-512-native",
"parameterDtype": "torch.float16",
"placement": {
"mps:0": 458
},
"precision": "fp16",
"predictionDevices": [
"mps:0",
"mps:0"
],
"provider": "Metal/MPS",
"requestedDevice": "mps",
"requestedPrecision": "fp16",
"reused": false,
"selectionReason": "Explicit device selection; actual placement is reported after inference",
"torchVersion": "2.11.0",
"version": "4.57.6",
"totalMs": 3888.3075,
"imagesPerSecond": 1.3265581586218618,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Native output for same fixture and configuration as spatial qualification"
},
"runtime": {
"acceleration": "partial",
"accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
"artifactRevision": "aeb262f5a0bc609236d27f1277c3173ce57aeb27",
"cpuThreads": 4,
"device": "mps",
"durationMs": 420.436417,
"engine": "transformers",
"fallback": false,
"fallbackReason": null,
"family": {
"catalogId": "eomt-native",
"defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
"displayName": "EoMT Desktop",
"id": "eomt",
"variantId": "ade20k-semantic-large-512",
"variantName": "ADE20K Semantic Large 512"
},
"inferenceMs": 377.3945000139065,
"loadMs": 3123.454625,
"modelId": "eomt-ade20k-semantic-large-512-native",
"parameterDtype": "torch.float16",
"placement": {
"mps:0": 458
},
"precision": "fp16",
"predictionDevices": [
"mps:0",
"mps:0"
],
"provider": "Metal/MPS",
"requestedDevice": "mps",
"requestedPrecision": "fp16",
"reused": false,
"selectionReason": "Explicit device selection; actual placement is reported after inference",
"torchVersion": "2.11.0",
"version": "4.57.6",
"totalMs": 3543.8910419999997,
"imagesPerSecond": 2.649747147780774,
"grade": "fast"
}
}
]EoMT CITYSCAPES Semantic Large 1024 Desktop
26 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1508 ms / 339 ms |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1550 ms / 339 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1683 ms / 279 ms |
FP16 | mpsAccelerated | Passed | Moderate | Not recorded | 1700 ms / 3339 ms |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1700 ms / 270 ms |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1708 ms / 270 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1717 ms / 279 ms |
FP16 | mpsAccelerated | Passed | Moderate | Not recorded | 3748 ms / 3339 ms |
FP32 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3149 ms / 339 ms |
FP32 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3180 ms / 339 ms |
FP32 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3195 ms / 339 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3392 ms / 279 ms |
FP32 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3400 ms / 339 ms |
BF16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3416 ms / 270 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3416 ms / 279 ms |
BF16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3449 ms / 270 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3460 ms / 279 ms |
BF16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3481 ms / 270 ms |
BF16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3738 ms / 270 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3807 ms / 279 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 15971 ms / 75 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 6419 ms / 75 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 6420 ms / 75 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 15741 ms / 75 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 15567 ms / 75 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 15231 ms / 75 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-14T21:51:44.238813+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
- Runtime
- transformers 4.57.6
- Artifact revision
- 8d6b6d1a3f7b50d441afd7d247c2ed10db186e8f
- Fixture
{ "file": "model-samples/desktop/mobilenet-dog.jpg", "size": 83281, "sha1": "f1d042827c2f2154a6d8043b8e8f07c1be612941", "sha256": "d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1", "url": "https://assets.localai.tools/samples/v1/files/d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1/mobilenet-dog.jpg" }- Rerun when
- Checkpoint or runtime revision; Preprocessing or postprocessing; Fixture or quality gate
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 74.86766699003056,
"inferenceMs": 15971.039334021043,
"totalMs": 16045.907001011074,
"imagesPerSecond": 0.06261333273845422,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 74.86766699003056,
"inferenceMs": 6419.192541972734,
"totalMs": 6494.060208962765,
"imagesPerSecond": 0.1557828330372346,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": false,
"pointRecall": null
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 74.86766699003056,
"inferenceMs": 6419.963833002839,
"totalMs": 6494.83149999287,
"imagesPerSecond": 0.15576411737077736,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 74.86766699003056,
"inferenceMs": 15741.11304199323,
"totalMs": 15815.980708983261,
"imagesPerSecond": 0.06352790919754263,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 74.86766699003056,
"inferenceMs": 15566.559666011017,
"totalMs": 15641.427333001047,
"imagesPerSecond": 0.06424027026237926,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 74.86766699003056,
"inferenceMs": 15231.452291016467,
"totalMs": 15306.319958006497,
"imagesPerSecond": 0.06565362126300993,
"grade": "slow"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 269.6552079869434,
"inferenceMs": 3738.159666012507,
"totalMs": 4007.8148739994504,
"imagesPerSecond": 0.2675113128773067,
"grade": "slow"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 269.6552079869434,
"inferenceMs": 1699.8922500060871,
"totalMs": 1969.5474579930305,
"imagesPerSecond": 0.5882725802158455,
"grade": "moderate"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 269.6552079869434,
"inferenceMs": 1708.4945410024375,
"totalMs": 1978.149748989381,
"imagesPerSecond": 0.5853106205497518,
"grade": "moderate"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 269.6552079869434,
"inferenceMs": 3481.155708024744,
"totalMs": 3750.8109160116874,
"imagesPerSecond": 0.28726092248468077,
"grade": "slow"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 269.6552079869434,
"inferenceMs": 3449.4774999911897,
"totalMs": 3719.132707978133,
"imagesPerSecond": 0.2898989774545722,
"grade": "slow"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 269.6552079869434,
"inferenceMs": 3415.9150830237195,
"totalMs": 3685.570291010663,
"imagesPerSecond": 0.29274732412692595,
"grade": "slow"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 279.33120797388256,
"inferenceMs": 3806.640749971848,
"totalMs": 4085.9719579457305,
"imagesPerSecond": 0.2626988112832543,
"grade": "slow"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 279.33120797388256,
"inferenceMs": 1717.1941670239903,
"totalMs": 1996.5253749978729,
"imagesPerSecond": 0.5823453277465211,
"grade": "moderate"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 279.33120797388256,
"inferenceMs": 1682.9552080016583,
"totalMs": 1962.2864159755409,
"imagesPerSecond": 0.5941928788392415,
"grade": "moderate"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 279.33120797388256,
"inferenceMs": 3459.8387500154786,
"totalMs": 3739.169957989361,
"imagesPerSecond": 0.28903081104445294,
"grade": "slow"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 279.33120797388256,
"inferenceMs": 3392.320708022453,
"totalMs": 3671.6519159963354,
"imagesPerSecond": 0.29478344946428964,
"grade": "slow"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 279.33120797388256,
"inferenceMs": 3416.326874983497,
"totalMs": 3695.6580829573795,
"imagesPerSecond": 0.2927120373997674,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 338.5887499898672,
"inferenceMs": 3399.805249995552,
"totalMs": 3738.393999985419,
"imagesPerSecond": 0.2941344949100565,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 338.5887499898672,
"inferenceMs": 1549.7517080511898,
"totalMs": 1888.340458041057,
"imagesPerSecond": 0.6452646542054781,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 338.5887499898672,
"inferenceMs": 1508.0046249786392,
"totalMs": 1846.5933749685064,
"imagesPerSecond": 0.6631279396866339,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 338.5887499898672,
"inferenceMs": 3194.946291972883,
"totalMs": 3533.5350419627503,
"imagesPerSecond": 0.31299430682526397,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 338.5887499898672,
"inferenceMs": 3148.639541992452,
"totalMs": 3487.228291982319,
"imagesPerSecond": 0.3175974850926258,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 338.5887499898672,
"inferenceMs": 3180.060707964003,
"totalMs": 3518.6494579538703,
"imagesPerSecond": 0.31445940560054225,
"grade": "slow"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Native output for same fixture and configuration as spatial qualification"
},
"runtime": {
"acceleration": "partial",
"accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
"artifactRevision": "8d6b6d1a3f7b50d441afd7d247c2ed10db186e8f",
"cpuThreads": 4,
"device": "mps",
"durationMs": 3759.5588749999997,
"engine": "transformers",
"fallback": false,
"fallbackReason": null,
"family": {
"catalogId": "eomt-native",
"defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
"displayName": "EoMT Desktop",
"id": "eomt",
"variantId": "cityscapes-semantic-large-1024",
"variantName": "CITYSCAPES Semantic Large 1024"
},
"inferenceMs": 3747.7345000370406,
"loadMs": 3339.24625,
"modelId": "eomt-cityscapes-semantic-large-1024-native",
"parameterDtype": "torch.float16",
"placement": {
"mps:0": 458
},
"precision": "fp16",
"predictionDevices": [
"mps:0",
"mps:0"
],
"provider": "Metal/MPS",
"requestedDevice": "mps",
"requestedPrecision": "fp16",
"reused": false,
"selectionReason": "Explicit device selection; actual placement is reported after inference",
"torchVersion": "2.11.0",
"version": "4.57.6",
"totalMs": 7098.805125,
"imagesPerSecond": 0.26682786627230837,
"grade": "moderate"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Native output for same fixture and configuration as spatial qualification"
},
"runtime": {
"acceleration": "partial",
"accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
"artifactRevision": "8d6b6d1a3f7b50d441afd7d247c2ed10db186e8f",
"cpuThreads": 4,
"device": "mps",
"durationMs": 1739.9657080000002,
"engine": "transformers",
"fallback": false,
"fallbackReason": null,
"family": {
"catalogId": "eomt-native",
"defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
"displayName": "EoMT Desktop",
"id": "eomt",
"variantId": "cityscapes-semantic-large-1024",
"variantName": "CITYSCAPES Semantic Large 1024"
},
"inferenceMs": 1699.535499967169,
"loadMs": 3339.24625,
"modelId": "eomt-cityscapes-semantic-large-1024-native",
"parameterDtype": "torch.float16",
"placement": {
"mps:0": 458
},
"precision": "fp16",
"predictionDevices": [
"mps:0",
"mps:0"
],
"provider": "Metal/MPS",
"requestedDevice": "mps",
"requestedPrecision": "fp16",
"reused": false,
"selectionReason": "Explicit device selection; actual placement is reported after inference",
"torchVersion": "2.11.0",
"version": "4.57.6",
"totalMs": 5079.211958,
"imagesPerSecond": 0.5883960647008065,
"grade": "moderate"
}
}
]EoMT COCO Instance Large 1280 Desktop
26 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
FP16 | mpsAccelerated | Passed | Moderate | Not recorded | 3946 ms / 5072 ms |
FP16 | mpsAccelerated | Passed | Moderate | Not recorded | 4364 ms / 5072 ms |
FP32 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3377 ms / 352 ms |
FP32 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3379 ms / 352 ms |
FP32 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3381 ms / 352 ms |
FP32 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3398 ms / 352 ms |
FP32 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3529 ms / 352 ms |
FP32 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3736 ms / 352 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3803 ms / 275 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3825 ms / 275 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3828 ms / 275 ms |
BF16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3864 ms / 269 ms |
BF16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3890 ms / 269 ms |
BF16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3900 ms / 269 ms |
BF16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3926 ms / 269 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3971 ms / 275 ms |
BF16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 4021 ms / 269 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 4031 ms / 275 ms |
BF16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 4153 ms / 269 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 4158 ms / 275 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 13996 ms / 65 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 13407 ms / 65 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 13596 ms / 65 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 13492 ms / 65 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 13487 ms / 65 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 13694 ms / 65 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-14T21:32:39.471043+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
- Runtime
- transformers 4.57.6
- Artifact revision
- 057a34b54f9f1cf5395de72edaeaefca43e66494
- Fixture
{ "file": "model-samples/desktop/mobilenet-dog.jpg", "size": 83281, "sha1": "f1d042827c2f2154a6d8043b8e8f07c1be612941", "sha256": "d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1", "url": "https://assets.localai.tools/samples/v1/files/d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1/mobilenet-dog.jpg" }- Rerun when
- Checkpoint or runtime revision; Preprocessing or postprocessing; Fixture or quality gate
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 64.69133298378438,
"inferenceMs": 13995.703165943269,
"totalMs": 14060.394498927053,
"imagesPerSecond": 0.07145050078179498,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 64.69133298378438,
"inferenceMs": 13407.404666999355,
"totalMs": 13472.09599998314,
"imagesPerSecond": 0.07458565060405573,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": false,
"pointRecall": null
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 64.69133298378438,
"inferenceMs": 13595.744749996811,
"totalMs": 13660.436082980596,
"imagesPerSecond": 0.07355242529102604,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 64.69133298378438,
"inferenceMs": 13491.55170796439,
"totalMs": 13556.243040948175,
"imagesPerSecond": 0.07412045861334658,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 64.69133298378438,
"inferenceMs": 13486.85204202775,
"totalMs": 13551.543375011533,
"imagesPerSecond": 0.0741462868343034,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 64.69133298378438,
"inferenceMs": 13693.713375017978,
"totalMs": 13758.404708001763,
"imagesPerSecond": 0.07302621083221607,
"grade": "slow"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 269.2592079984024,
"inferenceMs": 4153.020208002999,
"totalMs": 4422.279416001402,
"imagesPerSecond": 0.24078861886416272,
"grade": "slow"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 269.2592079984024,
"inferenceMs": 4020.541208970826,
"totalMs": 4289.8004169692285,
"imagesPerSecond": 0.24872273358839145,
"grade": "slow"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 269.2592079984024,
"inferenceMs": 3889.7433749516495,
"totalMs": 4159.002582950052,
"imagesPerSecond": 0.2570863688436593,
"grade": "slow"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 269.2592079984024,
"inferenceMs": 3925.9483750211075,
"totalMs": 4195.20758301951,
"imagesPerSecond": 0.2547155246264856,
"grade": "slow"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 269.2592079984024,
"inferenceMs": 3900.380084000062,
"totalMs": 4169.639291998465,
"imagesPerSecond": 0.25638526975925974,
"grade": "slow"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 269.2592079984024,
"inferenceMs": 3863.787041977048,
"totalMs": 4133.04624997545,
"imagesPerSecond": 0.25881343592071093,
"grade": "slow"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 274.85849999357015,
"inferenceMs": 4031.470707966946,
"totalMs": 4306.329207960516,
"imagesPerSecond": 0.24804843503484014,
"grade": "slow"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 274.85849999357015,
"inferenceMs": 3971.031333960127,
"totalMs": 4245.889833953697,
"imagesPerSecond": 0.25182374952522624,
"grade": "slow"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 274.85849999357015,
"inferenceMs": 3824.672083952464,
"totalMs": 4099.530583946034,
"imagesPerSecond": 0.2614603234080626,
"grade": "slow"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 274.85849999357015,
"inferenceMs": 3827.7053750352934,
"totalMs": 4102.563875028864,
"imagesPerSecond": 0.26125312740163015,
"grade": "slow"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 274.85849999357015,
"inferenceMs": 3803.3817499526776,
"totalMs": 4078.240249946248,
"imagesPerSecond": 0.26292390975805735,
"grade": "slow"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 274.85849999357015,
"inferenceMs": 4158.285583020188,
"totalMs": 4433.144083013758,
"imagesPerSecond": 0.24048372340836052,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 351.8199999816716,
"inferenceMs": 3735.617583966814,
"totalMs": 4087.4375839484856,
"imagesPerSecond": 0.2676933539160907,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 351.8199999816716,
"inferenceMs": 3529.098958999384,
"totalMs": 3880.9189589810558,
"imagesPerSecond": 0.28335844690610007,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 351.8199999816716,
"inferenceMs": 3378.588374995161,
"totalMs": 3730.4083749768324,
"imagesPerSecond": 0.2959816020800203,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 351.8199999816716,
"inferenceMs": 3397.867874999065,
"totalMs": 3749.6878749807365,
"imagesPerSecond": 0.2943022026718079,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 351.8199999816716,
"inferenceMs": 3377.2522910148837,
"totalMs": 3729.0722909965552,
"imagesPerSecond": 0.2960986961680302,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 351.8199999816716,
"inferenceMs": 3381.327167036943,
"totalMs": 3733.1471670186147,
"imagesPerSecond": 0.2957418642444765,
"grade": "slow"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Native output for same fixture and configuration as spatial qualification"
},
"runtime": {
"acceleration": "partial",
"accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
"artifactRevision": "057a34b54f9f1cf5395de72edaeaefca43e66494",
"cpuThreads": 4,
"device": "mps",
"durationMs": 4374.495083,
"engine": "transformers",
"fallback": false,
"fallbackReason": null,
"family": {
"catalogId": "eomt-native",
"defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
"displayName": "EoMT Desktop",
"id": "eomt",
"variantId": "coco-instance-large-1280",
"variantName": "COCO Instance Large 1280"
},
"inferenceMs": 4363.544624997303,
"loadMs": 5071.599959,
"modelId": "eomt-coco-instance-large-1280-native",
"parameterDtype": "torch.float16",
"placement": {
"mps:0": 458
},
"precision": "fp16",
"predictionDevices": [
"mps:0",
"mps:0"
],
"provider": "Metal/MPS",
"requestedDevice": "mps",
"requestedPrecision": "fp16",
"reused": false,
"selectionReason": "Explicit device selection; actual placement is reported after inference",
"torchVersion": "2.11.0",
"version": "4.57.6",
"totalMs": 9446.095042,
"imagesPerSecond": 0.2291714846391924,
"grade": "moderate"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Native output for same fixture and configuration as spatial qualification"
},
"runtime": {
"acceleration": "partial",
"accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
"artifactRevision": "057a34b54f9f1cf5395de72edaeaefca43e66494",
"cpuThreads": 4,
"device": "mps",
"durationMs": 3957.007667,
"engine": "transformers",
"fallback": false,
"fallbackReason": null,
"family": {
"catalogId": "eomt-native",
"defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
"displayName": "EoMT Desktop",
"id": "eomt",
"variantId": "coco-instance-large-1280",
"variantName": "COCO Instance Large 1280"
},
"inferenceMs": 3946.1205839761533,
"loadMs": 5071.599959,
"modelId": "eomt-coco-instance-large-1280-native",
"parameterDtype": "torch.float16",
"placement": {
"mps:0": 458
},
"precision": "fp16",
"predictionDevices": [
"mps:0",
"mps:0"
],
"provider": "Metal/MPS",
"requestedDevice": "mps",
"requestedPrecision": "fp16",
"reused": false,
"selectionReason": "Explicit device selection; actual placement is reported after inference",
"torchVersion": "2.11.0",
"version": "4.57.6",
"totalMs": 9028.607626,
"imagesPerSecond": 0.25341344206780153,
"grade": "moderate"
}
}
]EoMT COCO Instance Large 640 Desktop
26 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 478 ms / 279 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 480 ms / 279 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 480 ms / 279 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 480 ms / 319 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 482 ms / 279 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 486 ms / 319 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 489 ms / 319 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 489 ms / 319 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 520 ms / 402 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 541 ms / 402 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 579 ms / 402 ms |
FP16 | mpsAccelerated | Passed | Fast | Not recorded | 590 ms / 5102 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 599 ms / 279 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 612 ms / 319 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 654 ms / 402 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 668 ms / 279 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 675 ms / 319 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 854 ms / 402 ms |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1080 ms / 402 ms |
FP16 | mpsAccelerated | Passed | Moderate | Not recorded | 1149 ms / 5102 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1879 ms / 67 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1879 ms / 67 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1907 ms / 67 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1934 ms / 67 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 2022 ms / 67 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 2159 ms / 67 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-14T21:14:05.010047+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
- Runtime
- transformers 4.57.6
- Artifact revision
- a43666870d47a7010bfea2c4554f7ca829714558
- Fixture
{ "file": "model-samples/desktop/mobilenet-dog.jpg", "size": 83281, "sha1": "f1d042827c2f2154a6d8043b8e8f07c1be612941", "sha256": "d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1", "url": "https://assets.localai.tools/samples/v1/files/d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1/mobilenet-dog.jpg" }- Rerun when
- Checkpoint or runtime revision; Preprocessing or postprocessing; Fixture or quality gate
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 67.03858397668228,
"inferenceMs": 2159.350500034634,
"totalMs": 2226.3890840113163,
"imagesPerSecond": 0.4631022152188637,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 67.03858397668228,
"inferenceMs": 2021.743458986748,
"totalMs": 2088.7820429634303,
"imagesPerSecond": 0.4946225969249221,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 67.03858397668228,
"inferenceMs": 1879.0390830254182,
"totalMs": 1946.0776670021005,
"imagesPerSecond": 0.5321869082094408,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 67.03858397668228,
"inferenceMs": 1879.0461250464432,
"totalMs": 1946.0847090231255,
"imagesPerSecond": 0.5321849137552617,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 67.03858397668228,
"inferenceMs": 1907.3512500035577,
"totalMs": 1974.38983398024,
"imagesPerSecond": 0.5242872805929871,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 67.03858397668228,
"inferenceMs": 1934.0937500237487,
"totalMs": 2001.132334000431,
"imagesPerSecond": 0.5170380184454456,
"grade": "moderate"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 318.8823329983279,
"inferenceMs": 675.1152499928139,
"totalMs": 993.9975829911418,
"imagesPerSecond": 1.481228575433075,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 318.8823329983279,
"inferenceMs": 612.3045000131242,
"totalMs": 931.1868330114521,
"imagesPerSecond": 1.6331743437759576,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 318.8823329983279,
"inferenceMs": 480.3947500186041,
"totalMs": 799.277083016932,
"imagesPerSecond": 2.0816214164731677,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 318.8823329983279,
"inferenceMs": 488.61841700272635,
"totalMs": 807.5007500010543,
"imagesPerSecond": 2.046586794935362,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 318.8823329983279,
"inferenceMs": 486.34987499099225,
"totalMs": 805.2322079893202,
"imagesPerSecond": 2.0561329434258027,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 318.8823329983279,
"inferenceMs": 488.843375002034,
"totalMs": 807.7257080003619,
"imagesPerSecond": 2.0456449880206296,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 278.6241670255549,
"inferenceMs": 667.5229580141604,
"totalMs": 946.1471250397153,
"imagesPerSecond": 1.498075815961354,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 278.6241670255549,
"inferenceMs": 598.7348750350066,
"totalMs": 877.3590420605615,
"imagesPerSecond": 1.6701883282505174,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 278.6241670255549,
"inferenceMs": 480.26133299572393,
"totalMs": 758.8855000212789,
"imagesPerSecond": 2.082199692742917,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 278.6241670255549,
"inferenceMs": 482.31491597834975,
"totalMs": 760.9390830039047,
"imagesPerSecond": 2.073334178296257,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 278.6241670255549,
"inferenceMs": 478.00366597948596,
"totalMs": 756.6278330050409,
"imagesPerSecond": 2.092034164530688,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 278.6241670255549,
"inferenceMs": 479.8966670059599,
"totalMs": 758.5208340315148,
"imagesPerSecond": 2.083781923802319,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 402.2555000265129,
"inferenceMs": 1079.9683749792166,
"totalMs": 1482.2238750057295,
"imagesPerSecond": 0.9259530400779046,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 402.2555000265129,
"inferenceMs": 854.4880410190672,
"totalMs": 1256.74354104558,
"imagesPerSecond": 1.1702913932035777,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 402.2555000265129,
"inferenceMs": 654.0843340335414,
"totalMs": 1056.3398340600543,
"imagesPerSecond": 1.528854840221139,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 402.2555000265129,
"inferenceMs": 579.4954590382986,
"totalMs": 981.7509590648115,
"imagesPerSecond": 1.7256390613647767,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 402.2555000265129,
"inferenceMs": 541.144874994643,
"totalMs": 943.4003750211559,
"imagesPerSecond": 1.8479339751852946,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 402.2555000265129,
"inferenceMs": 520.3491250285879,
"totalMs": 922.6046250551008,
"imagesPerSecond": 1.9217866465040374,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Native output for same fixture and configuration as spatial qualification"
},
"runtime": {
"acceleration": "partial",
"accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
"artifactRevision": "a43666870d47a7010bfea2c4554f7ca829714558",
"cpuThreads": 4,
"device": "mps",
"durationMs": 1160.2174169999998,
"engine": "transformers",
"fallback": false,
"fallbackReason": null,
"family": {
"catalogId": "eomt-native",
"defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
"displayName": "EoMT Desktop",
"id": "eomt",
"variantId": "coco-instance-large-640",
"variantName": "COCO Instance Large 640"
},
"inferenceMs": 1148.9204170065932,
"loadMs": 5102.389333,
"modelId": "eomt-coco-instance-large-640-native",
"parameterDtype": "torch.float16",
"placement": {
"mps:0": 458
},
"precision": "fp16",
"predictionDevices": [
"mps:0",
"mps:0"
],
"provider": "Metal/MPS",
"requestedDevice": "mps",
"requestedPrecision": "fp16",
"reused": false,
"selectionReason": "Explicit device selection; actual placement is reported after inference",
"torchVersion": "2.11.0",
"version": "4.57.6",
"totalMs": 6262.60675,
"imagesPerSecond": 0.8703823042899771,
"grade": "moderate"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Native output for same fixture and configuration as spatial qualification"
},
"runtime": {
"acceleration": "partial",
"accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
"artifactRevision": "a43666870d47a7010bfea2c4554f7ca829714558",
"cpuThreads": 4,
"device": "mps",
"durationMs": 601.5315,
"engine": "transformers",
"fallback": false,
"fallbackReason": null,
"family": {
"catalogId": "eomt-native",
"defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
"displayName": "EoMT Desktop",
"id": "eomt",
"variantId": "coco-instance-large-640",
"variantName": "COCO Instance Large 640"
},
"inferenceMs": 590.0762910023332,
"loadMs": 5102.389333,
"modelId": "eomt-coco-instance-large-640-native",
"parameterDtype": "torch.float16",
"placement": {
"mps:0": 458
},
"precision": "fp16",
"predictionDevices": [
"mps:0",
"mps:0"
],
"provider": "Metal/MPS",
"requestedDevice": "mps",
"requestedPrecision": "fp16",
"reused": false,
"selectionReason": "Explicit device selection; actual placement is reported after inference",
"torchVersion": "2.11.0",
"version": "4.57.6",
"totalMs": 5703.920833,
"imagesPerSecond": 1.694696118532995,
"grade": "fast"
}
}
]EoMT COCO Panoptic 7B 640 Desktop
32 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 3921 ms / 276451 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 4262 ms / 276451 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3876 ms / 33594 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3944 ms / 33594 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 4067 ms / 33594 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 4185 ms / 33594 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 4238 ms / 33594 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 4418 ms / 31635 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 4430 ms / 31635 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 4516 ms / 31635 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 4583 ms / 31635 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 4674 ms / 31635 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 4677 ms / 31635 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 4811 ms / 33594 ms |
BF16 | mps | Failed | — | macos · aarch64 | 5727 ms / 26465 ms |
BF16 | mps | Failed | — | macos · aarch64 | 4617 ms / 26465 ms |
BF16 | mps | Failed | — | macos · aarch64 | 4081 ms / 26465 ms |
BF16 | mps | Failed | — | macos · aarch64 | 4086 ms / 26465 ms |
BF16 | mps | Failed | — | macos · aarch64 | 4110 ms / 26465 ms |
BF16 | mps | Failed | — | macos · aarch64 | 4511 ms / 26465 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 95358 ms / 151 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 44109 ms / 151 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 50881 ms / 151 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 41249 ms / 151 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 42040 ms / 151 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 52235 ms / 151 ms |
FP32 | mps | Failed | — | macos · aarch64 | 6598 ms / 19066 ms |
FP32 | mps | Failed | — | macos · aarch64 | 4892 ms / 19066 ms |
FP32 | mps | Failed | — | macos · aarch64 | 4718 ms / 19066 ms |
FP32 | mps | Failed | — | macos · aarch64 | 4914 ms / 19066 ms |
FP32 | mps | Failed | — | macos · aarch64 | 5084 ms / 19066 ms |
FP32 | mps | Failed | — | macos · aarch64 | 5315 ms / 19066 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-15T06:53:24.543030+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
- Runtime
- transformers 4.57.6
- Artifact revision
- a45f5da9787a3bad985155039050226d9fbecca5
- Fixture
{ "file": "model-samples/desktop/mobilenet-dog.jpg", "size": 83281, "sha1": "f1d042827c2f2154a6d8043b8e8f07c1be612941", "sha256": "d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1", "url": "https://assets.localai.tools/samples/v1/files/d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1/mobilenet-dog.jpg" }- Rerun when
- Checkpoint or runtime revision; Preprocessing or postprocessing; Fixture or quality gate
Per-case quality & failure details
[
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "Metal/MPS",
"loadMs": 33593.50700001232,
"inferenceMs": 4811.2799159716815,
"totalMs": 38404.786915984005,
"peakRssBytes": 22430482432,
"imagesPerSecond": 0.20784490145342976,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "Metal/MPS",
"loadMs": 33593.50700001232,
"inferenceMs": 3944.016749970615,
"totalMs": 37537.52374998294,
"peakRssBytes": 22430482432,
"imagesPerSecond": 0.2535486189320698,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "Metal/MPS",
"loadMs": 33593.50700001232,
"inferenceMs": 3875.861707958393,
"totalMs": 37469.368707970716,
"peakRssBytes": 22430482432,
"imagesPerSecond": 0.25800714146912873,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "Metal/MPS",
"loadMs": 33593.50700001232,
"inferenceMs": 4238.054167013615,
"totalMs": 37831.56116702594,
"peakRssBytes": 22430482432,
"imagesPerSecond": 0.23595734282571934,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "Metal/MPS",
"loadMs": 33593.50700001232,
"inferenceMs": 4185.032167006284,
"totalMs": 37778.53916701861,
"peakRssBytes": 22430482432,
"imagesPerSecond": 0.2389467894377832,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "Metal/MPS",
"loadMs": 33593.50700001232,
"inferenceMs": 4067.2992920153774,
"totalMs": 37660.8062920277,
"peakRssBytes": 22430482432,
"imagesPerSecond": 0.24586339194736084,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "Metal/MPS",
"loadMs": 31635.103542008437,
"inferenceMs": 4677.024625008926,
"totalMs": 36312.12816701736,
"peakRssBytes": 11318706176,
"imagesPerSecond": 0.213811147081162,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "Metal/MPS",
"loadMs": 31635.103542008437,
"inferenceMs": 4430.008624971379,
"totalMs": 36065.112166979816,
"peakRssBytes": 11318706176,
"imagesPerSecond": 0.22573319482114118,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "Metal/MPS",
"loadMs": 31635.103542008437,
"inferenceMs": 4417.87679202389,
"totalMs": 36052.98033403233,
"peakRssBytes": 11318706176,
"imagesPerSecond": 0.22635307571397578,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "Metal/MPS",
"loadMs": 31635.103542008437,
"inferenceMs": 4516.275916015729,
"totalMs": 36151.37945802417,
"peakRssBytes": 11318706176,
"imagesPerSecond": 0.2214213698622299,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "Metal/MPS",
"loadMs": 31635.103542008437,
"inferenceMs": 4674.305790977087,
"totalMs": 36309.409332985524,
"peakRssBytes": 11318706176,
"imagesPerSecond": 0.21393551143579898,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "Metal/MPS",
"loadMs": 31635.103542008437,
"inferenceMs": 4583.446374977939,
"totalMs": 36218.549916986376,
"peakRssBytes": 11318706176,
"imagesPerSecond": 0.21817643715855914,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "Metal/MPS",
"loadMs": 26465.21195897367,
"inferenceMs": 5726.736209006049,
"totalMs": 32191.948167979717,
"peakRssBytes": 17485021184,
"imagesPerSecond": 0.1746195325755302,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "Metal/MPS",
"loadMs": 26465.21195897367,
"inferenceMs": 4616.539833019488,
"totalMs": 31081.751791993156,
"peakRssBytes": 17485021184,
"imagesPerSecond": 0.21661244918706601,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "Metal/MPS",
"loadMs": 26465.21195897367,
"inferenceMs": 4081.294458999764,
"totalMs": 30546.506417973433,
"peakRssBytes": 17485021184,
"imagesPerSecond": 0.24502030178069487,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "Metal/MPS",
"loadMs": 26465.21195897367,
"inferenceMs": 4086.222457990516,
"totalMs": 30551.434416964184,
"peakRssBytes": 17485021184,
"imagesPerSecond": 0.24472480641491326,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "Metal/MPS",
"loadMs": 26465.21195897367,
"inferenceMs": 4109.718374966178,
"totalMs": 30574.930333939847,
"peakRssBytes": 17485021184,
"imagesPerSecond": 0.24332567557216855,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "Metal/MPS",
"loadMs": 26465.21195897367,
"inferenceMs": 4511.223124980461,
"totalMs": 30976.43508395413,
"peakRssBytes": 17485021184,
"imagesPerSecond": 0.22166937265031225,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "PyTorch CPU",
"loadMs": 150.91558400308713,
"inferenceMs": 95358.02529199282,
"totalMs": 95508.9408759959,
"peakRssBytes": 29611065344,
"imagesPerSecond": 0.01048679434098946,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "PyTorch CPU",
"loadMs": 150.91558400308713,
"inferenceMs": 44109.261416015215,
"totalMs": 44260.1770000183,
"peakRssBytes": 29611065344,
"imagesPerSecond": 0.022670975842658737,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "PyTorch CPU",
"loadMs": 150.91558400308713,
"inferenceMs": 50880.95858297311,
"totalMs": 51031.874166976195,
"peakRssBytes": 29611065344,
"imagesPerSecond": 0.019653717772814164,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "PyTorch CPU",
"loadMs": 150.91558400308713,
"inferenceMs": 41248.72304103337,
"totalMs": 41399.63862503646,
"peakRssBytes": 29611065344,
"imagesPerSecond": 0.024243174728226635,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "PyTorch CPU",
"loadMs": 150.91558400308713,
"inferenceMs": 42040.499666996766,
"totalMs": 42191.41525099985,
"peakRssBytes": 29611065344,
"imagesPerSecond": 0.02378658693214901,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "PyTorch CPU",
"loadMs": 150.91558400308713,
"inferenceMs": 52234.89391698968,
"totalMs": 52385.809500992764,
"peakRssBytes": 29611065344,
"imagesPerSecond": 0.019144290818110473,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "Metal/MPS",
"loadMs": 19066.379999974743,
"inferenceMs": 6598.004333034623,
"totalMs": 25664.384333009366,
"peakRssBytes": 1586331648,
"imagesPerSecond": 0.15156097958184722,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "Metal/MPS",
"loadMs": 19066.379999974743,
"inferenceMs": 4891.629500023555,
"totalMs": 23958.009499998298,
"peakRssBytes": 1586331648,
"imagesPerSecond": 0.20443085478881518,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "Metal/MPS",
"loadMs": 19066.379999974743,
"inferenceMs": 4717.508417030331,
"totalMs": 23783.888417005073,
"peakRssBytes": 1586331648,
"imagesPerSecond": 0.21197630435379267,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "Metal/MPS",
"loadMs": 19066.379999974743,
"inferenceMs": 4913.599958003033,
"totalMs": 23979.979957977775,
"peakRssBytes": 1586331648,
"imagesPerSecond": 0.2035167715213056,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "Metal/MPS",
"loadMs": 19066.379999974743,
"inferenceMs": 5083.821790991351,
"totalMs": 24150.201790966094,
"peakRssBytes": 1586331648,
"imagesPerSecond": 0.1967024103346862,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"attentionImplementation": "sdpa",
"provider": "Metal/MPS",
"loadMs": 19066.379999974743,
"inferenceMs": 5315.2464590384625,
"totalMs": 24381.626459013205,
"peakRssBytes": 1586331648,
"imagesPerSecond": 0.18813803041992935,
"grade": "slow",
"engine": "transformers",
"version": "4.57.6",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Native output for same fixture and configuration as spatial qualification"
},
"runtime": {
"acceleration": "partial",
"accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5",
"attentionImplementation": "sdpa",
"cpuThreads": 4,
"device": "mps",
"durationMs": 4272.326999999999,
"engine": "transformers",
"fallback": false,
"fallbackReason": null,
"family": {
"catalogId": "eomt-native",
"defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
"displayName": "EoMT Desktop",
"id": "eomt",
"variantId": "coco-panoptic-7b-640",
"variantName": "COCO Panoptic 7B 640"
},
"inferenceMs": 4261.654041998554,
"loadMs": 276451.09150000004,
"modelId": "eomt-coco-panoptic-7b-640-native",
"parameterDtype": "torch.float16",
"placement": {
"mps:0": 602
},
"precision": "fp16",
"predictionDevices": [
"mps:0",
"mps:0"
],
"provider": "Metal/MPS",
"requestedDevice": "mps",
"requestedPrecision": "fp16",
"reused": false,
"selectionReason": "Explicit device selection; actual placement is reported after inference",
"torchVersion": "2.11.0",
"version": "4.57.6",
"totalMs": 280723.4185,
"imagesPerSecond": 0.23465067556986346,
"grade": "moderate"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Native output for same fixture and configuration as spatial qualification"
},
"runtime": {
"acceleration": "partial",
"accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
"artifactRevision": "a45f5da9787a3bad985155039050226d9fbecca5",
"attentionImplementation": "sdpa",
"cpuThreads": 4,
"device": "mps",
"durationMs": 3933.489583,
"engine": "transformers",
"fallback": false,
"fallbackReason": null,
"family": {
"catalogId": "eomt-native",
"defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
"displayName": "EoMT Desktop",
"id": "eomt",
"variantId": "coco-panoptic-7b-640",
"variantName": "COCO Panoptic 7B 640"
},
"inferenceMs": 3921.0635410272516,
"loadMs": 276451.09150000004,
"modelId": "eomt-coco-panoptic-7b-640-native",
"parameterDtype": "torch.float16",
"placement": {
"mps:0": 602
},
"precision": "fp16",
"predictionDevices": [
"mps:0",
"mps:0"
],
"provider": "Metal/MPS",
"requestedDevice": "mps",
"requestedPrecision": "fp16",
"reused": false,
"selectionReason": "Explicit device selection; actual placement is reported after inference",
"torchVersion": "2.11.0",
"version": "4.57.6",
"totalMs": 280384.58108300006,
"imagesPerSecond": 0.2550328474753605,
"grade": "moderate"
}
}
]EoMT COCO Panoptic Base 640 2x Desktop
26 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 318 ms / 138 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 325 ms / 158 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 328 ms / 158 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 330 ms / 158 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 331 ms / 138 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 334 ms / 138 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 339 ms / 138 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 346 ms / 158 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 407 ms / 241 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 415 ms / 241 ms |
FP16 | mpsAccelerated | Passed | Fast | Not recorded | 420 ms / 3109 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 424 ms / 241 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 434 ms / 241 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 460 ms / 138 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 471 ms / 158 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 572 ms / 241 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 588 ms / 158 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 588 ms / 138 ms |
FP16 | mpsAccelerated | Passed | Fast | Not recorded | 678 ms / 3109 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 701 ms / 241 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 806 ms / 53 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 835 ms / 53 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 838 ms / 53 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 838 ms / 53 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 923 ms / 53 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 2434 ms / 53 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-14T21:11:18.630023+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
- Runtime
- transformers 4.57.6
- Artifact revision
- 1ac497947a2410fdb05279c5907c5209af37b9f1
- Fixture
{ "file": "model-samples/desktop/mobilenet-dog.jpg", "size": 83281, "sha1": "f1d042827c2f2154a6d8043b8e8f07c1be612941", "sha256": "d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1", "url": "https://assets.localai.tools/samples/v1/files/d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1/mobilenet-dog.jpg" }- Rerun when
- Checkpoint or runtime revision; Preprocessing or postprocessing; Fixture or quality gate
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 52.826541010290384,
"inferenceMs": 2433.665375050623,
"totalMs": 2486.4919160609134,
"imagesPerSecond": 0.41090283415779744,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 52.826541010290384,
"inferenceMs": 922.5171250291169,
"totalMs": 975.3436660394073,
"imagesPerSecond": 1.0839907172112795,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 52.826541010290384,
"inferenceMs": 806.1512910062447,
"totalMs": 858.9778320165351,
"imagesPerSecond": 1.240461946977461,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 52.826541010290384,
"inferenceMs": 838.0493330187164,
"totalMs": 890.8758740290068,
"imagesPerSecond": 1.1932471760318992,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 52.826541010290384,
"inferenceMs": 838.1419579964131,
"totalMs": 890.9684990067035,
"imagesPerSecond": 1.1931153075673604,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 52.826541010290384,
"inferenceMs": 835.1196660078131,
"totalMs": 887.9462070181035,
"imagesPerSecond": 1.1974331831752654,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 241.47554201772436,
"inferenceMs": 701.2341669760644,
"totalMs": 942.7097089937888,
"imagesPerSecond": 1.4260571533647668,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 241.47554201772436,
"inferenceMs": 571.9608750077896,
"totalMs": 813.436417025514,
"imagesPerSecond": 1.7483713374387382,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 241.47554201772436,
"inferenceMs": 424.3493750109337,
"totalMs": 665.8249170286581,
"imagesPerSecond": 2.356548775343982,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 241.47554201772436,
"inferenceMs": 415.3332499554381,
"totalMs": 656.8087919731624,
"imagesPerSecond": 2.407705138240899,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 241.47554201772436,
"inferenceMs": 434.4510420341976,
"totalMs": 675.926584051922,
"imagesPerSecond": 2.3017553262567283,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 241.47554201772436,
"inferenceMs": 407.0659999852069,
"totalMs": 648.5415420029312,
"imagesPerSecond": 2.4566040888611202,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 138.0069589940831,
"inferenceMs": 588.3609169977717,
"totalMs": 726.3678759918548,
"imagesPerSecond": 1.6996370273924692,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 138.0069589940831,
"inferenceMs": 459.55174998380244,
"totalMs": 597.5587089778855,
"imagesPerSecond": 2.1760334935842294,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 138.0069589940831,
"inferenceMs": 317.93570797890425,
"totalMs": 455.94266697298735,
"imagesPerSecond": 3.1452899907246414,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 138.0069589940831,
"inferenceMs": 338.94779096590355,
"totalMs": 476.95474995998666,
"imagesPerSecond": 2.950306881039962,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 138.0069589940831,
"inferenceMs": 333.6381250410341,
"totalMs": 471.6450840351172,
"imagesPerSecond": 2.9972593805849086,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 138.0069589940831,
"inferenceMs": 330.9533340507187,
"totalMs": 468.9602930448018,
"imagesPerSecond": 3.0215740320861966,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 157.62016701046377,
"inferenceMs": 587.7712920191698,
"totalMs": 745.3914590296336,
"imagesPerSecond": 1.7013420246584374,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 157.62016701046377,
"inferenceMs": 471.26441705040634,
"totalMs": 628.8845840608701,
"imagesPerSecond": 2.1219509978260045,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 157.62016701046377,
"inferenceMs": 327.58645905414596,
"totalMs": 485.20662606460974,
"imagesPerSecond": 3.0526292292036175,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 157.62016701046377,
"inferenceMs": 345.6185840186663,
"totalMs": 503.23875102913007,
"imagesPerSecond": 2.8933629331285946,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 157.62016701046377,
"inferenceMs": 330.1146670128219,
"totalMs": 487.73483402328566,
"imagesPerSecond": 3.0292504390941204,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 157.62016701046377,
"inferenceMs": 324.7161660110578,
"totalMs": 482.33633302152157,
"imagesPerSecond": 3.079612611482812,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Native output for same fixture and configuration as spatial qualification"
},
"runtime": {
"acceleration": "partial",
"accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
"artifactRevision": "1ac497947a2410fdb05279c5907c5209af37b9f1",
"cpuThreads": 4,
"device": "mps",
"durationMs": 689.8155830000001,
"engine": "transformers",
"fallback": false,
"fallbackReason": null,
"family": {
"catalogId": "eomt-native",
"defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
"displayName": "EoMT",
"id": "eomt",
"variantId": "coco-panoptic-base-640-2x",
"variantName": "COCO Panoptic Base 640 2x"
},
"inferenceMs": 678.021790983621,
"loadMs": 3109.4634170000004,
"modelId": "eomt-coco-panoptic-base-640-2x-native",
"parameterDtype": "torch.float16",
"placement": {
"mps:0": 242
},
"precision": "fp16",
"predictionDevices": [
"mps:0",
"mps:0"
],
"provider": "Metal/MPS",
"requestedDevice": "mps",
"requestedPrecision": "fp16",
"reused": false,
"selectionReason": "Explicit device selection; actual placement is reported after inference",
"torchVersion": "2.11.0",
"version": "4.57.6",
"totalMs": 3799.2790000000005,
"imagesPerSecond": 1.4748788509426491,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Native output for same fixture and configuration as spatial qualification"
},
"runtime": {
"acceleration": "partial",
"accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
"artifactRevision": "1ac497947a2410fdb05279c5907c5209af37b9f1",
"cpuThreads": 4,
"device": "mps",
"durationMs": 431.868792,
"engine": "transformers",
"fallback": false,
"fallbackReason": null,
"family": {
"catalogId": "eomt-native",
"defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
"displayName": "EoMT",
"id": "eomt",
"variantId": "coco-panoptic-base-640-2x",
"variantName": "COCO Panoptic Base 640 2x"
},
"inferenceMs": 419.69904105644673,
"loadMs": 3109.4634170000004,
"modelId": "eomt-coco-panoptic-base-640-2x-native",
"parameterDtype": "torch.float16",
"placement": {
"mps:0": 242
},
"precision": "fp16",
"predictionDevices": [
"mps:0",
"mps:0"
],
"provider": "Metal/MPS",
"requestedDevice": "mps",
"requestedPrecision": "fp16",
"reused": false,
"selectionReason": "Explicit device selection; actual placement is reported after inference",
"torchVersion": "2.11.0",
"version": "4.57.6",
"totalMs": 3541.332209,
"imagesPerSecond": 2.3826597208391207,
"grade": "fast"
}
}
]EoMT COCO Panoptic Giant 640 Desktop
26 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1076 ms / 699 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1080 ms / 1055 ms |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1081 ms / 888 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1125 ms / 1055 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1128 ms / 1055 ms |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1132 ms / 699 ms |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1135 ms / 888 ms |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1135 ms / 888 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1137 ms / 1055 ms |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1147 ms / 699 ms |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1151 ms / 699 ms |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1160 ms / 888 ms |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1195 ms / 888 ms |
FP16 | mpsAccelerated | Passed | Moderate | Not recorded | 1197 ms / 6759 ms |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1214 ms / 699 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1231 ms / 1055 ms |
FP32 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1301 ms / 888 ms |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 1319 ms / 699 ms |
FP16 | mpsAccelerated | Passed | Moderate | Not recorded | 1355 ms / 6759 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 2935 ms / 1055 ms |
FP32 | cpu | Passed | Slow | macos · aarch64 | 4149 ms / 90 ms |
FP32 | cpu | Passed | Slow | macos · aarch64 | 4194 ms / 90 ms |
FP32 | cpu | Passed | Slow | macos · aarch64 | 4275 ms / 90 ms |
FP32 | cpu | Passed | Slow | macos · aarch64 | 4306 ms / 90 ms |
FP32 | cpu | Passed | Slow | macos · aarch64 | 4307 ms / 90 ms |
FP32 | cpu | Passed | Slow | macos · aarch64 | 4765 ms / 90 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-14T22:18:18.399974+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
- Runtime
- transformers 4.57.6
- Artifact revision
- b141595af7ff6d9f3e78c847903f6761b8575298
- Fixture
{ "file": "model-samples/desktop/mobilenet-dog.jpg", "size": 83281, "sha1": "f1d042827c2f2154a6d8043b8e8f07c1be612941", "sha256": "d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1", "url": "https://assets.localai.tools/samples/v1/files/d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1/mobilenet-dog.jpg" }- Rerun when
- Checkpoint or runtime revision; Preprocessing or postprocessing; Fixture or quality gate
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 90.40616598213091,
"inferenceMs": 4764.8558329674415,
"totalMs": 4855.261998949572,
"imagesPerSecond": 0.20986993836857037,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 90.40616598213091,
"inferenceMs": 4305.557249987032,
"totalMs": 4395.963415969163,
"imagesPerSecond": 0.23225797311207785,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 90.40616598213091,
"inferenceMs": 4148.856999992859,
"totalMs": 4239.2631659749895,
"imagesPerSecond": 0.24103024037746332,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 90.40616598213091,
"inferenceMs": 4275.357208040077,
"totalMs": 4365.763374022208,
"imagesPerSecond": 0.23389858468888572,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 90.40616598213091,
"inferenceMs": 4193.52383405203,
"totalMs": 4283.930000034161,
"imagesPerSecond": 0.23846293465172488,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 90.40616598213091,
"inferenceMs": 4306.714584003203,
"totalMs": 4397.1207499853335,
"imagesPerSecond": 0.2321955589335744,
"grade": "slow"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 699.3644169997424,
"inferenceMs": 1319.4539999822155,
"totalMs": 2018.8184169819579,
"imagesPerSecond": 0.7578892481386079,
"grade": "moderate"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 699.3644169997424,
"inferenceMs": 1214.0130000188947,
"totalMs": 1913.377417018637,
"imagesPerSecond": 0.823714408317239,
"grade": "moderate"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 699.3644169997424,
"inferenceMs": 1075.6408749730326,
"totalMs": 1775.005291972775,
"imagesPerSecond": 0.9296783185420235,
"grade": "moderate"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 699.3644169997424,
"inferenceMs": 1146.9129999750294,
"totalMs": 1846.2774169747718,
"imagesPerSecond": 0.8719057156225206,
"grade": "moderate"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 699.3644169997424,
"inferenceMs": 1132.3877499671653,
"totalMs": 1831.7521669669077,
"imagesPerSecond": 0.8830897367346088,
"grade": "moderate"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 699.3644169997424,
"inferenceMs": 1151.298000011593,
"totalMs": 1850.6624170113355,
"imagesPerSecond": 0.8685848494394417,
"grade": "moderate"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 1054.9896669690497,
"inferenceMs": 2935.386458004359,
"totalMs": 3990.376124973409,
"imagesPerSecond": 0.3406706456906721,
"grade": "moderate"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 1054.9896669690497,
"inferenceMs": 1230.5514580220915,
"totalMs": 2285.5411249911413,
"imagesPerSecond": 0.812643789482266,
"grade": "moderate"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 1054.9896669690497,
"inferenceMs": 1079.6370829921216,
"totalMs": 2134.6267499611713,
"imagesPerSecond": 0.9262371733551295,
"grade": "moderate"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 1054.9896669690497,
"inferenceMs": 1128.2044170075096,
"totalMs": 2183.1940839765593,
"imagesPerSecond": 0.8863641951096383,
"grade": "moderate"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 1054.9896669690497,
"inferenceMs": 1125.0781249837019,
"totalMs": 2180.0677919527516,
"imagesPerSecond": 0.8888271647930993,
"grade": "moderate"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 1054.9896669690497,
"inferenceMs": 1137.2789589804597,
"totalMs": 2192.2686259495094,
"imagesPerSecond": 0.8792917446538124,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 888.4135420084931,
"inferenceMs": 1300.6567499833182,
"totalMs": 2189.0702919918112,
"imagesPerSecond": 0.7688423559965577,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 888.4135420084931,
"inferenceMs": 1194.7302499902435,
"totalMs": 2083.1437919987366,
"imagesPerSecond": 0.8370090235918662,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 888.4135420084931,
"inferenceMs": 1080.7537920190953,
"totalMs": 1969.1673340275884,
"imagesPerSecond": 0.9252801215083144,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 888.4135420084931,
"inferenceMs": 1134.982957970351,
"totalMs": 2023.396499978844,
"imagesPerSecond": 0.8810704979996034,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 888.4135420084931,
"inferenceMs": 1159.804124967195,
"totalMs": 2048.217666975688,
"imagesPerSecond": 0.8622145571591969,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 888.4135420084931,
"inferenceMs": 1135.193291993346,
"totalMs": 2023.6068340018392,
"imagesPerSecond": 0.8809072490589219,
"grade": "moderate"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Native output for same fixture and configuration as spatial qualification"
},
"runtime": {
"acceleration": "partial",
"accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
"artifactRevision": "b141595af7ff6d9f3e78c847903f6761b8575298",
"cpuThreads": 4,
"device": "mps",
"durationMs": 1367.108792,
"engine": "transformers",
"fallback": false,
"fallbackReason": null,
"family": {
"catalogId": "eomt-native",
"defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
"displayName": "EoMT Desktop",
"id": "eomt",
"variantId": "coco-panoptic-giant-640",
"variantName": "COCO Panoptic Giant 640"
},
"inferenceMs": 1355.2602909621783,
"loadMs": 6759.449958,
"modelId": "eomt-coco-panoptic-giant-640-native",
"parameterDtype": "torch.float16",
"placement": {
"mps:0": 746
},
"precision": "fp16",
"predictionDevices": [
"mps:0",
"mps:0"
],
"provider": "Metal/MPS",
"requestedDevice": "mps",
"requestedPrecision": "fp16",
"reused": false,
"selectionReason": "Explicit device selection; actual placement is reported after inference",
"torchVersion": "2.11.0",
"version": "4.57.6",
"totalMs": 8126.55875,
"imagesPerSecond": 0.7378656385557063,
"grade": "moderate"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Native output for same fixture and configuration as spatial qualification"
},
"runtime": {
"acceleration": "partial",
"accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
"artifactRevision": "b141595af7ff6d9f3e78c847903f6761b8575298",
"cpuThreads": 4,
"device": "mps",
"durationMs": 1207.791042,
"engine": "transformers",
"fallback": false,
"fallbackReason": null,
"family": {
"catalogId": "eomt-native",
"defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
"displayName": "EoMT Desktop",
"id": "eomt",
"variantId": "coco-panoptic-giant-640",
"variantName": "COCO Panoptic Giant 640"
},
"inferenceMs": 1197.1005420200527,
"loadMs": 6759.449958,
"modelId": "eomt-coco-panoptic-giant-640-native",
"parameterDtype": "torch.float16",
"placement": {
"mps:0": 746
},
"precision": "fp16",
"predictionDevices": [
"mps:0",
"mps:0"
],
"provider": "Metal/MPS",
"requestedDevice": "mps",
"requestedPrecision": "fp16",
"reused": false,
"selectionReason": "Explicit device selection; actual placement is reported after inference",
"torchVersion": "2.11.0",
"version": "4.57.6",
"totalMs": 7967.241,
"imagesPerSecond": 0.8353517226819942,
"grade": "moderate"
}
}
]EoMT COCO Panoptic Large 1280 Desktop
26 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
FP16 | mpsAccelerated | Passed | Moderate | Not recorded | 3807 ms / 4377 ms |
FP16 | mpsAccelerated | Passed | Moderate | Not recorded | 4119 ms / 4377 ms |
FP32 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3351 ms / 385 ms |
FP32 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3365 ms / 385 ms |
FP32 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3438 ms / 385 ms |
FP32 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3442 ms / 385 ms |
FP32 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3513 ms / 385 ms |
FP32 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3611 ms / 385 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3778 ms / 268 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3778 ms / 268 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3790 ms / 268 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3824 ms / 268 ms |
BF16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3878 ms / 266 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 3910 ms / 268 ms |
BF16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 4023 ms / 266 ms |
BF16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 4025 ms / 266 ms |
BF16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 4058 ms / 266 ms |
BF16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 4095 ms / 266 ms |
BF16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 4245 ms / 266 ms |
FP16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 4667 ms / 268 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 15172 ms / 65 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 13861 ms / 65 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 14248 ms / 65 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 14027 ms / 65 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 14685 ms / 65 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 13938 ms / 65 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-14T21:26:17.333945+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
- Runtime
- transformers 4.57.6
- Artifact revision
- d1bb11420d0d054623ffb2b6899ff0430e4cc655
- Fixture
{ "file": "model-samples/desktop/mobilenet-dog.jpg", "size": 83281, "sha1": "f1d042827c2f2154a6d8043b8e8f07c1be612941", "sha256": "d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1", "url": "https://assets.localai.tools/samples/v1/files/d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1/mobilenet-dog.jpg" }- Rerun when
- Checkpoint or runtime revision; Preprocessing or postprocessing; Fixture or quality gate
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 65.29920798493549,
"inferenceMs": 15171.545999997761,
"totalMs": 15236.845207982697,
"imagesPerSecond": 0.06591286082513592,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 65.29920798493549,
"inferenceMs": 13860.936082957778,
"totalMs": 13926.235290942714,
"imagesPerSecond": 0.07214519957490566,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": false,
"pointRecall": null
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 65.29920798493549,
"inferenceMs": 14248.368541011587,
"totalMs": 14313.667748996522,
"imagesPerSecond": 0.07018347378660682,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 65.29920798493549,
"inferenceMs": 14026.922458957415,
"totalMs": 14092.22166694235,
"imagesPerSecond": 0.07129147558389849,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 65.29920798493549,
"inferenceMs": 14684.519000002183,
"totalMs": 14749.818207987119,
"imagesPerSecond": 0.06809892785727958,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 65.29920798493549,
"inferenceMs": 13937.820542021655,
"totalMs": 14003.11975000659,
"imagesPerSecond": 0.0717472288429215,
"grade": "slow"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 266.3343329913914,
"inferenceMs": 4057.889040966984,
"totalMs": 4324.223373958375,
"imagesPerSecond": 0.2464335495387776,
"grade": "slow"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 266.3343329913914,
"inferenceMs": 4022.585916041862,
"totalMs": 4288.920249033254,
"imagesPerSecond": 0.248596306175103,
"grade": "slow"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 266.3343329913914,
"inferenceMs": 3877.560082997661,
"totalMs": 4143.8944159890525,
"imagesPerSecond": 0.25789413409344797,
"grade": "slow"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 266.3343329913914,
"inferenceMs": 4024.6608330053277,
"totalMs": 4290.995165996719,
"imagesPerSecond": 0.2484681421597635,
"grade": "slow"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 266.3343329913914,
"inferenceMs": 4094.9511250364594,
"totalMs": 4361.285458027851,
"imagesPerSecond": 0.24420315883284113,
"grade": "slow"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 266.3343329913914,
"inferenceMs": 4244.703958975151,
"totalMs": 4511.038291966543,
"imagesPerSecond": 0.2355876898989775,
"grade": "slow"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 268.4909590170719,
"inferenceMs": 4667.368875001557,
"totalMs": 4935.859834018629,
"imagesPerSecond": 0.21425347487660623,
"grade": "slow"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 268.4909590170719,
"inferenceMs": 3909.7971249721013,
"totalMs": 4178.288083989173,
"imagesPerSecond": 0.2557677465188518,
"grade": "slow"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 268.4909590170719,
"inferenceMs": 3777.9040840105154,
"totalMs": 4046.3950430275872,
"imagesPerSecond": 0.26469703247162074,
"grade": "slow"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 268.4909590170719,
"inferenceMs": 3778.4952499787323,
"totalMs": 4046.986208995804,
"imagesPerSecond": 0.26465561919275365,
"grade": "slow"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 268.4909590170719,
"inferenceMs": 3790.1447499752976,
"totalMs": 4058.6357089923695,
"imagesPerSecond": 0.2638421659242744,
"grade": "slow"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 268.4909590170719,
"inferenceMs": 3823.8740409724414,
"totalMs": 4092.3649999895133,
"imagesPerSecond": 0.2615148902095353,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 384.88262495957315,
"inferenceMs": 3610.5999590363353,
"totalMs": 3995.4825839959085,
"imagesPerSecond": 0.2769622808800172,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 384.88262495957315,
"inferenceMs": 3438.265999953728,
"totalMs": 3823.1486249133013,
"imagesPerSecond": 0.2908442802312148,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 384.88262495957315,
"inferenceMs": 3350.5487089860253,
"totalMs": 3735.4313339455985,
"imagesPerSecond": 0.2984585770438268,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 384.88262495957315,
"inferenceMs": 3365.4015000211075,
"totalMs": 3750.2841249806806,
"imagesPerSecond": 0.2971413663403098,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 384.88262495957315,
"inferenceMs": 3441.6352080297656,
"totalMs": 3826.517832989339,
"imagesPerSecond": 0.2905595565930041,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 384.88262495957315,
"inferenceMs": 3512.5767920399085,
"totalMs": 3897.4594169994816,
"imagesPerSecond": 0.2846912848328807,
"grade": "slow"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Native output for same fixture and configuration as spatial qualification"
},
"runtime": {
"acceleration": "partial",
"accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
"artifactRevision": "d1bb11420d0d054623ffb2b6899ff0430e4cc655",
"cpuThreads": 4,
"device": "mps",
"durationMs": 4129.924792,
"engine": "transformers",
"fallback": false,
"fallbackReason": null,
"family": {
"catalogId": "eomt-native",
"defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
"displayName": "EoMT Desktop",
"id": "eomt",
"variantId": "coco-panoptic-large-1280",
"variantName": "COCO Panoptic Large 1280"
},
"inferenceMs": 4118.7245000037365,
"loadMs": 4377.031083000001,
"modelId": "eomt-coco-panoptic-large-1280-native",
"parameterDtype": "torch.float16",
"placement": {
"mps:0": 458
},
"precision": "fp16",
"predictionDevices": [
"mps:0",
"mps:0"
],
"provider": "Metal/MPS",
"requestedDevice": "mps",
"requestedPrecision": "fp16",
"reused": false,
"selectionReason": "Explicit device selection; actual placement is reported after inference",
"torchVersion": "2.11.0",
"version": "4.57.6",
"totalMs": 8506.955875,
"imagesPerSecond": 0.2427936124397475,
"grade": "moderate"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Native output for same fixture and configuration as spatial qualification"
},
"runtime": {
"acceleration": "partial",
"accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
"artifactRevision": "d1bb11420d0d054623ffb2b6899ff0430e4cc655",
"cpuThreads": 4,
"device": "mps",
"durationMs": 3817.920584,
"engine": "transformers",
"fallback": false,
"fallbackReason": null,
"family": {
"catalogId": "eomt-native",
"defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
"displayName": "EoMT Desktop",
"id": "eomt",
"variantId": "coco-panoptic-large-1280",
"variantName": "COCO Panoptic Large 1280"
},
"inferenceMs": 3806.836832955014,
"loadMs": 4377.031083000001,
"modelId": "eomt-coco-panoptic-large-1280-native",
"parameterDtype": "torch.float16",
"placement": {
"mps:0": 458
},
"precision": "fp16",
"predictionDevices": [
"mps:0",
"mps:0"
],
"provider": "Metal/MPS",
"requestedDevice": "mps",
"requestedPrecision": "fp16",
"reused": false,
"selectionReason": "Explicit device selection; actual placement is reported after inference",
"torchVersion": "2.11.0",
"version": "4.57.6",
"totalMs": 8194.951667000001,
"imagesPerSecond": 0.262685280163101,
"grade": "moderate"
}
}
]EoMT COCO Panoptic Large 640 Desktop
32 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 397 ms / 371 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 403 ms / 282 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 408 ms / 286 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 441 ms / 371 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 444 ms / 282 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 445 ms / 371 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 446 ms / 282 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 448 ms / 371 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 455 ms / 286 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 455 ms / 282 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 456 ms / 286 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 457 ms / 286 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 507 ms / 282 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 516 ms / 286 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 529 ms / 371 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 537 ms / 371 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 603 ms / 282 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 615 ms / 286 ms |
FP16 | mpsAccelerated | Passed | Fast | Not recorded | 964 ms / 3879 ms |
FP16 | mpsAccelerated | Passed | Moderate | Not recorded | 1294 ms / 3879 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1777 ms / 78 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1799 ms / 116 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1815 ms / 78 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1830 ms / 78 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1830 ms / 78 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1859 ms / 116 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1874 ms / 116 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1893 ms / 116 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1925 ms / 78 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1927 ms / 116 ms |
FP32 | cpu | Passed | Slow | macos · aarch64 | 6040 ms / 116 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 5296 ms / 78 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-14T21:19:31.165285+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
- Runtime
- transformers 4.57.6
- Artifact revision
- dcd130bed9b1ebda7041fd660fddb16f905b9c3b
- Fixture
{ "file": "model-samples/desktop/mobilenet-dog.jpg", "size": 83281, "sha1": "f1d042827c2f2154a6d8043b8e8f07c1be612941", "sha256": "d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1", "url": "https://assets.localai.tools/samples/v1/files/d4cb12ea8a768582562e1f1c0c930ea6fc4757f5b66551b986b41498efd75ca1/mobilenet-dog.jpg" }- Rerun when
- Checkpoint or runtime revision; Preprocessing or postprocessing; Fixture or quality gate
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": false,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 77.70008296938613,
"inferenceMs": 5296.401540981606,
"totalMs": 5374.101623950992,
"imagesPerSecond": 0.18880743694796703,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 77.70008296938613,
"inferenceMs": 1924.6754579944536,
"totalMs": 2002.3755409638397,
"imagesPerSecond": 0.5195681151574604,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 77.70008296938613,
"inferenceMs": 1776.6283339587972,
"totalMs": 1854.3284169281833,
"imagesPerSecond": 0.562863926509455,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 77.70008296938613,
"inferenceMs": 1814.7650830214843,
"totalMs": 1892.4651659908704,
"imagesPerSecond": 0.5510355083177239,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 77.70008296938613,
"inferenceMs": 1829.777541977819,
"totalMs": 1907.4776249472052,
"imagesPerSecond": 0.5465145226993513,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 77.70008296938613,
"inferenceMs": 1830.203665944282,
"totalMs": 1907.903748913668,
"imagesPerSecond": 0.5463872784256808,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 115.972374973353,
"inferenceMs": 6039.950290985871,
"totalMs": 6155.922665959224,
"imagesPerSecond": 0.16556427649618535,
"grade": "slow"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 115.972374973353,
"inferenceMs": 1926.5651250025257,
"totalMs": 2042.5374999758787,
"imagesPerSecond": 0.5190584979569216,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 115.972374973353,
"inferenceMs": 1798.6600840231404,
"totalMs": 1914.6324589964934,
"imagesPerSecond": 0.5559694179476408,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 115.972374973353,
"inferenceMs": 1873.5294159851037,
"totalMs": 1989.5017909584567,
"imagesPerSecond": 0.5337519611210368,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 115.972374973353,
"inferenceMs": 1859.203708008863,
"totalMs": 1975.176082982216,
"imagesPerSecond": 0.5378646759859156,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "cpu",
"precision": "fp32",
"fallback": false,
"provider": "PyTorch CPU",
"loadMs": 115.972374973353,
"inferenceMs": 1892.8136669564992,
"totalMs": 2008.7860419298522,
"imagesPerSecond": 0.5283140213203997,
"grade": "moderate"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 281.87954099848866,
"inferenceMs": 603.4478329820558,
"totalMs": 885.3273739805445,
"imagesPerSecond": 1.657144073346496,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 281.87954099848866,
"inferenceMs": 506.68249995214865,
"totalMs": 788.5620409506373,
"imagesPerSecond": 1.9736225350084926,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 281.87954099848866,
"inferenceMs": 402.5952079682611,
"totalMs": 684.4747489667498,
"imagesPerSecond": 2.4838845078325815,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 281.87954099848866,
"inferenceMs": 444.11112496163696,
"totalMs": 725.9906659601256,
"imagesPerSecond": 2.2516886963513505,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 281.87954099848866,
"inferenceMs": 455.49562497762963,
"totalMs": 737.3751659761183,
"imagesPerSecond": 2.195410768323213,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "bf16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 281.87954099848866,
"inferenceMs": 445.60620799893513,
"totalMs": 727.4857489974238,
"imagesPerSecond": 2.2441339057879324,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 285.9305420424789,
"inferenceMs": 615.2746669831686,
"totalMs": 901.2052090256475,
"imagesPerSecond": 1.6252903843794302,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 285.9305420424789,
"inferenceMs": 516.4917499641888,
"totalMs": 802.4222920066677,
"imagesPerSecond": 1.9361393479553846,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 285.9305420424789,
"inferenceMs": 407.85075002349913,
"totalMs": 693.781292065978,
"imagesPerSecond": 2.4518773103699893,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 285.9305420424789,
"inferenceMs": 457.03470800071955,
"totalMs": 742.9652500431985,
"imagesPerSecond": 2.188017633003106,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 285.9305420424789,
"inferenceMs": 455.0348339835182,
"totalMs": 740.9653760259971,
"imagesPerSecond": 2.197633950890496,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp16",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 285.9305420424789,
"inferenceMs": 456.17958396906033,
"totalMs": 742.1101260115393,
"imagesPerSecond": 2.1921191459278973,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 371.2309999973513,
"inferenceMs": 537.4406250193715,
"totalMs": 908.6716250167228,
"imagesPerSecond": 1.8606706554123182,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 371.2309999973513,
"inferenceMs": 528.5210000001825,
"totalMs": 899.7519999975339,
"imagesPerSecond": 1.8920724058261726,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": null
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 371.2309999973513,
"inferenceMs": 397.4365000030957,
"totalMs": 768.667500000447,
"imagesPerSecond": 2.516125217467975,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 371.2309999973513,
"inferenceMs": 441.2139590131119,
"totalMs": 812.4449590104632,
"imagesPerSecond": 2.2664740758355792,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 371.2309999973513,
"inferenceMs": 444.7900840314105,
"totalMs": 816.0210840287618,
"imagesPerSecond": 2.248251559334181,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"pointRecall": 1
},
"runtime": {
"device": "mps",
"precision": "fp32",
"fallback": false,
"provider": "Metal/MPS",
"loadMs": 371.2309999973513,
"inferenceMs": 448.1362079968676,
"totalMs": 819.3672079942189,
"imagesPerSecond": 2.231464412281968,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Native output for same fixture and configuration as spatial qualification"
},
"runtime": {
"acceleration": "partial",
"accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
"artifactRevision": "dcd130bed9b1ebda7041fd660fddb16f905b9c3b",
"cpuThreads": 4,
"device": "mps",
"durationMs": 1306.739292,
"engine": "transformers",
"fallback": false,
"fallbackReason": null,
"family": {
"catalogId": "eomt-native",
"defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
"displayName": "EoMT",
"id": "eomt",
"variantId": "coco-panoptic-large-640",
"variantName": "COCO Panoptic Large 640"
},
"inferenceMs": 1293.7580840080045,
"loadMs": 3879.1905,
"modelId": "eomt-coco-panoptic-large-640-native",
"parameterDtype": "torch.float16",
"placement": {
"mps:0": 458
},
"precision": "fp16",
"predictionDevices": [
"mps:0",
"mps:0"
],
"provider": "Metal/MPS",
"requestedDevice": "mps",
"requestedPrecision": "fp16",
"reused": false,
"selectionReason": "Explicit device selection; actual placement is reported after inference",
"torchVersion": "2.11.0",
"version": "4.57.6",
"totalMs": 5185.929792,
"imagesPerSecond": 0.7729420301684569,
"grade": "moderate"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Native output for same fixture and configuration as spatial qualification"
},
"runtime": {
"acceleration": "partial",
"accelerationDetails": "Model inputs and predictions use the selected device. Image processing and visualization use CPU.",
"artifactRevision": "dcd130bed9b1ebda7041fd660fddb16f905b9c3b",
"cpuThreads": 4,
"device": "mps",
"durationMs": 974.563083,
"engine": "transformers",
"fallback": false,
"fallbackReason": null,
"family": {
"catalogId": "eomt-native",
"defaultModelId": "eomt-coco-panoptic-small-640-2x-native",
"displayName": "EoMT",
"id": "eomt",
"variantId": "coco-panoptic-large-640",
"variantName": "COCO Panoptic Large 640"
},
"inferenceMs": 963.8849589973688,
"loadMs": 3879.1905,
"modelId": "eomt-coco-panoptic-large-640-native",
"parameterDtype": "torch.float16",
"placement": {
"mps:0": 458
},
"precision": "fp16",
"predictionDevices": [
"mps:0",
"mps:0"
],
"provider": "Metal/MPS",
"requestedDevice": "mps",
"requestedPrecision": "fp16",
"reused": false,
"selectionReason": "Explicit device selection; actual placement is reported after inference",
"torchVersion": "2.11.0",
"version": "4.57.6",
"totalMs": 4853.753583,
"imagesPerSecond": 1.037468206828539,
"grade": "fast"
}
}
]Release validation still required
- Managed runtime, full precision lifecycle, backup and forced recovery require qualification.