MobileNet 2 Desktop
Classify still images into1,000 ImageNet categories with the exact original TorchVision V1 checkpoint, using built-in Hugging Face Transformers MobileNetV2.
Desktop app
Run MobileNet 2 Desktop on your computer
Download the desktop app to load this model, provide input and view results locally.
Public desktop downloads are coming soon.Device requirements
If your device falls short
MemoryAdvisory only: close other apps, reduce image/tile/token/batch settings or choose a smaller verified model/precision. CPU can need more RAM than GPU.
GPUIf the selected GPU runtime fails, existing CPU fallback may recover when enabled, but CPU has its own memory/runtime needs.
PlatformUse a compatible OS/architecture/runtime or another model. These estimates do not add hardware detection or block runs.
Benchmark evidence
Saved desktop measurements apply to the recorded hardware and runtime. Fallback cases report the device that actually ran; they do not prove GPU support on the requested device.
MobileNetV2 ImageNet V1 Desktop
170 configurations testedQuality criteria
{ "frozenBeforeMatrix": true, "version": 1, "scope": "One independently identified Labrador photograph, mirror/downsize robustness derivatives and blank closed-set negative. Exact original TorchVision V1 architecture/full-logit and preprocessing parity independently checked. Not broad ImageNet quality.", "minimumExpectedScore": 0.05, "expectedTop5": "Labrador retriever", "maximumWallInferenceMs": 250, "requireFiniteNormalizedScores": true, "normalizationTolerance": 0.00001, "cpuGpuScoreTolerance": { "fp32": 0.0001, "fp16": 0.02, "bf16": 0.03 }, "matrix": { "cpuThreads": [ 1, 4 ], "mpsPrecisions": [ "fp32", "fp16", "bf16" ], "coldRepetitions": 3, "warmRepeats": 5, "warmupExcluded": 1, "sustainedPublicApiCalls": 20 }, "blank": "Require1000 unique indices, finite probabilities summing to1; no invented unknown class or confidence calibration claim." }
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 12 ms / 1869 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 12 ms / 1921 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 12 ms / 1871 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 13 ms / 1838 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 13 ms / 1851 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 13 ms / 1947 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 14 ms / 1863 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 14 ms / 1838 ms |
BF16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 14 ms / 1851 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 15 ms / 1868 ms |
BF16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 15 ms / 1842 ms |
BF16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 15 ms / 1920 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 15 ms / 1869 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 15 ms / 1871 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 15 ms / 1868 ms |
BF16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 16 ms / 1925 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 16 ms / 1947 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 16 ms / 1921 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 16 ms / 1851 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 19 ms / 1863 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 63 ms / 1947 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 64 ms / 1947 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 64 ms / 1869 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 64 ms / 1947 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 64 ms / 1868 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 64 ms / 1947 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 65 ms / 1921 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 65 ms / 1869 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 65 ms / 1838 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 65 ms / 1838 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 65 ms / 1869 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 65 ms / 1838 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 66 ms / 1851 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 66 ms / 1947 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 66 ms / 1921 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 66 ms / 1871 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 66 ms / 1871 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 67 ms / 1869 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 67 ms / 1851 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 67 ms / 1869 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 67 ms / 1863 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 67 ms / 1947 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 67 ms / 1947 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 67 ms / 1871 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 67 ms / 1838 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 67 ms / 1851 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 68 ms / 1921 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 68 ms / 1851 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 69 ms / 1921 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 69 ms / 1851 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 69 ms / 1871 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 69 ms / 1869 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 70 ms / 1871 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 70 ms / 1921 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 70 ms / 1871 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 71 ms / 1921 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 71 ms / 1851 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 71 ms / 1851 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 71 ms / 1869 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 72 ms / 1871 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 72 ms / 1921 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 72 ms / 1838 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 73 ms / 1838 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 74 ms / 1838 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 82 ms / 1947 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 82 ms / 1869 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 82 ms / 1868 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 83 ms / 1838 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 84 ms / 1871 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 85 ms / 1863 ms |
FP32 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 86 ms / 1851 ms |
FP16 | mpsAccelerated | Passed | Very fast | macos · aarch64 | 87 ms / 1921 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 15 ms / 1554 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 15 ms / 1598 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 15 ms / 1554 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 15 ms / 1598 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 16 ms / 2593 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 16 ms / 2593 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 16 ms / 1685 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 16 ms / 1685 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 22 ms / 1726 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 22 ms / 1726 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 22 ms / 1711 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 22 ms / 1540 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 22 ms / 1604 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 23 ms / 1540 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 23 ms / 1604 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 23 ms / 1711 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 62 ms / 1598 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 62 ms / 1598 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 62 ms / 1598 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 63 ms / 1554 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 63 ms / 1598 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 63 ms / 1598 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 63 ms / 1598 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 64 ms / 1598 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 65 ms / 1685 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 65 ms / 1554 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 65 ms / 1554 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 66 ms / 1685 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 66 ms / 1554 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 66 ms / 1685 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 66 ms / 1554 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 66 ms / 1554 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 67 ms / 1685 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 67 ms / 1554 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 68 ms / 1598 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 68 ms / 1554 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 68 ms / 1685 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 68 ms / 1685 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 70 ms / 1604 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 70 ms / 1685 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 70 ms / 1604 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 70 ms / 1604 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 70 ms / 1604 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 70 ms / 1540 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 70 ms / 1604 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 71 ms / 1540 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 71 ms / 2593 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 71 ms / 1540 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 71 ms / 1726 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 71 ms / 1540 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 72 ms / 1604 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 73 ms / 1711 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 73 ms / 1711 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 73 ms / 1711 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 74 ms / 1711 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 74 ms / 1685 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 74 ms / 1711 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 74 ms / 1711 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 74 ms / 1711 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 75 ms / 1540 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 75 ms / 1540 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 75 ms / 1540 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 75 ms / 1604 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 75 ms / 1540 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 76 ms / 1726 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 76 ms / 1604 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 78 ms / 1711 ms |
FP32 | cpu | Passed | Very fast | macos · aarch64 | 94 ms / 2593 ms |
BF16 | mps | Failed | — | macos · aarch64 | 82 ms / 1920 ms |
BF16 | mps | Failed | — | macos · aarch64 | 64 ms / 1920 ms |
BF16 | mps | Failed | — | macos · aarch64 | 15 ms / 1920 ms |
BF16 | mps | Failed | — | macos · aarch64 | 82 ms / 1851 ms |
BF16 | mps | Failed | — | macos · aarch64 | 67 ms / 1851 ms |
BF16 | mps | Failed | — | macos · aarch64 | 70 ms / 1851 ms |
BF16 | mps | Failed | — | macos · aarch64 | 12 ms / 1851 ms |
BF16 | mps | Failed | — | macos · aarch64 | 65 ms / 1851 ms |
BF16 | mps | Failed | — | macos · aarch64 | 65 ms / 1851 ms |
BF16 | mps | Failed | — | macos · aarch64 | 71 ms / 1851 ms |
BF16 | mps | Failed | — | macos · aarch64 | 66 ms / 1851 ms |
BF16 | mps | Failed | — | macos · aarch64 | 65 ms / 1851 ms |
BF16 | mps | Failed | — | macos · aarch64 | 87 ms / 1925 ms |
BF16 | mps | Failed | — | macos · aarch64 | 67 ms / 1925 ms |
BF16 | mps | Failed | — | macos · aarch64 | 72 ms / 1925 ms |
BF16 | mps | Failed | — | macos · aarch64 | 12 ms / 1925 ms |
BF16 | mps | Failed | — | macos · aarch64 | 66 ms / 1925 ms |
BF16 | mps | Failed | — | macos · aarch64 | 67 ms / 1925 ms |
BF16 | mps | Failed | — | macos · aarch64 | 69 ms / 1925 ms |
BF16 | mps | Failed | — | macos · aarch64 | 65 ms / 1925 ms |
BF16 | mps | Failed | — | macos · aarch64 | 67 ms / 1925 ms |
BF16 | mps | Failed | — | macos · aarch64 | 82 ms / 1842 ms |
BF16 | mps | Failed | — | macos · aarch64 | 67 ms / 1842 ms |
BF16 | mps | Failed | — | macos · aarch64 | 64 ms / 1842 ms |
BF16 | mps | Failed | — | macos · aarch64 | 14 ms / 1842 ms |
BF16 | mps | Failed | — | macos · aarch64 | 67 ms / 1842 ms |
BF16 | mps | Failed | — | macos · aarch64 | 68 ms / 1842 ms |
BF16 | mps | Failed | — | macos · aarch64 | 70 ms / 1842 ms |
BF16 | mps | Failed | — | macos · aarch64 | 67 ms / 1842 ms |
BF16 | mps | Failed | — | macos · aarch64 | 71 ms / 1842 ms |
Test setup & sample details
- Quality gate
- { "frozenBeforeMatrix": true, "version": 1, "scope": "One independently identified Labrador photograph, mirror/downsize robustness derivatives and blank closed-set negative. Exact original TorchVision V1 architecture/full-logit and preprocessing parity independently checked. Not broad ImageNet quality.", "minimumExpectedScore": 0.05, "expectedTop5": "Labrador retriever", "maximumWallInferenceMs": 250, "requireFiniteNormalizedScores": true, "normalizationTolerance": 0.00001, "cpuGpuScoreTolerance": { "fp32": 0.0001, "fp16": 0.02, "bf16": 0.03 }, "matrix": { "cpuThreads": [ 1, 4 ], "mpsPrecisions": [ "fp32", "fp16", "bf16" ], "coldRepetitions": 3, "warmRepeats": 5, "warmupExcluded": 1, "sustainedPublicApiCalls": 20 }, "blank": "Require1000 unique indices, finite probabilities summing to1; no invented unknown class or confidence calibration claim." }
- Tested
- 2026-11-13T11:37:20.609519+00:00
- Environment
- { "os": "macos", "osVersion": "27.2", "hardware": "AppleM3Max;Mac15,11;96GiB", "concurrentUploadActivity": "none; excluded", "transport": "Pinned verified local snapshot, not Internet transfer. GPU synthetic graph compilation included in load/total, not hidden from initialization." }
- Runtime
- transformers 5.22.1
- Artifact revision
- sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889
- Fixture
{ "file": "research/mobilenet-v2-family-expansion/fixture-dog.png", "size": 709899, "sha1": "409baef01589c0a55c12bba3b4f5330adc74b1c7", "sha256": "cc986ccf120f7892c9ee0202001c30a91a3039cc5859d851b981476507a25fe3", "url": "https://assets.localai.tools/samples/v1/files/cc986ccf120f7892c9ee0202001c30a91a3039cc5859d851b981476507a25fe3/fixture-dog.png" }- Rerun when
- Artifact/runtime/preprocessing/config/host changes; Fixture checksum or frozen quality gate changes
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-0-0-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 2592.594999994617,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 31.572624982800335,
"inferenceMs": 94.45062500890344,
"imagesPerSecond": 10.587542431886867,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 24.0095829940401,
"totalMs": 2687.0456250035204,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 388628480,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.6782526969909668,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.8856543563306332,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-0-1-mirrored-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-2-mirrored-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 2592.594999994617,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 14.398291998077184,
"inferenceMs": 70.6326249637641,
"imagesPerSecond": 14.15776350536341,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 12.870709004346281,
"totalMs": 2663.227624958381,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 394330112,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.1767652928829193,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.734110951423645,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-0-2-small-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-3-small-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 2592.594999994617,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.396292051766068,
"inferenceMs": 15.78654203331098,
"imagesPerSecond": 63.34509469457674,
"inputWidth": 112,
"inputHeight": 92,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 12.68924999749288,
"totalMs": 2608.381542027928,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 394395648,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"expectedLabel": null,
"minimumExpectedScore": null,
"probabilitySum": 1.000000161377102,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-0-3-blank",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-4-blank",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 2592.594999994617,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.926416053436697,
"inferenceMs": 15.646499989088625,
"imagesPerSecond": 63.91205705412511,
"inputWidth": 224,
"inputHeight": 224,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 12.840624956879765,
"totalMs": 2608.2414999837056,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 394493952,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-1-0-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1726.0015839710832,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 22.461500018835068,
"inferenceMs": 75.82729199202731,
"imagesPerSecond": 13.187863811688576,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 20.92533401446417,
"totalMs": 1801.8288759631105,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 388907008,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.6782526969909668,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.8856543563306332,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-1-1-mirrored-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-2-mirrored-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1726.0015839710832,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 21.231083024758846,
"inferenceMs": 70.84262499120086,
"imagesPerSecond": 14.115795400356873,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 19.81049997266382,
"totalMs": 1796.844208962284,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 394608640,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.1767652928829193,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.734110951423645,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-1-2-small-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-3-small-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1726.0015839710832,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 20.114333019591868,
"inferenceMs": 22.278166958130896,
"imagesPerSecond": 44.886996397835524,
"inputWidth": 112,
"inputHeight": 92,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 19.5686660008505,
"totalMs": 1748.279750929214,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 394657792,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"expectedLabel": null,
"minimumExpectedScore": null,
"probabilitySum": 1.000000161377102,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-1-3-blank",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-4-blank",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1726.0015839710832,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 20.673791994340718,
"inferenceMs": 22.192624979652464,
"imagesPerSecond": 45.060014347868275,
"inputWidth": 224,
"inputHeight": 224,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 19.75708396639675,
"totalMs": 1748.1942089507356,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 394706944,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-5-0-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1685.346667014528,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 15.583999978844076,
"inferenceMs": 73.72254104120657,
"imagesPerSecond": 13.564372387016052,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 13.827292015776038,
"totalMs": 1759.0692080557346,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 389890048,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1685.346667014528,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.993084023240954,
"inferenceMs": 68.11508402461186,
"imagesPerSecond": 14.681035989600664,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 12.4688750365749,
"totalMs": 1753.46175103914,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 395591680,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.6782526969909668,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.8856543563306332,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-5-2-mirrored-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-2-mirrored-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1685.346667014528,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 14.209041022695601,
"inferenceMs": 67.97733303392306,
"imagesPerSecond": 14.710786013051809,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 12.684207991696894,
"totalMs": 1753.3240000484511,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 401293312,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.1767652928829193,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.734110951423645,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-5-3-small-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-3-small-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1685.346667014528,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.942208955995739,
"inferenceMs": 16.46499999333173,
"imagesPerSecond": 60.73489222016378,
"inputWidth": 112,
"inputHeight": 92,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 13.202667003497481,
"totalMs": 1701.8116670078598,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 401768448,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"expectedLabel": null,
"minimumExpectedScore": null,
"probabilitySum": 1.000000161377102,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-5-4-blank",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-4-blank",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1685.346667014528,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 14.322916977107525,
"inferenceMs": 15.983749995939434,
"imagesPerSecond": 62.56354111231991,
"inputWidth": 224,
"inputHeight": 224,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 13.226291979663074,
"totalMs": 1701.3304170104675,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 401801216,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-5-5-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1685.346667014528,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.989083003252745,
"inferenceMs": 69.85920801525936,
"imagesPerSecond": 14.314505251499128,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 12.439250014722347,
"totalMs": 1755.2058750297874,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 407502848,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-5-6-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1685.346667014528,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.534541998524219,
"inferenceMs": 66.08854199294001,
"imagesPerSecond": 15.131215939168793,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 12.099582992959768,
"totalMs": 1751.435209007468,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 413204480,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-5-7-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1685.346667014528,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.569791975896806,
"inferenceMs": 65.61612500809133,
"imagesPerSecond": 15.240156285923419,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 12.139207974541932,
"totalMs": 1750.9627920226194,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 418906112,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-5-8-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1685.346667014528,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.499084045179188,
"inferenceMs": 65.0119159836322,
"imagesPerSecond": 15.381795550399808,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 12.080667016562074,
"totalMs": 1750.3585829981603,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 424607744,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-5-9-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1685.346667014528,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 14.236624992918223,
"inferenceMs": 66.87537499237806,
"imagesPerSecond": 14.953187180093904,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 12.652125034946948,
"totalMs": 1752.2220420069061,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 430309376,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-6-0-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1597.8443750063889,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 14.783209015149623,
"inferenceMs": 67.67366698477417,
"imagesPerSecond": 14.776796419573198,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 13.154375017620623,
"totalMs": 1665.518041991163,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 389251072,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-6-1-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1597.8443750063889,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.60254199244082,
"inferenceMs": 62.744875031057745,
"imagesPerSecond": 15.93755664514457,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 12.163792038336396,
"totalMs": 1660.5892500374466,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 394952704,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.6782526969909668,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.8856543563306332,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-6-2-mirrored-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-2-mirrored-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1597.8443750063889,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.576416007708758,
"inferenceMs": 63.708834000863135,
"imagesPerSecond": 15.69641032806301,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 12.130958028137684,
"totalMs": 1661.553209007252,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 400654336,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.1767652928829193,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.734110951423645,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-6-3-small-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-3-small-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1597.8443750063889,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 12.777374999132007,
"inferenceMs": 14.96241701534018,
"imagesPerSecond": 66.83412171808556,
"inputWidth": 112,
"inputHeight": 92,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 12.165124993771315,
"totalMs": 1612.806792021729,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 401129472,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"expectedLabel": null,
"minimumExpectedScore": null,
"probabilitySum": 1.000000161377102,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-6-4-blank",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-4-blank",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1597.8443750063889,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.295874989125878,
"inferenceMs": 14.857292000669986,
"imagesPerSecond": 67.30701664575922,
"inputWidth": 224,
"inputHeight": 224,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 12.275375018361956,
"totalMs": 1612.7016670070589,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 401162240,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-6-5-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1597.8443750063889,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.535666977986693,
"inferenceMs": 62.15587502811104,
"imagesPerSecond": 16.08858373480758,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 12.096541002392769,
"totalMs": 1660.0002500345,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 406863872,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-6-6-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1597.8443750063889,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.484709023032337,
"inferenceMs": 62.79137497767806,
"imagesPerSecond": 15.925754139887744,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 12.045749987009913,
"totalMs": 1660.635749984067,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 412565504,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-6-7-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1597.8443750063889,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.522291032131761,
"inferenceMs": 62.929917010478675,
"imagesPerSecond": 15.89069313143201,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 12.118624988943338,
"totalMs": 1660.7742920168675,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 418267136,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-6-8-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1597.8443750063889,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.490499986801296,
"inferenceMs": 62.47695896308869,
"imagesPerSecond": 16.00590068077415,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 12.086290982551873,
"totalMs": 1660.3213339694776,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 423968768,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-6-9-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1597.8443750063889,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.481334026437253,
"inferenceMs": 62.45874997694045,
"imagesPerSecond": 16.010566980114017,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 12.073666031938046,
"totalMs": 1660.3031249833293,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 429670400,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-7-0-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1554.1714999708347,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 14.590832986868918,
"inferenceMs": 67.72133300546557,
"imagesPerSecond": 14.766395692761884,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 13.013040996156633,
"totalMs": 1621.8928329763003,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 389431296,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-7-1-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1554.1714999708347,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.68158299010247,
"inferenceMs": 62.707790988497436,
"imagesPerSecond": 15.946981774297091,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 12.200542027130723,
"totalMs": 1616.8792909593321,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 395132928,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.6782526969909668,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.8856543563306332,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-7-2-mirrored-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-2-mirrored-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1554.1714999708347,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.526124996133149,
"inferenceMs": 66.21274998178706,
"imagesPerSecond": 15.102831407471628,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 12.078040977939963,
"totalMs": 1620.3842499526218,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 400834560,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.1767652928829193,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.734110951423645,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-7-3-small-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-3-small-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1554.1714999708347,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 12.678832979872823,
"inferenceMs": 14.951165998354554,
"imagesPerSecond": 66.88441557735729,
"inputWidth": 112,
"inputHeight": 92,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 12.063250003848225,
"totalMs": 1569.1226659691893,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 400850944,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"expectedLabel": null,
"minimumExpectedScore": null,
"probabilitySum": 1.000000161377102,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-7-4-blank",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-4-blank",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1554.1714999708347,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.156250002793968,
"inferenceMs": 14.657250023446977,
"imagesPerSecond": 68.22562202325234,
"inputWidth": 224,
"inputHeight": 224,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 12.168958957772702,
"totalMs": 1568.8287499942817,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 401047552,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-7-5-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1554.1714999708347,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.50125001044944,
"inferenceMs": 65.13345899293199,
"imagesPerSecond": 15.353092181216965,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 12.08529097493738,
"totalMs": 1619.3049589637667,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 406749184,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-7-6-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1554.1714999708347,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.495249964762479,
"inferenceMs": 65.85912499576807,
"imagesPerSecond": 15.183924779812324,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 12.076041952241212,
"totalMs": 1620.0306249666028,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 412450816,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-7-7-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1554.1714999708347,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.713083986658603,
"inferenceMs": 66.39987498056144,
"imagesPerSecond": 15.060269319674925,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 12.276833003852516,
"totalMs": 1620.5713749513961,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 418152448,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-7-8-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1554.1714999708347,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.48604098893702,
"inferenceMs": 67.05587496981025,
"imagesPerSecond": 14.912936419817322,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 12.048000004142523,
"totalMs": 1621.227374940645,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 423854080,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-7-9-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1554.1714999708347,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.544499990530312,
"inferenceMs": 65.41541597107425,
"imagesPerSecond": 15.286916473055609,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 12.123916007112712,
"totalMs": 1619.586915941909,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 429555712,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-8-0-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1539.5254999748431,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 22.85416697850451,
"inferenceMs": 75.42512501822785,
"imagesPerSecond": 13.258181537761216,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 21.31387498229742,
"totalMs": 1614.950624993071,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 388825088,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-8-1-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1539.5254999748431,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 21.572500001639128,
"inferenceMs": 71.06962503166869,
"imagesPerSecond": 14.070708823275753,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 20.1997910044156,
"totalMs": 1610.5951250065118,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 394526720,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.6782526969909668,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.8856543563306332,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-8-2-mirrored-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-2-mirrored-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1539.5254999748431,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 21.37724997010082,
"inferenceMs": 74.51175001915544,
"imagesPerSecond": 13.420702100580385,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 20.030250016134232,
"totalMs": 1614.0372499939986,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 400228352,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.1767652928829193,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.734110951423645,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-8-3-small-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-3-small-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1539.5254999748431,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 20.395292027387768,
"inferenceMs": 22.606207989156246,
"imagesPerSecond": 44.23563653310101,
"inputWidth": 112,
"inputHeight": 92,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 19.86729202326387,
"totalMs": 1562.1317079639994,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 400703488,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"expectedLabel": null,
"minimumExpectedScore": null,
"probabilitySum": 1.000000161377102,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-8-4-blank",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-4-blank",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1539.5254999748431,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 20.824458973947912,
"inferenceMs": 22.32341602211818,
"imagesPerSecond": 44.79601146209853,
"inputWidth": 224,
"inputHeight": 224,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 19.935208023525774,
"totalMs": 1561.8489159969613,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 400752640,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-8-5-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1539.5254999748431,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 21.22562500881031,
"inferenceMs": 74.7109999647364,
"imagesPerSecond": 13.384909858949822,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 19.883500004652888,
"totalMs": 1614.2364999395795,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 406470656,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-8-6-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1539.5254999748431,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 21.302625013049692,
"inferenceMs": 70.3744160127826,
"imagesPerSecond": 14.209709389536716,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 19.93854099418968,
"totalMs": 1609.8999159876257,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 412172288,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-8-7-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1539.5254999748431,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 21.297542029060423,
"inferenceMs": 74.69212496653199,
"imagesPerSecond": 13.388292279113488,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 19.951374968513846,
"totalMs": 1614.217624941375,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 417873920,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-8-8-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1539.5254999748431,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 21.411666995845735,
"inferenceMs": 70.77533396659419,
"imagesPerSecond": 14.129216267237933,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 20.058666996192187,
"totalMs": 1610.3008339414373,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 423575552,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-8-9-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1539.5254999748431,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 21.41191699774936,
"inferenceMs": 70.51637495169416,
"imagesPerSecond": 14.181103335005949,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 20.06208396051079,
"totalMs": 1610.0418749265373,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 429277184,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-9-0-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1604.4509580242448,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 22.94316701591015,
"inferenceMs": 75.91704203514382,
"imagesPerSecond": 13.172272959964324,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 21.397167001850903,
"totalMs": 1680.3680000593886,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 389758976,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-9-1-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1604.4509580242448,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 21.50320803048089,
"inferenceMs": 70.05025004036725,
"imagesPerSecond": 14.275466531864465,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 20.129333017393947,
"totalMs": 1674.501208064612,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 395476992,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.6782526969909668,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.8856543563306332,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-9-2-mirrored-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-2-mirrored-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1604.4509580242448,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 21.519499947316945,
"inferenceMs": 70.13745798030868,
"imagesPerSecond": 14.257716615289269,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 20.13575000455603,
"totalMs": 1674.5884160045534,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 401178624,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.1767652928829193,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.734110951423645,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-9-3-small-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-3-small-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1604.4509580242448,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 20.482584019191563,
"inferenceMs": 22.654500033240765,
"imagesPerSecond": 44.14134050774496,
"inputWidth": 112,
"inputHeight": 92,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 19.940457947086543,
"totalMs": 1627.1054580574855,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 401653760,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"expectedLabel": null,
"minimumExpectedScore": null,
"probabilitySum": 1.000000161377102,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-9-4-blank",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-4-blank",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1604.4509580242448,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 20.910542050842196,
"inferenceMs": 22.395458014216274,
"imagesPerSecond": 44.651911086846994,
"inputWidth": 224,
"inputHeight": 224,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 20.02629201160744,
"totalMs": 1626.846416038461,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 401719296,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-9-5-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1604.4509580242448,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 21.277541993185878,
"inferenceMs": 69.723583990708,
"imagesPerSecond": 14.342349356758097,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 19.935334043111652,
"totalMs": 1674.1745420149527,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 407420928,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-9-6-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1604.4509580242448,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 21.194167027715594,
"inferenceMs": 70.0700000161305,
"imagesPerSecond": 14.271442839586047,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 19.831417012028396,
"totalMs": 1674.5209580403753,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 413122560,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-9-7-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1604.4509580242448,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 21.21124998666346,
"inferenceMs": 70.48150000628084,
"imagesPerSecond": 14.188120285619442,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 19.864458008669317,
"totalMs": 1674.9324580305256,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 418824192,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-9-8-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1604.4509580242448,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 22.648375015705824,
"inferenceMs": 71.54545799130574,
"imagesPerSecond": 13.977127662269222,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 21.261957997921854,
"totalMs": 1675.9964160155505,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 425033728,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-9-9-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1604.4509580242448,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 23.757583985570818,
"inferenceMs": 75.02475002547726,
"imagesPerSecond": 13.328934780328028,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 22.238458041101694,
"totalMs": 1679.475708049722,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 433455104,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-10-0-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1711.292041989509,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 22.882958000991493,
"inferenceMs": 77.9949999996461,
"imagesPerSecond": 12.821334701000545,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 21.35183400241658,
"totalMs": 1789.2870419891551,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 389758976,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-10-1-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1711.292041989509,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 21.48425002815202,
"inferenceMs": 73.41987499967217,
"imagesPerSecond": 13.620290146291655,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 20.09541599545628,
"totalMs": 1784.7119169891812,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 395460608,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.6782526969909668,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.8856543563306332,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-10-2-mirrored-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-2-mirrored-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1711.292041989509,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 21.35545900091529,
"inferenceMs": 73.4402080415748,
"imagesPerSecond": 13.61651916119159,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 19.98466602526605,
"totalMs": 1784.7322500310838,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 401162240,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.1767652928829193,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.734110951423645,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-10-3-small-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-3-small-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1711.292041989509,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 20.478916994761676,
"inferenceMs": 22.737208986654878,
"imagesPerSecond": 43.98077180831336,
"inputWidth": 112,
"inputHeight": 92,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 19.926165987271816,
"totalMs": 1734.029250976164,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 401653760,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"expectedLabel": null,
"minimumExpectedScore": null,
"probabilitySum": 1.000000161377102,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-10-4-blank",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-4-blank",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1711.292041989509,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 20.79166698968038,
"inferenceMs": 22.30308298021555,
"imagesPerSecond": 44.83685062226924,
"inputWidth": 224,
"inputHeight": 224,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 19.893124990630895,
"totalMs": 1733.5951249697246,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 401719296,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-10-5-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1711.292041989509,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 21.479499991983175,
"inferenceMs": 73.91058298526332,
"imagesPerSecond": 13.52986216059729,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 20.123499969486147,
"totalMs": 1785.2026249747723,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 407437312,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-10-6-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1711.292041989509,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 21.24650002224371,
"inferenceMs": 72.759083006531,
"imagesPerSecond": 13.743988498456448,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 19.893291988410056,
"totalMs": 1784.05112499604,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 413138944,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-10-7-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1711.292041989509,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 21.347208006773144,
"inferenceMs": 73.64745903760195,
"imagesPerSecond": 13.578200973497717,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 20.00658301403746,
"totalMs": 1784.939501027111,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 418840576,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-10-8-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1711.292041989509,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 21.152624976821244,
"inferenceMs": 73.97512497846037,
"imagesPerSecond": 13.518057594240956,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 19.79841699358076,
"totalMs": 1785.2671669679694,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 424542208,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996475219726562,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965188231319189,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-10-9-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1711.292041989509,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": "cpu",
"parameterDtype": "torch.float32",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 21.243249997496605,
"inferenceMs": 74.31808300316334,
"imagesPerSecond": 13.455675383303888,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"modelInferenceMs": 19.889249990228564,
"totalMs": 1785.6101249926724,
"requestedDevice": "cpu",
"actualDevice": "cpu",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 4,
"residentMemoryBytes": 430243840,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "bf97be5a7b5fb6d0aab5fa3772b0db3ae78d481ac627d69f4fd9c7876bbe9fb4",
"reuseScope": "CPU mathematical/valid-input execution unchanged by MPS-only init prewarming; original measurement retained.",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996474027633667,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965187039226294,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-11-0-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 1.1920928955078125e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1868.3387500350364,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 153.23866601102054,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 28.761500027030706,
"inferenceMs": 81.81133295875043,
"imagesPerSecond": 12.223245409095139,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77348864,
"modelInferenceMs": 11.123917007353157,
"totalMs": 1950.1500829937868,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 481050624,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.6782530546188354,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.8856547214090824,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-11-1-mirrored-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-2-mirrored-dog",
"sameClassSet": true,
"maximumScoreError": 3.5762786865234375e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1868.3387500350364,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 153.23866601102054,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 15.281916013918817,
"inferenceMs": 64.12649998674169,
"imagesPerSecond": 15.594177137482202,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77348864,
"modelInferenceMs": 12.083832989446819,
"totalMs": 1932.465250021778,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 486817792,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.17676469683647156,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7341110333800316,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-11-2-small-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-3-small-dog",
"sameClassSet": true,
"maximumScoreError": 6.556510925292969e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1868.3387500350364,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 153.23866601102054,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 12.429666996467859,
"inferenceMs": 14.562209020368755,
"imagesPerSecond": 68.67090004004608,
"inputWidth": 112,
"inputHeight": 92,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77348864,
"modelInferenceMs": 10.186666972003877,
"totalMs": 1882.900959055405,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 486817792,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"expectedLabel": null,
"minimumExpectedScore": null,
"probabilitySum": 0.9999999803803803,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-11-3-blank",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-4-blank",
"sameClassSet": true,
"maximumScoreError": 8.754432201385498e-8,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1868.3387500350364,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 153.23866601102054,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.531083008274436,
"inferenceMs": 14.915250008925796,
"imagesPerSecond": 67.04547355234179,
"inputWidth": 224,
"inputHeight": 224,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77365248,
"modelInferenceMs": 7.66912498511374,
"totalMs": 1883.2540000439622,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 487653376,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5947906374931335,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7952595055103302,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-12-0-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.004856884479522705,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1862.9496659850702,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 158.50349998800084,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 28.662958007771522,
"inferenceMs": 84.62858299026266,
"imagesPerSecond": 11.816338696289641,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 12.30958296218887,
"totalMs": 1947.5782489753328,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 486555648,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.6771113276481628,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.8851615022867918,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-12-1-mirrored-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-2-mirrored-dog",
"sameClassSet": true,
"maximumScoreError": 0.0020411089062690735,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1862.9496659850702,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 158.50349998800084,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 14.828417042735964,
"inferenceMs": 67.12212495040148,
"imagesPerSecond": 14.898217253088001,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 11.392084008548409,
"totalMs": 1930.0717909354717,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 492290048,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.17434847354888916,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7355400174856186,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-12-2-small-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-3-small-dog",
"sameClassSet": true,
"maximumScoreError": 0.00694487988948822,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1862.9496659850702,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 158.50349998800084,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 16.769165988080204,
"inferenceMs": 19.01737501611933,
"imagesPerSecond": 52.58349268247533,
"inputWidth": 112,
"inputHeight": 92,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 13.739624992012978,
"totalMs": 1881.9670410011895,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 492290048,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"expectedLabel": null,
"minimumExpectedScore": null,
"probabilitySum": 0.9999999157334969,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-12-3-blank",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-4-blank",
"sameClassSet": true,
"maximumScoreError": 0.00015655066817998886,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1862.9496659850702,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 158.50349998800084,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 12.412000040058047,
"inferenceMs": 13.896041025873274,
"imagesPerSecond": 71.96294240482473,
"inputWidth": 224,
"inputHeight": 224,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 7.142083020880818,
"totalMs": 1876.8457070109434,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 493273088,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.5648589134216309,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7866855692118406,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-13-0-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.03478860855102539,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1919.5253750076517,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 161.75641701556742,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 27.85195899195969,
"inferenceMs": 81.5963750355877,
"imagesPerSecond": 12.255446391630228,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 10.921165987383574,
"totalMs": 2001.1217500432394,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 482803712,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.6382553577423096,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.8794247079640627,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-13-1-mirrored-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-2-mirrored-dog",
"sameClassSet": true,
"maximumScoreError": 0.03999733924865723,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1919.5253750076517,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 161.75641701556742,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.873125019017607,
"inferenceMs": 63.943916000425816,
"imagesPerSecond": 15.638704392038498,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 10.723292012698948,
"totalMs": 1983.4692910080776,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 488587264,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.14329007267951965,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7653137668967247,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-13-2-small-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-3-small-dog",
"sameClassSet": true,
"maximumScoreError": 0.08517764508724213,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1919.5253750076517,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 161.75641701556742,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 12.566166988108307,
"inferenceMs": 14.67937498819083,
"imagesPerSecond": 68.1227913861778,
"inputWidth": 112,
"inputHeight": 92,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 10.090791969560087,
"totalMs": 1934.2047499958426,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 488587264,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"expectedLabel": null,
"minimumExpectedScore": null,
"probabilitySum": 1.0000000203799573,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-13-3-blank",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-4-blank",
"sameClassSet": true,
"maximumScoreError": 0.0020814710296690464,
"threshold": 0.03,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1919.5253750076517,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 161.75641701556742,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.332458038348705,
"inferenceMs": 14.700750005431473,
"imagesPerSecond": 68.02374026022692,
"inputWidth": 224,
"inputHeight": 224,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 7.597957970574498,
"totalMs": 1934.2261250130832,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 489521152,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996474027633667,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965187039226294,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-14-0-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 1.1920928955078125e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1851.3883340056054,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 151.15295798750594,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 31.989249982871115,
"inferenceMs": 86.43779199337587,
"imagesPerSecond": 11.569013702671104,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77348864,
"modelInferenceMs": 15.980958007276058,
"totalMs": 1937.8261259989813,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 481361920,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996474027633667,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965187039226294,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-14-1-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 1.1920928955078125e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1851.3883340056054,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 151.15295798750594,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 17.69629103364423,
"inferenceMs": 68.78341699484736,
"imagesPerSecond": 14.538387938402524,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77348864,
"modelInferenceMs": 14.707583002746105,
"totalMs": 1920.1717510004528,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 487211008,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.6782530546188354,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.8856547214090824,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-14-2-mirrored-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-2-mirrored-dog",
"sameClassSet": true,
"maximumScoreError": 3.5762786865234375e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1851.3883340056054,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 151.15295798750594,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 15.134582994505763,
"inferenceMs": 67.38445803057402,
"imagesPerSecond": 14.840217302723943,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77348864,
"modelInferenceMs": 12.085082998964936,
"totalMs": 1918.7727920361795,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 492912640,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.17676469683647156,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7341110333800316,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-14-3-small-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-3-small-dog",
"sameClassSet": true,
"maximumScoreError": 6.556510925292969e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1851.3883340056054,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 151.15295798750594,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 10.62304200604558,
"inferenceMs": 12.858500005677342,
"imagesPerSecond": 77.76956873340401,
"inputWidth": 112,
"inputHeight": 92,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77348864,
"modelInferenceMs": 8.777124981861562,
"totalMs": 1864.2468340112828,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 492912640,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"expectedLabel": null,
"minimumExpectedScore": null,
"probabilitySum": 0.9999999803803803,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-14-4-blank",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-4-blank",
"sameClassSet": true,
"maximumScoreError": 8.754432201385498e-8,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1851.3883340056054,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 151.15295798750594,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 14.067500014789402,
"inferenceMs": 16.284417011775076,
"imagesPerSecond": 61.40840039142398,
"inputWidth": 224,
"inputHeight": 224,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77365248,
"modelInferenceMs": 8.97529098438099,
"totalMs": 1867.6727510173805,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 493764608,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996474027633667,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965187039226294,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-14-5-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 1.1920928955078125e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1851.3883340056054,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 151.15295798750594,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 20.260709046851844,
"inferenceMs": 70.779166999273,
"imagesPerSecond": 14.128451102148057,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77365248,
"modelInferenceMs": 16.85229199938476,
"totalMs": 1922.1675010048784,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 499531776,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996474027633667,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965187039226294,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-14-6-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 1.1920928955078125e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1851.3883340056054,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 151.15295798750594,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 16.116750019136816,
"inferenceMs": 67.97887501306832,
"imagesPerSecond": 14.71045232519307,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77365248,
"modelInferenceMs": 11.82383397826925,
"totalMs": 1919.3672090186737,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 505233408,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996474027633667,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965187039226294,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-14-7-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 1.1920928955078125e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1851.3883340056054,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 151.15295798750594,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 19.159374991431832,
"inferenceMs": 70.9148749592714,
"imagesPerSecond": 14.101413851104311,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77365248,
"modelInferenceMs": 16.092375037260354,
"totalMs": 1922.3032089648768,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 510935040,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996474027633667,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965187039226294,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-14-8-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 1.1920928955078125e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1851.3883340056054,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 151.15295798750594,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 15.87454101536423,
"inferenceMs": 66.80795800639316,
"imagesPerSecond": 14.968276682013022,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77365248,
"modelInferenceMs": 12.863292009569705,
"totalMs": 1918.1962920119986,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 516636672,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996474027633667,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965187039226294,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-14-9-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 1.1920928955078125e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1851.3883340056054,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 151.15295798750594,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 14.810332970228046,
"inferenceMs": 65.63704099971801,
"imagesPerSecond": 15.235299836327115,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77365248,
"modelInferenceMs": 11.80787495104596,
"totalMs": 1917.0253750053234,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 522338304,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996474027633667,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965187039226294,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-15-0-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 1.1920928955078125e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1837.6554590067826,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 151.9484170130454,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 28.50974997272715,
"inferenceMs": 82.56608300143853,
"imagesPerSecond": 12.11151072750511,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77348864,
"modelInferenceMs": 12.429416994564235,
"totalMs": 1920.2215420082211,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 480460800,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996474027633667,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965187039226294,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-15-1-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 1.1920928955078125e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1837.6554590067826,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 151.9484170130454,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 14.994582976214588,
"inferenceMs": 65.41324994759634,
"imagesPerSecond": 15.287422667443016,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77348864,
"modelInferenceMs": 11.90058299107477,
"totalMs": 1903.068708954379,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 486244352,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.6782530546188354,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.8856547214090824,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-15-2-mirrored-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-2-mirrored-dog",
"sameClassSet": true,
"maximumScoreError": 3.5762786865234375e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1837.6554590067826,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 151.9484170130454,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 19.468416983727366,
"inferenceMs": 72.24412501091138,
"imagesPerSecond": 13.841956004712705,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77348864,
"modelInferenceMs": 15.988250030204654,
"totalMs": 1909.899584017694,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 491945984,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.17676469683647156,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7341110333800316,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-15-3-small-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-3-small-dog",
"sameClassSet": true,
"maximumScoreError": 6.556510925292969e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1837.6554590067826,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 151.9484170130454,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 10.428041976410896,
"inferenceMs": 12.650292017497122,
"imagesPerSecond": 79.04955858859702,
"inputWidth": 112,
"inputHeight": 92,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77348864,
"modelInferenceMs": 8.145542000420392,
"totalMs": 1850.3057510242797,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 491962368,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"expectedLabel": null,
"minimumExpectedScore": null,
"probabilitySum": 0.9999999803803803,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-15-4-blank",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-4-blank",
"sameClassSet": true,
"maximumScoreError": 8.754432201385498e-8,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1837.6554590067826,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 151.9484170130454,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 12.518915988039225,
"inferenceMs": 14.01541696395725,
"imagesPerSecond": 71.34999997300474,
"inputWidth": 224,
"inputHeight": 224,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77365248,
"modelInferenceMs": 7.530584000051022,
"totalMs": 1851.6708759707399,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 492830720,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996474027633667,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965187039226294,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-15-5-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 1.1920928955078125e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1837.6554590067826,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 151.9484170130454,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 20.506541011855006,
"inferenceMs": 73.0871669948101,
"imagesPerSecond": 13.68229254351875,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77365248,
"modelInferenceMs": 16.384249960538,
"totalMs": 1910.7426260015927,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 498548736,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996474027633667,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965187039226294,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-15-6-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 1.1920928955078125e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1837.6554590067826,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 151.9484170130454,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 15.607832989189774,
"inferenceMs": 65.45970804290846,
"imagesPerSecond": 15.276572870513046,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77365248,
"modelInferenceMs": 11.303125007543713,
"totalMs": 1903.115167049691,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 504250368,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996474027633667,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965187039226294,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-15-7-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 1.1920928955078125e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1837.6554590067826,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 151.9484170130454,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 21.449957974255085,
"inferenceMs": 73.68070801021531,
"imagesPerSecond": 13.572073708376378,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77365248,
"modelInferenceMs": 12.034957995638251,
"totalMs": 1911.336167016998,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 509952000,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996474027633667,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965187039226294,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-15-8-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 1.1920928955078125e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1837.6554590067826,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 151.9484170130454,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 17.103959049563855,
"inferenceMs": 67.38029199186713,
"imagesPerSecond": 14.841134854694621,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77365248,
"modelInferenceMs": 12.568208039738238,
"totalMs": 1905.0357509986497,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 515653632,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996474027633667,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965187039226294,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-15-9-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 1.1920928955078125e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1837.6554590067826,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 151.9484170130454,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 14.95358400279656,
"inferenceMs": 65.05608302541077,
"imagesPerSecond": 15.371352738980644,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77365248,
"modelInferenceMs": 11.743374983780086,
"totalMs": 1902.7115420321934,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 521355264,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996474027633667,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965187039226294,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-16-0-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 1.1920928955078125e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1870.5207500024699,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 148.39041704544798,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 29.399582999758422,
"inferenceMs": 83.90729100210592,
"imagesPerSecond": 11.917915452363989,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77348864,
"modelInferenceMs": 12.873041967395693,
"totalMs": 1954.4280410045758,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 480657408,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996474027633667,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965187039226294,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-16-1-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 1.1920928955078125e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1870.5207500024699,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 148.39041704544798,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 14.926500036381185,
"inferenceMs": 66.31899997591972,
"imagesPerSecond": 15.078635087427399,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77348864,
"modelInferenceMs": 11.717875022441149,
"totalMs": 1936.8397499783896,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 486408192,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.6782530546188354,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.8856547214090824,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-16-2-mirrored-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-2-mirrored-dog",
"sameClassSet": true,
"maximumScoreError": 3.5762786865234375e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1870.5207500024699,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 148.39041704544798,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 18.061208014842123,
"inferenceMs": 70.4507080372423,
"imagesPerSecond": 14.194321503076603,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77348864,
"modelInferenceMs": 14.845917001366615,
"totalMs": 1940.9714580397122,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 492109824,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.17676469683647156,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7341110333800316,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-16-3-small-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-3-small-dog",
"sameClassSet": true,
"maximumScoreError": 6.556510925292969e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1870.5207500024699,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 148.39041704544798,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 10.160667006857693,
"inferenceMs": 12.282792013138533,
"imagesPerSecond": 81.41471409190436,
"inputWidth": 112,
"inputHeight": 92,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77348864,
"modelInferenceMs": 7.82595801865682,
"totalMs": 1882.8035420156084,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 492126208,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"expectedLabel": null,
"minimumExpectedScore": null,
"probabilitySum": 0.9999999803803803,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-16-4-blank",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-4-blank",
"sameClassSet": true,
"maximumScoreError": 8.754432201385498e-8,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1870.5207500024699,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 148.39041704544798,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.46887502586469,
"inferenceMs": 14.871333027258515,
"imagesPerSecond": 67.24346756051007,
"inputWidth": 224,
"inputHeight": 224,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77365248,
"modelInferenceMs": 7.759415952023119,
"totalMs": 1885.3920830297284,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 493076480,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996474027633667,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965187039226294,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-16-5-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 1.1920928955078125e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1870.5207500024699,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 148.39041704544798,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 15.422833967022598,
"inferenceMs": 66.38387497514486,
"imagesPerSecond": 15.06389918296296,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77365248,
"modelInferenceMs": 11.81220798753202,
"totalMs": 1936.9046249776147,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 498827264,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996474027633667,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965187039226294,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-16-6-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 1.1920928955078125e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1870.5207500024699,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 148.39041704544798,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 18.90499994624406,
"inferenceMs": 71.56574999680743,
"imagesPerSecond": 13.973164538129069,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77365248,
"modelInferenceMs": 15.908083005342633,
"totalMs": 1942.0864999992773,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 504528896,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996474027633667,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965187039226294,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-16-7-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 1.1920928955078125e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1870.5207500024699,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 148.39041704544798,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 15.3202919755131,
"inferenceMs": 67.3425419954583,
"imagesPerSecond": 14.84945430286017,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77365248,
"modelInferenceMs": 12.414041033480316,
"totalMs": 1937.8632919979282,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 510230528,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996474027633667,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965187039226294,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-16-8-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 1.1920928955078125e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1870.5207500024699,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 148.39041704544798,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 19.675666000694036,
"inferenceMs": 69.69441601540893,
"imagesPerSecond": 14.348351807394543,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77365248,
"modelInferenceMs": 11.74454198917374,
"totalMs": 1940.2151660178788,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 515932160,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5996474027633667,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7965187039226294,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-16-9-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 1.1920928955078125e-7,
"threshold": 0.0001,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1870.5207500024699,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float32",
"initializationWarmupMs": 148.39041704544798,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 17.114082991611212,
"inferenceMs": 68.89462500112131,
"imagesPerSecond": 14.51492043078432,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 77365248,
"modelInferenceMs": 13.799749955069274,
"totalMs": 1939.4153750035912,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp32",
"actualPrecision": "fp32",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 521633792,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5947906374931335,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7952595055103302,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-17-0-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.004856884479522705,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1920.7933750003576,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 170.77391705242917,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 29.03645799960941,
"inferenceMs": 87.06041699042544,
"imagesPerSecond": 11.486276250089361,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 11.64262497331947,
"totalMs": 2007.853791990783,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 482607104,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5947906374931335,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7952595055103302,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-17-1-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.004856884479522705,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1920.7933750003576,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 170.77391705242917,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 17.653167014941573,
"inferenceMs": 69.9372089584358,
"imagesPerSecond": 14.298540289108582,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 13.669500011019409,
"totalMs": 1990.7305839587934,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 488374272,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.6771113276481628,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.8851615022867918,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-17-2-mirrored-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-2-mirrored-dog",
"sameClassSet": true,
"maximumScoreError": 0.0020411089062690735,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1920.7933750003576,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 170.77391705242917,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 16.470916976686567,
"inferenceMs": 68.6940000159666,
"imagesPerSecond": 14.557312134503295,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 13.152749976143241,
"totalMs": 1989.4873750163242,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 494075904,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.17434847354888916,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7355400174856186,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-17-3-small-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-3-small-dog",
"sameClassSet": true,
"maximumScoreError": 0.00694487988948822,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1920.7933750003576,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 170.77391705242917,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 9.585499996319413,
"inferenceMs": 11.923583981115371,
"imagesPerSecond": 83.86740107536498,
"inputWidth": 112,
"inputHeight": 92,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 7.245082990266383,
"totalMs": 1932.716958981473,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 494075904,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"expectedLabel": null,
"minimumExpectedScore": null,
"probabilitySum": 0.9999999157334969,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-17-4-blank",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-4-blank",
"sameClassSet": true,
"maximumScoreError": 0.00015655066817998886,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1920.7933750003576,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 170.77391705242917,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 14.240499993320554,
"inferenceMs": 15.869208029471338,
"imagesPerSecond": 63.01511695749783,
"inputWidth": 224,
"inputHeight": 224,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 8.712666982319206,
"totalMs": 1936.662583029829,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 494944256,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5947906374931335,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7952595055103302,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-17-5-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.004856884479522705,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1920.7933750003576,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 170.77391705242917,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 17.060625017620623,
"inferenceMs": 71.76149997394532,
"imagesPerSecond": 13.935048742892404,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 12.856624962296337,
"totalMs": 1992.554874974303,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 500678656,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5947906374931335,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7952595055103302,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-17-6-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.004856884479522705,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1920.7933750003576,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 170.77391705242917,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 14.57508298335597,
"inferenceMs": 67.61341699166223,
"imagesPerSecond": 14.789963952321996,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 10.348334035370499,
"totalMs": 1988.4067919920199,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 506380288,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5947906374931335,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7952595055103302,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-17-7-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.004856884479522705,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1920.7933750003576,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 170.77391705242917,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 19.133708032313734,
"inferenceMs": 70.63133298652247,
"imagesPerSecond": 14.15802247694823,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 16.286708996631205,
"totalMs": 1991.42470798688,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 512081920,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5947906374931335,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7952595055103302,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-17-8-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.004856884479522705,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1920.7933750003576,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 170.77391705242917,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 15.490707999560982,
"inferenceMs": 65.99987501977012,
"imagesPerSecond": 15.151543843082312,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 12.470125046093017,
"totalMs": 1986.7932500201277,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 517783552,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5947906374931335,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7952595055103302,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-17-9-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.004856884479522705,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1920.7933750003576,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 170.77391705242917,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.890542031731457,
"inferenceMs": 64.66254201950505,
"imagesPerSecond": 15.464903926887938,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 10.734125040471554,
"totalMs": 1985.4559170198627,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 523485184,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5947906374931335,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7952595055103302,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-18-0-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.004856884479522705,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1947.2181249875575,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 156.43379098037258,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 27.556917048059404,
"inferenceMs": 81.71591605059803,
"imagesPerSecond": 12.237518078862456,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 11.867459048517048,
"totalMs": 2028.9340410381556,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 482410496,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5947906374931335,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7952595055103302,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-18-1-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.004856884479522705,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1947.2181249875575,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 156.43379098037258,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 16.71625004382804,
"inferenceMs": 67.13833298999816,
"imagesPerSecond": 14.894620635710059,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 12.281499977689236,
"totalMs": 2014.3564579775557,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 488161280,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.6771113276481628,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.8851615022867918,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-18-2-mirrored-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-2-mirrored-dog",
"sameClassSet": true,
"maximumScoreError": 0.0020411089062690735,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1947.2181249875575,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 156.43379098037258,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 14.733334013726562,
"inferenceMs": 67.31574999867007,
"imagesPerSecond": 14.855364458091257,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 11.61608297843486,
"totalMs": 2014.5338749862276,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 493862912,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.17434847354888916,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7355400174856186,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-18-3-small-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-3-small-dog",
"sameClassSet": true,
"maximumScoreError": 0.00694487988948822,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1947.2181249875575,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 156.43379098037258,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 10.68995799869299,
"inferenceMs": 12.960666033905,
"imagesPerSecond": 77.15652863703207,
"inputWidth": 112,
"inputHeight": 92,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 8.686834014952183,
"totalMs": 1960.1787910214625,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 493879296,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"expectedLabel": null,
"minimumExpectedScore": null,
"probabilitySum": 0.9999999157334969,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-18-4-blank",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-4-blank",
"sameClassSet": true,
"maximumScoreError": 0.00015655066817998886,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1947.2181249875575,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 156.43379098037258,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 14.12404200527817,
"inferenceMs": 15.568999981041998,
"imagesPerSecond": 64.23020111874085,
"inputWidth": 224,
"inputHeight": 224,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 7.345040969084948,
"totalMs": 1962.7871249685995,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 494845952,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5947906374931335,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7952595055103302,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-18-5-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.004856884479522705,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1947.2181249875575,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 156.43379098037258,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 15.411709027830511,
"inferenceMs": 65.73279097210616,
"imagesPerSecond": 15.213107266727073,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 12.365541013423353,
"totalMs": 2012.9509159596637,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 500613120,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5947906374931335,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7952595055103302,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-18-6-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.004856884479522705,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1947.2181249875575,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 156.43379098037258,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 15.225042006932199,
"inferenceMs": 63.75216704327613,
"imagesPerSecond": 15.685741306976778,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 10.941000015009195,
"totalMs": 2010.9702920308337,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 506314752,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5947906374931335,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7952595055103302,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-18-7-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.004856884479522705,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1947.2181249875575,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 156.43379098037258,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 15.180249989498407,
"inferenceMs": 64.1694160294719,
"imagesPerSecond": 15.583747864258532,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 12.179125042166561,
"totalMs": 2011.3875410170294,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 512016384,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5947906374931335,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7952595055103302,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-18-8-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.004856884479522705,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1947.2181249875575,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 156.43379098037258,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 14.229416032321751,
"inferenceMs": 63.40629199985415,
"imagesPerSecond": 15.77130547237016,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 11.313540977425873,
"totalMs": 2010.6244169874117,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 517718016,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5947906374931335,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7952595055103302,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-18-9-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.004856884479522705,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1947.2181249875575,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 156.43379098037258,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 14.512875000946224,
"inferenceMs": 63.517624977976084,
"imagesPerSecond": 15.743661705656297,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 11.440625006798655,
"totalMs": 2010.7357499655336,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 523419648,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5947906374931335,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7952595055103302,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-19-0-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.004856884479522705,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1868.5227090027183,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 152.38958405097947,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 26.953916996717453,
"inferenceMs": 81.77645795512944,
"imagesPerSecond": 12.228458226311266,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 11.797334009315819,
"totalMs": 1950.2991669578478,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 482672640,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5947906374931335,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7952595055103302,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-19-1-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.004856884479522705,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1868.5227090027183,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 152.38958405097947,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 14.887750032357872,
"inferenceMs": 66.82537501910701,
"imagesPerSecond": 14.964375429454387,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 11.742915958166122,
"totalMs": 1935.3480840218253,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 488488960,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.6771113276481628,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.8851615022867918,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-19-2-mirrored-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-2-mirrored-dog",
"sameClassSet": true,
"maximumScoreError": 0.0020411089062690735,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1868.5227090027183,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 152.38958405097947,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 18.163125030696392,
"inferenceMs": 69.09437500871718,
"imagesPerSecond": 14.472958180370496,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 11.41558401286602,
"totalMs": 1937.6170840114355,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 494190592,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.17434847354888916,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7355400174856186,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-19-3-small-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-3-small-dog",
"sameClassSet": true,
"maximumScoreError": 0.00694487988948822,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1868.5227090027183,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 152.38958405097947,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 9.43645799998194,
"inferenceMs": 11.658833012916148,
"imagesPerSecond": 85.77187775930557,
"inputWidth": 112,
"inputHeight": 92,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 7.52349995309487,
"totalMs": 1880.1815420156345,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 494190592,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"expectedLabel": null,
"minimumExpectedScore": null,
"probabilitySum": 0.9999999157334969,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-19-4-blank",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-4-blank",
"sameClassSet": true,
"maximumScoreError": 0.00015655066817998886,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1868.5227090027183,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 152.38958405097947,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.255583995487541,
"inferenceMs": 14.75645904429257,
"imagesPerSecond": 67.76693494004411,
"inputWidth": 224,
"inputHeight": 224,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 7.495541998650879,
"totalMs": 1883.279168047011,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 494993408,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5947906374931335,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7952595055103302,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-19-5-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.004856884479522705,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1868.5227090027183,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 152.38958405097947,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 14.428667025640607,
"inferenceMs": 65.4175000381656,
"imagesPerSecond": 15.28642946331768,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 11.462374997790903,
"totalMs": 1933.940209040884,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 500744192,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5947906374931335,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7952595055103302,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-19-6-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.004856884479522705,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1868.5227090027183,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 152.38958405097947,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 19.650625006761402,
"inferenceMs": 71.12195901572704,
"imagesPerSecond": 14.060355111687407,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 10.523000033572316,
"totalMs": 1939.6446680184454,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 506445824,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5947906374931335,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7952595055103302,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-19-7-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.004856884479522705,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1868.5227090027183,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 152.38958405097947,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 16.29204099299386,
"inferenceMs": 66.57016603276134,
"imagesPerSecond": 15.021744117445458,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 12.216042028740048,
"totalMs": 1935.0928750354797,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 512147456,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5947906374931335,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7952595055103302,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-19-8-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.004856884479522705,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1868.5227090027183,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 152.38958405097947,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 14.532166998833418,
"inferenceMs": 64.74391696974635,
"imagesPerSecond": 15.445466490192148,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 11.621250014286488,
"totalMs": 1933.2666259724647,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 517849088,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.5947906374931335,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7952595055103302,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-19-9-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.004856884479522705,
"threshold": 0.02,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1868.5227090027183,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.float16",
"initializationWarmupMs": 152.38958405097947,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.645624974742532,
"inferenceMs": 63.74758301535621,
"imagesPerSecond": 15.686869253679925,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 10.3615410043858,
"totalMs": 1932.2702920180745,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "fp16",
"actualPrecision": "fp16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 523550720,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.5648589134216309,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7866855692118406,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-20-0-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.03478860855102539,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1851.4650420402177,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 151.59141598269343,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 27.94224995886907,
"inferenceMs": 81.9772919639945,
"imagesPerSecond": 12.198500048516033,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 12.125792040023953,
"totalMs": 1933.4423340042122,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 484163584,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.5648589134216309,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7866855692118406,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-20-1-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.03478860855102539,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1851.4650420402177,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 151.59141598269343,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 14.37824999447912,
"inferenceMs": 66.76937500014901,
"imagesPerSecond": 14.976926173081122,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 11.411166982725263,
"totalMs": 1918.2344170403667,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 489930752,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.6382553577423096,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.8794247079640627,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-20-2-mirrored-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-2-mirrored-dog",
"sameClassSet": true,
"maximumScoreError": 0.03999733924865723,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1851.4650420402177,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 151.59141598269343,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 19.400999997742474,
"inferenceMs": 69.69770899740979,
"imagesPerSecond": 14.347673896098414,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 13.745417003519833,
"totalMs": 1921.1627510376275,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 495632384,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.14329007267951965,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7653137668967247,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-20-3-small-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-3-small-dog",
"sameClassSet": true,
"maximumScoreError": 0.08517764508724213,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1851.4650420402177,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 151.59141598269343,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 9.601083991583437,
"inferenceMs": 11.728083016350865,
"imagesPerSecond": 85.26542646448158,
"inputWidth": 112,
"inputHeight": 92,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 7.637708971742541,
"totalMs": 1863.1931250565685,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 495632384,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"expectedLabel": null,
"minimumExpectedScore": null,
"probabilitySum": 1.0000000203799573,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-20-4-blank",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-4-blank",
"sameClassSet": true,
"maximumScoreError": 0.0020814710296690464,
"threshold": 0.03,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1851.4650420402177,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 151.59141598269343,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 12.63408304657787,
"inferenceMs": 14.106749964412302,
"imagesPerSecond": 70.88805022579564,
"inputWidth": 224,
"inputHeight": 224,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 6.863625021651387,
"totalMs": 1865.57179200463,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 496500736,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.5648589134216309,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7866855692118406,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-20-5-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.03478860855102539,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1851.4650420402177,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 151.59141598269343,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 15.30591701157391,
"inferenceMs": 65.33841695636511,
"imagesPerSecond": 15.304931563735758,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 12.365374946966767,
"totalMs": 1916.8034589965828,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 502267904,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.5648589134216309,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7866855692118406,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-20-6-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.03478860855102539,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1851.4650420402177,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 151.59141598269343,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.259208004456013,
"inferenceMs": 65.4699580045417,
"imagesPerSecond": 15.274181173762619,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 10.218875017017126,
"totalMs": 1916.9350000447594,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 507969536,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.5648589134216309,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7866855692118406,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-20-7-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.03478860855102539,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1851.4650420402177,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 151.59141598269343,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 18.12916697235778,
"inferenceMs": 70.66575001226738,
"imagesPerSecond": 14.15112695791671,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 14.995375007856637,
"totalMs": 1922.130792052485,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 513671168,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.5648589134216309,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7866855692118406,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-20-8-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.03478860855102539,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1851.4650420402177,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 151.59141598269343,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 16.11387502634898,
"inferenceMs": 66.49395800195634,
"imagesPerSecond": 15.038960381491783,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 13.0245839827694,
"totalMs": 1917.959000042174,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 519372800,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.5648589134216309,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7866855692118406,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-20-9-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.03478860855102539,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1851.4650420402177,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 151.59141598269343,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.181333022657782,
"inferenceMs": 65.3403329779394,
"imagesPerSecond": 15.304482766220767,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 10.27045800583437,
"totalMs": 1916.805375018157,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 525074432,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.5648589134216309,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7866855692118406,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-21-0-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.03478860855102539,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1924.6139170136303,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 170.77341699041426,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 29.51391600072384,
"inferenceMs": 86.5593749913387,
"imagesPerSecond": 11.552763638832442,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 12.592499959282577,
"totalMs": 2011.173292004969,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 482983936,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.5648589134216309,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7866855692118406,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-21-1-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.03478860855102539,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1924.6139170136303,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 170.77341699041426,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 15.305083012208343,
"inferenceMs": 66.90949999028817,
"imagesPerSecond": 14.945560796974252,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 11.964583012741059,
"totalMs": 1991.5234170039184,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 488734720,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.6382553577423096,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.8794247079640627,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-21-2-mirrored-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-2-mirrored-dog",
"sameClassSet": true,
"maximumScoreError": 0.03999733924865723,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1924.6139170136303,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 170.77341699041426,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 18.88287498150021,
"inferenceMs": 72.13416596641764,
"imagesPerSecond": 13.863056245296496,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 10.52841596538201,
"totalMs": 1996.748082980048,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 494436352,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.14329007267951965,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7653137668967247,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-21-3-small-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-3-small-dog",
"sameClassSet": true,
"maximumScoreError": 0.08517764508724213,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1924.6139170136303,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 170.77341699041426,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 9.65924997581169,
"inferenceMs": 11.91633299458772,
"imagesPerSecond": 83.91843367034055,
"inputWidth": 112,
"inputHeight": 92,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 7.575541036203504,
"totalMs": 1936.530250008218,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 494436352,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"expectedLabel": null,
"minimumExpectedScore": null,
"probabilitySum": 1.0000000203799573,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-21-4-blank",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-4-blank",
"sameClassSet": true,
"maximumScoreError": 0.0020814710296690464,
"threshold": 0.03,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1924.6139170136303,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 170.77341699041426,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.955624948721379,
"inferenceMs": 15.513458987697959,
"imagesPerSecond": 64.46015687365349,
"inputWidth": 224,
"inputHeight": 224,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 7.4517079629004,
"totalMs": 1940.1273760013282,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 495288320,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.5648589134216309,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7866855692118406,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-21-5-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.03478860855102539,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1924.6139170136303,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 170.77341699041426,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.750542013440281,
"inferenceMs": 65.77950000064448,
"imagesPerSecond": 15.202304669238934,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 10.688792041037232,
"totalMs": 1990.3934170142747,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 501055488,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.5648589134216309,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7866855692118406,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-21-6-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.03478860855102539,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1924.6139170136303,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 170.77341699041426,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 14.925291005056351,
"inferenceMs": 67.48504197457805,
"imagesPerSecond": 14.818098511025672,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 11.662958015222102,
"totalMs": 1992.0989589882083,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 506757120,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.5648589134216309,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7866855692118406,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-21-7-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.03478860855102539,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1924.6139170136303,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 170.77341699041426,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 16.55412500258535,
"inferenceMs": 69.26679197931662,
"imagesPerSecond": 14.436932495713164,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 13.399042014498264,
"totalMs": 1993.880708992947,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 512458752,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.5648589134216309,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7866855692118406,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-21-8-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.03478860855102539,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1924.6139170136303,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 170.77341699041426,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.956790964584798,
"inferenceMs": 64.74154198076576,
"imagesPerSecond": 15.446033094131318,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 10.921458015218377,
"totalMs": 1989.355458994396,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 518160384,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.5648589134216309,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7866855692118406,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-21-9-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.03478860855102539,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1924.6139170136303,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 170.77341699041426,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 16.6953339939937,
"inferenceMs": 67.16970796696842,
"imagesPerSecond": 14.887663356996626,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 10.360583022702485,
"totalMs": 1991.7836249805987,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 523862016,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.5648589134216309,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7866855692118406,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-22-0-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.03478860855102539,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1842.0752920210361,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 151.7079170444049,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 28.24341697851196,
"inferenceMs": 82.49775000149384,
"imagesPerSecond": 12.121542708521048,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 12.400457984767854,
"totalMs": 1924.57304202253,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 482017280,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.5648589134216309,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7866855692118406,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-22-1-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.03478860855102539,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1842.0752920210361,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 151.7079170444049,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 15.12404199456796,
"inferenceMs": 66.73345802119002,
"imagesPerSecond": 14.984986986325028,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 12.163666950073093,
"totalMs": 1908.8087500422262,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 487800832,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.6382553577423096,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.8794247079640627,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-22-2-mirrored-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-2-mirrored-dog",
"sameClassSet": true,
"maximumScoreError": 0.03999733924865723,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1842.0752920210361,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 151.7079170444049,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.805750000756234,
"inferenceMs": 64.16424998315051,
"imagesPerSecond": 15.585002556136779,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 10.66379202529788,
"totalMs": 1906.2395420041867,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 493502464,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.14329007267951965,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7653137668967247,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-22-3-small-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-3-small-dog",
"sameClassSet": true,
"maximumScoreError": 0.08517764508724213,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1842.0752920210361,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 151.7079170444049,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 11.459375033155084,
"inferenceMs": 13.610999973025173,
"imagesPerSecond": 73.4699876557079,
"inputWidth": 112,
"inputHeight": 92,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75202560,
"modelInferenceMs": 8.321375004015863,
"totalMs": 1855.6862919940613,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 493518848,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"expectedLabel": null,
"minimumExpectedScore": null,
"probabilitySum": 1.0000000203799573,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-22-4-blank",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-4-blank",
"sameClassSet": true,
"maximumScoreError": 0.0020814710296690464,
"threshold": 0.03,
"passed": true
}
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1842.0752920210361,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 151.7079170444049,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 13.148250000085682,
"inferenceMs": 14.579082955606282,
"imagesPerSecond": 68.5914198475328,
"inputWidth": 224,
"inputHeight": 224,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 7.516915968153626,
"totalMs": 1856.6543749766424,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 494387200,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "excellent",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.5648589134216309,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7866855692118406,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-22-5-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.03478860855102539,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1842.0752920210361,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 151.7079170444049,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 16.3794580148533,
"inferenceMs": 67.1726250438951,
"imagesPerSecond": 14.887016836792265,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 12.939417036250234,
"totalMs": 1909.2479170649312,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 504905728,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.5648589134216309,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7866855692118406,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-22-6-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.03478860855102539,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1842.0752920210361,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 151.7079170444049,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 15.840417006984353,
"inferenceMs": 68.14783299341798,
"imagesPerSecond": 14.673980904082224,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 11.946292011998594,
"totalMs": 1910.2231250144541,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 510607360,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.5648589134216309,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7866855692118406,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-22-7-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.03478860855102539,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1842.0752920210361,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 151.7079170444049,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 16.14262501243502,
"inferenceMs": 70.38599997758865,
"imagesPerSecond": 14.207370788486436,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 12.988208036404103,
"totalMs": 1912.4612919986248,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 516308992,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.5648589134216309,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7866855692118406,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-22-8-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.03478860855102539,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1842.0752920210361,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 151.7079170444049,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 15.983082994353026,
"inferenceMs": 67.42341700010002,
"imagesPerSecond": 14.831642246765936,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 13.016375014558434,
"totalMs": 1909.4987090211362,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 522010624,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"score": 0.5648589134216309,
"expectedLabel": "Labrador retriever",
"minimumExpectedScore": 0.05,
"probabilitySum": 0.7866855692118406,
"speedPassed": true,
"method": "Frozen known-object top5/ranking; complete normalized1000-class blank negative",
"referenceParity": {
"caseId": "mobilenet-v2-imagenet-v1-native-22-9-dog",
"referenceCaseId": "mobilenet-v2-imagenet-v1-native-5-1-dog",
"sameClassSet": true,
"maximumScoreError": 0.03478860855102539,
"threshold": 0.03,
"passed": false
},
"reason": "Frozen cross-backend confidence-score consistency threshold failed"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 1842.0752920210361,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": "mps:0",
"parameterDtype": "torch.bfloat16",
"initializationWarmupMs": 151.7079170444049,
"initializationWarmupScope": "Synthetic224x224 GPU graph compilation inside measured load; not quality evidence. CPU unchanged.",
"mappedTensors": 314,
"accelerationDetails": "Built-in HF MobileNetV2 with exact original TorchVision state mapping; parameters and classifier output on selected device, CPU encoded-image decode/orientation/resize/crop/normalize and label assembly.",
"durationMs": 19.864666974171996,
"inferenceMs": 71.37083302950487,
"imagesPerSecond": 14.011325881352649,
"inputWidth": 700,
"inputHeight": 577,
"modelInputShape": [
1,
3,
224,
224
],
"driverAllocatedMemoryBytes": 75218944,
"modelInferenceMs": 13.550625008065253,
"totalMs": 1913.446125050541,
"requestedDevice": "mps",
"actualDevice": "mps",
"requestedPrecision": "bf16",
"actualPrecision": "bf16",
"fallback": false,
"cpuThreads": 1,
"residentMemoryBytes": 527712256,
"memoryMeasurementScope": "After-call RSS and Metal driver allocation, not peak/minimum",
"grade": "good",
"helperSourceSha256": "5507850091c27d5c959ac41dafd09afc85e8d0c97a7a75352ed34b59b4b0b7d5",
"artifactRevision": "sha256:b03531047ffacf1e2488318dcd2aba1126cde36e3bfe1aa5cb07700aeeee9889"
}
}
]Release validation still required
- Sample publication, complete dependency backup and forced model/runtime source recovery excluded/pending