PaddleOCR-VL 1.5
Read document images or rasterized PDF pages with OCR, table, formula, chart, seal and text-spotting modes. Returns complete native task text and per-page metadata with incremental streaming.
Desktop app
Run PaddleOCR-VL 1.5 on your computer
Download the desktop app to load this model, provide input and view results locally.
Public desktop downloads are coming soon.Device requirements
If your device falls short
MemoryAdvisory only: close other apps, reduce image/tile/token/batch settings or choose a smaller verified model/precision. CPU can need more RAM than GPU.
GPUIf the selected GPU runtime fails, existing CPU fallback may recover when enabled, but CPU has its own memory/runtime needs.
PlatformUse a compatible OS/architecture/runtime or another model. These estimates do not add hardware detection or block runs.
Benchmark evidence
Saved desktop measurements apply to the recorded hardware and runtime. Fallback cases report the device that actually ran; they do not prove GPU support on the requested device.
PaddleOCR-VL 1.5 Desktop
65 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 526 ms / 0 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 590 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 611 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 641 ms / 0 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 648 ms / 0 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 694 ms / 0 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 720 ms / 0 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 990 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 1082 ms / 0 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 1777 ms / 0 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2218 ms / 0 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2354 ms / 0 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2375 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2529 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2530 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2599 ms / 0 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2610 ms / 0 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2610 ms / 0 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2637 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2641 ms / 0 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2642 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2666 ms / 0 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2675 ms / 0 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2714 ms / 625 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2759 ms / 688 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2765 ms / 880 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2811 ms / 668 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2818 ms / 0 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2833 ms / 0 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2883 ms / 0 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2945 ms / 0 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2960 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 3041 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 3046 ms / 671 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 3092 ms / 0 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 3186 ms / 756 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 3211 ms / 610 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 3236 ms / 0 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 3246 ms / 0 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 3325 ms / 0 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 3402 ms / 629 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 3465 ms / 749 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 6723 ms / 0 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 7928 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 8402 ms / 0 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 10558 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 12872 ms / 0 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 13038 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 1743 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 1806 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 2605 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 4294 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 5725 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 5770 ms / 391 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 5805 ms / 373 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 5901 ms / 389 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 5903 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 5978 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 6297 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 6307 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 6407 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 6446 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 11647 ms / 389 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 15207 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 18428 ms / 0 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-07T09:11:26.053460+00:00
- Environment
- { "hardware": "Apple M3 Max", "os": "macos", "osVersion": "27.2", "arch": "aarch64", "runtime": "Transformers5.22.1/Torch2.10.0/PDFium5.14.0", "runtimeProvisioning": "Private probe venv from app-owned CPython; managed public/packaged provisioning verified separately", "concurrentUploads": "Persistent background mirror may run during later timed cases after passing CPU registration; no agent builds overlap matrix." }
- Runtime
- transformers 5.22.1
- Artifact revision
- 1e5216083435c7851caad5d691ef55aab81b5949
- Fixture
{ "file": "model-samples/desktop/docling/receipt.png", "size": 25773, "sha1": "05f77f558dde3e17f3541910abeecde08c193cd5", "sha256": "cc0123c5e9e48de091ce0d3462d267b2485295096206353c9bf4beeec594ca0e", "url": "https://assets.localai.tools/samples/v1/files/cc0123c5e9e48de091ce0d3462d267b2485295096206353c9bf4beeec594ca0e/receipt.png" }- Rerun when
- Exact original checkpoint revision; Pinned framework/helper/decoder/preprocessing/task prompts; Fixture/checksum/quality requirements; Relevant OS/backend behavior
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 388.74454199685715,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 5916.075708002609,
"inferenceMs": 5900.729624998348,
"pagesPerSecond": 0.16947056780292316,
"tokensPerSecond": 8.642998957949082,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 6304.820249999466,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 4736090112,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 5983.392750000348,
"inferenceMs": 5977.7974159951555,
"pagesPerSecond": 0.16728569578558136,
"tokensPerSecond": 7.5278563103511615,
"outputTokens": 45,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 5983.392750000348,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 4741169152,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 15221.303999998781,
"inferenceMs": 15207.213624998985,
"pagesPerSecond": 0.13151653217471873,
"tokensPerSecond": 4.537320360027796,
"outputTokens": 69,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "application/pdf",
"activationPlacement": {
"visionOutput": "cpu"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 15221.303999998781,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 5145673728,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 0,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 0,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 4300.2383330021985,
"inferenceMs": 4294.157208001707,
"pagesPerSecond": 0.23287456689676056,
"tokensPerSecond": 0.23287456689676056,
"outputTokens": 1,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 4300.2383330021985,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 5146525696,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 6302.357125001436,
"inferenceMs": 6296.709041998838,
"pagesPerSecond": 0.158813118619589,
"tokensPerSecond": 7.464216575120682,
"outputTokens": 47,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"task": "table",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 6302.357125001436,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 5146836992,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 1808.2702079991577,
"inferenceMs": 1805.67970899574,
"pagesPerSecond": 0.5538080729478693,
"tokensPerSecond": 9.414737240113777,
"outputTokens": 17,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"task": "formula",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 1808.2702079991577,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 5194596352,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2607.2904169996036,
"inferenceMs": 2604.7106250043726,
"pagesPerSecond": 0.3839198068301047,
"tokensPerSecond": 9.981914977582722,
"outputTokens": 26,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"task": "chart",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 2607.2904169996036,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 5194596352,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 1745.8881249986007,
"inferenceMs": 1742.928874999052,
"pagesPerSecond": 0.5737468776518744,
"tokensPerSecond": 2.2949875106074975,
"outputTokens": 4,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"task": "seal",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 1745.8881249986007,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 5194596352,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 18465.141041997413,
"inferenceMs": 18427.70337499678,
"pagesPerSecond": 0.054266122025646873,
"tokensPerSecond": 5.318079958513393,
"outputTokens": 98,
"maxNewTokens": 768,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"task": "spotting",
"spottingCoordinateSpace": "normalized0-1000 coordinates relative to image; small images doubled proportionally",
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 18465.141041997413,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 5488771072,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 6312.492416000168,
"inferenceMs": 6306.541874997492,
"pagesPerSecond": 0.15856550544198164,
"tokensPerSecond": 8.086840777541063,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 6312.492416000168,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 5488771072,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 6452.339499999653,
"inferenceMs": 6446.487665998575,
"pagesPerSecond": 0.15512323172111395,
"tokensPerSecond": 7.911284817776812,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 6452.339499999653,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 5488771072,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 5908.255583999562,
"inferenceMs": 5902.500208001584,
"pagesPerSecond": 0.16941973142912792,
"tokensPerSecond": 8.640406302885523,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 5908.255583999562,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 5488771072,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 5730.931500002043,
"inferenceMs": 5725.214416997915,
"pagesPerSecond": 0.1746659473627823,
"tokensPerSecond": 8.907963315501897,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 5730.931500002043,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 5488771072,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 6412.397000000055,
"inferenceMs": 6406.776874995558,
"pagesPerSecond": 0.1560847239589085,
"tokensPerSecond": 7.960320921904334,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 6412.397000000055,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 5488771072,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 390.7074590024422,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 5786.331542003609,
"inferenceMs": 5770.4580839999835,
"pagesPerSecond": 0.17329646718563754,
"tokensPerSecond": 8.838119826467514,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 6177.039001006051,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 4735254528,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 373.26424999628216,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 5819.0070830023615,
"inferenceMs": 5804.62166700454,
"pagesPerSecond": 0.17227651643247363,
"tokensPerSecond": 8.786102338056155,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 6192.271332998644,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 4727242752,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 389.1543339996133,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 11669.657958998869,
"inferenceMs": 11647.211457995581,
"pagesPerSecond": 0.08585746069833047,
"tokensPerSecond": 4.378730495614854,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 12058.812292998482,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 4730208256,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 879.9164999945788,
"placement": {
"mps:0/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2784.2296670060023,
"inferenceMs": 2764.9755840029684,
"pagesPerSecond": 0.3616668464581011,
"tokensPerSecond": 18.445009169363157,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 9401565184,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 3664.146167000581,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 4090544128,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2385.429665999254,
"inferenceMs": 2375.402290999773,
"pagesPerSecond": 0.4209813233694888,
"tokensPerSecond": 18.944159551626996,
"outputTokens": 45,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 9409953792,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 2385.429665999254,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 4090544128,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 6744.216416998825,
"inferenceMs": 6722.636207996402,
"pagesPerSecond": 0.2975023395764078,
"tokensPerSecond": 10.263830715386069,
"outputTokens": 69,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "application/pdf",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 13973815296,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 6744.216416998825,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 4090544128,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 0,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 0,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 1787.2494159964845,
"inferenceMs": 1777.4316669965629,
"pagesPerSecond": 0.5626095329390425,
"tokensPerSecond": 0.5626095329390425,
"outputTokens": 1,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 13973815296,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 1787.2494159964845,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 4090544128,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2363.0493329983437,
"inferenceMs": 2353.6475410001003,
"pagesPerSecond": 0.42487245119763556,
"tokensPerSecond": 19.96900520628887,
"outputTokens": 47,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "table",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 13973815296,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 2363.0493329983437,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 4090544128,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 725.1362499955576,
"inferenceMs": 720.0769579940243,
"pagesPerSecond": 1.3887404518341757,
"tokensPerSecond": 23.608587681180985,
"outputTokens": 17,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "formula",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 14313406464,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 725.1362499955576,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 4090544128,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 698.7486660000286,
"inferenceMs": 693.624000006821,
"pagesPerSecond": 1.4417032859159518,
"tokensPerSecond": 37.48428543381475,
"outputTokens": 26,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "chart",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 14313406464,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 698.7486660000286,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 4090544128,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 530.855666998832,
"inferenceMs": 525.6155830065836,
"pagesPerSecond": 1.9025311127190734,
"tokensPerSecond": 7.6101244508762935,
"outputTokens": 4,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "seal",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 14333591552,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 530.855666998832,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 4090544128,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 10602.692915999796,
"inferenceMs": 10558.232249997673,
"pagesPerSecond": 0.09471282467765571,
"tokensPerSecond": 9.28185681841026,
"outputTokens": 98,
"maxNewTokens": 768,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "spotting",
"spottingCoordinateSpace": "normalized0-1000 coordinates relative to image; small images doubled proportionally",
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 25573720064,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 10602.692915999796,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 4090544128,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2620.6909169995924,
"inferenceMs": 2609.924457996385,
"pagesPerSecond": 0.3831528521586754,
"tokensPerSecond": 19.540795460092447,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 26389512192,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 2620.6909169995924,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 4090544128,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2650.542082999891,
"inferenceMs": 2636.8180830031633,
"pagesPerSecond": 0.3792449719781447,
"tokensPerSecond": 19.341493570885383,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 26112688128,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 2650.542082999891,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 4090544128,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2891.1976250019507,
"inferenceMs": 2882.719458997599,
"pagesPerSecond": 0.34689466464687746,
"tokensPerSecond": 17.69162789699075,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 26112688128,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 2891.1976250019507,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 4090544128,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2651.3857089958037,
"inferenceMs": 2642.307999994955,
"pagesPerSecond": 0.3784570156098037,
"tokensPerSecond": 19.30130779609999,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 26112688128,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 2651.3857089958037,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 4090544128,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2841.873958001088,
"inferenceMs": 2833.0375000005006,
"pagesPerSecond": 0.3529780315297003,
"tokensPerSecond": 18.001879608014715,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 26112688128,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 2841.873958001088,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 4090544128,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 748.9979999954812,
"placement": {
"mps:0/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 3484.226041000511,
"inferenceMs": 3464.8762500000885,
"pagesPerSecond": 0.2886105961215713,
"tokensPerSecond": 14.719140402200136,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 9401565184,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 4233.224040995992,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 4089724928,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 755.9577080028248,
"placement": {
"mps:0/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 3205.4879999996047,
"inferenceMs": 3185.6034170050407,
"pagesPerSecond": 0.3139122700151278,
"tokensPerSecond": 16.009525770771518,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 9401565184,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 3961.4457080024295,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 3692920832,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 624.5338330045342,
"placement": {
"mps:0/torch.float16": 608
},
"parameterDtype": "torch.float16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2746.5641669987235,
"inferenceMs": 2713.725000001432,
"pagesPerSecond": 0.36849717639019147,
"tokensPerSecond": 18.793355995899766,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 6220185600,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp16",
"cpuThreads": 4,
"totalMs": 3371.0980000032578,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 2278178816,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float16": 608
},
"parameterDtype": "torch.float16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2620.531625005242,
"inferenceMs": 2609.8952920001466,
"pagesPerSecond": 0.3831571339529218,
"tokensPerSecond": 17.24207102788148,
"outputTokens": 45,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 6220185600,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp16",
"cpuThreads": 4,
"totalMs": 2620.531625005242,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 2278178816,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float16": 608
},
"parameterDtype": "torch.float16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 7956.767417002993,
"inferenceMs": 7928.077251002833,
"pagesPerSecond": 0.2522679757878264,
"tokensPerSecond": 8.70324516468001,
"outputTokens": 69,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "application/pdf",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 11098619904,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp16",
"cpuThreads": 4,
"totalMs": 7956.767417002993,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 2278178816,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 0,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 0,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float16": 608
},
"parameterDtype": "torch.float16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2226.61325000081,
"inferenceMs": 2217.5999589962885,
"pagesPerSecond": 0.4509379592758523,
"tokensPerSecond": 0.4509379592758523,
"outputTokens": 1,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 11098619904,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp16",
"cpuThreads": 4,
"totalMs": 2226.61325000081,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 2278178816,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float16": 608
},
"parameterDtype": "torch.float16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2687.4211250033113,
"inferenceMs": 2675.1999170010095,
"pagesPerSecond": 0.3738038393485875,
"tokensPerSecond": 17.568780449383613,
"outputTokens": 47,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "table",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 11098619904,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp16",
"cpuThreads": 4,
"totalMs": 2687.4211250033113,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 2278178816,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float16": 608
},
"parameterDtype": "torch.float16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 997.2469170024851,
"inferenceMs": 989.9464170011925,
"pagesPerSecond": 1.010155684010921,
"tokensPerSecond": 17.172646628185657,
"outputTokens": 17,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "formula",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 11438211072,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp16",
"cpuThreads": 4,
"totalMs": 997.2469170024851,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 2278178816,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float16": 608
},
"parameterDtype": "torch.float16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 652.645082998788,
"inferenceMs": 647.919416995137,
"pagesPerSecond": 1.5434018085732188,
"tokensPerSecond": 40.12844702290369,
"outputTokens": 26,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "chart",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 11438211072,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp16",
"cpuThreads": 4,
"totalMs": 652.645082998788,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 2278178816,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float16": 608
},
"parameterDtype": "torch.float16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 595.6199590000324,
"inferenceMs": 589.6281659952365,
"pagesPerSecond": 1.695984109429533,
"tokensPerSecond": 6.783936437718132,
"outputTokens": 4,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "seal",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 11458396160,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp16",
"cpuThreads": 4,
"totalMs": 595.6199590000324,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 2278178816,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float16": 608
},
"parameterDtype": "torch.float16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 13088.572000000568,
"inferenceMs": 13038.214624997636,
"pagesPerSecond": 0.07669761763874794,
"tokensPerSecond": 7.516366528597298,
"outputTokens": 98,
"maxNewTokens": 768,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "spotting",
"spottingCoordinateSpace": "normalized0-1000 coordinates relative to image; small images doubled proportionally",
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 23296212992,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp16",
"cpuThreads": 4,
"totalMs": 13088.572000000568,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 2278178816,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float16": 608
},
"parameterDtype": "torch.float16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 3263.587749999715,
"inferenceMs": 3245.739209000021,
"pagesPerSecond": 0.3080962257309914,
"tokensPerSecond": 15.71290751228056,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 24105713664,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp16",
"cpuThreads": 4,
"totalMs": 3263.587749999715,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 2278178816,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float16": 608
},
"parameterDtype": "torch.float16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2954.562333005015,
"inferenceMs": 2944.877625006484,
"pagesPerSecond": 0.3395726842801484,
"tokensPerSecond": 17.318206898287567,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 23828889600,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp16",
"cpuThreads": 4,
"totalMs": 2954.562333005015,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 2278178816,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float16": 608
},
"parameterDtype": "torch.float16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 3101.5857919992413,
"inferenceMs": 3091.5265000003274,
"pagesPerSecond": 0.32346479967093733,
"tokensPerSecond": 16.496704783217805,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 23828889600,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp16",
"cpuThreads": 4,
"totalMs": 3101.5857919992413,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 2278178816,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float16": 608
},
"parameterDtype": "torch.float16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 3338.0426250005257,
"inferenceMs": 3324.526083000819,
"pagesPerSecond": 0.3007947524049411,
"tokensPerSecond": 15.340532372651994,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 23828889600,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp16",
"cpuThreads": 4,
"totalMs": 3338.0426250005257,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 2278178816,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float16": 608
},
"parameterDtype": "torch.float16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2971.272125003452,
"inferenceMs": 2959.5401249971474,
"pagesPerSecond": 0.33789033355341447,
"tokensPerSecond": 17.232407011224137,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 23828889600,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp16",
"cpuThreads": 4,
"totalMs": 2971.272125003452,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 2278178816,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 628.8276659979601,
"placement": {
"mps:0/torch.float16": 608
},
"parameterDtype": "torch.float16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 3434.5222920019296,
"inferenceMs": 3402.118584002892,
"pagesPerSecond": 0.29393449267233124,
"tokensPerSecond": 14.990659126288893,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 6220185600,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp16",
"cpuThreads": 4,
"totalMs": 4063.3499579998897,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 2271461376,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 609.9582079987158,
"placement": {
"mps:0/torch.float16": 608
},
"parameterDtype": "torch.float16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 3242.5538749957923,
"inferenceMs": 3210.8138330004294,
"pagesPerSecond": 0.3114475182964824,
"tokensPerSecond": 15.883823433120602,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 6220185600,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp16",
"cpuThreads": 4,
"totalMs": 3852.512082994508,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 2484518912,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 671.1113749988726,
"placement": {
"mps:0/torch.bfloat16": 608
},
"parameterDtype": "torch.bfloat16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 3079.5422079972923,
"inferenceMs": 3045.572999995784,
"pagesPerSecond": 0.3283454377883519,
"tokensPerSecond": 16.74561732720595,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 6220185600,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 3750.653582996165,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 652083200,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.bfloat16": 608
},
"parameterDtype": "torch.bfloat16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 3052.264666002884,
"inferenceMs": 3041.485375004413,
"pagesPerSecond": 0.3287867198764844,
"tokensPerSecond": 14.795402394441798,
"outputTokens": 45,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 6220185600,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 3052.264666002884,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 653869056,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.bfloat16": 608
},
"parameterDtype": "torch.bfloat16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 8426.683875004528,
"inferenceMs": 8402.286416996503,
"pagesPerSecond": 0.23803044799262196,
"tokensPerSecond": 8.212050455745457,
"outputTokens": 69,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "application/pdf",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 11098619904,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 8426.683875004528,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 699875328,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 0,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 0,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.bfloat16": 608
},
"parameterDtype": "torch.bfloat16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2540.542540999013,
"inferenceMs": 2529.773041002045,
"pagesPerSecond": 0.3952923775343496,
"tokensPerSecond": 0.3952923775343496,
"outputTokens": 1,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 11098619904,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 2540.542540999013,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 706609152,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.bfloat16": 608
},
"parameterDtype": "torch.bfloat16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2678.306084002543,
"inferenceMs": 2665.670582995517,
"pagesPerSecond": 0.37514012660794016,
"tokensPerSecond": 17.63158595057319,
"outputTokens": 47,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "table",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 11098619904,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 2678.306084002543,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 708984832,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.bfloat16": 608
},
"parameterDtype": "torch.bfloat16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 1089.2737090034643,
"inferenceMs": 1081.8890829978045,
"pagesPerSecond": 0.9243091696878036,
"tokensPerSecond": 15.713255884692662,
"outputTokens": 17,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "formula",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 11438211072,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 1089.2737090034643,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 735297536,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.bfloat16": 608
},
"parameterDtype": "torch.bfloat16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 616.6852919996018,
"inferenceMs": 611.3256659955368,
"pagesPerSecond": 1.6357893273980433,
"tokensPerSecond": 42.53052251234912,
"outputTokens": 26,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "chart",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 11438211072,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 616.6852919996018,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 735510528,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.bfloat16": 608
},
"parameterDtype": "torch.bfloat16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 646.6551659977995,
"inferenceMs": 641.2211670030956,
"pagesPerSecond": 1.5595243130755418,
"tokensPerSecond": 6.238097252302167,
"outputTokens": 4,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "seal",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 11458396160,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 646.6551659977995,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 748453888,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.bfloat16": 608
},
"parameterDtype": "torch.bfloat16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 12918.690250000509,
"inferenceMs": 12871.705915997154,
"pagesPerSecond": 0.07768977993485576,
"tokensPerSecond": 7.6135984336158655,
"outputTokens": 98,
"maxNewTokens": 768,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "spotting",
"spottingCoordinateSpace": "normalized0-1000 coordinates relative to image; small images doubled proportionally",
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 23296212992,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 12918.690250000509,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 849870848,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.bfloat16": 608
},
"parameterDtype": "torch.bfloat16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 3285.92029200081,
"inferenceMs": 3235.837415995775,
"pagesPerSecond": 0.3090390126081989,
"tokensPerSecond": 15.760989643018144,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 24105713664,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 3285.92029200081,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 855097344,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.bfloat16": 608
},
"parameterDtype": "torch.bfloat16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2542.8487499957555,
"inferenceMs": 2528.7505830056034,
"pagesPerSecond": 0.3954522073946207,
"tokensPerSecond": 20.168062577125657,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 23828889600,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 2542.8487499957555,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 855621632,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.bfloat16": 608
},
"parameterDtype": "torch.bfloat16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2608.3291670001927,
"inferenceMs": 2598.6844159997418,
"pagesPerSecond": 0.38481009615601564,
"tokensPerSecond": 19.6253149039568,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 23828889600,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 2608.3291670001927,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 856342528,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.bfloat16": 608
},
"parameterDtype": "torch.bfloat16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2652.5270830024965,
"inferenceMs": 2641.469333000714,
"pagesPerSecond": 0.3785771757811771,
"tokensPerSecond": 19.307435964840035,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 23828889600,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 2652.5270830024965,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 856342528,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.bfloat16": 608
},
"parameterDtype": "torch.bfloat16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2828.4447920013918,
"inferenceMs": 2818.497332998959,
"pagesPerSecond": 0.3547989874930882,
"tokensPerSecond": 18.0947483621475,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 23828889600,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 2828.4447920013918,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 856408064,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 668.3111660022405,
"placement": {
"mps:0/torch.bfloat16": 608
},
"parameterDtype": "torch.bfloat16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2841.5618750004796,
"inferenceMs": 2810.736709005141,
"pagesPerSecond": 0.35577861021139523,
"tokensPerSecond": 18.144709120781158,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 6220185600,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 3509.87304100272,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 653524992,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"repetitive": false,
"completeGeneration": true,
"pageCoverage": true,
"blankNoHallucination": true,
"taskStructureAndContent": true,
"scope": "Authored task-specific fixture only; no broad multilingual or layout claims"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 687.7966669999296,
"placement": {
"mps:0/torch.bfloat16": 608
},
"parameterDtype": "torch.bfloat16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2790.7250420030323,
"inferenceMs": 2759.316708004917,
"pagesPerSecond": 0.3624085619091674,
"tokensPerSecond": 18.48283665736754,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"task": "ocr",
"spottingCoordinateSpace": null,
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 6220185600,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 3478.521709002962,
"artifactRevision": "1e5216083435c7851caad5d691ef55aab81b5949",
"fallback": false,
"grade": "fast",
"sampledPeakHelperRssBytes": 649674752,
"memoryScope": "Process RSS sampled200ms including init; not GPU-memory minimum"
}
}
]Release validation still required
- Complete public dependency archive and forced model/runtime-source recovery pending