Granite Docling Desktop
Convert page images or locally rendered PDFs into DocTags, Markdown and structured DoclingDocument JSON using native Granite Docling and Docling Core.
Desktop app
Run Granite Docling Desktop on your computer
Download the desktop app to load this model, provide input and view results locally.
Public desktop downloads are coming soon.Device requirements
If your device falls short
MemoryAdvisory only: close other apps, reduce image/tile/token/batch settings or choose a smaller verified model/precision. CPU can need more RAM than GPU.
GPUIf the selected GPU runtime fails, existing CPU fallback may recover when enabled, but CPU has its own memory/runtime needs.
PlatformUse a compatible OS/architecture/runtime or another model. These estimates do not add hardware detection or block runs.
Benchmark evidence
Saved desktop measurements apply to the recorded hardware and runtime. Fallback cases report the device that actually ran; they do not prove GPU support on the requested device.
Docling — Image to Text Desktop
37 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 1120 ms / 0 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 1938 ms / 0 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2262 ms / 0 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2617 ms / 0 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2639 ms / 0 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2659 ms / 0 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2680 ms / 0 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2744 ms / 0 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2757 ms / 0 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2768 ms / 0 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2836 ms / 400 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2873 ms / 463 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2919 ms / 452 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 4646 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 4728 ms / 642 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 2996 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 4297 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 5075 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 5859 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 5925 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 5929 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 5951 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 5990 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 6042 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 6048 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 6049 ms / 233 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 6050 ms / 231 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 6079 ms / 234 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 9914 ms / 0 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 10072 ms / 222 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 11126 ms / 0 ms |
FP16 | mps | Failed | — | macos · aarch64 | 13651 ms / 399 ms |
FP16 | mps | Failed | — | macos · aarch64 | 13649 ms / 0 ms |
FP16 | mps | Failed | — | macos · aarch64 | 30191 ms / 0 ms |
BF16 | mps | Failed | — | macos · aarch64 | 3414 ms / 0 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 17932 ms / 229 ms |
FP32 | mps | Failed | — | macos · aarch64 | 11112 ms / 402 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-01T18:24:15.587596+00:00
- Environment
- { "hardware": "Apple M3 Max", "os": "macOS-27.2-arm64-arm-64bit-Mach-O", "python": "3.13.1", "runtime": "Transformers5.22.1/Torch2.10.0/DoclingCore2.105.0/PDFium5.14.0" }
- Runtime
- transformers 5.22.1
- Artifact revision
- 982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe
- Fixture
{ "file": "model-samples/desktop/docling/receipt.png", "size": 25773, "sha1": "05f77f558dde3e17f3541910abeecde08c193cd5", "sha256": "cc0123c5e9e48de091ce0d3462d267b2485295096206353c9bf4beeec594ca0e", "url": "https://assets.localai.tools/samples/v1/files/cc0123c5e9e48de091ce0d3462d267b2485295096206353c9bf4beeec594ca0e/receipt.png" }- Rerun when
- artifact revision/checksums; Transformers/Torch/DoclingCore/PDFium runtime or driver/backend changes; PDF rendering/image processor/DocTags conversion/output budget changes; fixtures/independent content/table annotations/quality gates
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 221.53637500014156,
"placement": {
"cpu/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 10546.207333001803,
"inferenceMs": 10071.50191600158,
"pagesPerSecond": 0.09929005706797336,
"tokensPerSecond": 15.687829016739792,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 158,
"inferenceMs": 10071.50191600158,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 158,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 10767.743708001944,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 10054.900209001062,
"inferenceMs": 9914.482208998379,
"pagesPerSecond": 0.10086255428371242,
"tokensPerSecond": 15.936283576826561,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 158,
"inferenceMs": 9914.482208998379,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 158,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 10054.900209001062,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 232.50833400015836,
"placement": {
"cpu/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 6573.0403330017,
"inferenceMs": 6049.472582999442,
"pagesPerSecond": 0.1653036667709272,
"tokensPerSecond": 26.117979349806497,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 158,
"inferenceMs": 6049.472582999442,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 158,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 6805.548667001858,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 1,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 5221.63804199954,
"inferenceMs": 5074.659333000454,
"pagesPerSecond": 0.19705756276032382,
"tokensPerSecond": 22.85867728019756,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 116,
"inferenceMs": 5074.659333000454,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 116,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 5221.63804199954,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 11417.920667001454,
"inferenceMs": 11125.83399900177,
"pagesPerSecond": 0.17976180483902993,
"tokensPerSecond": 22.739868312137286,
"pageCount": 2,
"pages": [
{
"page": 1,
"width": 864,
"height": 1013,
"inputTokens": 1142,
"outputTokens": 158,
"inferenceMs": 6258.920791002311,
"outputLimitReached": false,
"eosReached": true
},
{
"page": 2,
"width": 864,
"height": 1013,
"inputTokens": 1142,
"outputTokens": 95,
"inferenceMs": 4866.91320799946,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 253,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "application/pdf",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 11417.920667001454,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 0,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 0,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 3134.4827079992683,
"inferenceMs": 2996.0566250010743,
"pagesPerSecond": 0.3337720628025985,
"tokensPerSecond": 1.335088251210394,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 4,
"inferenceMs": 2996.0566250010743,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 4,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 3134.4827079992683,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 6000.5071249979665,
"inferenceMs": 5858.795374999318,
"pagesPerSecond": 0.17068355113872133,
"tokensPerSecond": 26.96800107991797,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 158,
"inferenceMs": 5858.795374999318,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 158,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 6000.5071249979665,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 6185.75141600013,
"inferenceMs": 6042.393957999593,
"pagesPerSecond": 0.16549731893533504,
"tokensPerSecond": 26.148576391782935,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 158,
"inferenceMs": 6042.393957999593,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 158,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 6185.75141600013,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 6095.4652079999505,
"inferenceMs": 5951.139791999594,
"pagesPerSecond": 0.1680350378165118,
"tokensPerSecond": 26.549535975008865,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 158,
"inferenceMs": 5951.139791999594,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 158,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 6095.4652079999505,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 234.13920899838558,
"placement": {
"cpu/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 6593.717832998664,
"inferenceMs": 6079.362917000253,
"pagesPerSecond": 0.16449092012645153,
"tokensPerSecond": 25.98956537997934,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 158,
"inferenceMs": 6079.362917000253,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 158,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 6827.857041997049,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 6196.763749998354,
"inferenceMs": 6047.7085419988725,
"pagesPerSecond": 0.1653518837846446,
"tokensPerSecond": 26.125597637973847,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 158,
"inferenceMs": 6047.7085419988725,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 158,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 6196.763749998354,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 6067.744291001873,
"inferenceMs": 5925.4142090030655,
"pagesPerSecond": 0.16876457319736424,
"tokensPerSecond": 26.66480256518355,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 158,
"inferenceMs": 5925.4142090030655,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 158,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 6067.744291001873,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 231.2269579997519,
"placement": {
"cpu/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 6541.495584002405,
"inferenceMs": 6049.515958999109,
"pagesPerSecond": 0.1653024815171245,
"tokensPerSecond": 26.117792079705673,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 158,
"inferenceMs": 6049.515958999109,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 158,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 6772.722542002157,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 6077.96750000125,
"inferenceMs": 5928.538042000582,
"pagesPerSecond": 0.16867564868699914,
"tokensPerSecond": 26.650752492545866,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 158,
"inferenceMs": 5928.538042000582,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 158,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 6077.96750000125,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 6130.192207998334,
"inferenceMs": 5990.318083000602,
"pagesPerSecond": 0.1669360434862069,
"tokensPerSecond": 26.375894870820687,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 158,
"inferenceMs": 5990.318083000602,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 158,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 6130.192207998334,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 399.5997919992078,
"placement": {
"mps:0/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 3308.3390409992717,
"inferenceMs": 2836.1237499993877,
"pagesPerSecond": 0.35259392330825334,
"tokensPerSecond": 55.70983988270403,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 158,
"inferenceMs": 2836.1237499993877,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 158,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"driverAllocatedMemoryBytes": 15888875520,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 3707.9388329984795,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 1,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 2410.981000000902,
"inferenceMs": 2261.7572499984817,
"pagesPerSecond": 0.442134097282399,
"tokensPerSecond": 51.287555284758284,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 116,
"inferenceMs": 2261.7572499984817,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 116,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"driverAllocatedMemoryBytes": 15888875520,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 2410.981000000902,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 4961.91925000312,
"inferenceMs": 4646.09120899695,
"pagesPerSecond": 0.430469379535014,
"tokensPerSecond": 54.45437651117927,
"pageCount": 2,
"pages": [
{
"page": 1,
"width": 864,
"height": 1013,
"inputTokens": 1142,
"outputTokens": 158,
"inferenceMs": 2686.3760839987663,
"outputLimitReached": false,
"eosReached": true
},
{
"page": 2,
"width": 864,
"height": 1013,
"inputTokens": 1142,
"outputTokens": 95,
"inferenceMs": 1959.7151249981835,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 253,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "application/pdf",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"driverAllocatedMemoryBytes": 15888875520,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 4961.91925000312,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 0,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 0,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 1264.5018750008603,
"inferenceMs": 1120.4285829990113,
"pagesPerSecond": 0.8925156098065042,
"tokensPerSecond": 3.570062439226017,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 4,
"inferenceMs": 1120.4285829990113,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 4,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"driverAllocatedMemoryBytes": 15888875520,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 1264.5018750008603,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 2760.5528339990997,
"inferenceMs": 2616.954083001474,
"pagesPerSecond": 0.38212363239215336,
"tokensPerSecond": 60.37553391796023,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 158,
"inferenceMs": 2616.954083001474,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 158,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"driverAllocatedMemoryBytes": 15888875520,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 2760.5528339990997,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 2906.097792001674,
"inferenceMs": 2757.1589590006624,
"pagesPerSecond": 0.36269218237691014,
"tokensPerSecond": 57.305364815551805,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 158,
"inferenceMs": 2757.1589590006624,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 158,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"driverAllocatedMemoryBytes": 15888875520,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 2906.097792001674,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 2811.8897909989755,
"inferenceMs": 2658.6438750018715,
"pagesPerSecond": 0.3761316095783216,
"tokensPerSecond": 59.428794313374816,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 158,
"inferenceMs": 2658.6438750018715,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 158,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"driverAllocatedMemoryBytes": 15888875520,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 2811.8897909989755,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 463.0552919988986,
"placement": {
"mps:0/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 3353.3787920023315,
"inferenceMs": 2873.0370410012256,
"pagesPerSecond": 0.348063733856877,
"tokensPerSecond": 54.994069949386564,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 158,
"inferenceMs": 2873.0370410012256,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 158,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"driverAllocatedMemoryBytes": 15888875520,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 3816.43408400123,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 2829.662083000585,
"inferenceMs": 2679.934499999945,
"pagesPerSecond": 0.3731434480954742,
"tokensPerSecond": 58.956664799084926,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 158,
"inferenceMs": 2679.934499999945,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 158,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"driverAllocatedMemoryBytes": 15888875520,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 2829.662083000585,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 2896.321749998606,
"inferenceMs": 2744.022708000557,
"pagesPerSecond": 0.36442847104886167,
"tokensPerSecond": 57.579698425720146,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 158,
"inferenceMs": 2744.022708000557,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 158,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"driverAllocatedMemoryBytes": 15888875520,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 2896.321749998606,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 452.1019579988206,
"placement": {
"mps:0/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 3400.96066699698,
"inferenceMs": 2918.955207998806,
"pagesPerSecond": 0.34258833340768724,
"tokensPerSecond": 54.12895667841458,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 158,
"inferenceMs": 2918.955207998806,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 158,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"driverAllocatedMemoryBytes": 15888875520,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 3853.0626249958004,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 2796.6863330002525,
"inferenceMs": 2639.353167000081,
"pagesPerSecond": 0.37888070929765394,
"tokensPerSecond": 59.86315206902932,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 158,
"inferenceMs": 2639.353167000081,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 158,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"driverAllocatedMemoryBytes": 15888875520,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 2796.6863330002525,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 2915.7769170014944,
"inferenceMs": 2767.9037079979025,
"pagesPerSecond": 0.36128424450261176,
"tokensPerSecond": 57.08291063141266,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 158,
"inferenceMs": 2767.9037079979025,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 158,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"driverAllocatedMemoryBytes": 15888875520,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 2915.7769170014944,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"phraseRecall": 0,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 0,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": false,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 398.6139170010574,
"placement": {
"mps:0/torch.float16": 470
},
"parameterDtype": "torch.float16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 14322.553667003376,
"inferenceMs": 13650.904040998284,
"pagesPerSecond": 0.0732552215587086,
"tokensPerSecond": 75.01334687611761,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 1024,
"inferenceMs": 13650.904040998284,
"outputLimitReached": true,
"eosReached": false
}
],
"outputTokens": 1024,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": true,
"allPagesReachedEos": false,
"structuredImagesIncluded": false,
"driverAllocatedMemoryBytes": 44600983552,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp16",
"cpuThreads": 4,
"totalMs": 14721.167584004434,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "moderate"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"phraseRecall": 0,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 0,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": false,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": false,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float16": 470
},
"parameterDtype": "torch.float16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 13801.189333997172,
"inferenceMs": 13649.457999999868,
"pagesPerSecond": 0.07326298231036057,
"tokensPerSecond": 75.02129388580923,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 1024,
"inferenceMs": 13649.457999999868,
"outputLimitReached": true,
"eosReached": false
}
],
"outputTokens": 1024,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": true,
"allPagesReachedEos": false,
"structuredImagesIncluded": false,
"driverAllocatedMemoryBytes": 44600983552,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp16",
"cpuThreads": 4,
"totalMs": 13801.189333997172,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "moderate"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"phraseRecall": 0,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 0,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": false,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float16": 470
},
"parameterDtype": "torch.float16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 30508.169500000804,
"inferenceMs": 30191.073416997824,
"pagesPerSecond": 0.06624474633200632,
"tokensPerSecond": 67.83462024397447,
"pageCount": 2,
"pages": [
{
"page": 1,
"width": 864,
"height": 1013,
"inputTokens": 1142,
"outputTokens": 1024,
"inferenceMs": 14096.000292000099,
"outputLimitReached": true,
"eosReached": false
},
{
"page": 2,
"width": 864,
"height": 1013,
"inputTokens": 1142,
"outputTokens": 1024,
"inferenceMs": 16095.073124997725,
"outputLimitReached": true,
"eosReached": false
}
],
"outputTokens": 2048,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "application/pdf",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": true,
"allPagesReachedEos": false,
"structuredImagesIncluded": false,
"driverAllocatedMemoryBytes": 44600983552,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp16",
"cpuThreads": 4,
"totalMs": 30508.169500000804,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "moderate"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": true,
"tableCount": 0,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 641.6284580009233,
"placement": {
"mps:0/torch.bfloat16": 470
},
"parameterDtype": "torch.bfloat16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 5354.282874999626,
"inferenceMs": 4728.248666997388,
"pagesPerSecond": 0.2114947986936222,
"tokensPerSecond": 33.41617819359231,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 158,
"inferenceMs": 4728.248666997388,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 158,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"driverAllocatedMemoryBytes": 9706471424,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 5995.911333000549,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"phraseRecall": 0.3333333333333333,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 0.5,
"minimumTextTokenRecall": 0.9,
"tableStructureAndCells": false,
"tableCount": 1,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored document content and table annotations; full EOS-completed pages required"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.bfloat16": 470
},
"parameterDtype": "torch.bfloat16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 3572.004499998002,
"inferenceMs": 3413.6911669993424,
"pagesPerSecond": 0.2929380401095295,
"tokensPerSecond": 28.70792793073389,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 98,
"inferenceMs": 3413.6911669993424,
"outputLimitReached": false,
"eosReached": true
}
],
"outputTokens": 98,
"maxNewTokens": 1024,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"structuredImagesIncluded": false,
"driverAllocatedMemoryBytes": 9706471424,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 3572.004499998002,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"passed": false,
"scope": "Original receipt content with splitting disabled",
"expectedPartial": false,
"validDocumentShape": true,
"fullContentMatched": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 229.39883299841313,
"placement": {
"cpu/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 18381.32745799885,
"inferenceMs": 17932.495584002027,
"pagesPerSecond": 0.05576468681203089,
"tokensPerSecond": 57.10303929551963,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 82,
"outputTokens": 1024,
"inferenceMs": 17932.495584002027,
"outputLimitReached": true,
"eosReached": false
}
],
"outputTokens": 1024,
"maxNewTokens": 1024,
"doImageSplitting": false,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": true,
"allPagesReachedEos": false,
"structuredImagesIncluded": false,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 18610.726290997263,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"scope": "Explicit partial-output behavior, not full-page extraction quality",
"expectedPartial": true,
"validDocumentShape": true,
"fullContentMatched": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 4452.096750002966,
"inferenceMs": 4296.683958000358,
"pagesPerSecond": 0.23273762040096427,
"tokensPerSecond": 3.7238019264154283,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 16,
"inferenceMs": 4296.683958000358,
"outputLimitReached": true,
"eosReached": false
}
],
"outputTokens": 16,
"maxNewTokens": 16,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": true,
"allPagesReachedEos": false,
"structuredImagesIncluded": false,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 4452.096750002966,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "fail",
"quality": {
"passed": false,
"scope": "Original receipt content with splitting disabled",
"expectedPartial": false,
"validDocumentShape": true,
"fullContentMatched": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 402.03674999793293,
"placement": {
"mps:0/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 11528.050332999555,
"inferenceMs": 11111.59483299707,
"pagesPerSecond": 0.08999608202329272,
"tokensPerSecond": 92.15598799185175,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 82,
"outputTokens": 1024,
"inferenceMs": 11111.59483299707,
"outputLimitReached": true,
"eosReached": false
}
],
"outputTokens": 1024,
"maxNewTokens": 1024,
"doImageSplitting": false,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": true,
"allPagesReachedEos": false,
"structuredImagesIncluded": false,
"driverAllocatedMemoryBytes": 1884651520,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 11930.087082997488,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "moderate"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Explicit partial-output behavior, not full-page extraction quality",
"expectedPartial": true,
"validDocumentShape": true,
"fullContentMatched": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"doclingCoreVersion": "2.105.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 470
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language model parameters and input tensors on selected device; image/PDF rendering, tokenization, decoding and Docling Core conversion on CPU",
"durationMs": 2094.0821250005683,
"inferenceMs": 1938.2313329988392,
"pagesPerSecond": 0.5159342865708377,
"tokensPerSecond": 8.254948585133404,
"pageCount": 1,
"pages": [
{
"page": 1,
"width": 768,
"height": 900,
"inputTokens": 1142,
"outputTokens": 16,
"inferenceMs": 1938.2313329988392,
"outputLimitReached": true,
"eosReached": false
}
],
"outputTokens": 16,
"maxNewTokens": 16,
"doImageSplitting": true,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": true,
"allPagesReachedEos": false,
"structuredImagesIncluded": false,
"driverAllocatedMemoryBytes": 6772047872,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 2094.0821250005683,
"artifactRevision": "982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe",
"fallback": false,
"grade": "fast"
}
}
]