A model is still running

A model is still running or downloading. Leaving will stop the current operation. Stay on this page to let it finish.

Skip to content
LocalAI.toolsBeta

Join the waitlist

Get an email when the new apps and model runs launch.

No account needed.

Back to models

Mixedbread Edge ColBERT v0 Desktop

Encode English queries and passages as 48-number vectors for each retained token.

Desktop inferencePrivate · Runs on your computer
About this model

Encode English queries and passages as 48-number vectors for each retained token.

Created by
Mixedbread
License
Apache-2.0
Model released
Oct 16, 2025
Source repository created
Oct 13, 2025
Model weights · FP32
67.8 MB

Best for

  • Late-interaction text retrieval using MaxSim.
  • Token-level query and passage comparison.

Keep in mind

  • Measured on three small English retrieval fixtures. Evaluate on your own corpus before relying on rankings.
  • Queries are truncated to 48 tokens and documents to 512 tokens, including special tokens.
  • Retrieval scores measure relevance and can reflect training-data biases. Scores do not establish factual accuracy.
  • The selected runtime returns complete token matrices after inference. Incremental streaming is unavailable.

Processed on your deviceYour input is processed locally in the desktop app. Model and runtime files download from external hosts.

Target platforms · verification required

  • macos · aarch64, x86_64
  • windows · x86_64

Recommended configuration

FP32 · cpu (verified platform only)

Sources & technical details
Original model
mixedbread-ai/mxbai-edge-colbert-v0-17m
Release verification
Verified
Release evidence
Mixedbread's dated release announcement explicitly introduces both checkpoints.
Source updated
Aug 19, 2026
Native model files
mixedbread-ai/mxbai-edge-colbert-v0-17m
Files updated
Aug 19, 2026
Artifact revision
592c6417c1c6687572043408ed1ae5196bce16b1

Desktop app

Sample result

Mixedbread Edge ColBERT v0 17M Desktop

Run Mixedbread Edge ColBERT v0 Desktop on your computer

Download the desktop app to load this model, provide input and view results locally.

Public desktop downloads are coming soon.

Device requirements

macOS Apple SiliconCPU / FP32Tested8.0 GiBRAM estimate
macOS Apple SiliconMPS / FP32Tested8.0 GiBRAM estimate
macOS Apple SiliconMPS / FP16Tested8.0 GiBRAM estimate
macOS Apple SiliconMPS / BF16Tested8.0 GiBRAM estimate
If your device falls short

MemoryAdvisory only: close other apps, reduce image/tile/token/batch settings or choose a smaller verified model/precision. CPU can need more RAM than GPU.

GPUIf the selected GPU runtime fails, existing CPU fallback may recover when enabled, but CPU has its own memory/runtime needs.

PlatformUse a compatible OS/architecture/runtime or another model. These estimates do not add hardware detection or block runs.

Benchmark evidence

Saved desktop measurements apply to the recorded hardware and runtime. Fallback cases report the device that actually ran; they do not prove GPU support on the requested device.

Mixedbread Edge ColBERT v0 17M Desktop

4 configurations tested
Passed4
Failed0
TestedNov 14, 2026
Runtimetransformers 5.22.1

Quality criteria

Three held-out ranking fixtures, exact token IDs and agreement with original native FP32 token vectors on browser.

PrecisionDeviceResultPlatformInference / load
BF16mpsAcceleratedPassedSlowmacos · aarch64464 ms / 2035 ms
FP32mpsAcceleratedPassedSlowmacos · aarch64467 ms / 1897 ms
FP16mpsAcceleratedPassedSlowmacos · aarch64492 ms / 1908 ms
FP32cpuPassedNot measuredmacos · aarch648 ms / 1979 ms
Test setup & sample details
Quality gate
Three held-out ranking fixtures, exact token IDs and agreement with original native FP32 token vectors on browser.
Tested
2026-11-14T12:27:39.903333+00:00
Environment
Apple M3 Max on macOS 27.2. Managed native or isolated Chrome module workers.
Runtime
transformers 5.22.1
Artifact revision
592c6417c1c6687572043408ed1ae5196bce16b1
Fixture
{
  "file": "model-samples/mxbai-edge-colbert-v0-astronomy-query.json",
  "size": 163,
  "sha1": "040b125553ac4ebfb935837e3627846506d3fc44",
  "sha256": "0790ccf44917ad2edb477b1eac3d6d7c2e3aeb2e87b74aa25a8b4f3251485cbe",
  "url": "https://assets.localai.tools/samples/v1/files/0790ccf44917ad2edb477b1eac3d6d7c2e3aeb2e87b74aa25a8b4f3251485cbe/mxbai-edge-colbert-v0-astronomy-query.json",
  "durationSeconds": 0
}
Rerun when
artifact revision; runtime version; retrieval fixture; tokenizer or projection semantics
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "scores": [
        11.569320791985172,
        11.758439178301645,
        11.709889965380437,
        11.722880817316915
      ],
      "winner": 1,
      "expected": 1,
      "grade": "retrieval-pass"
    },
    "runtime": {
      "inferenceMs": 7.722500013187528,
      "durationMs": 7.908416970167309,
      "embeddingsPerSecond": 129.4917446801326,
      "outputDevice": "cpu",
      "peakResidentMemoryBytes": 399065088,
      "loadMs": 1979.3045410187915,
      "totalMs": 1987.027041031979,
      "grade": "interactive",
      "version": "5.22.1",
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "artifactRevision": "592c6417c1c6687572043408ed1ae5196bce16b1",
      "acceleration": "cpu",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "provider": "PyTorch CPU",
      "placement": {
        "cpu/torch.float32": 46
      },
      "cpuThreads": 4,
      "engine": "transformers"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scores": [
        11.569321051071174,
        11.758438946328058,
        11.709890006975224,
        11.722881057004145
      ],
      "winner": 1,
      "expected": 1,
      "grade": "retrieval-pass"
    },
    "runtime": {
      "inferenceMs": 466.62333398126066,
      "durationMs": 467.2564170323312,
      "embeddingsPerSecond": 2.1430561379516426,
      "outputDevice": "mps",
      "peakResidentMemoryBytes": 442974208,
      "loadMs": 1897.2673340467736,
      "totalMs": 2363.8906680280343,
      "grade": "slow",
      "version": "5.22.1",
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "artifactRevision": "592c6417c1c6687572043408ed1ae5196bce16b1",
      "acceleration": "partial",
      "requestedDevice": "mps",
      "requestedPrecision": "fp32",
      "provider": "Metal/MPS",
      "placement": {
        "mps:0/torch.float32": 46
      },
      "cpuThreads": 4,
      "engine": "transformers"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scores": [
        11.569295235377067,
        11.758409362463102,
        11.709864571510934,
        11.72285209089974
      ],
      "winner": 1,
      "expected": 1,
      "grade": "retrieval-pass"
    },
    "runtime": {
      "inferenceMs": 492.23183299181983,
      "durationMs": 494.24425000324845,
      "embeddingsPerSecond": 2.0315630419957387,
      "outputDevice": "mps",
      "peakResidentMemoryBytes": 481525760,
      "loadMs": 1908.3251249976456,
      "totalMs": 2400.5569579894654,
      "grade": "slow",
      "version": "5.22.1",
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "artifactRevision": "592c6417c1c6687572043408ed1ae5196bce16b1",
      "acceleration": "partial",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "provider": "Metal/MPS",
      "placement": {
        "mps:0/torch.float16": 46
      },
      "cpuThreads": 4,
      "engine": "transformers"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scores": [
        11.56811596645032,
        11.75804946251657,
        11.709197176694467,
        11.722652320194442
      ],
      "winner": 1,
      "expected": 1,
      "grade": "retrieval-pass"
    },
    "runtime": {
      "inferenceMs": 463.667458971031,
      "durationMs": 464.4362499821,
      "embeddingsPerSecond": 2.1567180975330813,
      "outputDevice": "mps",
      "peakResidentMemoryBytes": 477315072,
      "loadMs": 2035.4351670248434,
      "totalMs": 2499.1026259958744,
      "grade": "slow",
      "version": "5.22.1",
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "artifactRevision": "592c6417c1c6687572043408ed1ae5196bce16b1",
      "acceleration": "partial",
      "requestedDevice": "mps",
      "requestedPrecision": "bf16",
      "provider": "Metal/MPS",
      "placement": {
        "mps:0/torch.bfloat16": 46
      },
      "cpuThreads": 4,
      "engine": "transformers"
    }
  }
]

Mixedbread Edge ColBERT v0 32M Desktop

4 configurations tested
Passed4
Failed0
TestedNov 14, 2026
Runtimetransformers 5.22.1

Quality criteria

Three held-out ranking fixtures, exact token IDs and agreement with original native FP32 token vectors on browser.

PrecisionDeviceResultPlatformInference / load
BF16mpsAcceleratedPassedSlowmacos · aarch64260 ms / 1883 ms
FP16mpsAcceleratedPassedSlowmacos · aarch64267 ms / 1923 ms
FP32mpsAcceleratedPassedNot measuredmacos · aarch64244 ms / 2225 ms
FP32cpuPassedSlowmacos · aarch64258 ms / 1932 ms
Test setup & sample details
Quality gate
Three held-out ranking fixtures, exact token IDs and agreement with original native FP32 token vectors on browser.
Tested
2026-11-14T12:27:54.273079+00:00
Environment
Apple M3 Max on macOS 27.2. Managed native or isolated Chrome module workers.
Runtime
transformers 5.22.1
Artifact revision
bb13a29ec9b1e7edd4ba8f7a0776c48b55cbad66
Fixture
{
  "file": "model-samples/mxbai-edge-colbert-v0-astronomy-query.json",
  "size": 163,
  "sha1": "040b125553ac4ebfb935837e3627846506d3fc44",
  "sha256": "0790ccf44917ad2edb477b1eac3d6d7c2e3aeb2e87b74aa25a8b4f3251485cbe",
  "url": "https://assets.localai.tools/samples/v1/files/0790ccf44917ad2edb477b1eac3d6d7c2e3aeb2e87b74aa25a8b4f3251485cbe/mxbai-edge-colbert-v0-astronomy-query.json",
  "durationSeconds": 0
}
Rerun when
artifact revision; runtime version; retrieval fixture; tokenizer or projection semantics
Per-case quality & failure details
[
  {
    "precision": "fp32",
    "device": "cpu",
    "status": "pass",
    "quality": {
      "passed": true,
      "scores": [
        11.208053477273115,
        11.530794457560223,
        11.410383620652931,
        11.475587789753598
      ],
      "winner": 1,
      "expected": 1,
      "grade": "retrieval-pass"
    },
    "runtime": {
      "inferenceMs": 257.8439579810947,
      "durationMs": 258.02358298096806,
      "embeddingsPerSecond": 3.878314651349405,
      "outputDevice": "cpu",
      "peakResidentMemoryBytes": 431882240,
      "loadMs": 1931.9567080237903,
      "totalMs": 2189.800666004885,
      "grade": "slow",
      "version": "5.22.1",
      "device": "cpu",
      "precision": "fp32",
      "fallback": false,
      "artifactRevision": "bb13a29ec9b1e7edd4ba8f7a0776c48b55cbad66",
      "acceleration": "cpu",
      "requestedDevice": "cpu",
      "requestedPrecision": "fp32",
      "provider": "PyTorch CPU",
      "placement": {
        "cpu/torch.float32": 65
      },
      "cpuThreads": 4,
      "engine": "transformers"
    }
  },
  {
    "precision": "fp32",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scores": [
        11.20805355991609,
        11.530794549951084,
        11.410383489551636,
        11.475587768749135
      ],
      "winner": 1,
      "expected": 1,
      "grade": "retrieval-pass"
    },
    "runtime": {
      "inferenceMs": 244.0697499550879,
      "durationMs": 244.68599999090657,
      "embeddingsPerSecond": 4.097189431234364,
      "outputDevice": "mps",
      "peakResidentMemoryBytes": 440123392,
      "loadMs": 2225.2276659710333,
      "totalMs": 2469.297415926121,
      "grade": "interactive",
      "version": "5.22.1",
      "device": "mps",
      "precision": "fp32",
      "fallback": false,
      "artifactRevision": "bb13a29ec9b1e7edd4ba8f7a0776c48b55cbad66",
      "acceleration": "partial",
      "requestedDevice": "mps",
      "requestedPrecision": "fp32",
      "provider": "Metal/MPS",
      "placement": {
        "mps:0/torch.float32": 65
      },
      "cpuThreads": 4,
      "engine": "transformers"
    }
  },
  {
    "precision": "fp16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scores": [
        11.207926836298777,
        11.530546287330043,
        11.410290769382218,
        11.475508406875395
      ],
      "winner": 1,
      "expected": 1,
      "grade": "retrieval-pass"
    },
    "runtime": {
      "inferenceMs": 266.5492499945685,
      "durationMs": 267.17304199701175,
      "embeddingsPerSecond": 3.751651899303326,
      "outputDevice": "mps",
      "peakResidentMemoryBytes": 566345728,
      "loadMs": 1922.8624170063995,
      "totalMs": 2189.411667000968,
      "grade": "slow",
      "version": "5.22.1",
      "device": "mps",
      "precision": "fp16",
      "fallback": false,
      "artifactRevision": "bb13a29ec9b1e7edd4ba8f7a0776c48b55cbad66",
      "acceleration": "partial",
      "requestedDevice": "mps",
      "requestedPrecision": "fp16",
      "provider": "Metal/MPS",
      "placement": {
        "mps:0/torch.float16": 65
      },
      "cpuThreads": 4,
      "engine": "transformers"
    }
  },
  {
    "precision": "bf16",
    "device": "mps",
    "status": "pass",
    "quality": {
      "passed": true,
      "scores": [
        11.21024335739344,
        11.532356846599706,
        11.412626143986673,
        11.476217925606408
      ],
      "winner": 1,
      "expected": 1,
      "grade": "retrieval-pass"
    },
    "runtime": {
      "inferenceMs": 260.248959006276,
      "durationMs": 261.0681669902988,
      "embeddingsPerSecond": 3.8424745436767904,
      "outputDevice": "mps",
      "peakResidentMemoryBytes": 563904512,
      "loadMs": 1882.7967920224182,
      "totalMs": 2143.045751028694,
      "grade": "slow",
      "version": "5.22.1",
      "device": "mps",
      "precision": "bf16",
      "fallback": false,
      "artifactRevision": "bb13a29ec9b1e7edd4ba8f7a0776c48b55cbad66",
      "acceleration": "partial",
      "requestedDevice": "mps",
      "requestedPrecision": "bf16",
      "provider": "Metal/MPS",
      "placement": {
        "mps:0/torch.bfloat16": 65
      },
      "cpuThreads": 4,
      "engine": "transformers"
    }
  }
]
Release validation still required
  • Public backup, sample publication and dependent recovery are deferred.
  • Windows benchmarking is deferred.

Something not quite right?

Tell us what happened with Mixedbread Edge ColBERT v0 Desktop. Loading, results, or the page itself—we’re here for all of it.

Attachments Optional

Up to 3 files · 50 MB each

Browser, OS, CPU/GPU, memory, screen, storage, and network details are included automatically. Model inputs, results, and browsing history are not collected.