[Bug] ColBert endpoint is not working

[IKACE]

What happened?

Hi,

We encounter the following error when trying to use the colbert endpoint stated in dspy retrieve example. Could someone please take a look at this? Thanks!

Error

> GET /wiki17_abstracts?query=Academy%20Award%20Danny%20Boyle&k=1 HTTP/1.1
> Host: 20.102.90.50:2017
> User-Agent: curl/7.76.1
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< date: Sat, 06 Dec 2025 05:12:00 GMT
< server: uvicorn
< content-length: 177
< content-type: application/json
< 
* Connection #0 to host 20.102.90.50 left intact
{"error":true,"message":"Exception occurred when connecting to server on port 2171: Cannot connect to host localhost:2171 ssl:default [Connect call failed ('127.0.0.1', 2171)]"}

Steps to reproduce

curl -v "http://20.102.90.50:2017/wiki17_abstracts?query=Academy%20Award%20Danny%20Boyle&k=1"

DSPy version

3.0.0b3

[akshatvishu]

Hey! this looks like a similar issue to what people were facing in #8946 and potentially caused by OOM bug detailed in #9091 . The public hosted instance (20.102.90.50) is currently unreliable because the internal search process keeps crashing due to an unbounded cache.

For now, the people have moved to running the server locally. nielsgl has built a dedicated tool, colbert-server, that mimics the original 2017 Wikipedia API exactly.

It automatically pulls the required index (approx. 13GB) from Hugging Face and serves it locally.

Option 1: Using uv (Recommended / Easiest) If you have uv installed, you can run it in one line without managing venvs:

# Optional: Check if your system (torch/faiss) is ready before downloading data
uvx colbert-server doctor

# Start the server (downloads data on first run, then caches it)
uvx colbert-server serve --from-cache

Option 2: Using standard pip

pip install colbert-server
colbert-server serve --from-cache

If you don't want to install the package, @nielsgl also provided a standalone script in the repository that you can run manually.

For full documentation, check the repo here: https://github.com/nielsgl/colbert-server

[TomeHirata]

Let me close this issue since it's duplicated with #8946