Agentic healthcare LLMs

Enabling LLMs to access medical databases via model context protocol

Model Context Protocol (MCP) is a tool-wrapper that enables Large Language Models (LLMs) to search the web, use a calculator, retrieve up-to-date weather reports, and (perhaps in the future) query medical records to help doctors answer questions and get up to speed on new patients.

DICOM MCP is exactly this kind of tool-wrapper for medical imaging systems. It lets AI assistants query DICOM-compliant databases to find patient scans, retrieve diagnostic image reports, and extract clinical information. Here’s what it looks like in action when used with Claude

Example interaction

The LLM retrieves the answer by:

1. Finding the patient in the DICOM database
2. Locating relevant imaging studies
3. Downloading and extracting the text from the clinical image report
4. Providing an answer based on retrieved information

NOTE: This example uses synthetically generated PDF-reports written in Danish.

How It Works

Model Context Protocol

Model Context Protocol provides a standardized way for language models to use external tools. It’s like giving an LLM the ability to call functions when it needs specific information or capabilities.

The MCP specification defines:

• Tools: Functions the model can execute (like query_patients or extract_pdf_text)
• Resources: Data sources the model can access
• Prompts: Templates for common interactions

MCP is LLM-agnostic, meaning it works with most tool-enabled LLM like Claude, GPT, and open-weights alternatives. This allows organizations to use off-the-shelf vanilla langugage models without any finetuning or custom prompt-engineering. MCPs are simple plug n’ play extensions to LLMs, similar to how users can install browser-extensions in their web-browser.

DICOM Standard

DICOM is the standard format for storing and transmitting medical images. It’s used in radiology, cardiology, nuclear medicine, and other specialties that generate medical images. Beyond just pixel data, DICOM files contain patient information, study details, and sometimes embedded documents. Some of these documents are so-called imaging reports, which are written by medical professionals, and which summarize the clinical findings and impressions of the particular medical image.

DICOM databases include PACS (Picture Archiving and Communication Systems), VNAs (Vendor Neutral Archives), the open source ORTHANC, and other specialty archives that follow the DICOM standard. At Rigshospitalet we use PACS.

DICOM MCP Implementation

The DICOM MCP server creates a bridge between these two worlds. It:

1. Gives the LLM a crash-course in how to interact with DICOM servers
2. Translates LLM requests into DICOM networking operations
3. Handles the complexity of DICOM queries and retrievals
4. Extracts useful information from DICOM objects

Unlike vector database approaches that may require creating secondary databases of medical information, the MCP approach works directly with existing clinical systems. It teaches and enables the LLM to interact with the same underlying data that clinicians access through their workstations — just through a different interface.

Getting Started

Clone the DICOM-MCP repository and follow the steps in the README, which will guide you through:

• Installing DICOM MCP python dependencies
• Setting up an Orthanc test DICOM server using Docker
• Populating the test server with John Doe sample data
• Configuring integration with Claude Desktop or other LLM clients

The complete implementation is available on GitHub.

Conclusion

DICOM MCP enables large language models to access medical imaging reports directly from clinical systems. Since the MCP approach is LLM-agnostic, healthcare organizations have the flexibility to use the AI models that best fit their requirements, including open-weights models, such as LLaMa , which can be run on premise for maximum data privacy.

Future work includes expanding the DICOM-MCP to cover more DICOM functionalities such as sending DICOM images between DICOM nodes - for instance, sending an abdominal CT image to an AI spleen-segmentation node. This could enable LLMs to answer questions like:

“How has the size of John Doe’s spleen changed over time?”

Disclaimer

This implementation is a research project demonstrating what’s possible with MCP and DICOM — it should not be connected with live patient databases or used with real patient data.

References

Model Context Protocol. https://modelcontextprotocol.io/

DICOM Standard. https://www.dicomstandard.org/

DICOM MCP. https://github.com/ChristianHinge/dicom-mcp