Ollama OpenAI Bridge

ollama-oai-bridge is a Spring Boot-based proxy and translation layer designed to bridge the gap between OpenAI-compatible clients and Ollama (or other Llama-based) servers.

Overview

The ollama-oai-bridge is specifically designed to work with a llama-server in the downstream. It simulates Ollama's model discovery endpoints while using OpenAI-compatible endpoints for the actual LLM requests.

This setup provides an easy way to use local models with VS Code GitHub Copilot BYOK (Bring Your Own Key) agents. By using llama.cpp as the upstream server, it leverages superior model and device support compared to Ollama, while still being compatible with tools that expect the Ollama API for model discovery.

Key Features

OpenAI-to-Ollama Discovery: Translates model lists and model details into the format expected by Ollama clients.
Model Metadata Normalization: Automatically infers model families (Llama, Gemma, Mistral, etc.), quantization levels, and capabilities (Vision, Tools, Thinking) from model names and metadata.
Dynamic Modelfile Generation: Can generate Ollama-compatible Modelfile content for any model served by the upstream.
Request Caching: Includes a caching filter to allow inspection of request bodies without consuming the input stream.
Configurable Upstream: Easily point the bridge to any upstream LLM server via configuration.

Configuration

The application is configured via src/main/resources/application.properties.

Property	Default	Description
`server.port`	`11434`	The port on which the bridge runs.
`bridge.upstream-base-url`	`http://localhost:5001`	The base URL of the upstream LLM server.
`bridge.version`	`0.24.0`	The ollama version to report.
`bridge.reported-context-offset`	`0`	Tokens to subtract from the downstream context length before reporting it to clients.

API Endpoints

Discovery API (Ollama Compatible)

GET /api/version: Returns the bridge version.
GET /api/tags: Returns a list of available models.
POST /api/show: Returns detailed information about a specific model.

Proxying

All other requests are proxied to the bridge.upstream-base-url.

Getting Started

License

This project is licensed under the Apache License 2.0. See the LICENSE file for details.

Prerequisites

Java 25
Maven

Building and Running

Clone the repository:

git clone <repository-url>
cd ollama-oai-bridge

Build the project:
```
./mvnw clean package
```
Run the application: You can run it using Maven:
```
./mvnw spring-boot:run
```
You can also override any of the bridge settings with JVM system properties, for example:
```
java -Dserver.port=11435 -Dbridge.upstream-base-url=http://localhost:5002 -Dbridge.version=0.24.1 -Dbridge.reported-context-offset=5000 -jar target/ollama-oai-bridge.jar
```
If you use VS Code GitHub Copilot BYOK, configure the Ollama provider to point at the bridge endpoint and keep the upstream server behind it.

Or, after building, you can use the provided start.sh script. Copy the generated .jar file from the target/ directory and start.sh into the same directory, then run:
```
./start.sh
```
The start.sh launcher also accepts JVM -D options directly, so this works too:
```
./start.sh -Dserver.port=11435 -Dbridge.upstream-base-url=http://localhost:5002 -Dbridge.version=0.24.1 -Dbridge.reported-context-offset=5000
```

Docker

You can build and run the application as a Docker image:

cd docker
./build.sh
./start-docker.sh

The build.sh script builds a multi-stage Docker image that compiles the JAR inside Docker and packages it into a minimal runtime image. The start-docker.sh script runs the container with host networking.

Architecture

GatewayProxyConfiguration: Sets up the Spring Cloud Gateway routes to proxy requests.
LlamaServerClient: Handles low-level communication with the upstream server and normalizes model data.
OllamaDiscoveryService: Implements the logic for translating upstream data into Ollama-compatible JSON responses.
OllamaDiscoveryController: Exposes the discovery endpoints.
RequestBodyCachingFilter: Ensures request bodies can be read multiple times during the request lifecycle.

4.4 KiB Raw Permalink Blame History