Ollama OpenAI Bridge

ollama-oai-bridge is a Spring Boot-based proxy and translation layer designed to bridge the gap between OpenAI-compatible clients and Ollama (or other Llama-based) servers.

Overview

The ollama-oai-bridge is specifically designed to work with a llama-server in the downstream. It simulates Ollama's model discovery endpoints while using OpenAI-compatible endpoints for the actual LLM requests.

This setup provides an easy way to use local models with VS Code GitHub Copilot BYOK (Bring Your Own Key) agents. By using llama.cpp as the upstream server, it leverages superior model and device support compared to Ollama, while still being compatible with tools that expect the Ollama API for model discovery.

Key Features

OpenAI-to-Ollama Discovery: Translates model lists and model details into the format expected by Ollama clients.
Model Metadata Normalization: Automatically infers model families (Llama, Gemma, Mistral, etc.), quantization levels, and capabilities (Vision, Tools, Thinking) from model names and metadata.
Dynamic Modelfile Generation: Can generate Ollama-compatible Modelfile content for any model served by the upstream.
Request Caching: Includes a caching filter to allow inspection of request bodies without consuming the input stream.
Configurable Upstream: Easily point the bridge to any upstream LLM server via configuration.

Configuration

The application is configured via src/main/resources/application.properties.

Property	Default	Description
`server.port`	`11434`	The port on which the bridge runs.
`bridge.upstream-base-url`	`http://localhost:5001`	The base URL of the upstream LLM server.
`bridge.version`	`0.24.0`	The ollama version to report.

API Endpoints

Discovery API (Ollama Compatible)

GET /api/version: Returns the bridge version.
GET /api/tags: Returns a list of available models.
POST /api/show: Returns detailed information about a specific model.

Proxying

All other requests are proxied to the bridge.upstream-base-url.

Getting Started

License

This project is licensed under the Apache License 2.0. See the LICENSE file for details.

Prerequisites

Java 25
Maven

Building and Running

Clone the repository:

git clone <repository-url>
cd ollama-oai-bridge

Build the project:
```
./mvnw clean package
```
Run the application: You can run it using Maven:
```
./mvnw spring-boot:run
```
Or, after building, you can use the provided start.sh script. Copy the generated .jar file from the target/ directory and start.sh into the same directory, then run:
```
./start.sh
```

Architecture

GatewayProxyConfiguration: Sets up the Spring Cloud Gateway routes to proxy requests.
LlamaServerClient: Handles low-level communication with the upstream server and normalizes model data.
OllamaDiscoveryService: Implements the logic for translating upstream data into Ollama-compatible JSON responses.
OllamaDiscoveryController: Exposes the discovery endpoints.
RequestBodyCachingFilter: Ensures request bodies can be read multiple times during the request lifecycle.

3.3 KiB Raw Blame History