# Ollama OpenAI Bridge

`ollama-oai-bridge` is a Spring Boot-based proxy and translation layer designed to bridge the gap between OpenAI-compatible clients and Ollama (or other Llama-based) servers.

## Overview

The `ollama-oai-bridge` is specifically designed to work with a `llama-server` in the downstream. It simulates Ollama's model discovery endpoints while using OpenAI-compatible endpoints for the actual LLM requests.

This setup provides an easy way to use local models with VS Code GitHub Copilot BYOK (Bring Your Own Key) agents. By using `llama.cpp` as the upstream server, it leverages superior model and device support compared to Ollama, while still being compatible with tools that expect the Ollama API for model discovery.

## Key Features

- **OpenAI-to-Ollama Discovery**: Translates model lists and model details into the format expected by Ollama clients.
- **Model Metadata Normalization**: Automatically infers model families (Llama, Gemma, Mistral, etc.), quantization levels, and capabilities (Vision, Tools, Thinking) from model names and metadata.
- **Dynamic Modelfile Generation**: Can generate Ollama-compatible `Modelfile` content for any model served by the upstream.
- **Request Caching**: Includes a caching filter to allow inspection of request bodies without consuming the input stream.
- **Configurable Upstream**: Easily point the bridge to any upstream LLM server via configuration.

## Configuration

The application is configured via `src/main/resources/application.properties`.

| Property | Default | Description |
|----------|---------|-------------|
| `server.port` | `11434` | The port on which the bridge runs. |
| `bridge.upstream-base-url` | `http://localhost:5001` | The base URL of the upstream LLM server. |
| `bridge.version` | `0.24.0` | The ollama version to report. |

## API Endpoints

### Discovery API (Ollama Compatible)

- `GET /api/version`: Returns the bridge version.
- `GET /api/tags`: Returns a list of available models.
- `POST /api/show`: Returns detailed information about a specific model.

### Proxying

- All other requests are proxied to the `bridge.upstream-base-url`.

## Getting Started

### Prerequisites

- Java 25
- Maven

### Building and Running

1.  **Clone the repository**:
    ```bash
    git clone <repository-url>
    cd ollama-oai-bridge
    ```

2.  **Build the project**:
    ```bash
    ./mvnw clean package
    ```

3.  **Run the application**:
    You can run it using Maven:
    ```bash
    ./mvnw spring-boot:run
    ```

    Or, after building, you can use the provided `start.sh` script. Copy the generated `.jar` file from the `target/` directory and `start.sh` into the same directory, then run:
    ```bash
    ./start.sh
    ```

## Architecture

- **`GatewayProxyConfiguration`**: Sets up the Spring Cloud Gateway routes to proxy requests.
- **`LlamaServerClient`**: Handles low-level communication with the upstream server and normalizes model data.
- **`OllamaDiscoveryService`**: Implements the logic for translating upstream data into Ollama-compatible JSON responses.
- **`OllamaDiscoveryController`**: Exposes the discovery endpoints.
- **`RequestBodyCachingFilter`**: Ensures request bodies can be read multiple times during the request lifecycle.