A self-hosted AI chatbot does not have to run its language model on the same server as its interface. Open WebUI provides an open-source chat interface that can run in a Docker container, while an OpenAI-compatible cloud API handles inference. This arrangement is useful for developers who want to control the application, user accounts and deployment without buying a GPU for every experiment. The distinction is important: the chat application is self-hosted, but prompts sent to a cloud model leave the local infrastructure.
This guide builds a working configuration pattern using Open WebUI and an OpenAI-compatible provider, with AI/ ML API as the example. It covers Docker deployment, API connection, model selection, troubleshooting and the security decisions required before inviting other users. Commands and configuration are based on published product documentation; no live deployment or performance benchmark is claimed.
What the Architecture Looks Like
The browser connects to Open WebUI running on a machine or private server. Open WebUI manages the chat interface, accounts, settings and its application data. When a user selects a cloud model, the Open WebUI backend sends an authenticated request to the provider’s API and displays the response. In this setup, Docker hosts the interface; it does not download or execute the cloud model.
This separation offers flexibility. A small server can host the interface while different cloud models handle coding, summarization or general chat. It also creates a clear trust boundary: any text and attachments forwarded to the provider are processed outside the self-hosted environment. Model selection should therefore consider privacy, retention policies and cost as well as response quality.
Prerequisites
A Linux, macOS or Windows machine with Docker and Docker Compose installed is sufficient for the interface. A domain and HTTPS reverse proxy are recommended for any deployment accessed beyond localhost. The cloud connection requires an API account, a valid API key and a text model that supports the OpenAI Chat Completions protocol. No dedicated GPU is required for cloud inference.
Before starting, generate a persistent secret for Open WebUI sessions using `openssl rand -hex 32` and keep the result out of version control. Use a named Docker volume for persistent application data. If the server will be public, plan access controls and TLS before opening the port to the internet.
Step 1: Deploy Open WebUI with Docker Compose
Create a directory for the project and add a `compose.yaml` file. The configuration below binds the application to localhost on port 3000, persists its data and sets a stable session secret. Replace the example secret with a generated value before starting.
services:
open-webui:
image: ghcr.io/open-webui/open-webui:v0.12.4
container_name: open-webui
ports:
– “127.0.0.1:3000:8080”
environment:
WEBUI_SECRET_KEY: “REPLACE_WITH_GENERATED_SECRET”
volumes:
– open-webui:/app/backend/data
restart: unless-stopped
volumes:
open-webui:
From the directory containing the file, run:
docker compose up -d
docker compose logs -f open-webui
Wait until the application reports that startup has completed. Open `http://localhost:3000` in a browser on the same machine and create the initial administrator account. The first account has elevated privileges, so it should use a strong password. The official quick-start documentation also describes Python and other installation options, but Docker Compose makes the deployment easier to reproduce and back up.
For a remote server, keep the application bound to loopback and publish it through an HTTPS reverse proxy such as Caddy or Nginx. Exposing port 3000 directly without TLS is not an appropriate production default. The Compose example pins Open WebUI v0.12.4, a published release, instead of tracking the moving `main` tag. Pinning makes updates intentional; a successful deployment and compatibility with the chosen provider still need to be checked on the target machine.
Step 2: Obtain an API Key and Select a Chat Model
Open WebUI supports OpenAI-compatible endpoints, which makes it possible to connect multiple hosted model providers without replacing the interface. AI/ML API documents the base URL `https://api.aimlapi.com/v1` and a standard chat-completions endpoint. Its model catalogue lists model identifiers and capabilities across different providers.
Before choosing a model, consult the AI/ML API Models catalogue and filter for text/chat models that support `/v1/chat/completions`. The full catalogue includes image, video and other model types, but not every model belongs in a chat selector. Copy the exact model ID from the model’s documentation rather than using its marketing name. Check pricing, context length, rate limits and supported parameters.
Generate an API key in the provider dashboard. Treat it as a server-side credential: do not put it in a browser script, screenshot, public repository or shared Compose file. For an initial test, use Open WebUI’s administrator connection settings to store the key rather than hardcoding it into the article’s sample configuration.
Step 3: Connect Open WebUI to the Cloud API
Sign in as the Open WebUI administrator and open Settings → Admin → Connections. Under Manage OpenAI API Connections, add a new connection. Enter the following values:
Base URL: https://api.aimlapi.com/v1
API Key: <your AI/ML API key>
Provider: Default (OpenAI-compatible)
Model IDs: deepseek/deepseek-v4-pro (enter manually if discovery is unavailable)
Save the connection and use Verify Connection. If automatic model discovery does not populate the selector, enter deepseek/deepseek-v4-pro manually in the Model IDs allowlist and test a chat completion. A failed model-list request does not by itself establish that the chat endpoint is incompatible.
The provider’s complete model catalogue and Open WebUI’s automatic model-discovery request may use different routes. Use the documented manual Model IDs option if the model is not listed. The actual chat-completions request is the decisive compatibility check.
Step 4: Send a First Chat Message
Create a new chat in Open WebUI, select the configured model and send a small prompt such as: `Explain the difference between HTTP and HTTPS in two sentences.` A successful result should appear in the conversation area. Check that the selected model is the intended cloud model and that no unexpected fallback is active.
For independent API diagnosis, the same provider can be queried with curl. First export the API key in the same terminal session (the credential saved in Open WebUI is not automatically available in your shell):
export AIML_API_KEY=’PASTE_YOUR_PRIVATE_AIMLAPI_KEY_HERE’
Then run the request below. The example uses the same model ID, deepseek/deepseek-v4-pro, as the Open WebUI connection. Never commit or share the key.
curl https://api.aimlapi.com/v1/chat/completions
-H “Authorization: Bearer $AIML_API_KEY”
-H “Content-Type: application/json”
-d ‘{
“model”: “deepseek/deepseek-v4-pro”,
“messages”: [
{“role”: “user”, “content”: “Explain HTTP in one sentence.”}
],
“stream”: false
}’
This direct request separates provider problems from Open WebUI configuration problems. If curl succeeds but the chat interface fails, inspect the Open WebUI connection settings and container logs. If both fail, investigate the API key, model identifier, quota and endpoint compatibility first. The request above is a template, not a transcript of a completed test.
Step 5: Configure the Chatbot for Real Users
A chatbot used by a team needs more than a functioning API connection. Keep account registration restricted until access policies are ready. Assign administrative rights sparingly and review which users can create connections, upload files or use tools. If the deployment handles internal documents, establish which categories of information may be sent to the cloud provider.
Open WebUI offers additional features such as retrieval, tools and model-specific options, but they should be enabled deliberately. Retrieval can send selected document excerpts to an external model. Tool calling may permit operations beyond generating text, depending on configuration. Validate each feature with the chosen provider because OpenAI-compatible chat support does not guarantee identical tool-calling, embeddings or multimodal behavior.
For production, put the service behind HTTPS, use backups for the persistent Docker volume, keep credentials out of logs and track provider spending. Update the application on a tested schedule and record the image version used for each deployment. A self-hosted interface offers control over the application layer, but it does not eliminate third-party data processing when cloud inference is enabled.
Common Problems and How to Diagnose Them
The model list is empty
Check the provider connection and the Model IDs allowlist. Enter deepseek/deepseek-v4-pro manually if discovery does not populate the selector. Confirm the API key and the model’s chat-completions support.
Authentication fails with 401 or 403
Check that the API key belongs to the intended provider, has not been revoked and is pasted without extra spaces. Verify the configured base URL. Avoid exposing the key in screenshots or diagnostic logs. Authorization errors can also reflect account restrictions, so consult the provider dashboard if the credential appears correct.
The response hangs or times out
Inspect `docker compose logs -f open-webui`, confirm outbound HTTPS connectivity and retry with a short prompt. Some models take longer to respond than others. Distinguish a browser or reverse-proxy timeout from an error returned by the API itself; increasing every timeout without investigation can conceal reliability problems.
Chat works, but tools or document search do not
OpenAI compatibility is not a blanket guarantee for function calling, embeddings, structured output or provider-specific features. Check the selected model’s documented capabilities and whether Open WebUI needs a separate embedding model for retrieval. Test the basic chat endpoint before adding more complex functionality.
Open WebUI with Local Models vs Cloud Models
Both approaches use the same self-hosted interface but differ in operating requirements. Local inference with Ollama or another runtime can keep generation on infrastructure under the operator’s control, subject to the application’s other integrations. Cloud inference removes the need to host model weights and high-performance accelerators, but introduces external API costs, network dependence and provider data-processing considerations.
A hybrid installation can offer both. Routine internal tasks might use a local model, while demanding requests use a cloud endpoint. That choice should be based on documented privacy requirements, acceptable latency, model quality and the total cost of ownership rather than a blanket claim that one option is always safer or cheaper.
Final Thoughts
Open WebUI makes it practical to self-host the chat application while connecting to hosted language models through an OpenAI-compatible interface. Docker Compose handles the application deployment, and a configurable provider connection keeps model selection separate from the UI. The most important implementation details are a valid model ID, a compatible endpoint, secure credential storage and a clear understanding of where prompts are processed. Start with one text model and a simple chat test, then expand to additional models and features only after the baseline connection is reliable.
