OpenAI-Compatible API: How to Switch an App to Open-Source Models in 10 Minutes

If your app already talks to the OpenAI SDK, switching to open-weight models mostly means changing three values: the base URL, the API key and the model name. The rest of your code, including the way you build messages and read responses, can stay as it is.

Nebula Block's Inference API follows the OpenAI format, so any OpenAI SDK works against https://inference.nebulablock.com/v1. This guide walks through the switch step by step, then covers what to test before you move real traffic. Model names and limits are as of November 2026; the catalog changes often.

Before you start

You need three things, and they take a couple of minutes:

1. An account with credit. Sign up, then add at least $5. Free accounts have a daily request cap of 0 on most models, and a $5 deposit moves you to Tier 2, which opens the catalog.
2. An API key. Create one under API Keys in the console and store it as an environment variable, not in your code:

export NEBULA_API_KEY="sk-..."

3. A model ID from the Model Catalog. More on choosing one below.

Step 1: change three values

Test the endpoint with cURL first. If this works, your key, credit and model ID are all fine:

curl https://inference.nebulablock.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $NEBULA_API_KEY" \
  -d '{
    "model": "deepseek-ai/DeepSeek-V3.2",
    "messages": [{"role": "user", "content": "Explain what an inference endpoint is, in two sentences."}]
  }'

Then in Python, point the SDK at the new base URL. Reading the values from environment variables means you can switch providers, or switch back, without a code change. Set them like this:

export LLM_BASE_URL="https://inference.nebulablock.com/v1"
export LLM_API_KEY="$NEBULA_API_KEY"
export LLM_MODEL="deepseek-ai/DeepSeek-V3.2"

Then read them in your code:

import os
from openai import OpenAI

client = OpenAI(
    base_url=os.environ["LLM_BASE_URL"],   # https://inference.nebulablock.com/v1
    api_key=os.environ["LLM_API_KEY"],     # your Nebula Block key
)
MODEL = os.environ["LLM_MODEL"]            # e.g. deepseek-ai/DeepSeek-V3.2

response = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": "Explain what an inference endpoint is, in two sentences."}],
)
print(response.choices[0].message.content)

In JavaScript it's the same idea:

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: process.env.LLM_BASE_URL,
  apiKey: process.env.LLM_API_KEY,
});

const response = await client.chat.completions.create({
  model: process.env.LLM_MODEL,
  messages: [{ role: "user", content: "Explain what an inference endpoint is, in two sentences." }],
});
console.log(response.choices[0].message.content);

The JavaScript example uses top-level await, so run it as an ES module (for example, in a .mjs file).

That's the whole switch. The Python and JavaScript examples follow the Quickstart in the docs.

Step 2: pick a model

Nebula Block serves dozens of text and multimodal models through the same endpoint. A few open-weight options from the catalog, with their model IDs and context lengths:

Don't hard-code a list from a blog post, including this one. Call the live list from your own account:

for model in client.models.list():
    print(model.id)

The List Models endpoint returns what's available right now.

Step 3: streaming and token usage

If your app streams responses, set stream=True as you would with any OpenAI-compatible API:

stream = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": "Write a haiku about GPUs."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

Non-streaming responses include a usage object with prompt and completion token counts. Log it from day one, because it's how you'll compare cost before and after the switch.

Step 4: handle rate limits

Nebula Block applies limits per tier: requests per minute, tokens per minute and requests per day, plus a daily cap for each model. A 429 can mean different things, so read the message:

• Rate limit exceeded (per minute): retry with exponential backoff.
• Daily limit reached: waiting a few seconds won't help. Upgrade your tier, switch models, or wait for the reset at 00:00 UTC.

Both cases come back as a 429, so a small wrapper should retry the first and stop immediately on the second:

import time
from openai import RateLimitError

def chat(messages, retries=4):
    for attempt in range(retries):
        try:
            return client.chat.completions.create(model=MODEL, messages=messages)
        except RateLimitError as e:
            if "daily" in str(e).lower():
                raise  # waiting won't help: upgrade, switch models or wait for the reset
            time.sleep(2 ** attempt)
    raise RuntimeError("Still rate limited. Check your tier in the console.")

The Tiers and Rate Limits page lists the numbers for each tier.

Step 5: test before you move real traffic

The switch takes minutes. Making sure the new model is good enough for your app is the part worth spending time on. Before you send production traffic:

1. Build a small test set of real prompts, including the awkward ones, and save the current model's answers.
2. Run the same prompts through the new model and compare quality, format and latency.
3. Check the details that often differ between models: context length, whether tool calling and structured output are supported (the catalog mentions this for many models, so check the one you pick), how the model follows your system prompt, and how it formats JSON.
4. Compare cost per request using the usage counts. Different models use different tokenizers, so the same text can be a different number of tokens.
5. Move a small share of traffic first, then increase it as the results hold up.

One more point if you're switching for data-location reasons: keep your fallback inside the same boundary. A fallback that quietly sends requests back to the old provider defeats the purpose. Models marked with a Canadian flag in the catalog are hosted in Canada, and the data policy describes how Nebula Block handles customer data.

Common snags

• A 429 on a model that works for someone else. Daily caps are set per model and per tier, so check the Limits page in the console.
• A 402 response. Your credit balance can't cover the request. Add credit.
• Model not found. IDs include the provider prefix, like deepseek-ai/DeepSeek-V3.2. List the models and copy the ID exactly.
• Answers read differently. Different models have different defaults. Adjust the system prompt and temperature, and rerun your test set.

The bottom line

Switching an OpenAI-compatible app is a small change: base URL, key, model. The ten minutes cover the technical switch. Reserve the rest of your time for a proper comparison on your own prompts, because that's what tells you whether the new model is actually better for your app.

Learn more at

• Email: contact@nebulablock.com
• Website: nebulablock.com
• Docs: docs.nebulablock.com
• Book a call: nebulablock.com/contact