Quickstart

1. Launch an endpoint

In the console

1. Sign in at inferoptimal.com/console and add credit on Billing.
2. Choose a prepared model in the catalog, or a supported Hugging Face model under Models. Check the quoted price, then launch.
3. When the endpoint is ready, copy its base URL, model name and key from the endpoint page. The key is shown once: save it.

With the REST API

Create a full-access platform key (vbc_…) in API keys. It manages endpoints; it never goes into your app. These commands need curl and jq.

export INFEROPTIMAL_API_KEY="vbc_..."
API=https://www.inferoptimal.com/console/api/deployments
AUTH="Authorization: Bearer $INFEROPTIMAL_API_KEY"

# Quote: free, starts nothing. Read the price and any blockers.
curl -sS "$API/plan" -H "$AUTH" -H "Content-Type: application/json" \
  -d '{"model": "Qwen/Qwen3-8B", "max_runtime_hours": 2}' | tee plan.json | jq '{plan_id, price, blockers}'

# Launch the quoted plan.
ID=$(jq -n --arg plan "$(jq -r .plan_id plan.json)" --arg key "quickstart-$(date +%s)" \
  '{plan_id: $plan, idempotency_key: $key,
    usage_plan: {agent_tool: "none", agents: 1, calls: "My app sends chat completions."}}' |
  curl -sS "$API" -H "$AUTH" -H "Content-Type: application/json" -d @- | jq -r .deployment_id)
echo "$ID"

usage_plan says how you will call the endpoint. idempotency_key makes a retried launch return the same endpoint instead of a second one. max_runtime_hours is a ceiling; billing stops when you stop.

Wait until it can answer, then get the connection and the endpoint key:

until [ "$(curl -sS "$API/$ID" -H "$AUTH" | tee status.json | jq -r .inference_ready)" = true ]; do
  jq -r .status status.json
  jq -e '.status | test("failed|terminat")' status.json > /dev/null && break
  sleep 20
done

curl -sS -X POST "$API/$ID/connection" -H "$AUTH" -H "Content-Type: application/json" -d '{}' |
  jq '{base_url, model, api_modes}'
curl -sS -X POST "$API/$ID/key" -H "$AUTH" -H "Content-Type: application/json" \
  -d "{\"idempotency_key\": \"key-$ID\"}" | jq -r .api_key

The endpoint key (vbe_…) is returned once; the same request returns it again for ten minutes. Store it as a secret.

With the command-line tool

Node 20 or newer. launch shows the price and asks before it creates anything, then follows the boot and prints the key once.

curl -fsSL https://www.inferoptimal.com/cli/install.sh | sh
inferoptimal login
inferoptimal hardware ls --model Qwen/Qwen3-8B     # pick an option id from the first column
inferoptimal launch Qwen/Qwen3-8B --on OPTION_ID
inferoptimal use ENDPOINT_ID --with python         # generate Python connection setup
inferoptimal stop ENDPOINT_ID

2. Send your first request

Set three values from the endpoint page or the connection above. Every example on this page reads them.

export OPENAI_BASE_URL="PASTE_BASE_URL"   # ends in /v1
export OPENAI_API_KEY="PASTE_ENDPOINT_KEY"  # vbe_...
export MODEL="PASTE_MODEL"

curl:

curl "$OPENAI_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"model\": \"$MODEL\", \"messages\": [{\"role\": \"user\", \"content\": \"Hello\"}]}"

Python (pip install openai). The client reads OPENAI_BASE_URL and OPENAI_API_KEY itself, so existing OpenAI code only needs the model name changed.

import os
from openai import OpenAI

client = OpenAI()  # reads OPENAI_BASE_URL and OPENAI_API_KEY
reply = client.chat.completions.create(
    model=os.environ["MODEL"],
    messages=[{"role": "user", "content": "Hello"}],
)
print(reply.choices[0].message.content)

TypeScript (npm install openai, save as hello.ts, run npx tsx hello.ts):

import OpenAI from 'openai';

const client = new OpenAI(); // reads OPENAI_BASE_URL and OPENAI_API_KEY
const reply = await client.chat.completions.create({
  model: process.env.MODEL!,
  messages: [{ role: 'user', content: 'Hello' }],
});
console.log(reply.choices[0].message.content);

Pass the key as Authorization: Bearer. GET $OPENAI_BASE_URL/models lists the model name. A 401 means a wrong key; 410 means the endpoint was stopped.

3. Streaming and tool calls

Streaming works as it does with OpenAI. Use it for long answers.

import os
from openai import OpenAI

client = OpenAI()
stream = client.chat.completions.create(
    model=os.environ["MODEL"],
    messages=[{"role": "user", "content": "Write a haiku about GPUs"}],
    stream=True,
)
for chunk in stream:
    if chunk.choices:
        print(chunk.choices[0].delta.content or "", end="", flush=True)
print()

import OpenAI from 'openai';

const client = new OpenAI();
const stream = await client.chat.completions.create({
  model: process.env.MODEL!,
  messages: [{ role: 'user', content: 'Write a haiku about GPUs' }],
  stream: true,
});
for await (const chunk of stream) process.stdout.write(chunk.choices[0]?.delta?.content ?? '');
process.stdout.write('\n');

Tool calls use the OpenAI tools field. The connection's usage.call.tool_calls says whether your endpoint's model supports them.

import json, os
from openai import OpenAI

client = OpenAI()
tools = [{"type": "function", "function": {
    "name": "get_weather",
    "description": "Current weather for a city",
    "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]},
}}]
messages = [{"role": "user", "content": "What is the weather in Paris?"}]
reply = client.chat.completions.create(model=os.environ["MODEL"], messages=messages, tools=tools)
call = reply.choices[0].message.tool_calls[0]
print(call.function.name, json.loads(call.function.arguments))

messages += [reply.choices[0].message,
             {"role": "tool", "tool_call_id": call.id, "content": json.dumps({"temp_c": 18})}]
final = client.chat.completions.create(model=os.environ["MODEL"], messages=messages, tools=tools)
print(final.choices[0].message.content)

Some chat endpoints also serve /completions, /responses or Anthropic's /messages. Use only the interfaces listed in your connection's api_modes; support varies by endpoint. The connection's non_streaming_carried says whether a long non-streaming answer is carried past the proxy timeout; where it is false, stream long generations.

4. Claude Code, Codex and Cursor

Connect the InferOptimal MCP server to manage endpoints from a compatible coding agent. Your main coding model stays unchanged.

To delegate work to your dedicated endpoint's model, ask your agent to call the MCP query actions get_connection and get_codex_config with your service_id. The latter returns setup commands for supported clients after checking the endpoint's tool-calling capabilities.

1. Run the returned installer on the machine containing your project. It needs Node 20 or newer and your saved dedicated key.
2. Reload MCP tools if your client requires it. Confirm the InferOptimal agent tool appears.
3. Start the endpoint and delegate a small task that reads a project file, uses a tool and returns a result.

This installs InferOptimal's Chat Completions worker as a tool. It does not make every model a native replacement for Claude Code, Codex or Cursor. Direct model configuration requires an endpoint and client combination with the required API and tool behavior; use the endpoint's returned configuration.

5. LangChain and LlamaIndex

LangChain (pip install langchain-openai):

import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model=os.environ["MODEL"],
    base_url=os.environ["OPENAI_BASE_URL"],
    api_key=os.environ["OPENAI_API_KEY"],
)
print(llm.invoke("Hello").content)

LlamaIndex (pip install llama-index-llms-openai-like):

import os
from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model=os.environ["MODEL"],
    api_base=os.environ["OPENAI_BASE_URL"],
    api_key=os.environ["OPENAI_API_KEY"],
    is_chat_model=True,
)
print(llm.complete("Hello"))

6. Embeddings

Launch the Document search endpoint (BGE-M3) from the catalog, or quote it with the API:

curl -sS "$API/plan" -H "$AUTH" -H "Content-Type: application/json" \
  -d '{"managed_endpoint": "bge-m3-document-search", "max_runtime_hours": 2}' | tee plan.json | jq '{plan_id, price, blockers}'

Launch it as in step 1, then set the three values from its connection and call /embeddings:

import os
from openai import OpenAI

client = OpenAI()
result = client.embeddings.create(
    model=os.environ["MODEL"],
    input=["InferOptimal runs models on dedicated GPUs.", "Stop the endpoint when you are done."],
)
print(len(result.data), len(result.data[0].embedding))

7. Speech to text

Launch a Whisper model (for example openai/whisper-large-v3-turbo under Models, or "model": "openai/whisper-large-v3-turbo" in the quote). The endpoint speaks the OpenAI audio API:

import os
from openai import OpenAI

client = OpenAI()
with open("recording.wav", "rb") as audio:
    text = client.audio.transcriptions.create(model=os.environ["MODEL"], file=audio).text
print(text)

Direct uploads are limited to 100 MB. For recordings up to 512 MiB, upload to storage first and transcribe by reference:

# 1. Ask for an upload URL for this exact file size.
curl -sS "$OPENAI_BASE_URL/audio/uploads" -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" -d "{\"bytes\": $(wc -c < meeting.wav)}" | tee upload.json

# 2. Upload the file straight to storage.
curl -sS -X PUT --upload-file meeting.wav "$(jq -r .upload_url upload.json)"

# 3. Transcribe it by its upload_id.
curl -sS "$OPENAI_BASE_URL/audio/transcriptions" -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"upload_id\": \"$(jq -r .upload_id upload.json)\", \"model\": \"$MODEL\"}"

Limits, chunking and audio metering are in the API reference.

8. Manage endpoints from a coding agent (MCP)

Connect the hosted MCP server and your agent can quote, launch, connect and stop endpoints for you. You sign in once in the browser; no platform key is needed.

Claude Code:

claude mcp add --transport http --scope user inferoptimal https://www.inferoptimal.com/mcp
claude mcp login inferoptimal

Codex:

codex mcp add inferoptimal --url https://www.inferoptimal.com/mcp
codex mcp login inferoptimal

Cursor: add this to ~/.cursor/mcp.json, then sign in when Cursor asks.

{"mcpServers":{"inferoptimal":{"url":"https://www.inferoptimal.com/mcp"}}}

Any other MCP client:

{"mcpServers":{"inferoptimal":{"type":"http","url":"https://www.inferoptimal.com/mcp"}}}

Then ask your agent to launch a model. The agent guide has the full tool list.

The agent tools on Tools run ~/.inferoptimal/agents.mjs. Get it once (Node 20 or newer):

mkdir -p ~/.inferoptimal && curl -fsSL https://www.inferoptimal.com/agents/inferoptimal-agents.mjs -o ~/.inferoptimal/agents.mjs

Or ask your agent to call get_codex_config over MCP. It returns the same download and install in one paste.

9. Stop and billing

An endpoint bills for the time it runs, from your credit balance, until you stop it. The quote's price.billing_increment says how, for example by the minute once live. It is never stopped for idling unless you ask: after 30 idle minutes you get one email with what it costs and a Stop link.

Stop it with the Stop button on its page, from your agent, or with the API:

curl -sS -X POST "$API/$ID/stop" -H "$AUTH" -H "Content-Type: application/json" -d '{}' |
  jq '{status, cost, cleanup}'

Billing stops when Stop is accepted (cost.metering_stopped: true). The hardware is released in the background; cleanup.provider_confirmed turns true when it is gone. The key stops working.

To stop automatically, set "idle_stop_minutes" (2 or more) in the quote. max_runtime_hours and max_spend_usd also end it.

Your balance is on Billing, where you add credit. With a platform key, GET https://www.inferoptimal.com/console/api/billing returns it in nano-dollars (availableNanos) along with each endpoint's charges.