Deploy a model. Use it. Stop when you're done.

One dedicated key for your application

Each optimized endpoint can have one dedicated key (vbs_…). Embed it in your application: the same key starts the endpoint, runs inference and stops it, and the URL and key stay the same every time. For example, your application calls Start when a user signs in, sends requests while they work and calls Stop when they are done. The dedicated key cannot change limits, price or keys, delete the endpoint or launch anything else. Dedicated keys are for optimized endpoints; for a custom model, contact us.

1. Get the key once

Create it from an optimized endpoint on the Endpoints page, or with your platform key. GET /console/api/services returns capabilities.canCreate. Choose startup behavior before quoting: for private_only quote rapid_coldboot: false; for qualified_bridge quote rapid_coldboot: true and approve the returned startup provider and capabilities.

POST /console/api/deployments/plan
Authorization: Bearer $INFEROPTIMAL_API_KEY
Content-Type: application/json

{
  "managed_endpoint": "OFFERING_FROM_CATALOG",
  "rapid_coldboot": false,
  "max_runtime_hours": 2,
  "max_spend_usd": 21.60
}

Review the price, startup behavior and blockers, then create the dedicated key with that plan and the same selector. Startup and spending options belong in policy, not selector.

POST /console/api/services
Authorization: Bearer $INFEROPTIMAL_API_KEY
Content-Type: application/json

{
  "name": "My application",
  "plan_id": "plan_FROM_QUOTE",
  "selector": {"managed_endpoint": "OFFERING_FROM_CATALOG"},
  "idempotency_key": "persisted-service-creation-id",
  "policy": {
    "startup": "private_only",
    "maxHourlyUsd": 10.80,
    "maxSpendUsd": 21.60,
    "totalBudgetUsd": 100,
    "maxRuntimeHours": 2,
    "maxConcurrentRequests": 4,
    "idleSeconds": null
  }
}

Use your quoted selector, price and supported concurrency; these example amounts are not an offer. Save the returned service_url, base_url, model and api_key in your application's secrets. The key is returned at creation and on identical creation retries for ten minutes; it is not included in listing or inspection. Creation starts no hardware. An endpoint is never stopped for idling by default: omit idleSeconds or send null, and its owner gets one warning email after 30 idle minutes. To opt in to stopping on idle, set a whole number of seconds from 120. Idle time is billed at the hourly rate, and active requests reset the idle clock.

2. Start

Read the endpoint's state with its dedicated key, then start it with the generation and admission.epoch you read.

curl "https://www.inferoptimal.com/services/svc_YOUR_ENDPOINT" \
  -H "Authorization: Bearer $INFEROPTIMAL_DEDICATED_KEY"

It returns state (stopped, starting, ready, stopping or attention_required), generation, admission.epoch, accepts_requests, idle_seconds, idle (for example "summary": "Idle for 41 minutes at $2.00/h", or null while serving or stopped) and the URLs.

curl -X POST "https://www.inferoptimal.com/services/svc_YOUR_ENDPOINT/start" \
  -H "Authorization: Bearer $INFEROPTIMAL_DEDICATED_KEY" \
  -H "Content-Type: application/json" \
  -d '{"idempotency_key":"user-login-7f3a9c","expected_generation":3,"expected_epoch":2,"expires_at":1792420000000}'

expires_at is Unix milliseconds, at most five minutes ahead. Save the whole body before sending and retry it unchanged after a lost response. Start is idempotent and deduplicated: a retried body and simultaneous Starts share one allocation. A Start prepared before a Stop returns 409 service_start_stale; read the state again. Right after a Stop, Start can return 409 service_cleanup_pending with Retry-After while the previous hardware is released. Start keeps the endpoint's hourly ceiling, per-start cap, total authorization, daily start limit and your balance hold, and returns 402 or 429 when one is reached. When state is attention_required, Start with the key returns the reason; fix it, then press Start in the console. Private prewarm is complete when an unstopped replica has phase: "ready" and inference_backend: "private"; state: "ready" alone can mean startup-bridge availability.

3. Send requests

curl "https://www.inferoptimal.com/services/svc_YOUR_ENDPOINT/v1/chat/completions" \
  -H "Authorization: Bearer $INFEROPTIMAL_DEDICATED_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"svc_YOUR_ENDPOINT","messages":[{"role":"user","content":"Hello"}]}'

Any OpenAI-compatible client works with base_url and the endpoint ID as model.

Supported api_modes identify /chat/completions for text/vision-language or /embeddings for synchronous embedding calls. This release uses one private replica; embedding services admit one active request, which may contain a batch. Asynchronous media jobs, private repository credentials and custom containers use the explicit deployment API.

Keep the inference HTTP connection open: validated requests wait in order for startup or an available concurrency slot. Waiting and inference share policy.requestTimeoutSeconds (default 1800, maximum 3600); X-InferOptimal-Request-Timeout-Ms can set a shorter total deadline. There are 256 waiting places by default, increasing to four times the concurrency limit with a ceiling of 1024. A full waiting queue returns 429 service_queue_full with Retry-After: 5; expiration before dispatch returns 504 service_request_timeout. Input validation also has a bounded memory allowance: large concurrent bodies may wait unread before joining the service queue. Bodies are limited to 8 MiB; validation overload returns 429 service_payload_capacity. FIFO order starts after validation; the oldest requests can fill available slots in parallel. The queue stores metadata only: a disconnected HTTP request is not a saved job. Your client or network may have an earlier timeout. No inference is automatically repeated after execution may have begun. If an approved startup provider explicitly refuses a request before execution, the same connection can keep waiting for its private GPU within the original deadline.

A dedicated key does not remove cold-start latency. Choose qualified_bridge at creation only after approving a quote that offers startup inference and its provider/data handling; no fallback model is selected silently. The serviceChat helper in inferoptimal.mjs also understands explicit retry-safe 503 service_starting responses, without retrying ambiguous inference failures.

4. Stop

curl -X POST "https://www.inferoptimal.com/services/svc_YOUR_ENDPOINT/stop" \
  -H "Authorization: Bearer $INFEROPTIMAL_DEDICATED_KEY"

Stop cancels waiting requests, lets active ones finish, then releases the hardware and ends billing. The URL and key stay the same, and requests return 410 until the next Start. An endpoint is never stopped for idling unless you set idleSeconds: after 30 idle minutes its owner gets one warning email with the hourly cost and a Stop link, and using it again re-arms the warning. Stopping an endpoint that is already stopped changes nothing.

LatentSync 1.6: batch video + audio → MP4

Quote model: "ByteDance/LatentSync-1.6", then launch and retrieve the connection and inference key using the lifecycle above. This prepared endpoint runs on InferOptimal cloud. Use its host as ENDPOINT, without /v1; media jobs use file uploads rather than Chat Completions.

Already launched? Reuse that endpoint and its inference key. The optional workload-sizing form is not needed to submit a job. Upload the actual files as multipart fields named video and audio; file paths, URLs or base64 inside JSON are not accepted. Large-file JSON requests use completed upload IDs instead. Let your HTTP client set the multipart Content-Type and boundary.

curl "$ENDPOINT/jobs" \
  -H "Authorization: Bearer $INFEROPTIMAL_ENDPOINT_KEY" \
  -H "Idempotency-Key: customer-job-001" \
  -F video=@face.mp4 -F audio=@speech.wav \
  -F steps=20 -F guidance=1.5

The 202 response includes id, status: "queued" and status_url. Save the returned ID as JOB_ID. Reuse the idempotency key with identical inputs when retrying this submission; different inputs with the same key return 409. Use a new key for a different job.

# Poll until status is "completed"; stop on "failed" or "cancelled".
curl "$ENDPOINT/jobs/$JOB_ID" \
  -H "Authorization: Bearer $INFEROPTIMAL_ENDPOINT_KEY"

# Download the result after completion.
curl -f "$ENDPOINT/jobs/$JOB_ID/result" \
  -H "Authorization: Bearer $INFEROPTIMAL_ENDPOINT_KEY" \
  -o lipsynced.mp4

New LatentSync 1.6 endpoints process video in bounded windows, with no fixed clip-duration limit. Multipart uploads allow 32 MiB per input; the client below automatically uses chunked uploads for larger files, up to 2 GiB per input. Video dimensions must not exceed 1920 pixels on either side. Leave enough time in your accepted endpoint lease for the job to finish; runtime and spending limits still apply. Read /runtime for your endpoint’s input and parameter requirements. The endpoint accepts one running job and eight queued jobs; additional submissions return 429. Existing LatentSync 1.5 deployments and its model selector remain supported.

import { submitMediaJob } from './media-jobs.mjs';
const job = await submitMediaJob({
  baseUrl: process.env.ENDPOINT,
  apiKey: process.env.INFEROPTIMAL_ENDPOINT_KEY,
  videoPath: './face.mp4', audioPath: './speech.wav',
  idempotencyKey: 'customer-job-001',
  parameters: { steps: 20, guidance: 1.5 }
});
console.log('Save this job ID:', job.id);
const status = await job.wait({ timeoutMs: 60 * 60 * 1000 });
if (status.status !== 'completed') throw new Error(status.status);
await job.download('./lipsynced.mp4');

For larger batches, keep excess jobs in your client queue; on 429, wait for a submitted job to finish, or retry with exponential backoff, keeping the same job key and inputs. Download each completed result promptly: it expires one hour after that job finishes, and stopping the endpoint removes access to all results.

# When all jobs are finished, stop the deployment using your platform key.
curl -X POST \
  "https://www.inferoptimal.com/console/api/deployments/$DEPLOYMENT_ID/stop" \
  -H "Authorization: Bearer $INFEROPTIMAL_API_KEY" \
  -H "Content-Type: application/json" -d '{}'