Authentication

All API requests require an API key. Create one from the API Keys page.

Pass your key using either header:

Authorization: Bearer YOUR_API_KEY
# or
X-Api-Key: YOUR_API_KEY

Credits

Paid-provider chat completions require a positive prepaid balance. Usage is deducted after a successful response. Free-tier models keep working at $0. GET https://api.fastinfra.ai/v1/models is not gated.

When the wallet is empty, paid chat requests return HTTP 402:

HTTP/1.1 402 Payment Required

{
  "error": {
    "message": "Insufficient API credits. Add funds on the Billing page, or use a free-tier model.",
    "type": "insufficient_quota",
    "code": "insufficient_quota"
  }
}

Rate Limits

Chat completion requests are rate limited to protect GPU capacity and keep latency stable for all users. Production integrations must handle 429 Too Many Requests and honor the Retry-After response header.

Current limits

What is limited

• Rate limited: POST https://api.fastinfra.ai/v1/chat/completions (streaming and non-streaming)
• Not rate limited: GET https://api.fastinfra.ai/v1/models, account pages, and other non-chat routes

429 responses

Two conditions return HTTP 429 with Retry-After: 60 and type: "rate_limit_error":

Per-minute limit exceeded — too many chat requests on one key within 60 seconds:

HTTP/1.1 429 Too Many Requests
Retry-After: 60

{
  "error": {
    "message": "Rate limit exceeded. Maximum 120 requests per minute per API key.",
    "type": "rate_limit_error"
  }
}

Concurrent limit exceeded — too many in-flight chat requests on one key, or server-wide capacity is full:

HTTP/1.1 429 Too Many Requests
Retry-After: 60

{
  "error": {
    "message": "Too many concurrent requests. Wait for in-flight requests to finish before retrying.",
    "type": "rate_limit_error"
  }
}

Integration best practices

• On 429, wait at least the number of seconds in Retry-After (typically 60) before retrying.
• Avoid firing many parallel chat calls on the same API key; queue or serialize when possible.
• Use "stream": true for long generations so a single slot is held for the full response.
• Batch related prompts where your app allows, instead of many tiny back-to-back calls.

Retry example (Python)

import time
from openai import OpenAI

client = OpenAI(api_key="YOUR_API_KEY", base_url="https://api.fastinfra.ai/v1")

def chat_with_retry(messages, max_retries=3):
    for attempt in range(max_retries):
        try:
            return client.chat.completions.create(
                model="llama3.1:8b",
                messages=messages,
            )
        except Exception as exc:
            if getattr(exc, "status_code", None) != 429 or attempt == max_retries - 1:
                raise
            retry_after = int(getattr(exc, "response", {}).headers.get("Retry-After", 60))
            time.sleep(retry_after)

Chat Completions

Create a chat completion using the OpenAI-compatible endpoint. Pass any model ID from GET https://api.fastinfra.ai/v1/models or the pricing catalog.

Request body

{
  "model": "llama3.1:8b",
  "messages": [
    { "role": "system", "content": "You are a helpful assistant." },
    { "role": "user", "content": "Hello!" }
  ],
  "temperature": 0.7
}

Example (local / free-tier model)

curl https://api.fastinfra.ai/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3.1:8b",
    "messages": [{"role": "user", "content": "Summarize quantum computing in one sentence."}]
  }'

Example (provider-qualified model)

Append :provider to pin a specific upstream. See Provider Models.

curl https://api.fastinfra.ai/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "all-minilm:latest",
    "messages": [{"role": "user", "content": "Explain provider routing in one sentence."}]
  }'

Response

{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "created": 1234567890,
  "model": "llama3.1:8b",
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "Quantum computing uses quantum bits..."
    },
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 12,
    "completion_tokens": 24,
    "total_tokens": 36
  }
}

Provider Models

FastInfra aggregates models from multiple wholesale inference providers behind one OpenAI-compatible API. Use plain model IDs for automatic routing, or append :provider to pin a specific upstream.

Discover models

• GET https://api.fastinfra.ai/v1/models — full catalog with id and owned_by (provider key)
• GET https://api.fastinfra.ai/v1/models/count — total model count
• Pricing page — searchable catalog with per-token pricing

Model ID formats

Supported provider keys

Use these keys after the colon in a provider-qualified model ID:

Local Ollama models use provider key ollama and do not require a :provider suffix.

Routing behavior

When you send a plain model ID (no :provider suffix), the gateway resolves the provider chain in this order:

1. Free-tier models — routed to local Ollama when listed as free tier
2. Pricing catalog — cheapest enabled provider for that model
3. Configured routes — per-model primary and fallback chain
4. First enabled provider — any available wholesale provider
5. Ollama fallback — local inference when available

When you include :provider, routing skips automatic selection and uses only that provider. Use this when you need a specific upstream API or model variant.

When to pin a provider

• Omit :provider — let the gateway optimize for cost and availability (recommended for most apps)
• Include :provider — when you need a specific provider's latency, features, or model version

Request example

curl https://api.fastinfra.ai/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "all-minilm:latest",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Python example

from openai import OpenAI

client = OpenAI(api_key="YOUR_API_KEY", base_url="https://api.fastinfra.ai/v1")

response = client.chat.completions.create(
    model="all-minilm:latest",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)

Errors

• 502 — No inference provider available for the requested model. The model is not in the catalog or no provider is enabled for it.
• 503 — Inference provider is not reachable. Upstream providers are offline or misconfigured on this deployment.

If a model does not appear in GET https://api.fastinfra.ai/v1/models, it may not be enabled on this platform yet. Browse the pricing catalog for models available on this deployment.

List Models

Returns the aggregated model catalog from all enabled providers on this deployment. Each entry includes an id and owned_by provider key.

Response shape

{
  "object": "list",
  "data": [
    { "id": "anthropic/claude-3.5-sonnet", "object": "model", "owned_by": "openrouter" },
    { "id": "deepseek/deepseek-v4-flash-0731", "object": "model", "owned_by": "openrouter" },
    { "id": "llama3.1:8b", "object": "model", "owned_by": "ollama" }
  ],
  "total_count": 3
}

Use GET https://api.fastinfra.ai/v1/models/count for a lightweight count without downloading the full list.

Currently available (745)

• accounts/fireworks/models/glm-5p2 (fireworks)
• accounts/fireworks/models/kimi-k2p6 (fireworks)
• accounts/fireworks/models/kimi-k2p7-code (fireworks)
• accounts/fireworks/models/minimax-m2p7 (fireworks)
• accounts/fireworks/models/nemotron-3-ultra-nvfp4 (fireworks)
• accounts/fireworks/models/nemotron-lightning-3p5-30b-a3b (fireworks)
• accounts/fireworks/models/qwen3p7-plus (fireworks)
• accounts/fireworks/models/qwen3p8-2p4t-a95b (fireworks)
• accounts/fireworks/models/qwen3p8-max (fireworks)
• accounts/fireworks/routers/glm-5p2-fast (fireworks)
• accounts/fireworks/routers/kimi-k2p6-turbo (fireworks)
• accounts/fireworks/routers/kimi-k2p7-code-fast (fireworks)
• accounts/fireworks/routers/kimi-k3-fast (fireworks)
• agentica-org/DeepCoder-14B-Preview (together)
• ai21/jamba-large-1.7 (openrouter)
• aion-labs/aion-2.0 (openrouter)
• aion-labs/aion-3.0 (openrouter)
• aion-labs/aion-3.0-mini (openrouter)
• aion-labs/aion-rp-llama-3.1-8b (openrouter)
• alibaba/happyhorse-1.0-i2v (together)
• alibaba/happyhorse-1.0-r2v (together)
• alibaba/happyhorse-1.0-t2v (together)
• alibaba/happyhorse-1.1-i2v (together)
• alibaba/happyhorse-1.1-r2v (together)
• alibaba/happyhorse-1.1-t2v (together)
• all-minilm:latest (ollama)
• allenai/Molmo-7B-D-0924 (together)
• allenai/olmo-3-32b-think (openrouter)
• amazon/nova-2-lite-v1 (openrouter)
• amazon/nova-lite-v1 (openrouter)
• amazon/nova-micro-v1 (openrouter)
• amazon/nova-premier-v1 (openrouter)
• amazon/nova-pro-v1 (openrouter)
• anthracite-org/magnum-v4-72b (openrouter)
• anthropic/claude-3-haiku (openrouter)
• anthropic/claude-fable-5 (openrouter)
• anthropic/claude-haiku-4-5 (openrouter)
• anthropic/claude-opus-4 (openrouter)
• anthropic/claude-opus-4-7 (openrouter)
• anthropic/claude-opus-4-8 (openrouter)
• anthropic/claude-opus-4.1 (openrouter)
• anthropic/claude-opus-4.5 (openrouter)
• anthropic/claude-opus-4.6 (openrouter)
• anthropic/claude-opus-4.7-fast (openrouter)
• anthropic/claude-opus-4.8-fast (openrouter)
• anthropic/claude-opus-5 (openrouter)
• anthropic/claude-opus-5-fast (openrouter)
• anthropic/claude-sonnet-4 (openrouter)
• anthropic/claude-sonnet-4-6 (openrouter)
• anthropic/claude-sonnet-4.5 (openrouter)

Showing 50 of 745 models. Call GET https://api.fastinfra.ai/v1/models for the full list.

Code Examples

C# (.NET)

var client = new OpenAIClient(
    new ApiKeyCredential("YOUR_API_KEY"),
    new OpenAIClientOptions { Endpoint = new Uri("https://api.fastinfra.ai/v1") });

var chat = client.GetChatClient("llama3.1:8b");
var response = await chat.CompleteChatAsync("Hello!");
Console.WriteLine(response.Value.Content[0].Text);

Python (free-tier / local model)

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.fastinfra.ai/v1"
)

response = client.chat.completions.create(
    model="llama3.1:8b",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)

Python (provider model)

from openai import OpenAI

client = OpenAI(api_key="YOUR_API_KEY", base_url="https://api.fastinfra.ai/v1")

response = client.chat.completions.create(
    model="all-minilm:latest",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)

JavaScript

const response = await fetch("https://api.fastinfra.ai/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_API_KEY",
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    model: "all-minilm:latest",
    messages: [{ role: "user", content: "Hello!" }]
  })
});

const data = await response.json();
console.log(data.choices[0].message.content);