> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hyperbolic.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Text Generation APIs

> Complete guide to chat completions and text generation with large language models

# Text Generation APIs

Access 20+ large language models through Hyperbolic's OpenAI-compatible chat completions API. Generate text, have conversations, write code, and more with state-of-the-art open-source models.

## Chat Completions

The chat completions endpoint is the primary way to interact with text generation models.

### Endpoint

```text theme={null}
POST https://api.hyperbolic.xyz/v1/chat/completions
```

### Basic Request

<Tabs>
  <Tab title="Python">
    ```python theme={null}
    import requests

    url = "https://api.hyperbolic.xyz/v1/chat/completions"
    headers = {
        "Content-Type": "application/json",
        "Authorization": "Bearer YOUR_API_KEY"
    }
    data = {
        "model": "deepseek-ai/DeepSeek-R1",
        "messages": [
            {"role": "user", "content": "Explain quantum computing in simple terms."}
        ],
        "max_tokens": 512,
        "temperature": 0.7
    }

    response = requests.post(url, headers=headers, json=data)
    print(response.json()["choices"][0]["message"]["content"])
    ```
  </Tab>

  <Tab title="cURL">
    ```bash theme={null}
    curl -X POST "https://api.hyperbolic.xyz/v1/chat/completions" \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer YOUR_API_KEY" \
      -d '{
        "model": "deepseek-ai/DeepSeek-R1",
        "messages": [
          {"role": "user", "content": "Explain quantum computing in simple terms."}
        ],
        "max_tokens": 512,
        "temperature": 0.7
      }'
    ```
  </Tab>
</Tabs>

### Request Parameters

| Parameter     | Type         | Required | Description                                        |
| ------------- | ------------ | -------- | -------------------------------------------------- |
| `model`       | string       | Yes      | Model ID (e.g., `deepseek-ai/DeepSeek-R1`)         |
| `messages`    | array        | Yes      | Array of message objects with `role` and `content` |
| `max_tokens`  | integer      | No       | Maximum number of tokens to generate               |
| `temperature` | float        | No       | Sampling temperature (0-2). Higher = more creative |
| `top_p`       | float        | No       | Nucleus sampling threshold (0-1)                   |
| `stream`      | boolean      | No       | Enable streaming responses (default: false)        |
| `stop`        | string/array | No       | Stop sequence(s) to end generation                 |

### Messages Format

Messages are an array of objects with `role` and `content` fields:

```json theme={null}
{
  "messages": [
    {"role": "system", "content": "You are a helpful coding assistant."},
    {"role": "user", "content": "Write a Python function to reverse a string."},
    {"role": "assistant", "content": "Here's a function to reverse a string..."},
    {"role": "user", "content": "Can you make it more efficient?"}
  ]
}
```

**Roles:**

* `system`: Sets the behavior and context for the assistant
* `user`: Messages from the user
* `assistant`: Previous responses from the model (for conversation history)

### Response Format

```json theme={null}
{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1234567890,
  "model": "deepseek-ai/DeepSeek-R1",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Quantum computing uses quantum mechanics..."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 15,
    "completion_tokens": 150,
    "total_tokens": 165
  }
}
```

## Multi-turn Conversations

To maintain conversation context, include the full message history in each request:

<Tabs>
  <Tab title="Python">
    ```python theme={null}
    import requests

    url = "https://api.hyperbolic.xyz/v1/chat/completions"
    headers = {
        "Content-Type": "application/json",
        "Authorization": "Bearer YOUR_API_KEY"
    }

    # Maintain conversation history
    messages = [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is the capital of France?"},
        {"role": "assistant", "content": "The capital of France is Paris."},
        {"role": "user", "content": "What is its population?"}
    ]

    response = requests.post(url, headers=headers, json={
        "model": "meta-llama/Llama-3.3-70B-Instruct",
        "messages": messages,
        "max_tokens": 256
    })

    print(response.json()["choices"][0]["message"]["content"])
    ```
  </Tab>

  <Tab title="cURL">
    ```bash theme={null}
    curl -X POST "https://api.hyperbolic.xyz/v1/chat/completions" \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer YOUR_API_KEY" \
      -d '{
        "model": "meta-llama/Llama-3.3-70B-Instruct",
        "messages": [
          {"role": "system", "content": "You are a helpful assistant."},
          {"role": "user", "content": "What is the capital of France?"},
          {"role": "assistant", "content": "The capital of France is Paris."},
          {"role": "user", "content": "What is its population?"}
        ],
        "max_tokens": 256
      }'
    ```
  </Tab>
</Tabs>

## Streaming Responses

Enable real-time token streaming for responsive chat applications:

<Tabs>
  <Tab title="Python">
    ```python theme={null}
    import requests

    url = "https://api.hyperbolic.xyz/v1/chat/completions"
    headers = {
        "Content-Type": "application/json",
        "Authorization": "Bearer YOUR_API_KEY"
    }
    data = {
        "model": "meta-llama/Llama-3.3-70B-Instruct",
        "messages": [
            {"role": "user", "content": "Write a short story about a robot."}
        ],
        "max_tokens": 512,
        "stream": True
    }

    response = requests.post(url, headers=headers, json=data, stream=True)

    for line in response.iter_lines():
        if line:
            line = line.decode('utf-8')
            if line.startswith('data: ') and line != 'data: [DONE]':
                import json
                chunk = json.loads(line[6:])
                content = chunk['choices'][0]['delta'].get('content', '')
                print(content, end='', flush=True)
    ```
  </Tab>

  <Tab title="cURL">
    ```bash theme={null}
    curl -X POST "https://api.hyperbolic.xyz/v1/chat/completions" \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer YOUR_API_KEY" \
      -d '{
        "model": "meta-llama/Llama-3.3-70B-Instruct",
        "messages": [
          {"role": "user", "content": "Write a short story about a robot."}
        ],
        "max_tokens": 512,
        "stream": true
      }'
    ```
  </Tab>
</Tabs>

<Tip>
  For tool calling (function calling) capabilities, see the [Tool Calling](#tool-calling) section below.
</Tip>

## System Prompts

Use system prompts to set the model's behavior, persona, or instructions:

```python theme={null}
messages = [
    {
        "role": "system",
        "content": """You are an expert Python developer. Follow these guidelines:
- Write clean, well-documented code
- Include type hints
- Add brief comments explaining complex logic
- Suggest improvements when relevant"""
    },
    {"role": "user", "content": "Write a function to find prime numbers."}
]
```

## Tool Calling

Tool calling (also known as function calling) allows models to invoke external tools and APIs. The model generates structured JSON arguments that your application can use to call functions, then the results can be passed back to continue the conversation.

### Supported Models

The following models support tool calling:

* `meta-llama/Llama-3.3-70B-Instruct`
* `Qwen/Qwen3-Coder-480B-A35B-Instruct`

### Example

<Tabs>
  <Tab title="Python">
    ```python theme={null}
    import requests
    import json

    url = "https://api.hyperbolic.xyz/v1/chat/completions"
    headers = {
        "Content-Type": "application/json",
        "Authorization": "Bearer YOUR_API_KEY"
    }

    # Define available tools
    tools = [
        {
            "type": "function",
            "function": {
                "name": "get_weather",
                "description": "Get the current weather for a location",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {
                            "type": "string",
                            "description": "City and state, e.g. San Francisco, CA"
                        },
                        "unit": {
                            "type": "string",
                            "enum": ["celsius", "fahrenheit"],
                            "description": "Temperature unit"
                        }
                    },
                    "required": ["location"]
                }
            }
        }
    ]

    data = {
        "model": "meta-llama/Llama-3.3-70B-Instruct",
        "messages": [
            {"role": "user", "content": "What's the weather like in San Francisco?"}
        ],
        "tools": tools,
        "tool_choice": "auto"
    }

    response = requests.post(url, headers=headers, json=data)
    result = response.json()

    # Check if the model wants to call a tool
    message = result["choices"][0]["message"]
    if message.get("tool_calls"):
        tool_call = message["tool_calls"][0]
        print(f"Function: {tool_call['function']['name']}")
        print(f"Arguments: {tool_call['function']['arguments']}")
    ```
  </Tab>

  <Tab title="cURL">
    ```bash theme={null}
    curl -X POST "https://api.hyperbolic.xyz/v1/chat/completions" \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer YOUR_API_KEY" \
      -d '{
        "model": "meta-llama/Llama-3.3-70B-Instruct",
        "messages": [
          {"role": "user", "content": "What'\''s the weather like in San Francisco?"}
        ],
        "tools": [
          {
            "type": "function",
            "function": {
              "name": "get_weather",
              "description": "Get the current weather for a location",
              "parameters": {
                "type": "object",
                "properties": {
                  "location": {
                    "type": "string",
                    "description": "City and state, e.g. San Francisco, CA"
                  },
                  "unit": {
                    "type": "string",
                    "enum": ["celsius", "fahrenheit"],
                    "description": "Temperature unit"
                  }
                },
                "required": ["location"]
              }
            }
          }
        ],
        "tool_choice": "auto"
      }'
    ```
  </Tab>
</Tabs>

<Info>
  The `tool_choice` parameter controls how the model uses tools: `"auto"` lets the model decide, `"none"` disables tool use, or specify a tool name to force its use.
</Info>

## Text Completions (Base Models)

<Warning>
  **Sunset Notice:** Llama 3.1 405B BASE is being discontinued. This model will be removed in a future update. Please plan your migration accordingly.
</Warning>

For raw text completion without chat formatting, use the completions endpoint with base models like Llama 3.1 405B BASE.

### Endpoint

```text theme={null}
POST https://api.hyperbolic.xyz/v1/completions
```

### Example

<Tabs>
  <Tab title="Python">
    ```python theme={null}
    import requests

    url = "https://api.hyperbolic.xyz/v1/completions"
    headers = {
        "Content-Type": "application/json",
        "Authorization": "Bearer YOUR_API_KEY"
    }
    data = {
        "model": "meta-llama/Meta-Llama-3.1-405B",
        "prompt": "The key principles of machine learning are",
        "max_tokens": 256,
        "temperature": 0.7
    }

    response = requests.post(url, headers=headers, json=data)
    print(response.json()["choices"][0]["text"])
    ```
  </Tab>

  <Tab title="cURL">
    ```bash theme={null}
    curl -X POST "https://api.hyperbolic.xyz/v1/completions" \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer YOUR_API_KEY" \
      -d '{
        "model": "meta-llama/Meta-Llama-3.1-405B",
        "prompt": "The key principles of machine learning are",
        "max_tokens": 256,
        "temperature": 0.7
      }'
    ```
  </Tab>
</Tabs>

<Info>
  Base models are ideal for text completion, fill-in-the-middle tasks, and custom prompting strategies where chat formatting isn't needed.
</Info>

## Available Models

### Instruct Models (Chat Completions)

| Model            | Model ID                              | Context | Price           | Tools |
| ---------------- | ------------------------------------- | ------- | --------------- | ----- |
| Llama 3.3 70B    | `meta-llama/Llama-3.3-70B-Instruct`   | 131K    | \$0.40/M tokens | Yes   |
| Qwen3-Coder 480B | `Qwen/Qwen3-Coder-480B-A35B-Instruct` | 262K    | \$0.40/M tokens | Yes   |

## OpenAI SDK Compatibility

The API is fully compatible with OpenAI's SDK. Just change the base URL and API key:

<Tabs>
  <Tab title="Python">
    ```python theme={null}
    from openai import OpenAI

    client = OpenAI(
        api_key="YOUR_HYPERBOLIC_API_KEY",
        base_url="https://api.hyperbolic.xyz/v1"
    )

    response = client.chat.completions.create(
        model="meta-llama/Llama-3.3-70B-Instruct",
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Hello!"}
        ]
    )

    print(response.choices[0].message.content)
    ```
  </Tab>

  <Tab title="Node.js">
    ```javascript theme={null}
    import OpenAI from 'openai';

    const client = new OpenAI({
      apiKey: 'YOUR_HYPERBOLIC_API_KEY',
      baseURL: 'https://api.hyperbolic.xyz/v1'
    });

    const response = await client.chat.completions.create({
      model: 'meta-llama/Llama-3.3-70B-Instruct',
      messages: [
        { role: 'system', content: 'You are a helpful assistant.' },
        { role: 'user', content: 'Hello!' }
      ]
    });

    console.log(response.choices[0].message.content);
    ```
  </Tab>
</Tabs>

## Error Handling

| Error Code | Description                | Solution                                                               |
| ---------- | -------------------------- | ---------------------------------------------------------------------- |
| `401`      | Invalid or missing API key | Check your API key is correct and included in the Authorization header |
| `400`      | Invalid request parameters | Verify model ID, message format, and parameter values                  |
| `429`      | Rate limit exceeded        | Reduce request frequency or upgrade to Pro tier                        |
| `500`      | Server error               | Retry the request; contact support if persistent                       |

<Tip>
  Basic tier allows 60 requests/minute. Upgrade to Pro tier (minimum \$5 deposit) for 600 requests/minute.
</Tip>

## Next Steps

<CardGroup cols={3}>
  <Card title="Vision Language Models" icon="eye" href="/docs/inference/vlm-apis">
    Process images with multimodal AI models
  </Card>

  <Card title="Image APIs" icon="image" href="/docs/inference/image-apis">
    Generate images from text prompts
  </Card>

  <Card title="Audio APIs" icon="volume-high" href="/docs/inference/audio-apis">
    Text-to-speech and audio generation
  </Card>
</CardGroup>