> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hyperbolic.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Overview

> Deploy AI models instantly - OpenAI-compatible API with 25+ open-source models

# Serverless Inference

Deploy state-of-the-art AI models instantly with Hyperbolic's Serverless Inference platform. Access 25+ open-source models through a simple API, with no infrastructure to manage and pricing that scales with your usage.

## Upcoming Model Deprecations

<Warning>
  The following models are being sunset and will be removed in a future update. If you are currently using any of these models, please migrate to an alternative as soon as possible.
</Warning>

| Model                          | Category | Model ID                                     |
| ------------------------------ | -------- | -------------------------------------------- |
| Qwen3-Next 80B Thinking        | Text     | `Qwen/Qwen3-Next-80B-A3B-Thinking`           |
| Qwen3-Next 80B Instruct        | Text     | `Qwen/Qwen3-Next-80B-A3B-Instruct`           |
| GPT-OSS 120B                   | Text     | `openai/gpt-oss-120b`                        |
| GPT-OSS 20B                    | Text     | `openai/gpt-oss-20b`                         |
| Llama 3.1 405B BASE            | Text     | `meta-llama/Meta-Llama-3.1-405B`             |
| FLUX.1-dev                     | Image    | `FLUX.1-dev`                                 |
| SDXL 1.0                       | Image    | `SDXL1.0-base`                               |
| SDXL Turbo                     | Image    | `SDXL-turbo`                                 |
| Stable Diffusion 2             | Image    | `SD2`                                        |
| Stable Diffusion 1.5           | Image    | `SD1.5`                                      |
| Segmind SD 1B                  | Image    | `SSD`                                        |
| NVIDIA Nemotron Nano 12B v2 VL | Vision   | `nvidia/NVIDIA-Nemotron-Nano-12B-v2-VL-BF16` |
| Qwen2.5-VL-72B-Instruct        | Vision   | `Qwen/Qwen2.5-VL-72B-Instruct`               |
| Qwen2.5-VL-7B-Instruct         | Vision   | `Qwen/Qwen2.5-VL-7B-Instruct`                |
| Pixtral 12B                    | Vision   | `mistralai/Pixtral-12B-2409`                 |
| Melo TTS                       | Audio    | `MeloTTS`                                    |

## Why Serverless Inference?

Skip the complexity of GPU management, model deployment, and infrastructure scaling. Focus on building your application while we handle the AI infrastructure.

### Key Benefits

* **Instant Deployment**: Start using models in seconds, not hours
* **No Infrastructure**: Zero DevOps required - we handle everything
* **Pay Per Use**: Only pay for the tokens/images you generate
* **OpenAI Compatible**: Drop-in replacement for existing code
* **Privacy First**: Zero data retention policy

## Supported Model Categories

### Text Generation (LLMs)

Deploy the latest language models for chat, completion, and reasoning tasks.

**Available Models:**

* **Llama 3.1** (8B, 70B, 405B) - Meta's latest open models
* **Qwen 2.5** (7B, 72B) - Alibaba's multilingual models
* **Deepseek V2.5** - Efficient reasoning model
* **Hermes 3** - Fine-tuned for conversations
* **Mistral 7B** - Fast and efficient

**Pricing**: From \$0.10 per million input tokens

### Image Generation

Create stunning visuals with state-of-the-art diffusion models.

**Available Models:**

* **Stable Diffusion XL** - High-quality 1024x1024 images
* **Stable Diffusion 3.5** - Latest generation
* **FLUX.1 \[schnell/dev]** - Ultra-fast generation *(Sunset)*
* **ControlNet** - Guided image generation
* **Custom LoRA** - Use your fine-tuned models

**Pricing**: From \$0.0025 per image

### Vision-Language Models

Process and understand images with multimodal models.

**Available Models:**

* **Llama 3.2 Vision** (11B, 90B) - Image understanding
* **Qwen2-VL** (2B, 7B) - Multimodal reasoning

**Pricing**: From \$0.15 per million tokens

### Audio Generation

Generate natural-sounding speech and process audio.

**Available Models:**

* **Melo TTS** - Text-to-speech generation *(Sunset)*
* **Whisper** - Speech-to-text transcription (coming soon)

## Pricing Tiers

| Tier           | RPM Limit | IP Limit  | Min. Deposit  | Features                                                                      |
| -------------- | --------- | --------- | ------------- | ----------------------------------------------------------------------------- |
| **Basic**      | 60        | 100       | \$0           | API access, All models, Community support                                     |
| **Pro**        | 600       | 100       | ≥ \$5         | 10x rate limits, Priority queue, Email support                                |
| **Enterprise** | Unlimited | Unlimited | Contact Sales | Custom limits, Dedicated instances, SLA guarantees, Fine-tuning, 24/7 support |

> **Note:** Each source IP is capped at 600 RPM for DDoS protection. Need higher limits? [Contact sales](mailto:sales@hyperbolic.ai).

## Developer Experience

### OpenAI SDK Compatibility

Switch from OpenAI with just 2 lines of code:

```python theme={null}
from openai import OpenAI

client = OpenAI(
    api_key="YOUR_HYPERBOLIC_API_KEY",
    base_url="https://api.hyperbolic.xyz/v1"
)

# Your existing code works unchanged
response = client.chat.completions.create(
    model="meta-llama/Meta-Llama-3.1-70B-Instruct",
    messages=[{"role": "user", "content": "Hello!"}]
)
```

### REST API

Direct HTTP access for any platform:

```bash theme={null}
curl -X POST "https://api.hyperbolic.xyz/v1/chat/completions" \
  -H "Authorization: Bearer $HYPERBOLIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meta-llama/Meta-Llama-3.1-70B-Instruct",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

### Streaming Support

Real-time token streaming for chat applications:

```python theme={null}
stream = client.chat.completions.create(
    model="meta-llama/Meta-Llama-3.1-70B-Instruct",
    messages=messages,
    stream=True
)

for chunk in stream:
    print(chunk.choices[0].delta.content, end="")
```

## Advanced Features

### Function Calling

Enable models to call external tools and APIs. Supported on 18+ models including DeepSeek, Llama, Qwen, Kimi, and GPT-OSS families. [See full list →](/docs/inference/text-apis#tool-calling)

* Structured output generation
* Tool integration for agents
* JSON schema validation

### Custom Parameters

Fine-tune model behavior:

* Temperature, top\_p, top\_k controls
* Max tokens and stop sequences
* Presence and frequency penalties
* Custom system prompts

### Structured Output

Get reliable JSON responses:

* JSON mode for consistent formatting
* Schema enforcement
* Type validation

### Batch Processing

Optimize for throughput:

* Batch multiple requests
* Async processing
* Bulk pricing discounts

## Use Cases

### Chatbots & Assistants

Build conversational AI with streaming responses and context management.

### Content Generation

Create articles, summaries, and creative writing at scale.

### Code Generation

Generate, explain, and debug code across multiple languages.

### Image Creation

Design assets, generate product images, and create visual content.

### Data Processing

Extract insights, classify text, and analyze sentiment.

### Translation

Translate content across 100+ languages with context preservation.

## Getting Started

### Quick Start in 3 Steps

#### 1. Get Your API Key

Sign up at [app.hyperbolic.ai](https://app.hyperbolic.ai) and generate an API key

#### 2. Install SDK

```bash theme={null}
pip install openai
# or
npm install openai
```

#### 3. Make Your First Request

```python theme={null}
from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.hyperbolic.xyz/v1"
)

response = client.chat.completions.create(
    model="meta-llama/Meta-Llama-3.1-70B-Instruct",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain quantum computing in simple terms"}
    ]
)

print(response.choices[0].message.content)
```

## Integration Examples

### LangChain Integration

```python theme={null}
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    openai_api_key="YOUR_API_KEY",
    openai_api_base="https://api.hyperbolic.xyz/v1",
    model_name="meta-llama/Meta-Llama-3.1-70B-Instruct"
)
```

### Vercel AI SDK

```typescript theme={null}
import { OpenAI } from 'openai';

const openai = new OpenAI({
  apiKey: process.env.HYPERBOLIC_API_KEY,
  baseURL: 'https://api.hyperbolic.xyz/v1'
});
```

### Gradio Interface

Deploy interactive demos with one-click Hugging Face Spaces integration.

## Reliability & Compliance

### Infrastructure

* **99.9% Uptime SLA** for Enterprise tier
* **Global CDN** for low-latency access
* **Auto-scaling** to handle traffic spikes
* **Multi-region** deployment

### Security

* **Zero Data Retention**: Your data is never stored
* **Encrypted Connections**: TLS 1.3 for all API calls
* **API Key Rotation**: Regular key management
* **Compliance**: Pursuing SOC 2 Type II certification

### Support

* **Documentation**: Comprehensive guides and examples
* **Community Discord**: Active developer community
* **Email Support**: Pro tier and above
* **24/7 Support**: Enterprise tier

## Resources

* 🎮 **[Playground](https://app.hyperbolic.ai/playground)** - Test models before coding
* 📖 **[Text APIs](/docs/inference/text-apis)** - Text generation models
* 💡 **[Image APIs](/docs/inference/image-apis)** - Image generation models

## Pricing Calculator

Estimate your costs based on usage:

| Usage Level | Tokens/Month | Estimated Cost |
| ----------- | ------------ | -------------- |
| **Hobby**   | 1M tokens    | \~\$0.15       |
| **Startup** | 10M tokens   | \~\$1.50       |
| **Growth**  | 100M tokens  | \~\$12.00      |
| **Scale**   | 1B tokens    | \~\$100.00     |

*Based on Llama 3.1 70B pricing. Actual costs vary by model.*

[Get Your API Key →](https://app.hyperbolic.ai)

***

> **Migration Support**
>
> Moving from OpenAI, Anthropic, or another provider? Our team can help with migration strategies and code conversion. [Contact us →](mailto:support@hyperbolic.ai)