> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hyperbolic.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Managing Instances

> Create, monitor, and manage high-performance GPU instances through the Web UI

Master the full lifecycle of GPU instance management on Hyperbolic - from creation to termination, including monitoring, scaling, and troubleshooting.

## Creating Instances

### Web UI Method

<Steps>
  <Step title="Navigate to On-Demand GPU platform">
    Go to [app.hyperbolic.ai/gpus](https://app.hyperbolic.ai/gpus) and browse available GPUs.
  </Step>

  <Step title="Select GPU Configuration">
    * Choose GPU type (H100 80GB, H200 141GB, B200 192GB)
    * Select quantity
    * Pick region for optimal latency
    * Choose InfiniBand if needed for multi-GPU
  </Step>

  <Step title="Configure Instance">
    * **Storage**: Configure as needed for your storage needs
    * **Label**: Name your instance for easy identification
    * **SSH Keys** (Optional): Add SSH keys for secure access to your instance
  </Step>

  <Step title="Launch">
    Review pricing and click "Start Building". Instance will be ready in a few minutes (may take up to 25 minutes depending on configuration and region).
  </Step>
</Steps>

## Connecting to Instances

### SSH Connection

Basic SSH connection:

```bash theme={null}
# Standard connection
ssh ubuntu@<instance-ip>

# With specific key
ssh -i ~/.ssh/hyperbolic_key ubuntu@<instance-ip>
```

### File Transfer

<Tabs>
  <Tab title="SCP">
    ```bash theme={null}
    # Upload file
    scp model.pth ubuntu@<instance-ip>:/home/ubuntu/

    # Download file
    scp ubuntu@<instance-ip>:/home/ubuntu/results.csv ./

    # Upload directory
    scp -r dataset/ ubuntu@<instance-ip>:/home/ubuntu/
    ```
  </Tab>

  <Tab title="Rsync">
    ```bash theme={null}
    # Sync directory (upload)
    rsync -avz --progress dataset/ ubuntu@<instance-ip>:/home/ubuntu/dataset/

    # Sync with deletion of removed files
    rsync -avz --delete --progress local/ ubuntu@<instance-ip>:/home/ubuntu/remote/

    # Resume interrupted transfer
    rsync -avz --partial --progress large_file.tar ubuntu@<instance-ip>:/data/
    ```
  </Tab>

  <Tab title="SFTP">
    ```bash theme={null}
    # Interactive SFTP session
    sftp ubuntu@<instance-ip>

    # SFTP commands
    sftp> put model.pth
    sftp> get results/
    sftp> ls
    sftp> pwd
    sftp> exit
    ```
  </Tab>
</Tabs>

## Instance Lifecycle Management

### Terminating Instances

<Warning>
  Termination is permanent. All data on the instance will be lost. Always backup important data before terminating.
</Warning>

To terminate an instance:

1. Go to "My Instances" in the Web UI
2. Click "Terminate" in the instance actions
3. Confirm the deletion

## Monitoring and Metrics

### Real-Time GPU Monitoring

SSH into your instance and use these commands:

```bash theme={null}
# Basic GPU status
nvidia-smi

# Continuous monitoring (updates every 1 second)
watch -n 1 nvidia-smi

# Detailed GPU metrics
nvidia-smi -q

# Show running processes
nvidia-smi pmon

# GPU utilization over time
nvidia-smi dmon
```

### System Monitoring

```bash theme={null}
# System resources
htop

# Disk usage
df -h

# Memory usage
free -h

# Network statistics
nethogs  # Install with: sudo apt install nethogs

# Process monitoring
ps aux | grep python
```

### Setting Up Custom Monitoring

<Tabs>
  <Tab title="Prometheus + Grafana">
    ```bash theme={null}
    # Install DCGM exporter for GPU metrics
    docker run -d --gpus all --rm -p 9400:9400 nvidia/dcgm-exporter:latest

    # Install Prometheus
    docker run -d -p 9090:9090 \
      -v prometheus.yml:/etc/prometheus/prometheus.yml \
      prom/prometheus

    # Install Grafana
    docker run -d -p 3000:3000 grafana/grafana
    ```
  </Tab>

  <Tab title="Weights & Biases">
    ```python theme={null}
    import wandb
    import torch

    # Initialize W&B
    wandb.init(project="hyperbolic-training")

    # Log GPU metrics automatically
    wandb.watch(model)

    # Custom GPU logging
    for epoch in range(num_epochs):
        gpu_utilization = torch.cuda.utilization()
        gpu_memory = torch.cuda.memory_allocated() / 1024**3

        wandb.log({
            "gpu_utilization": gpu_utilization,
            "gpu_memory_gb": gpu_memory,
            "epoch": epoch
        })
    ```
  </Tab>
</Tabs>

## Managing Multiple Instances

### Viewing All Instances

Go to [app.hyperbolic.ai/instances](https://app.hyperbolic.ai/instances) to view all your instances with:

* Instance status
* GPU type and configuration
* Region
* Running time and costs
* Quick action buttons

### Batch Operations

**Parallel SSH Commands**:

```bash theme={null}
# Run command on multiple instances
for ip in 192.168.1.10 192.168.1.11 192.168.1.12; do
    ssh ubuntu@$ip "nvidia-smi" &
done
wait

# Using GNU parallel
parallel -j 4 ssh ubuntu@{} "python train.py" ::: \
  instance1.hyperbolic.ai \
  instance2.hyperbolic.ai \
  instance3.hyperbolic.ai
```

**Using Ansible**:

```yaml theme={null}
# inventory.yml
all:
  hosts:
    h100-1:
      ansible_host: 192.168.1.10
    h100-2:
      ansible_host: 192.168.1.11
    h100-3:
      ansible_host: 192.168.1.12
  vars:
    ansible_user: ubuntu
    ansible_ssh_private_key_file: ~/.ssh/hyperbolic_key

# Run command on all instances
ansible all -i inventory.yml -m shell -a "nvidia-smi"
```

## Instance States and Troubleshooting

### Instance States

| State           | Description                       | Billing |
| --------------- | --------------------------------- | ------- |
| **Pending**     | Instance is being provisioned     | No      |
| **Running**     | Instance is active and accessible | Yes     |
| **Starting**    | Instance is booting up            | No      |
| **Terminating** | Instance is being deleted         | No      |
| **Failed**      | Instance failed to start          | No      |

### Common Issues and Solutions

<AccordionGroup>
  <Accordion title="Instance stuck in 'Pending' state">
    **Solution**:

    * Wait up to 25 minutes for provisioning
    * Check region availability in the dashboard
    * Contact [support](mailto:support@hyperbolic.ai) if pending > 30 minutes

    Check the instance status in your dashboard for updates.
  </Accordion>

  <Accordion title="Cannot SSH to instance">
    **Solution**:

    * Verify instance is in "Running" state
    * Check SSH key is authorized
    * Confirm IP address is correct
    * Test network connectivity

    ```bash theme={null}
    # Debug SSH connection
    ssh -vvv ubuntu@<instance-ip>
    ```
  </Accordion>

  <Accordion title="GPU not available in instance">
    **Solution**:

    * Verify GPU driver is loaded
    * Check Docker runtime configuration
    * Restart instance if needed

    ```bash theme={null}
    # Check GPU availability
    nvidia-smi

    # Check driver
    nvidia-smi -q | grep "Driver Version"

    # For Docker containers
    docker run --gpus all nvidia/cuda:12.2.0-base nvidia-smi
    ```
  </Accordion>

  <Accordion title="Instance automatically terminated">
    **Possible causes**:

    * Insufficient account balance
    * Violation of terms of service
    * Hardware failure (rare)

    **Solution**:

    * Check billing status in your dashboard
    * Review instance logs
    * Contact support for clarification
  </Accordion>
</AccordionGroup>

<Info>
  Still having issues? Contact [**support@hyperbolic.ai**](mailto:support@hyperbolic.ai) or use the Intercom chat widget for immediate assistance.
</Info>

## Best Practices

<CardGroup cols={2}>
  <Card title="Use Instance Labels" icon="tag">
    Name your instances clearly to track different projects and experiments.
  </Card>

  <Card title="Implement Auto-shutdown" icon="clock">
    Set up scripts to automatically terminate idle instances to avoid unnecessary charges.
  </Card>

  <Card title="Regular Backups" icon="shield">
    Schedule regular backups of important data to external storage (S3, GCS, etc.).
  </Card>

  <Card title="Monitor Costs" icon="dollar-sign">
    Set up billing alerts and regularly review instance usage to optimize costs.
  </Card>
</CardGroup>

### Auto-shutdown Script Example

```python theme={null}
#!/usr/bin/env python3
import subprocess
import time
import sys

def get_gpu_utilization():
    """Get current GPU utilization percentage"""
    result = subprocess.run(
        ['nvidia-smi', '--query-gpu=utilization.gpu', '--format=csv,noheader,nounits'],
        capture_output=True,
        text=True
    )
    return int(result.stdout.strip())

def auto_shutdown(idle_minutes=30, threshold=5):
    """Shutdown instance if GPU idle for specified minutes"""
    idle_count = 0
    check_interval = 60  # Check every minute

    while True:
        util = get_gpu_utilization()

        if util < threshold:
            idle_count += 1
            print(f"GPU idle ({util}%). Idle count: {idle_count}/{idle_minutes}")

            if idle_count >= idle_minutes:
                print("Shutting down due to inactivity...")
                subprocess.run(['sudo', 'shutdown', '-h', 'now'])
                sys.exit(0)
        else:
            idle_count = 0
            print(f"GPU active ({util}%). Reset idle counter.")

        time.sleep(check_interval)

if __name__ == "__main__":
    auto_shutdown(idle_minutes=30, threshold=5)
```

## Advanced Configuration

### Custom Startup Scripts

Create a startup script that runs when instance boots:

```bash theme={null}
#!/bin/bash
# /home/ubuntu/startup.sh

# Mount additional storage
sudo mount /dev/nvme1n1 /data

# Start Jupyter
jupyter notebook --no-browser --port=8888 &

# Start TensorBoard
tensorboard --logdir=/home/ubuntu/logs --port=6006 &

# Start monitoring
python /home/ubuntu/monitor_gpu.py &
```

### Data Persistence Strategies

<Tabs>
  <Tab title="Cloud Storage">
    ```bash theme={null}
    # Install AWS CLI
    sudo apt-get update
    sudo apt-get install awscli -y

    # Configure AWS credentials
    aws configure

    # Sync data to S3
    aws s3 sync /home/ubuntu/models/ s3://my-bucket/models/

    # Download from S3
    aws s3 sync s3://my-bucket/dataset/ /home/ubuntu/dataset/
    ```
  </Tab>

  <Tab title="Git LFS">
    ```bash theme={null}
    # Install Git LFS
    curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash
    sudo apt-get install git-lfs

    # Initialize Git LFS
    git lfs install

    # Track large files
    git lfs track "*.pth"
    git lfs track "*.h5"

    # Push to repository
    git add .
    git commit -m "Save model checkpoint"
    git push
    ```
  </Tab>
</Tabs>

## Next Steps

<CardGroup>
  <Card title="Storage & Ports" icon="hard-drive" href="/docs/on-demand/storage-and-ports">
    Configure persistent storage and network settings
  </Card>
</CardGroup>