Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions get-started.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@ Now that you've created your account, you're ready to deploy your first Pod:

1. Open the [Pods page](https://www.console.runpod.io/pods) in the web interface.
2. Click the **Deploy** button.
3. Select **A40** from the list of graphics cards.
4. In the field under **Pod Name**, enter the name **quickstart-pod**.
3. Select any available GPU. (We recommend using an A40 for this tutorial for its low cost and high availability, but literally any GPU will work.)
4. In the field under **Pod Name**, enter the name `quickstart-pod`.
5. Keep all other fields (Pod Template, GPU Count, and Instance Pricing) on their default settings.
6. Click **Deploy On-Demand** to deploy and start your Pod. You'll be redirected back to the Pods page after a few seconds.

Expand Down Expand Up @@ -55,13 +55,13 @@ Take a minute to explore the other tabs:
3. Type `print("Hello, world!")` in the first line of the notebook.
4. Click the play button to run your code.

And that's it—congrats! You just ran your first line of code on Runpod.
And that's it. Congrats, you just ran your first line of code on Runpod!

## Step 5: Clean up

To avoid incurring unnecessary charges, follow these steps to clean up your Pod resources:

1. Return to the [Pods page](https://www.console.runpod.io/pods) and click your running Pod.
1. Return to the [Pods page](https://www.console.runpod.io/pods) and click on your running Pod.
2. Click the **Stop** button (pause icon) to stop your Pod.
3. Click **Stop Pod** in the modal that opens to confirm.

Expand All @@ -85,7 +85,7 @@ To learn more about how storage works, see the [Pod storage overview](/pods/stor
Now that you've learned the basics, you're ready to:

* [Generate API keys](/get-started/api-keys) for programmatic resource management.
* Experiment with various options for [accessing and managing Runpod resources](/get-started/connect-to-runpod).
* Experiment with various workflows for [accessing and managing Runpod resources](/get-started/connect-to-runpod).
* Learn how to [choose the right Pod](/pods/choose-a-pod) for your workload.
* Review options for [Pod pricing](/pods/pricing).
* [Explore our tutorials](/tutorials/introduction/overview) for specific AI/ML use cases.
Expand Down
2 changes: 1 addition & 1 deletion get-started/api-keys.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "Manage API keys"
description: "Learn how to create, edit, and disable Runpod API keys."
description: "Create, edit, and disable Runpod API keys."
---

<Note>
Expand Down
2 changes: 1 addition & 1 deletion get-started/connect-to-runpod.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "Choose a workflow"
description: "Review the available methods for accessing and managing Runpod resources."
description: "Review available methods for accessing and managing Runpod resources."
---

Runpod offers multiple ways to access and manage your compute resources. Choose the method that best fits your workflow:
Expand Down
6 changes: 3 additions & 3 deletions overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -47,12 +47,12 @@ Serverless provides pay-per-second computing with automatic scaling for producti
<Card title="Pricing" href="/serverless/pricing" icon="dollar-sign" iconType="solid">
Learn how Serverless billing works and how to optimize your costs.
</Card>
<Card title="vLLM quickstart" href="/serverless/vllm/get-started" icon="cloud-bolt" iconType="solid">
Deploy a large language model for text or image generation in minutes using vLLM.
</Card>
<Card title="Build your first worker" href="/serverless/workers/custom-worker" icon="code" iconType="solid">
Build a custom worker and deploy it as a Serverless endpoint.
</Card>
<Card title="vLLM quickstart" href="/serverless/vllm/get-started" icon="cloud-bolt" iconType="solid">
Deploy a large language model for text or image generation in minutes using vLLM.
</Card>
</CardGroup>

## Pods
Expand Down
4 changes: 2 additions & 2 deletions pods/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ Each Pod consists of these core components:
- **Container environment**: An Ubuntu Linux-based container that can run almost any compatible software.
- **Unique identifier**: Each Pod receives a dynamic ID (e.g., `2s56cp0pof1rmt`) for management and access.
- [Storage](#storage-options):
- **Container volume**: Houses the operating system and temporary storage.
- **Disk volume**: Persistent storage that is preserved between Pod starts and stops.
- **Container disk**: Houses the operating system and temporary storage.
- **Volume disk**: Persistent storage that is preserved between Pod starts and stops.
- **Network volume (optional)**: Permanent, portable storage that can be moved between machines and persists even after Pod deletion.
- **Hardware resources**: Allocated vCPU, system RAM, and multiple GPUs (based on your selection).
- **Network connectivity**: A proxy connection enabling web access to any [exposed port](/pods/configuration/expose-ports) on your container.
Expand Down
6 changes: 3 additions & 3 deletions pods/pricing.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -130,13 +130,13 @@ You can select your preferred pricing model directly from the Runpod console whe

Runpod offers [three types of storage](/pods/storage/types) for Pods::

- **Container volumes:** Temporary storage that is erased if the Pod is stopped, billed at \$0.10 per GB per month for storage on running Pods. Billed per-second.
- **Disk volumes:** Persistent storage that is billed at \$0.10 per GB per month on running Pods and \$0.20 per GB per month for volume storage on stopped Pods. Billed per-second.
- **Container disk:** Temporary storage that is erased if the Pod is stopped, billed at \$0.10 per GB per month for storage on running Pods. Billed per-second.
- **Volume disk:** Persistent storage that is billed at \$0.10 per GB per month on running Pods and \$0.20 per GB per month for volume storage on stopped Pods. Billed per-second.
- **Network volumes:** External storage that is billed at \$0.07 per GB per month for storage requirements below 1TB. For requirements exceeding 1TB, the rate is \$0.05 per GB per month. Billed hourly.

You are not charged for storage if the host machine is down or unavailable from the public internet.

Container and volume disk storage will be included in your Pod's displayed hourly cost during deployment.
Container disk and volume disk storage will be included in your Pod's displayed hourly cost during deployment.

<Note>
Runpod is not designed as a long-term cloud storage system. Storage is provided to support compute tasks. We recommend regularly backing up critical data to your local machine or to a dedicated cloud storage provider.
Expand Down
2 changes: 1 addition & 1 deletion pods/storage/create-network-volumes.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@ Network volumes are backed by high-performance storage servers co-located with R

Using network volumes provides significant flexibility that can lead to cost savings, especially if you need to frequently switch between Pods or share large datasets.

Network volume storage space costs less than for disk volumes (\$0.07/GB/month rather than \$0.10/GB/month), and storing data on a network volume can save you money compared to provisioning separate disk space for multiple Pods (even with just two Pods sharing one volume).
Network volume storage space costs less than for volume disks (\$0.07/GB/month rather than \$0.10/GB/month), and storing data on a network volume can save you money compared to provisioning separate disk space for multiple Pods (even with just two Pods sharing one volume).

For a deeper dive into potential benefits, read this [blog article on network volumes](https://blog.runpod.io/four-reasons-to-set-up-a/).

Expand Down
8 changes: 4 additions & 4 deletions serverless/storage/overview.mdx
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
---
title: "Storage options"
sidebarTitle: "Storage options"
description: "Explore storage options for your Serverless endpoints, including container volumes, network volumes, and S3-compatible storage."
description: "Explore storage options for your Serverless endpoints, including container disks, network volumes, and S3-compatible storage."
---

This guide explains the different types of storage available in Runpod Serverless, their characteristics, and when to use each option.

## Storage types

### Container volume
### Container disk

A worker's container disk holds temporary storage that exists only while a worker is running, and is completely lost when the worker is stopped or scaled down. It's created automatically when a Serverless worker launches and remains tightly coupled with the worker's lifecycle.

Container volumes provide fast read and write speeds since they are locally attached to workers. The cost of storage is included in the worker's running cost, making it an economical choice for temporary data.
Container disks provide fast read and write speeds since they are locally attached to workers. The cost of storage is included in the worker's running cost, making it an economical choice for temporary data.

Any data saved by a worker's handler function will be stored in the container disk by default. To persist data beyond the current worker session, use a network volume or S3-compatible storage.

Expand All @@ -36,7 +36,7 @@ To configure requests to send data to S3-compatible storage, see [S3-compatible

## Storage comparison table

| Feature | Container Volume | Network Volume | S3-Compatible Storage |
| Feature | Container disk | Network volume | S3-compatible storage |
| ----------------------- | ------------------------------------ | ------------------------------------ | ------------------------------ |
| **Persistence** | Temporary (erased when worker stops) | Permanent (independent of workers) | Permanent (external to Runpod) |
| **Sharing** | Not shareable | Can be attached to multiple workers | Accessible via S3 credentials |
Expand Down
59 changes: 39 additions & 20 deletions serverless/workers/custom-worker.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -24,20 +24,25 @@ In this tutorial you'll learn how to:
* You've [created a Runpod account](/get-started/manage-accounts).
* You've installed [Python 3.x](https://www.python.org/downloads/) and [Docker](https://docs.docker.com/get-started/get-docker/) on your local machine and configured them for your command line.

## Step 1: Create a Python virtual environment
## Step 1: Set up your project directory

Use the command line to set up your project directory and a virtual environment to manage dependencies.

First, set up a virtual environment to manage your project dependencies.
<Steps>
<Step title="Create a virtual environment">
Run this command in your local terminal:
<Step title="Create a directory for your project and navigate to it">
```sh
mkdir serverless-worker-tutorial
cd serverless-worker-tutorial
```
</Step>

<Step title="Create a virtual environment">
```sh
# Create a Python virtual environment
python3 -m venv venv
```
</Step>

<Step title="Activate the virtual environment">
<Step title="Activate the Python virtual environment">
<Tabs>
<Tab title="macOS/Linux">
```sh
Expand All @@ -58,11 +63,22 @@ First, set up a virtual environment to manage your project dependencies.
pip install runpod
```
</Step>

<Step title="Create the necessary project files">
```sh
touch rp_handler.py test_input.json Dockerfile
```

This creates the following empty files in your project directory, which we'll build out in the next few steps of this tutorial:
* `rp_handler.py`: Your handler function, which will contain your custom logic for processing requests to your Serverless endpoint.
* `test_input.json`: A test input file for local testing of your handler function.
* `Dockerfile`: Instructions for building your handler function into a Docker image for deployment to Runpod.
</Step>
</Steps>

## Step 2: Create a handler file

Create a file named `rp_handler.py` and add the following code:
Using a text editor of your choice, add the following code to your `rp_handler.py` file:

```python
import runpod
Expand All @@ -87,7 +103,7 @@ def handler(event):
print(f"Received prompt: {prompt}")
print(f"Sleeping for {seconds} seconds...")

# You can replace this sleep call with your own Python code
# You can replace this sleep call with your own custom Python code
time.sleep(seconds)

return prompt
Expand All @@ -97,11 +113,13 @@ if __name__ == '__main__':
runpod.serverless.start({'handler': handler })
```

This is a bare-bones handler that processes a JSON object and outputs a `prompt` string contained in the `input` object. You can replace the `time.sleep(seconds)` call with your own Python code for generating images, text, or running any machine learning workload.
This is a bare-bones handler that processes a JSON object and outputs a `prompt` string contained in the `input` object.

You can replace the `time.sleep(seconds)` call with your own custom Python code for generating images, text, or running any machine learning workload.

## Step 3: Create a test input file

You'll need to create an input file to properly test your handler locally. Create a file named `test_input.json` and add the following code:
Add the following lines to your `test_input.json` file:

```json
{
Expand All @@ -113,13 +131,13 @@ You'll need to create an input file to properly test your handler locally. Creat

## Step 4: Test your handler locally

Run your handler to verify that it works correctly:
Use the command line to test your handler locally:

```sh
python rp_handler.py
```

You should see output similar to this:
You should see output similar to this in your terminal:

```text
--- Starting Serverless Worker | Version 1.7.9 ---
Expand All @@ -138,7 +156,7 @@ INFO | Local testing complete, exiting.

## Step 5: Create a Dockerfile

Create a file named `Dockerfile` with the following content:
Add the following instructions to your `Dockerfile`:

```dockerfile
FROM python:3.10-slim
Expand All @@ -157,25 +175,26 @@ CMD ["python3", "-u", "rp_handler.py"]

## Step 6: Build and push your Docker image

<Info>
<Tip>

Instead of building and pushing your image via Docker Hub, you can also [deploy your worker from a GitHub repository](/serverless/workers/github-integration).

</Info>
</Tip>

Before you can deploy your worker on Runpod Serverless, you need to push it to Docker Hub:
<Steps>
<Step title="Build your Docker image">
Build your Docker image, specifying the platform for Runpod deployment, replacing `[YOUR_USERNAME]` with your Docker username:
Use the command line to build your Docker image, specifying the platform for Runpod deployment, replacing `DOCKER_USERNAME` with your Docker Hub username:

```sh
docker build --platform linux/amd64 --tag [YOUR_USERNAME]/serverless-test .
docker build --platform linux/amd64 \
--tag DOCKER_USERNAME/serverless-worker-tutorial .
```
</Step>

<Step title="Push the image to your container registry">
```sh
docker push [YOUR_USERNAME]/serverless-test:latest
docker push DOCKER_USERNAME/serverless-worker-tutorial:latest
```
</Step>
</Steps>
Expand All @@ -187,7 +206,7 @@ To deploy your worker to a Serverless endpoint:
1. Go to the [Serverless section](https://www.console.runpod.io/serverless) of the Runpod console.
2. Click **New Endpoint**.
3. Click **Import from Docker Registry**
4. In the **Container Image** field, enter your Docker image URL: `docker.io/yourusername/serverless-test:latest`.
4. In the **Container Image** field, enter your Docker image URL: `docker.io/DOCKER_USERNAME/serverless-worker-tutorial:latest`.
5. Click **Next** to proceed to endpoint configuration.
6. Configure your endpoint settings:
* (Optional) Enter a custom name for your endpoint, or use the randomly generated name.
Expand Down Expand Up @@ -239,7 +258,7 @@ Now that you've learned the basics, you're ready to:

* [Create more advanced handler functions.](/serverless/workers/handler-functions)
* [Send endpoint requests using cURL and the Serverless SDK.](/serverless/endpoints/send-requests)
* [Learn how to use endpoint operations like `/run` and `/status`.](/serverless/endpoints/operations)
* [Learn how to use endpoint operations like `/run` and `/status`.](/serverless/endpoints/send-requests#operation-overview)
* [Manage your Serverless endpoints using the Runpod console.](/serverless/endpoints/manage-endpoints)
* [Configure your endpoints for optimal performance and cost.](/serverless/endpoints/endpoint-configurations)
* [Learn more about local testing.](/serverless/development/local-testing)