From 3481bd162a4bded150dab2fe7ffcf2121690b741 Mon Sep 17 00:00:00 2001 From: Mo King Date: Tue, 21 Oct 2025 11:32:39 -0400 Subject: [PATCH 1/4] Update custom worker tutorial based on internal feedback --- serverless/workers/custom-worker.mdx | 59 ++++++++++++++++++---------- 1 file changed, 39 insertions(+), 20 deletions(-) diff --git a/serverless/workers/custom-worker.mdx b/serverless/workers/custom-worker.mdx index 3f76534a..dea70f5e 100644 --- a/serverless/workers/custom-worker.mdx +++ b/serverless/workers/custom-worker.mdx @@ -25,20 +25,25 @@ In this tutorial you'll learn how to: * You've [created a Runpod account](/get-started/manage-accounts). * You've installed [Python 3.x](https://www.python.org/downloads/) and [Docker](https://docs.docker.com/get-started/get-docker/) on your local machine and configured them for your command line. -## Step 1: Create a Python virtual environment +## Step 1: Set up your project directory + +Use the command line to set up your project directory and a virtual environment to manage dependencies. -First, set up a virtual environment to manage your project dependencies. - - Run this command in your local terminal: + + ```sh + mkdir serverless-worker-tutorial + cd serverless-worker-tutorial + ``` + + ```sh - # Create a Python virtual environment python3 -m venv venv ``` - + ```sh @@ -59,11 +64,22 @@ First, set up a virtual environment to manage your project dependencies. pip install runpod ``` + + + ```sh + touch rp_handler.py test_input.json Dockerfile + ``` + + This creates the following empty files in your project directory, which we'll build out in the next few steps of this tutorial: + * `rp_handler.py`: Your handler function, which will contain your custom logic for processing requests to your Serverless endpoint. + * `test_input.json`: A test input file for local testing of your handler function. + * `Dockerfile`: Instructions for building your handler function into a Docker image for deployment to Runpod. + ## Step 2: Create a handler file -Create a file named `rp_handler.py` and add the following code: +Using a text editor of your choice, add the following code to your `rp_handler.py` file: ```python import runpod @@ -88,7 +104,7 @@ def handler(event): print(f"Received prompt: {prompt}") print(f"Sleeping for {seconds} seconds...") - # You can replace this sleep call with your own Python code + # You can replace this sleep call with your own custom Python code time.sleep(seconds) return prompt @@ -98,11 +114,13 @@ if __name__ == '__main__': runpod.serverless.start({'handler': handler }) ``` -This is a bare-bones handler that processes a JSON object and outputs a `prompt` string contained in the `input` object. You can replace the `time.sleep(seconds)` call with your own Python code for generating images, text, or running any machine learning workload. +This is a bare-bones handler that processes a JSON object and outputs a `prompt` string contained in the `input` object. + +You can replace the `time.sleep(seconds)` call with your own custom Python code for generating images, text, or running any machine learning workload. ## Step 3: Create a test input file -You'll need to create an input file to properly test your handler locally. Create a file named `test_input.json` and add the following code: +Add the following lines to your `test_input.json` file: ```json { @@ -114,13 +132,13 @@ You'll need to create an input file to properly test your handler locally. Creat ## Step 4: Test your handler locally -Run your handler to verify that it works correctly: +Use the command line to test your handler locally: ```sh python rp_handler.py ``` -You should see output similar to this: +You should see output similar to this in your terminal: ```text --- Starting Serverless Worker | Version 1.7.9 --- @@ -139,7 +157,7 @@ INFO | Local testing complete, exiting. ## Step 5: Create a Dockerfile -Create a file named `Dockerfile` with the following content: +Add the following instructions to your `Dockerfile`: ```dockerfile FROM python:3.10-slim @@ -158,25 +176,26 @@ CMD ["python3", "-u", "rp_handler.py"] ## Step 6: Build and push your Docker image - + Instead of building and pushing your image via Docker Hub, you can also [deploy your worker from a GitHub repository](/serverless/workers/github-integration). - + Before you can deploy your worker on Runpod Serverless, you need to push it to Docker Hub: - Build your Docker image, specifying the platform for Runpod deployment, replacing `[YOUR_USERNAME]` with your Docker username: + Use the command line to build your Docker image, specifying the platform for Runpod deployment, replacing `DOCKER_USERNAME` with your Docker Hub username: ```sh - docker build --platform linux/amd64 --tag [YOUR_USERNAME]/serverless-test . + docker build --platform linux/amd64 \ + --tag DOCKER_USERNAME/serverless-worker-tutorial . ``` ```sh - docker push [YOUR_USERNAME]/serverless-test:latest + docker push DOCKER_USERNAME/serverless-worker-tutorial:latest ``` @@ -188,7 +207,7 @@ To deploy your worker to a Serverless endpoint: 1. Go to the [Serverless section](https://www.console.runpod.io/serverless) of the Runpod console. 2. Click **New Endpoint**. 3. Click **Import from Docker Registry** -4. In the **Container Image** field, enter your Docker image URL: `docker.io/yourusername/serverless-test:latest`. +4. In the **Container Image** field, enter your Docker image URL: `docker.io/DOCKER_USERNAME/serverless-worker-tutorial:latest`. 5. Click **Next** to proceed to endpoint configuration. 6. Configure your endpoint settings: * (Optional) Enter a custom name for your endpoint, or use the randomly generated name. @@ -240,7 +259,7 @@ Now that you've learned the basics, you're ready to: * [Create more advanced handler functions.](/serverless/workers/handler-functions) * [Send endpoint requests using cURL and the Serverless SDK.](/serverless/endpoints/send-requests) -* [Learn how to use endpoint operations like `/run` and `/status`.](/serverless/endpoints/operations) +* [Learn how to use endpoint operations like `/run` and `/status`.](/serverless/endpoints/send-requests#operation-overview) * [Manage your Serverless endpoints using the Runpod console.](/serverless/endpoints/manage-endpoints) * [Configure your endpoints for optimal performance and cost.](/serverless/endpoints/endpoint-configurations) * [Learn more about local testing.](/serverless/development/local-testing) From 21e66554e6cd2cd26a9407e7be799d4912a383d1 Mon Sep 17 00:00:00 2001 From: Mo King Date: Tue, 21 Oct 2025 11:48:23 -0400 Subject: [PATCH 2/4] Update quickstart based on feedback --- get-started.mdx | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/get-started.mdx b/get-started.mdx index 8e184141..4b7beecd 100644 --- a/get-started.mdx +++ b/get-started.mdx @@ -26,8 +26,8 @@ Now that you've created your account, you're ready to deploy your first Pod: 1. Open the [Pods page](https://www.console.runpod.io/pods) in the web interface. 2. Click the **Deploy** button. -3. Select **A40** from the list of graphics cards. -4. In the field under **Pod Name**, enter the name **quickstart-pod**. +3. Select any available GPU. (We recommend using an A40 for this tutorial for its low cost and high availability, but literally any GPU will work.) +4. In the field under **Pod Name**, enter the name `quickstart-pod`. 5. Keep all other fields (Pod Template, GPU Count, and Instance Pricing) on their default settings. 6. Click **Deploy On-Demand** to deploy and start your Pod. You'll be redirected back to the Pods page after a few seconds. @@ -55,13 +55,13 @@ Take a minute to explore the other tabs: 3. Type `print("Hello, world!")` in the first line of the notebook. 4. Click the play button to run your code. -And that's it—congrats! You just ran your first line of code on Runpod. +And that's it. Congrats, you just ran your first line of code on Runpod! ## Step 5: Clean up To avoid incurring unnecessary charges, follow these steps to clean up your Pod resources: -1. Return to the [Pods page](https://www.console.runpod.io/pods) and click your running Pod. +1. Return to the [Pods page](https://www.console.runpod.io/pods) and click on your running Pod. 2. Click the **Stop** button (pause icon) to stop your Pod. 3. Click **Stop Pod** in the modal that opens to confirm. @@ -85,7 +85,7 @@ To learn more about how storage works, see the [Pod storage overview](/pods/stor Now that you've learned the basics, you're ready to: * [Generate API keys](/get-started/api-keys) for programmatic resource management. -* Experiment with various options for [accessing and managing Runpod resources](/get-started/connect-to-runpod). +* Experiment with various workflows for [accessing and managing Runpod resources](/get-started/connect-to-runpod). * Learn how to [choose the right Pod](/pods/choose-a-pod) for your workload. * Review options for [Pod pricing](/pods/pricing). * [Explore our tutorials](/tutorials/introduction/overview) for specific AI/ML use cases. From 1159598d9fa643b46458fd1c6dca352b1b1e8a1d Mon Sep 17 00:00:00 2001 From: Mo King Date: Tue, 21 Oct 2025 11:51:11 -0400 Subject: [PATCH 3/4] Misc updates --- get-started/api-keys.mdx | 2 +- get-started/connect-to-runpod.mdx | 2 +- overview.mdx | 6 +++--- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/get-started/api-keys.mdx b/get-started/api-keys.mdx index 4d2c39ee..c51fd6e3 100644 --- a/get-started/api-keys.mdx +++ b/get-started/api-keys.mdx @@ -1,6 +1,6 @@ --- title: "Manage API keys" -description: "Learn how to create, edit, and disable Runpod API keys." +description: "Create, edit, and disable Runpod API keys." --- diff --git a/get-started/connect-to-runpod.mdx b/get-started/connect-to-runpod.mdx index a188cdc0..07e6b10b 100644 --- a/get-started/connect-to-runpod.mdx +++ b/get-started/connect-to-runpod.mdx @@ -1,6 +1,6 @@ --- title: "Choose a workflow" -description: "Review the available methods for accessing and managing Runpod resources." +description: "Review available methods for accessing and managing Runpod resources." --- Runpod offers multiple ways to access and manage your compute resources. Choose the method that best fits your workflow: diff --git a/overview.mdx b/overview.mdx index 4367394d..eb7edd0a 100644 --- a/overview.mdx +++ b/overview.mdx @@ -36,12 +36,12 @@ Serverless provides pay-per-second computing with automatic scaling for producti Learn how Serverless billing works and how to optimize your costs. - - Deploy a large language model for text or image generation in minutes using vLLM. - Build a custom worker and deploy it as a Serverless endpoint. + + Deploy a large language model for text or image generation in minutes using vLLM. + ## Pods From ccfa0b0436769ed9b209fad0e8ea1894b8e5bf22 Mon Sep 17 00:00:00 2001 From: Mo King Date: Tue, 21 Oct 2025 12:18:06 -0400 Subject: [PATCH 4/4] Fix storage terminology --- pods/overview.mdx | 4 ++-- pods/pricing.mdx | 6 +++--- pods/storage/create-network-volumes.mdx | 2 +- serverless/storage/overview.mdx | 8 ++++---- 4 files changed, 10 insertions(+), 10 deletions(-) diff --git a/pods/overview.mdx b/pods/overview.mdx index d4ea4054..21ee54b9 100644 --- a/pods/overview.mdx +++ b/pods/overview.mdx @@ -16,8 +16,8 @@ Each Pod consists of these core components: - **Container environment**: An Ubuntu Linux-based container that can run almost any compatible software. - **Unique identifier**: Each Pod receives a dynamic ID (e.g., `2s56cp0pof1rmt`) for management and access. - [Storage](#storage-options): - - **Container volume**: Houses the operating system and temporary storage. - - **Disk volume**: Persistent storage that is preserved between Pod starts and stops. + - **Container disk**: Houses the operating system and temporary storage. + - **Volume disk**: Persistent storage that is preserved between Pod starts and stops. - **Network volume (optional)**: Permanent, portable storage that can be moved between machines and persists even after Pod deletion. - **Hardware resources**: Allocated vCPU, system RAM, and multiple GPUs (based on your selection). - **Network connectivity**: A proxy connection enabling web access to any [exposed port](/pods/configuration/expose-ports) on your container. diff --git a/pods/pricing.mdx b/pods/pricing.mdx index c55e511a..9d75e435 100644 --- a/pods/pricing.mdx +++ b/pods/pricing.mdx @@ -130,13 +130,13 @@ You can select your preferred pricing model directly from the Runpod console whe Runpod offers [three types of storage](/pods/storage/types) for Pods:: -- **Container volumes:** Temporary storage that is erased if the Pod is stopped, billed at \$0.10 per GB per month for storage on running Pods. Billed per-second. -- **Disk volumes:** Persistent storage that is billed at \$0.10 per GB per month on running Pods and \$0.20 per GB per month for volume storage on stopped Pods. Billed per-second. +- **Container disk:** Temporary storage that is erased if the Pod is stopped, billed at \$0.10 per GB per month for storage on running Pods. Billed per-second. +- **Volume disk:** Persistent storage that is billed at \$0.10 per GB per month on running Pods and \$0.20 per GB per month for volume storage on stopped Pods. Billed per-second. - **Network volumes:** External storage that is billed at \$0.07 per GB per month for storage requirements below 1TB. For requirements exceeding 1TB, the rate is \$0.05 per GB per month. Billed hourly. You are not charged for storage if the host machine is down or unavailable from the public internet. -Container and volume disk storage will be included in your Pod's displayed hourly cost during deployment. +Container disk and volume disk storage will be included in your Pod's displayed hourly cost during deployment. Runpod is not designed as a long-term cloud storage system. Storage is provided to support compute tasks. We recommend regularly backing up critical data to your local machine or to a dedicated cloud storage provider. diff --git a/pods/storage/create-network-volumes.mdx b/pods/storage/create-network-volumes.mdx index 37d79b94..75190c4e 100644 --- a/pods/storage/create-network-volumes.mdx +++ b/pods/storage/create-network-volumes.mdx @@ -127,7 +127,7 @@ Network volumes are backed by high-performance storage servers co-located with R Using network volumes provides significant flexibility that can lead to cost savings, especially if you need to frequently switch between Pods or share large datasets. -Network volume storage space costs less than for disk volumes (\$0.07/GB/month rather than \$0.10/GB/month), and storing data on a network volume can save you money compared to provisioning separate disk space for multiple Pods (even with just two Pods sharing one volume). +Network volume storage space costs less than for volume disks (\$0.07/GB/month rather than \$0.10/GB/month), and storing data on a network volume can save you money compared to provisioning separate disk space for multiple Pods (even with just two Pods sharing one volume). For a deeper dive into potential benefits, read this [blog article on network volumes](https://blog.runpod.io/four-reasons-to-set-up-a/). diff --git a/serverless/storage/overview.mdx b/serverless/storage/overview.mdx index 70f8fe4d..195baa44 100644 --- a/serverless/storage/overview.mdx +++ b/serverless/storage/overview.mdx @@ -1,18 +1,18 @@ --- title: "Storage options" sidebarTitle: "Storage options" -description: "Explore storage options for your Serverless endpoints, including container volumes, network volumes, and S3-compatible storage." +description: "Explore storage options for your Serverless endpoints, including container disks, network volumes, and S3-compatible storage." --- This guide explains the different types of storage available in Runpod Serverless, their characteristics, and when to use each option. ## Storage types -### Container volume +### Container disk A worker's container disk holds temporary storage that exists only while a worker is running, and is completely lost when the worker is stopped or scaled down. It's created automatically when a Serverless worker launches and remains tightly coupled with the worker's lifecycle. -Container volumes provide fast read and write speeds since they are locally attached to workers. The cost of storage is included in the worker's running cost, making it an economical choice for temporary data. +Container disks provide fast read and write speeds since they are locally attached to workers. The cost of storage is included in the worker's running cost, making it an economical choice for temporary data. Any data saved by a worker's handler function will be stored in the container disk by default. To persist data beyond the current worker session, use a network volume or S3-compatible storage. @@ -36,7 +36,7 @@ To configure requests to send data to S3-compatible storage, see [S3-compatible ## Storage comparison table -| Feature | Container Volume | Network Volume | S3-Compatible Storage | +| Feature | Container disk | Network volume | S3-compatible storage | | ----------------------- | ------------------------------------ | ------------------------------------ | ------------------------------ | | **Persistence** | Temporary (erased when worker stops) | Permanent (independent of workers) | Permanent (external to Runpod) | | **Sharing** | Not shareable | Can be attached to multiple workers | Accessible via S3 credentials |