Infernet
Node
Deployment

Deployment

To deploy an Infernet Node in production, we have put together a best-practices repository: ritual-net/infernet-deploy (opens in a new tab).

Start by cloning the infernet-deploy repository locally:

git clone https://github.com/ritual-net/infernet-deploy

Deploying a single node locally

ℹ️

Infernet Nodes execute containerized workflows. As such, installing and running (opens in a new tab) a modern version of Docker is a prerequisite, regardless of your choice of how to run the node.

Running locally

You can run an Infernet Node locally via Docker Compose (opens in a new tab). First, you must create a configuration file (see: example configuration (opens in a new tab)).

# Create a configuration file
cd deploy
cp ../config.sample.json config.json

For filling in config.json properly, see Configuration. After you have configured your node, you can run it with:

# Run node
cd deploy
docker compose up -d

You can stop the node and related services with:

# Stop node
cd deploy
docker compose down
️⚠️

For your local node's REST API to be accessible from the public internet, you will have to expose node's local port (default: 4000) to the internet. This it NOT recommended. If planning to support off-chain workloads through the REST API, we highly recommend deploying a node cluster on AWS or GCP instead, as detailed in Deploying a node cluster.

Deploying a node cluster

Via infernet-deploy, node operators can use Terraform (opens in a new tab) for infrastructure procurement and Docker Compose (opens in a new tab) for deployment to deploy Infernet Nodes in production on Amazon Web Services (AWS) (opens in a new tab) and Google Cloud Platform (GCP) (opens in a new tab).

To deploy a cluster, operators need to specify:

  1. Cluster-level configurations via terraform.tfvars.
  2. Node-level configurations via a config.json, for each node being deployed.

General setup

Install Terraform

Follow the official instructions (opens in a new tab) to install Terraform. We use Terraform in the infernet-deploy scripts for infrastructure procurement.

Configure nodes

Once ready, you must create a configuration file (see: example configuration (opens in a new tab)) for each node being deployed.

  • These configuration files must have unique names
    • A straightforward approach would be 0.json, 1.json, etc.
  • These files must be placed under the top-level configs/ directory
  • The number and name of the .json files must match the number and name of keys in the nodes variable in terraform.tfvars
  • Each node strictly requires its own configuration .json file, even if identical

For more details around node configuration options, reference the configuration documentation.

Infernet Router

Along with your node cluster, you can opt to deploy an infernet-router (opens in a new tab), which is a lightweight REST server that helps route requests to Infernet nodes. The Infernet Router does not route requests directly. Instead, it returns an Infernet node IP to send requests to. As such, it can be used in a 2-step process, where the client:

  1. Requests an IP from the Infernet Router.
  2. Sends Infernet API request(s) directly to that IP.

The Infernet Router REST server is deployed automatically with Terraform, by configuring the router object and setting deploy = true in your .tfvars file. However, if you plan to use it, you need to understand its implications.

⚠️

IMPORTANT: When configuring a heterogeneous node cluster (i.e. 0.json, 1.json, etc. are not identical), container IDs should be reserved for a unique container setup at the cluster level, i.e. across nodes (and thus .json files). That is becuase the router uses container IDs to make routing decisions between services running across the cluster.

Example: Consider nodes A and B, each running a single LLM inference container; node A runs image1, and node B runs image2. If we set id: "llm-inference" in both nodes (containers[0].id attribute in 0.json, 1.json), the router will be unable to disambiguate between the two services, and will consider them interchangeable, which they are not. Any requests for "llm-inference" will be routed to either container, which is an error.

Therefore, re-using IDs across configuration files must imply an identical container configuration, including image, environment variables, command, etc. This will explicitly tell the router which containers are interchangeable, and allow it to distribute requests for those containers across all nodes running that container.

Deploying on AWS

Authenticate with AWS CLI

Follow the official AWS instructions (opens in a new tab) to authenticate with the CLI locally.

Setup terraform.tfvars

First, make a copy of the example configuration file terraform.tfvars.example (opens in a new tab):

cd procure/aws
cp terraform.tfvars.example terraform.tfvars

Then, configure your terraform.tfvars file. See variables.tf (opens in a new tab) for configuration descriptions.

Run Terraform

When ready to deploy, execute the Terraform scripts:

# Initialize
cd procure
make init provider=aws
 
# Print deployment plan
make plan provider=aws
 
# Deploy
make apply provider=aws
 
# WARNING: Destructive
# Destroy deployment
make destroy provider=aws

Deploying on GCP

Authenticate with gcloud CLI

Follow the official GCP instructions (opens in a new tab) to authenticate with the CLI locally.

Setup terraform.tfvars

First, make a copy of the example configuration file terraform.tfvars.example (opens in a new tab):

cd procure/gcp
cp terraform.tfvars.example terraform.tfvars

Then, configure your terraform.tfvars file. See variables.tf (opens in a new tab) for configuration descriptions.

Run Terraform

When ready to deploy, execute the Terraform scripts:

# Initialize
cd procure
make init provider=gcp
 
# Print deployment plan
make plan provider=gcp
 
# Deploy
make apply provider=gcp
 
# WARNING: Destructive
# Destroy deployment
make destroy provider=gcp
v1.0.0REST API