Infernet
Router
REST API

REST API Reference

The Router's REST API enables:

  1. Discover nodes that are running one or more specified containers
  2. Discover containers running across the network
  3. Collecting container resources, including hardware capabilities and supported models

GET /ips

Returns an Infernet node IP to send requests to.

  • Method: GET
  • URL: /api/v1/ips
  • Query Parameters:
    • container (string, repeatable): IDs of containers required for the job. Multiple can be specified by repeating this parameter (e.g., ?container=inference1&container=inference2). Only IPs of nodes running the specified containers will be returned.
    • n (integer, optional): Number of IPs to return. Defaults to 3.
    • offset (integer, optional): Number of node IPs to skip before returning.
  • Response:
    • Success:
      • Code: 200 OK
      • Content: string[]
        • An array of node IPs
    • Failure:
      • Code: 400
      • Content: {"error": "No containers specified"}
        • If no containers are specified

GET /containers

Returns all discoverable services (containers) running on the Infernet Network.

  • Method: GET
  • URL: /api/v1/containers
  • Response:

GET /resources

Retrieves resources of each service (container) running on every Infernet node, which include hardware specifications and supported model information.

If a model ID is provided, simply checks all containers for model support.

ℹ️

Router version >= v1.1 only

  • Method: GET
  • URL: /api/v1/resources
  • Query Parameters:
    • model_id (string, optional): Model IDs to support information for.
  • Response:
    • Code: 200 OK
    • Content:
      • Key: (string) - The node hostname / IP address.
      • Value: (object) - An object with:
        • Key: (string) - The container ID.
        • Value: One of
      Example: 
      {
  "localhost:8000": {
    "onnx-inference-service": {
      "compute_capability": [
        {
          "cached_models": [],
          "id": "ml",
          "models": [],
          "task": [],
          "type": "onnx"
        }
      ],
      "hardware_capabilities": [
        {
          "capability_id": "base",
          "cpu_info": {
            "architecture": "aarch64",
            "byte_order": "Little Endian",
            "cores": [],
            "model": "-",
            "num_cores": 12,
            "vendor_id": "Apple"
          },
          "disk_info": [
            {
              "available": 22042620,
              "filesystem": "overlay",
              "mount_point": "/",
              "size": 122713108,
              "used": 94404176
            },
            {
              "available": 65536,
              "filesystem": "tmpfs",
              "mount_point": "/dev",
              "size": 65536,
              "used": 0
            },
            {
              "available": 65536,
              "filesystem": "shm",
              "mount_point": "/dev/shm",
              "size": 65536,
              "used": 0
            },
            {
              "available": 22042620,
              "filesystem": "/dev/vda1",
              "mount_point": "/etc/hosts",
              "size": 122713108,
              "used": 94404176
            },
            {
              "available": 4576288,
              "filesystem": "tmpfs",
              "mount_point": "/sys/firmware",
              "size": 4576288,
              "used": 0
            }
          ],
          "os_info": {
            "name": "Linux",
            "version": "#1 SMP PREEMPT Wed Oct 25 16:32:24 UTC 2023"
          }
        }
      ],
      "service_id": "onnx-inference-service"
    }
  },
  "localhost:9000": {
    "torch-inference-service": {
      "compute_capability": [
        {
          "cached_models": [],
          "id": "ml",
          "models": [],
          "task": [],
          "type": "torch"
        }
      ],
      "hardware_capabilities": [
        {
          "capability_id": "base",
          "cpu_info": {
            "architecture": "aarch64",
            "byte_order": "Little Endian",
            "cores": [],
            "model": "-",
            "num_cores": 12,
            "vendor_id": "Apple"
          },
          "disk_info": [
            {
              "available": 22042620,
              "filesystem": "overlay",
              "mount_point": "/",
              "size": 122713108,
              "used": 94404176
            },
            {
              "available": 65536,
              "filesystem": "tmpfs",
              "mount_point": "/dev",
              "size": 65536,
              "used": 0
            },
            {
              "available": 65536,
              "filesystem": "shm",
              "mount_point": "/dev/shm",
              "size": 65536,
              "used": 0
            },
            {
              "available": 22042620,
              "filesystem": "/dev/vda1",
              "mount_point": "/etc/hosts",
              "size": 122713108,
              "used": 94404176
            },
            {
              "available": 4576288,
              "filesystem": "tmpfs",
              "mount_point": "/sys/firmware",
              "size": 4576288,
              "used": 0
            }
          ],
          "os_info": {
            "name": "Linux",
            "version": "#1 SMP PREEMPT Wed Oct 25 16:32:24 UTC 2023"
          }
        }
      ],
      "service_id": "torch-inference-service"
    }
  }
}

Data Types

Container

Information about a container.

  • id (string) - Container (service) ID
  • count (number) - Number of discoverable nodes running this container'
  • description (string, optional) - Description of the container
{
  "id": string,
  "count": number,
  "description": string
}

MLModel

Represents a machine learning model.

  • owner (string) - The organization that owns the model.
  • name (string) - The name of the model.
  • repository (string) - The repository where the model is stored.
  • storage (string) - The storage backend where the model is stored
  • files (string[]) - List of files associated with the model.
  • version (string) - The version of the model.
  • ml_type (string) - The type of machine learning model.
{
  "owner": string,
  "name": string,
  "repository": string,
  "storage": string,
  "files": string[],
  "version": string,
  "ml_type": string
}

CPUInfo

The CPU information of the hardware capability.

  • architecture (string) - CPU architecture.
  • byte_order (string) - CPU byte order.
  • cores (string[]) - CPU cores.
  • model (string) - CPU model.
  • num_cores (integer) - Number of CPU cores.
  • vendor_id (string) - CPU vendor ID.
{
  "architecture": string,
  "byte_order": string,
  "cores": string[],
  "model": string,
  "num_cores": integer,
  "vendor_id": string
}

DiskInfo

Disk information of the hardware capability.

  • available (integer) - Available disk space.
  • filesystem (string) - Filesystem type.
  • mount_point (string) - Disk mount point.
  • size (integer) - Total disk size.
  • used (integer) - Used disk space.
{
  "available": integer,
  "filesystem": string,
  "mount_point": string,
  "size": integer,
  "used": integer
}

OSInfo

OS information of the hardware capability.

  • name (string) - OS name.
  • version (string) - OS version.
{
  "name": string,
  "version": string
}

ServiceResources

  • service_id (string) - The unique identifier for the inference service.
  • compute_capability (object[]) - List of compute capabilities available for the service.
    • id (string) - Identifier for the compute capability.
    • type (string) - Type of inference service (e.g., CSS, ONNX).
    • task (string[]) - List of tasks supported by the compute capability.
    • models (MLModel[]) - List of MLModel objects supported by the compute capability.
    • cached_models (MLModel[]) - List of MLModel objects that are cached for faster access.
    • inference_engine (string, optional) - Inference engine used by the compute capability.
    • inference_engine_version (string, optional) - Version of the inference engine.
  • hardware_capabilities (HardwareCapability[]) - List of hardware capabilities available for the service.
    • capability_id (string) - Unique identifier for the hardware capability.
    • cpu_info (CPUInfo) - CPUInfo object.
    • disk_info (DiskInfo) - DiskInfo object.
    • os_info (OSInfo) - OSInfo object.

ModelSupport

Model support information for a container.

  • supported (boolean) - Whether the model ID is supported.
  • error (string, optional) - A message explaining why the model ID is unsupported or erroneous.
{
  "supported": boolean,
  "error": string
}
IntroductionREST Client