Skip to content
Models
Skip to content

Docs

Install, compile to .tic, and serve. We currently support vLLM workloads on NVIDIA GPUs.

API

OpenAI-compatible inference API served by isiro serve.

isiro serve owns the inference API on port 8000. It is OpenAI-compatible, so you can use OpenAI clients and SDKs with it. Prometheus /metrics is on the same port.

Base URL: http://HOST:8000/v1 (e.g. on localhost, http://127.0.0.1:8000/v1)

Smoke test:

curl http://127.0.0.1:8000/v1/models