Inference Instances
What an inference instance is
An inference instance is one dedicated GPU machine that runs one open-weight model and serves it over an OpenAI-compatible endpoint. You choose the model, describe your load, pick the hardware, and your application calls the endpoint with an API key. You pay for the GPU while the instance runs, from prepaid credit. There is no per-token or per-page charge.
Use it as the AI backend of a product or a pipeline:
- Chatbots and assistants behind a product or an internal tool.
- OCR and document pipelines that turn page images into text.
- Computer vision: questions about images, screenshots and scans with a vision model.
- Embeddings for search, retrieval and clustering.
Two kinds of instance
When you create an instance you first choose its kind.
| Coding instance | Inference instance | |
|---|---|---|
| For | A team coding with AI tools | Applications and pipelines |
| Models | Coding models | Chat, vision, embedding and OCR models |
| Sized by | Members and speed preference | Peak requests per minute and a typical request |
| Access | Squad members, invite links | The owner and the API keys the owner creates |
| Billing | GPU hours inside a weekly budget | GPU runtime from prepaid credit |
| Rate limits | None | Optional per-key requests per minute |
An inference instance has no members, no invite links and no cost splitting. The owner manages it and creates the API keys that applications use.
How it works in short
- Choose a model. The picker shows chat, vision, embedding and OCR models, with filters by category. See Inference Models.
- Describe your load. Give the peak requests per minute and pick a typical request: a short chat, a long document, an OCR page, an embedding batch. syndicAI estimates the GPU memory the load needs and whether a GPU can carry it. See Inference Sizing.
- Pick hardware. You see the cheapest GPU that can carry the load, with Balanced and Performance options when they differ in speed. Hardware from verified hosts is preselected.
- Create keys and call the endpoint.
https://chat.syndicai.dev/<your-instance>/v1. See Inference API. - Watch the load. The usage tab shows requests per minute against your peak, and per-key rejections. See Inference Limits.
What you should know before you start
- Availability is best effort. Instances run on rented GPUs, and a host can be interrupted. We restart the instance on a new host, but there is no SLA. Build clients that retry.
- Your prompts are not stored or logged by syndicAI. Requests are relayed through our proxy on Cloudflare to your GPU. The proxy reads a request only to enforce size limits and detect image content, and keeps nothing. The GPU is dedicated to you. You choose the region and a hardware trust tier, and verified hosts are the default. For sensitive documents, keep that default. For dedicated in-country nodes, see syndicAI Enterprise.
- A GPU is a fixed cost. It pays off for sustained or bulk work. For a few requests a day, a pay-per-token API is cheaper. The cost calculator shows where the break-even is for your load. See Inference Billing.