Pay for the GPU, not the token: why we built inference instances
We built syndicAI for one job: a squad of developers coding with AI on a GPU they share. You pay for the GPU by the hour, not by the token, and that turned out to be 5 to 70 times cheaper than token APIs for agentic coding.
Then people asked a different question. "Can I run my OCR on it?" "Can I put a chatbot behind it?" "I have two million documents to embed." None of those is coding, and all of them have the same shape as coding at its worst: a lot of tokens, sent steadily, to a model you could just as well run yourself.
So we built inference instances.
What an inference instance is
An inference instance is one dedicated GPU machine that runs one open-weight model behind an OpenAI-compatible endpoint. You choose the model (chat, vision, embeddings or OCR), tell us your peak requests per minute and what a typical request looks like, and we show you the GPU memory it needs, the speed to expect and whether a GPU can carry the load. You pick the hardware, create an API key per application, and call it.
You pay for the GPU while it runs, from prepaid credit. There is no per-token and no per-page charge.
Why GPU time and not tokens
A GPU costs the same per hour whether it handles one request or a thousand. So the more steady work you send it, the less each request costs. A token API does the opposite: every request has a price, however busy you are.
That makes the two a trade, and which one wins depends on the shape of your load. We would rather show you the numbers than claim a winner, so here are four loads at the defaults of our cost calculator, sized with the same capacity estimator as the instance wizard, with prices checked on 28 October 2026:
| Load | GPU instance | Per-token or per-page API | Cheaper |
|---|---|---|---|
| OCR, 60 pages a minute at peak, 8 hours a day | $180 a month (1x RTX 6000 Ada) | $648 (AWS Textract text detection) | GPU instance |
| Vision, 60 images a minute at peak, 8 hours a day | $180 a month (1x RTX 6000 Ada) | $356 (Gemini 2.5 Flash) | GPU instance |
| Chatbot, 30 chats a minute at peak, 8 hours a day | $506 a month (2x RTX PRO 5000) | $201 (Gemini 2.5 Flash) | API |
| Embeddings, 600 documents a minute at peak, 8 hours a day | $180 a month (1x RTX 6000 Ada) | $44 (OpenAI text-embedding-3-small) | API |
Two of those go to the API. That is on purpose. A chatbot at 30 chats a minute does not keep a GPU busy enough to beat a cheap, small model on a per-token price, and OpenAI's smallest embedding model is priced at $0.02 per million tokens, which is hard to beat with any GPU at modest volume.
Now make the load heavier. Read 600 pages a minute, around the clock at 70 percent of that on average, and the GPU instance costs about $1,900 a month against about $27,000 for per-page OCR. Embed 20,000 documents a minute the same way, and 14 small GPUs cost about $7,500 a month against $40,000 for the larger OpenAI embedding model. Bulk work is where a GPU wins, and it wins by a lot.
The rule of thumb: if the GPU would be busy for most of the hours it runs, rent the GPU. If it would sit idle most of the time, use the API, or pause the instance between jobs. The calculator shows the break-even rate for your own numbers.
What we are careful not to claim
Per-token prices come from different models with different quality. The calculator compares cost for a similar job, not quality, and says which open-weight model it assumed. The estimates are estimates: our capacity model is fitted to measured runs on two GPU classes, and the instance card says when it uses datasheet figures instead.
On data, here is exactly what happens. We do not store or log what you send. Requests are relayed through our proxy on Cloudflare to your GPU, and the proxy reads a request only to enforce size limits and to detect image content. The GPU is dedicated to you, and rented from GPU marketplaces and cloud providers; you choose the region and a hardware trust tier, and verified hosts are the default. Availability is best effort: a rented host can be interrupted, we bring the instance back on a new one, and there is no SLA. If you need dedicated nodes inside a specific country, that is syndicAI Enterprise.
Try it
Inference instances are in early access. Put your own numbers into the calculator, read how sizing works, and join the waiting list at app.syndicai.dev. If the calculator tells you to stay on the API, that is a fine answer too.