Oragon SDK · Python
Run AI models inside your own software.
Oragon is the Oragonlabs engine as a Python package. Local-first model serving for software that has to work: on hardware you don't control, with data that can't leave. One install, no API key.
That is the whole install.
The engine is inside the package: no compiler, no separate download. The model loads once and stays ready, so the second request doesn't pay for the first one's startup.
pip install oragon # the engine is inside the package
oragon doctor # check what this machine can run
oragon try # download a small model and chat with itfrom oragon import Oragon, Engine, LocalModel, user
client = Oragon(
engine=Engine.EMBEDDED,
model=LocalModel.QWEN2_5_0_5B,
)
reply = client.chat.generate(
model=LocalModel.QWEN2_5_0_5B,
messages=[user("Name one primary color.")],
)
print(reply.choices[0].message.content)Output
One primary color is red.
Why Oragon
An odd answer from the model becomes something your code can catch.
It says when it doesn't know
Valid data or a clear error. The exact label you asked for or “no match”. A cited answer or an honest refusal. Never a guess dressed up as an answer.
It fails like production software
Every error is a typed exception that says whether a retry is safe. Memory is checked before a model loads, context length before a word is generated.
It handles many requests at once
Requests are batched, shared prompts are cached, overflow waits in a bounded queue, and a request turned away is retried for you.
Your data stays yours
The engine ships inside the package. No API key, no telemetry. Prompts and answers never leave the machine.
What you can do
Start for free.
Three engines share one API (embedded, shared and hosted), so where the model runs is a deployment choice, not a rewrite.
Also in the box
Structured data
Pass a JSON schema and get data that matches it, or an ExtractionError. The schema is enforced while the model writes.
Your documents
Ask questions about your own files and get cited answers, or a refusal when the documents don't say. Searched entirely on your machine.
OpenAI-compatible API
oragon serve puts an OpenAI-compatible API in front of the model, with a browser playground. Any OpenAI client, in any language, can use it.
Decisions
Offer a set of labels and get one of them back, or an honest “no match”. Never stretched to fit.
Tool use
Hand it plain Python functions and act() runs the whole loop, retrying when a tool fails.
Typed errors
Every error says whether a retry is safe. Too little memory or too long a prompt is caught before any work starts.
Measured, not estimated
Built for the smallest machine, first.
A 16 GB MacBook with an Apple M4, running qwen2.5-1.5b and no GPU server. Servers do better. Speed per request is set by your hardware and answer quality by the model; what Oragon adds is everything around it.
- Concurrent, CPU only
- 16
- requests, 12+ tok/s each
- Flood test
- 6,000
- requests, none left hanging
- Long system prompt
- 10s → 0.5s
- warmed at startup
- Startup
- 0.8s
- to first answer, small model
Runs on
Same code on a laptop, a server or a GPU.
CPU is a first-class path, not a fallback. If a GPU is there it is used automatically. Every feature works the same on every platform; only the speed-up differs.
- Python:
- 3.9 to 3.13
- macOS 11+, Apple silicon:
- Accelerated on the Apple GPU
- Linux x86-64:
- NVIDIA, AMD or Intel GPU via Vulkan, or CPU
- Linux arm64:
- CPU
- Servers & GPUs:
- pip install "oragon[gpu]" (Linux, NVIDIA)
- Windows, Intel Macs:
- Coming soon
Pricing & your data
Free unless you need more than four at once.
No token limits, and commercial use is allowed on the free tier. You can ship a desktop app to someone who never connects.
- Free:
- Up to 4 lanes plus the queue, and every feature, including the HTTP API and GPU serving
- Paid:
- More than 4 lanes, many processes sharing one model, and Oragon's licensed models, encrypted on disk
- Prompts & answers:
- Never leave the machine. No code path could send them
- Telemetry:
- None. No crash reports, no update checks
- Air-gapped:
- Supported. Licences work offline for their whole validity period
Building on Oragon?
Tell us what you're shipping. If you need more lanes, a shared engine across processes or a licensed model, we'll set it up with you.
Get in touch: hello@oragonlabs.com