Hedged failover
When an attempt runs long, the next provider starts in parallel and the first answer wins. Errors fail over at the cost of one round trip.
← All products
Open source
Free LLM API for Python and Node.js
freelm
One OpenAI-compatible client over the free tiers of Gemini, Groq, OpenRouter, Cloudflare and more, with failover that races slow providers.
Verified facts
- Version 0.5.0, released October 10, 2026, on PyPI and npm
- Nine keyed providers plus two keyless fallbacks
- A hung provider cost 6.3 s instead of a 60 s failure (simulation, October 10, 2026)
- Free-only guard on mixed catalogues by default
- Zero runtime dependencies in JavaScript; MIT licensed
System brief
What it does.
freelm is an open-source client and local endpoint that pools nine free-tier LLM providers and two keyless fallbacks behind one OpenAI-compatible call. It rotates keys, tracks per-key and per-model quotas, races a slow provider against the next one, ranks providers by measured latency, and discovers which models are free today. Your own keys are called directly: there is no relay and nothing to host.
Latency-aware routing
Providers are ranked by measured latency within priority tiers. Unknown providers are tried; old samples expire after ten minutes.
Quota tracking
Per-key request buckets, daily counters, per-model benches and circuit breakers keep throttled keys out of rotation.
Local endpoint
freelm serve exposes /v1/chat/completions and /v1/models for OpenCode, Cline, Continue, Open WebUI, n8n and the OpenAI SDKs.
Live model discovery
Each provider's model list is read at runtime and cached, so retired free models drop out without a release.
freelm doctor
One tiny live request per key, with the fix and signup link for every failure.
Start
Install it and make the first call.
One free key is enough. Google AI Studio has the most generous free tier; add Groq and OpenRouter for failover. With no key at all, the CLI uses keyless endpoints and tells you so.
pip install freelm # Python 3.9+
npm install freelm # Node 20+, zero dependencies
export GEMINI_API_KEY=... # any free keys you have
freelm doctor # one tiny live request per keyimport freelm
llm = freelm.FreeLLM.from_env()
print(llm.text("Explain failover in one sentence."))freelm serve # OpenAI-compatible: http://127.0.0.1:4000/v1, model "auto"Measured October 10, 2026
How long a broken provider costs you.
Errors fail over at the cost of one round trip. A provider that hangs, or a stream that never starts, is raced: after a few seconds (three times that provider's usual latency, clamped) the next provider starts in parallel and the first answer wins. The slow one is cancelled and ranked lower, so the next call goes straight to the fast provider.
Run the simulation yourself ↗| Failure in front of a healthy provider | 0.4.0 | 0.5.0, first call | 0.5.0, next call |
|---|---|---|---|
| Rate limit (429), then healthy | 0.3 s | 0.3 s | 0.3 s |
| Server error (500), then healthy | 0.3 s | 0.3 s | 0.3 s |
| Connection refused, then healthy | 0.3 s | 0.3 s | 0.3 s |
| Provider hangs, then healthy | failed after 60 s | 6.3 s | 0.3 s |
| Unroutable host, then healthy | failed after 60 s | 6.3 s | 0.3 s |
| Slow provider (8 s), then healthy | 8.0 s every call | 6.3 s | 0.3 s |
| Stream stalls before the first token | failed after 60 s | 3.3 s | 0.3 s |
Simulation with fake providers on localhost over real HTTP (macOS arm64, Python 3.14 and Node 22; both clients gave the same times). The healthy provider answers in 0.3 s; times are wall-clock for one call with default settings and a 60 s deadline. Real providers add their own latency. Reproduce with benchmarks/failover.py or failover.mjs in the repository.
Providers, limits checked October 2026
Nine keyed providers and two keyless fallbacks.
| Provider | Free access | Environment variable | Notes |
|---|---|---|---|
| Google AI Studio (Gemini) | Free tier with per-model limits; Pro models have no free quota | GEMINI_API_KEY | Most generous free tier. A 429 sets aside only that model. |
| Groq | Per model: 30 requests/minute, 1,000 requests/day | GROQ_API_KEY | Fastest inference. Not xAI Grok, which is paid. |
| OpenRouter | :free models and the openrouter/free router; 20 requests/minute, 50 requests/day | OPENROUTER_API_KEY | Free-only guard blocks paid model ids by default. |
| Cloudflare Workers AIbeta | 10,000 Neurons/day on every account | CLOUDFLARE_API_TOKEN + CLOUDFLARE_ACCOUNT_ID | Workers Paid bills past the allowance; the Free plan stops. |
| Z.ai (GLM)beta | GLM-4.7-Flash, GLM-4.5-Flash and GLM-4.6V-Flash | ZAI_API_KEY | Every other GLM id is refused, so credit is never spent. |
| Coherebeta | Trial key: 20 requests/minute per model, 1,000 calls/month | COHERE_API_KEY | Trial keys are for non-commercial use. |
| Mistral | Experiment plan, low requests/minute | MISTRAL_API_KEY | Free account models only. |
| NVIDIA NIM | Free build credits | NVIDIA_API_KEY | Requests fail when credits run out; nothing is billed. |
| Cerebras | Trial credits only (no longer permanently free) | CEREBRAS_API_KEY | Supported for existing keys. |
| Kilo Gatewaykeyless | Free routes work without a key, about 200 requests/hour per IP | KILO_API_KEY (optional) | Keyless fallback; free routes may log prompts. |
| OVHcloud AI Endpointskeyless | Anonymous: 2 requests/minute per IP per model | none | Last-resort keyless fallback. |
Free tiers change often. Beta providers were added in 0.5.0 and have not yet been verified with live keys. freelm discovers current free models from each provider's catalogue, and freelm doctor reports which of your keys work today.
Routing
What happens on each kind of failure.
A 429 cools that key, or, where quotas are per model (Gemini, Groq, Z.ai, Cohere), sets aside only that model for that key. A spent daily quota rests the key for an hour instead of being retried every minute, and a spent monthly quota disables it. A 5xx or timeout trips a circuit breaker. A bad key is disabled, and a retired model is set aside for an hour.
Candidates are interleaved across providers, so every provider's best model is tried before any provider's second model. A request one provider rejects goes to the next; only a request that two providers reject is reported as a caller error. Routing ranks providers by measured latency, and samples expire after ten minutes so a provider that was slow once is tried again.
Free model ids change constantly, so freelm reads each provider's model list at runtime and caches it for an hour, with built-in lists as the fallback. Library or proxy: how the open-source gateways differ.
Questions
freelm, answered.
What is freelm?
freelm is an open-source (MIT) client for Python and Node.js that turns the free tiers of Gemini, Groq, OpenRouter, Cloudflare Workers AI, Mistral, NVIDIA NIM, Z.ai and Cohere into one OpenAI-compatible LLM. It rotates your keys, fails over between providers on rate limits, outages, slow responses and retired models, and can run as a local /v1 endpoint for tools that speak the OpenAI API.
Is freelm free to use?
The package is free and MIT licensed, and it calls each provider directly with your own keys, so there is no relay or markup. It is free-only by default: paid models on OpenRouter, Kilo and Z.ai are refused unless you opt in. Whether a provider ever bills you depends on your own account plan, for example Gemini with billing enabled or Cloudflare's Workers Paid plan.
Can I use it without an API key?
Yes, for trying it out. With no keys set, the freelm CLI falls back to keyless public endpoints (Kilo Gateway free routes and the OVHcloud anonymous tier) and says so. Their limits are low and free routes may log prompts, so one free Gemini or Groq key is the better start. The freelm doctor command checks every key you set and links to a fix.
How fast does freelm fail over?
An error such as a 429 or a 500 moves to the next provider at the cost of one round trip. A provider that hangs is raced: in the published simulation the call still answered in 6.3 seconds, and a stream that stalled before its first token answered in 3.3 seconds. Before 0.5.0 both cases failed after the 60 second deadline. Later calls go to the fast provider first.
Can I use it with OpenCode, Cline, Continue, Open WebUI or n8n?
Yes. Run freelm serve (or npx freelm serve, or the Docker image) and point the tool at http://127.0.0.1:4000/v1 with the model set to auto. Any client that accepts an OpenAI-compatible base URL works, including LangChain and the OpenAI SDKs. Tools that call it from their own servers need a public URL and an API key set with --api-key.
How is freelm different from LiteLLM, OpenRouter or a self-hosted free gateway?
Most free-tier tools are servers you run, often with a dashboard and an admin login to secure. freelm is a library you import, with one runtime dependency in Python and none in JavaScript, and a local endpoint only when you want one. LiteLLM covers more than a hundred paid and free providers and is the better choice for paid production traffic. OpenRouter is one hosted aggregator with one bill; freelm uses it as one of its pools and fails over to Gemini, Groq, Cloudflare and others directly.
Official links