SHAHRIAR LABSIntelligence in Motion
    Open Source • Python & Node.js

    One API. Six Free Providers. Zero Downtime.

    freelm is a free, always-up LLM client and API gateway for Python and Node.js. It pools free-tier LLM providers like OpenRouter, Gemini, NIM, Groq, Cerebras, and Mistral behind a single OpenAI-compatible interface, offering automatic failover and dynamic model discovery.

    Read the DocumentationView on GitHub
    PYTHON (PyPI)
    pip install freelm
    # Swap OpenAI() with the drop-in freelm shim
from freelm.compat import OpenAI

client = OpenAI() 

# Routes across 6 providers automatically
response = client.chat.completions.create(()
    model="auto", # virtual model routing
    messages=[{"role": "user", "content": "hi"}]
)

print(response.choices[0].message.content)
    NODE.JS (npm)
    npm install freelm
    // Built-in zero-dependency TypeScript/JS port
import { OpenAI } from "freelm/compat";

const client = new OpenAI();

// Handled securely with exponential backoff & failover
const response = await client.chat.completions.create({)
    model: "chat:fast",
    messages: [{ role: "user", content: "hi" }]
});

console.log(response.choices[0].message.content);

    Enterprise reliability. Free tier pricing.

    Stop managing multiple API keys, quotas, and SDKs. Write your AI logic once and let freelm handle the operations behind the scenes.

    🔀

    Multi-Provider Pooling

    Combine OpenRouter, Gemini (AI Studio), NVIDIA NIM, Groq, Cerebras, and Mistral. Multiply your API quota effortlessly.

    🛡️

    Zero-Downtime Failover

    If one provider rate-limits or crashes, freelm instantly retries with the next available provider. Streaming fails over seamlessly.

    🔌

    Drop-in OpenAI Shim

    Zero code rewrites. Replace your standard OpenAI import with freelm's compatibility wrapper and maintain identical SDK behavior.

    🌊

    Native Streaming

    Full support for async iteration and real-time token streaming across all underlying providers.

    🔍

    Live Model Discovery

    Free models constantly change. freelm dynamically queries endpoints, categorizes virtual models (e.g., chat:fast), and caches them.

    ⏱️

    Smart Circuit Breakers

    Dead or rate-limited keys are temporarily isolated into a cooldown state, preventing wasted requests and reducing overall latency.

    Start building immediately

    $ pip install freelm

    Python (PyPI) • View Package

    $ npm install freelm

    Node.js (npm) • View Package

    Frequently Asked Questions

    What is freelm?
    freelm is an open-source, always-up LLM client and API gateway for Python and Node.js. It pools free-tier LLM providers like OpenRouter, Gemini, NIM, Groq, Cerebras, and Mistral behind a single OpenAI-compatible interface, offering automatic failover and dynamic model discovery.
    How does the automatic failover work?
    When a provider hits a rate limit (429) or fails (5xx), freelm intercepts the error before throwing an exception. It trips a circuit breaker for that API key and instantly retries the request using the next available key or provider in your pool.
    Which routing strategies does freelm support?
    freelm supports four routing strategies: 'priority' (strict ordering), 'round_robin' (even load distribution), 'quota_aware' (prioritizing keys with the most remaining daily/RPM limits), and 'latency' (routing to the fastest provider).
    Is freelm actually free to use?
    Yes. The freelm package is free (MIT-licensed). It routes your requests to the free tiers of supported providers so you don't need a credit card. Your total throughput scales with the combined free quotas of the API keys you provide.
    Does it support the OpenAI SDK format?
    Yes, both the Python and Node.js versions provide a drop-in OpenAI shim. You can import OpenAI from the freelm compatibility module and use your existing chat completion and streaming code without changes.
    Do I need API keys for all supported providers?
    No. You only need to supply the keys you have. freelm automatically detects which keys are available in your environment variables and builds its routing pool dynamically.
    Explore the full API Documentation