Drop-in OpenAI Compatibility: Migrating to Free LLMs

Migrating off expensive OpenAI APIs shouldn't require rewriting your app. Here is how to do it in two lines.

The OpenAI Shim

If you already use the OpenAI SDK, the drop-in shim from freelm swaps in with almost no code change:

# Old code:
# from openai import OpenAI

# New code:
from freelm.compat import OpenAI

client = OpenAI()  # Backed by FreeLLM.from_env()
r = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "hi"}],
)
print(r.choices[0].message.content)

Virtual Models

Because names differ per provider, you ask by intent. Use model="auto", model="chat:fast", or model="chat:large". freelm automatically maps this to the correct concrete model for the active provider.

Frequently Asked Questions (FAQ)

Q: Do I need to change my prompt structure?
A: No. The standard OpenAI messages array works perfectly.

Q: Are all OpenAI SDK features supported?
A: Currently, we support chat completions, async, and streaming. Embeddings and vision are on the v2 roadmap.

Summary

Save money without the migration headache. Try the drop-in shim with pip install freelm today.