A self-hosted, OpenAI-compatible proxy routing across 11+ free-tier LLM providers. Features automatic failover, AES-256 key encryption, token rate tracking, and an admin dashboard.
- Client
- FreeLLMAPI ProxyDeveloper Utility
- Year
- 2026Self-Hosted Utility
- Role
- Lead EngineerFull-Stack API Architect
- Stack
- Express · SQLite · ReactVite · Tailwind CSS
- Status
- ReleasedOpen Source
Every serious AI lab now offers a free tier — a few million tokens a month. Stacked together, they add up to roughly 1.3 billion tokens per month of working inference capacity. The problem is managing multiple SDKs, rate limits, and failure points.
FreeLLMAPI collapses this into a single OpenAI-compatible `/v1/chat/completions` endpoint. Point any standard OpenAI client library at your local proxy, and it routes transparently across your configured API keys.
Built with Express, SQLite, and React, it tracks per-key RPM/TPM usage to avoid rate limits, and uses AES-256-GCM envelope encryption to keep credentials safe at rest.
Aggregating the
free tiers of intelligence.
To build an inference aggregation layer that runs happily on a Raspberry Pi (~40 MB memory footprint) while handling automatic failover for high-volume local experiments.
Includes a React-based admin dashboard to monitor live latency charts, adjust priorities in the fallback chain, and test models in a shared playground.
Quick Start
and setup guidelines.
1. Clone & Install: Run git clone https://github.com/byteWizard-zero/my-freellmapi-proxy.git and run npm install in the directory to pull the dependencies.
2. Setup Keys: Copy .env.example to .env and run node -e "console.log(require('crypto').randomBytes(32).toString('hex'))" to append a unique ENCRYPTION_KEY.
3. Launch: Run npm run dev to start both the Express proxy server (listening on port 3001) and the React admin dashboard (listening on port 5173).
4. Configure: Open http://localhost:5173 in your browser, enter your provider API keys (Google, Groq, Cerebras, etc.), prioritize the fallback list, and grab the unified API key.
5. Use in Code: Initialize any standard OpenAI SDK client by setting the base_url pointing to http://localhost:3001/v1 and passing your unified freellmapi-your-key as the token. Use the model auto to let the proxy select the best healthy backend dynamically.
Interactive 3 Proxy Dashboard.
Express server
SQLite key security.
FreeLLMAPI separates client query layers from rate limit counters, running a low-latency proxy engine on local device networks.
- Express
- TypeScript
- better-sqlite3
Receives client requests, queries local SQLite database for healthy keys, determines failover targets, and translates payloads between OpenAI schema and Gemini/other custom models.
- React
- Vite
- shadcn/ui
- Recharts
Admin interface displaying live rate tracking, request logs, database keys, and configuration for custom model fallback hierarchies.
Aggregated tiers,
failovers resolved.
Engineered by byteWizard-zero.
- Architecture & Design
- Zenith Soumya
- API Developer
Zenith Soumya is an AI Developer and IoT system designer. He specializes in low-overhead proxy systems and secure developer tools.
FreeLLMAPI is open-source on GitHub, serving as a unified local endpoint for LLM prototyping.