AI-OS – Unified AI Operating System
Architected and open-sourced a high-performance modular monolith system designed for agent routing, automated API key rotation, and real-time observability.
Dashboard
Main system overview.
Overview
AI-OS is a high-performance modular monolith system designed for agent routing, automated API key rotation, and real-time observability.
The Problem
Managing multiple LLM APIs, handling rate limits, and monitoring agent performance can become extremely complex in production AI applications.
The Approach
Architected a unified AI operating system with a smart key manager, fallback models, and a FastAPI core engine for sub-10ms event routing.
Agent routing architecture
AI-OS manages high-performance agent routing, handling API limits and model fallbacks automatically to ensure reliable LLM integrations.
My Contribution
Implemented a FastAPI core engine, a smart key manager using Celery, Redis, and Playwright, a Claude Code Proxy routing requests to fallback models (Gemini, Groq, OpenRouter), a Next.js admin dashboard, and Prometheus/Grafana observability pipelines. Containerized the entire 8-microservice stack using Docker Compose.
Key Capabilities
- Sub-10ms event routing via FastAPI core engine
- Automated API key rotation stack
- Smart key management using Celery, Redis, and Playwright
- Claude Code Proxy for fallback models (Gemini, Groq, OpenRouter)
- Next.js admin dashboard
- Prometheus/Grafana observability pipelines
- Docker Compose containerization of 8-microservice stack
Technology Stack
Project Context
Open-source high-performance agent routing and API management system.