openlit-cc

---

https://github.com/user-attachments/assets/6909bf4a-f5b4-4060-bde3-95e91fa36168

OpenLIT allows you to simplify your AI development workflow, especially for Generative AI and LLMs. It streamlines essential tasks like experimenting with LLMs, organizing and versioning prompts, and securely handling API keys. With just one line of code, you can enable OpenTelemetry-native observability, offering full-stack monitoring that includes LLMs, vector databases, and GPUs. This enables developers to confidently build AI features and applications, transitioning smoothly from testing to production.

This project proudly follows and maintains the Semantic Conventions with the OpenTelemetry community, consistently updating to align with the latest standards in Observability.

⚡ Features

• 📈 Analytics Dashboard: Monitor your AI application's health and performance with detailed dashboards that track metrics, costs, and user interactions, providing a clear view of overall efficiency.

• 🔌 OpenTelemetry-native Observability SDKs: Vendor-neutral SDKs (Python, TypeScript, Go) to send traces and metrics to your existing observability tools.

• 🛡️ 11 Built-in Evaluation Types: Automated LLM-as-a-Judge evaluation with hallucination, bias, toxicity, safety, instruction following, completeness, conciseness, sensitivity, relevance, coherence, and faithfulness detection. Context-aware evaluation that treats provided context as the source of truth.

• ⚙️ Rule Engine: Define conditional rules with AND/OR logic to match runtime trace attributes and dynamically retrieve contexts, prompts, and evaluation configs. SDK support across Python, TypeScript, and Go.

• 💲 Cost Tracking for Custom and Fine-Tuned Models: Tailor cost estimations for specific models using custom pricing files for precise budgeting.

• 🐛 Exceptions Monitoring Dashboard: Quickly spot and resolve issues by tracking common exceptions and errors with a dedicated monitoring dashboard.

• 💭 Prompt Management: Manage and version prompts using Prompt Hub for consistent and easy access across applications.

• 🔑 API Keys and Secrets Management: Securely handle your API keys and secrets centrally, avoiding insecure practices.

• 🎮 Experiment with different LLMs: Use OpenGround to explore, test and compare various LLMs side by side.

• 🚀 Fleet Hub for OpAMP Management: Centrally manage and monitor OpenTelemetry Collectors across your infrastructure using the OpAMP (Open Agent Management Protocol) with secure TLS communication.

🚀 Getting Started with LLM Observability

flowchart TB;
    subgraph " "
        direction LR;
        subgraph " "
            direction LR;
            OpenLIT_SDK[OpenLIT SDK] -->|Sends Traces & Metrics| OTC[OpenTelemetry Collector];
            OTC -->|Stores Data| ClickHouseDB[ClickHouse];
        end
        subgraph " "
            direction RL;
            OpenLIT_UI[OpenLIT] -->|Pulls Data| ClickHouseDB;
        end
    end

Step 1: Deploy OpenLIT Stack

1. Git Clone OpenLIT Repository

   Open your command line or terminal and run:

   git clone [email protected]:openlit/openlit.git
   

2. Self-host using Docker

   Deploy and run OpenLIT with the following command:

   docker compose up -d
   

For instructions on installing in Kubernetes using Helm, refer to the Kubernetes Helm installation guide.

Step 2: Install OpenLIT SDK

Open your command line or terminal and run:

pip install openlit

For instructions on using the TypeScript SDK, visit the TypeScript SDK Installation guide.

Step 3: Initialize OpenLIT in your Application

Integrate OpenLIT into your AI applications by adding the following lines to your code.

import openlit

openlit.init()

Configure the telemetry data destination as follows:

Purpose	Parameter/Environment Variable	For Sending to OpenLIT
Send data to an HTTP OTLP endpoint	otlp_endpoint or OTEL_EXPORTER_OTLP_ENDPOINT	"http://127.0.0.1:4318"
Authenticate telemetry backends	otlp_headers or OTEL_EXPORTER_OTLP_HEADERS	Not required by default

💡 Info: If the otlp_endpoint or OTEL_EXPORTER_OTLP_ENDPOINT is not provided, the OpenLIT SDK will output traces directly to your console, which is recommended during the development phase.

Example

---

Initialize using Function Arguments

Add the following two lines to your application code:

import openlit

openlit.init(
  otlp_endpoint="http://127.0.0.1:4318", 
)

---

---

Initialize using Environment Variables

Add the following two lines to your application code:

import openlit

openlit.init()

Then, configure the your OTLP endpoint using environment variable:

export OTEL_EXPORTER_OTLP_ENDPOINT = "http://127.0.0.1:4318"

---

Step 4: Visualize and Optimize

With the Observability data now being collected and sent to OpenLIT, the next step is to visualize and analyze this data to get insights into your AI application's performance, behavior, and identify areas of improvement.

Just head over to OpenLIT at 127.0.0.1:3000 on your browser to start exploring. You can login using the default credentials:

• Email: [email protected]
• Password: openlituser

🤖 AI Coding Agents (Claude Code, Cursor, Codex)

OpenLIT also ships a CLI that brings the same observability story to your local coding agents — Claude Code, Cursor, and Codex. No SDK to import: the CLI installs vendor hooks that emit OpenTelemetry traces for every session, prompt, tool call, file edit, subagent spawn, and code-impact event.

Install the CLI

# macOS + Linux
curl -fsSL https://raw.githubusercontent.com/openlit/openlit/main/cli/scripts/install.sh | sh

# Windows (PowerShell)
iwr -useb https://raw.githubusercontent.com/openlit/openlit/main/cli/scripts/install.ps1 | iex

Wire it into your agents

openlit configure --endpoint http://127.0.0.1:4318 [--api-key <key>]
openlit coding install --vendor=all   # or: cursor / claude-code / codex

Inspect what's running

openlit doctor    # config + OTLP reachability + installed plugins

Sessions, prompts, costs, and per-vendor breakdowns appear in the OpenLIT dashboard at http://127.0.0.1:3000/coding-agents. To detach later:

openlit coding uninstall --vendor=all

Coding-agent observability is OTel-native end-to-end: traces use the gen_ai.* semantic conventions alongside an coding_agent.* extension namespace. Anything that speaks OTLP (Datadog, Honeycomb, Grafana Tempo, raw OTel Collector) can consume the same data — OpenLIT is just one possible viewer.

📦 Supported Integrations

OpenLIT auto-instruments 50+ LLM providers, AI frameworks, and vector databases with a single line of code. Each integration produces OpenTelemetry-native traces and metrics. Click any card to view the integration docs.

_{ Python SDK     TypeScript SDK}

LLM Providers

<table> <tr> <td align="center" width="160">[<b>OpenAI</b>](https://docs.openlit.io/latest/sdk/integrations/openai)  </td> <td align="center" width="160">[<b>Anthropic</b>](https://docs.openlit.io/latest/sdk/integrations/anthropic)  </td> <td align="center" width="160">[<b>Cohere</b>](https://docs.openlit.io/latest/sdk/integrations/cohere)  </td> <td align="center" width="160">[<b>Mistral AI</b>](https://docs.openlit.io/latest/sdk/integrations/mistral)  </td> </tr> <tr> <td align="center" width="160">[<b>Groq</b>](https://docs.openlit.io/latest/sdk/integrations/groq)  </td> <td align="center" width="160">[<b>Google AI Studio</b>](https://docs.openlit.io/latest/sdk/integrations/google-ai-studio)  </td> <td align="center" width="160">[<b>Together AI</b>](https://docs.openlit.io/latest/sdk/integrations/together)  </td> <td align="center" width="160">[<b>Ollama</b>](https://docs.openlit.io/latest/sdk/integrations/ollama)  </td> </tr> <tr> <td align="center" width="160">[<b>AWS Bedrock</b>](https://docs.openlit.io/latest/sdk/integrations/bedrock)  </td> <td align="center" width="160">[<b>Azure AI Inference</b>](https://docs.openlit.io/latest/sdk/integrations/azure-ai-inference)  </td> <td align="center" width="160">[<b>Vertex AI</b>](https://docs.openlit.io/latest/sdk/integrations/vertexai) </td> <td align="center" width="160">[<b>vLLM</b>](https://docs.openlit.io/latest/sdk/integrations/vllm) </td> </tr> <tr> <td align="center" width="160">[<b>Reka</b>](https://docs.openlit.io/latest/sdk/integrations/reka) </td> <td align="center" width="160">[<b>LiteLLM</b>](https://docs.openlit.io/latest/sdk/integrations/litellm) </td> <td align="center" width="160">[<b>Hugging Face</b>](https://docs.openlit.io/latest/sdk/integrations/huggingface)  </td> <td align="center" width="160">[<b>AI21</b>](https://docs.openlit.io/latest/sdk/integrations/ai21) </td> </tr> <tr> <td align="center" width="160">[<b>GPT4All</b>](https://docs.openlit.io/latest/sdk/integrations/gpt4all) </td> <td align="center" width="160">[<b>PremAI</b>](https://docs.openlit.io/latest/sdk/integrations/premai) </td> <td align="center" width="160">[<b>Sarvam AI</b>](https://docs.openlit.io/latest/sdk/integrations/sarvam) </td> <td align="center" width="160">[<b>Julep</b>](https://docs.openlit.io/latest/sdk/integrations/julep-ai) </td> </tr> <tr> <td align="center" width="160">[<b>MultiOn</b>](https://docs.openlit.io/latest/sdk/integrations/multion) </td> <td align="center" width="160">[<b>Replicate</b>](https://docs.openlit.io/latest/sdk/integrations/replicate) </td> <td align="center" width="160">[<b>DigitalOcean (pydo)</b>](https://docs.openlit.io/latest/sdk/integrations/digitalocean) </td> <td align="center" width="160">[<b>DigitalOcean Gradient</b>](https://docs.openlit.io/latest/sdk/integrations/digitalocean-gradient) </td> </tr> </table>

AI Frameworks

<table> <tr> <td align="center" width="160">[<b>LangChain</b>](https://docs.openlit.io/latest/sdk/integrations/langchain)  </td> <td align="center" width="160">[<b>LlamaIndex</b>](https://docs.openlit.io/latest/sdk/integrations/llama-index)  </td> <td align="center" width="160">[<b>CrewAI</b>](https://docs.openlit.io/latest/sdk/integrations/crewai) </td> <td align="center" width="160">[<b>Pydantic AI</b>](https://docs.openlit.io/latest/sdk/integrations/pydantic) </td> </tr> <tr> <td align="center" width="160">[<b>Agno</b>](https://docs.openlit.io/latest/sdk/integrations/agno) </td> <td align="center" width="160">[<b>Browser Use</b>](https://docs.openlit.io/latest/sdk/integrations/browser-use) </td> <td align="center" width="160">[<b>Haystack</b>](https://docs.openlit.io/latest/sdk/integrations/haystack) </td> <td align="center" width="160">[<b>Letta</b>](https://docs.openlit.io/latest/sdk/integrations/letta) </td> </tr> <tr> <td align="center" width="160">[<b>Mem0</b>](https://docs.openlit.io/latest/sdk/integrations/mem0) </td> <td align="center" width="160">[<b>AG2 (AutoGen)</b>](https://docs.openlit.io/latest/sdk/integrations/ag2) </td> <td align="center" width="160">[<b>Controlflow</b>](https://docs.openlit.io/latest/sdk/integrations/controlflow) </td> <td align="center" width="160">[<b>Crawl4AI</b>](https://docs.openlit.io/latest/sdk/integrations/crawl4ai) </td> </tr> <tr> <td align="center" width="160">[<b>Dynamiq</b>](https://docs.openlit.io/latest/sdk/integrations/dynamiq) </td> <td align="center" width="160">[<b>OpenAI Agents</b>](https://docs.openlit.io/latest/sdk/integrations/openai-agents)  </td> <td align="center" width="160">[<b>Firecrawl</b>](https://docs.openlit.io/latest/sdk/integrations/firecrawl) </td> <td align="center" width="160">[<b>Google ADK</b>](https://docs.openlit.io/latest/sdk/integrations/google-adk)  </td> </tr> <tr> <td align="center" width="160">[<b>LangGraph</b>](https://docs.openlit.io/latest/sdk/integrations/langgraph)  </td> <td align="center" width="160">[<b>Smolagents</b>](https://docs.openlit.io/latest/sdk/integrations/smolagents) </td> <td align="center" width="160">[<b>Strands Agents</b>](https://docs.openlit.io/latest/sdk/integrations/strands)  </td> <td align="center" width="160">[<b>Claude Agent SDK</b>](https://docs.openlit.io/latest/sdk/integrations/claude-agent-sdk)  </td> </tr> <tr> <td align="center" width="160">[<b>MS Agent Framework</b>](https://docs.openlit.io/latest/sdk/integrations/agent-framework) </td> <td align="center" width="160">[<b>Vercel AI</b>](https://docs.openlit.io/latest/sdk/integrations/vercel-ai) </td> </tr> </table>

Vector Databases

<table> <tr> <td align="center" width="160">[<b>Pinecone</b>](https://docs.openlit.io/latest/sdk/integrations/pinecone)  </td> <td align="center" width="160">[<b>ChromaDB</b>](https://docs.openlit.io/latest/sdk/integrations/chromadb)  </td> <td align="center" width="160">[<b>Qdrant</b>](https://docs.openlit.io/latest/sdk/integrations/qdrant)  </td> <td align="center" width="160">[<b>Milvus</b>](https://docs.openlit.io/latest/sdk/integrations/milvus)  </td> </tr> <tr> <td align="center" width="160">[<b>Astra DB</b>](https://docs.openlit.io/latest/sdk/integrations/astradb) </td> <td align="center" width="160">[<b>PostgreSQL</b>](https://docs.openlit.io/latest/sdk/integrations/psycopg) </td> </tr> </table>

Specialized Tools

<table> <tr> <td align="center" width="160">[<b>ElevenLabs</b>](https://docs.openlit.io/latest/sdk/integrations/elevenlabs)  </td> <td align="center" width="160">[<b>AssemblyAI</b>](https://docs.openlit.io/latest/sdk/integrations/assemblyai) </td> <td align="center" width="160">[<b>MCP</b>](https://docs.openlit.io/latest/sdk/integrations/mcp) </td> </tr> </table>

🛣️ Roadmap

We are dedicated to continuously improving OpenLIT. Here's a look at what's been accomplished and what's on the horizon:

Feature	Status
OpenTelemetry-native Observability SDK for Tracing and Metrics	✅ Completed
OpenTelemetry-native GPU Monitoring	✅ Completed
Exceptions and Error Monitoring	✅ Completed
Prompt Hub for Managing and Versioning Prompts	✅ Completed
OpenGround for Testing and Comparing LLMs	✅ Completed
Vault for Central Management of LLM API Keys and Secrets	✅ Completed
Cost Tracking for Custom Models	✅ Completed
Real-Time Guardrails Implementation	✅ Completed
Programmatic Evaluation for LLM Response	✅ Completed
Fleet Hub for OpAMP Management	✅ Completed
Auto-Evaluation Metrics Based on Usage	🔜 Coming Soon
Human Feedback for LLM Events	🔜 Coming Soon
Dataset Generation Based on LLM Events	🔜 Coming Soon
Search over Traces	🔜 Coming Soon

🌱 Contributing

Whether it's big or small, we love contributions 💚. Check out our Contribution guide to get started

Unsure where to start? Here are a few ways to get involved:

• Join our Slack or Discord community to discuss ideas, share feedback, and connect with both our team and the wider OpenLIT community.

Your input helps us grow and improve, and we're here to support you every step of the way.

💚 Community & Support

Connect with OpenLIT community and maintainers for support, discussions, and updates:

• 🌟 If you like it, leave a star on our GitHub.
• 🌍 Join our Slack or Discord community for live interactions and questions.
• 🐞 Report bugs on our GitHub Issues to help us improve OpenLIT.
• 𝕏 Follow us on X for the latest updates and news.

License

OpenLIT is available under the Apache-2.0 license.

🙇‍♂️ Acknowledgments

This project is proudly supported by:

[
				![](https://raw.githubusercontent.com/openlit/openlit/9da454a69af920f50b99af869ac014d729188005/plugins/claude-code/docs/images/testmu-logo.png)
			](https://www.testmuai.com/?utm_medium=sponsor&utm_source=openlit)        

💻 Contributors