oss-llmops-stack.com• Feb 26, 2025• 1 min read
Open Source LLMOps StackSome background: I work on Langfuse and we've been collaborating with LiteLLM.(LiteLLM is a Python library and proxy/gateway that handles cost management, virtual keys, caching, and rate-limiting for OpenAI or other LLM APIs. Langfuse manages LLM tracing, evaluation, prompt management, and experiments.)We’ve each been building our open-source projects since early 2023 and learned that many devs and especially platform teams use the two together, so we created an integrated “OSS LLMOps stack.”This is a fully self-hostable, technology-agnostic setup that lets you (1) Use LLMs via a standardized interface without adding complexity to the application; (2) Keep LLM Tracing, Evaluation, Prompt Management in-house for compliance; (3) Track cost and usage via a single interface, create virtual API keys for attribution of costsIt also enables direct transfer of LLM traces from the LiteLLM proxy to Langfuse. This simplifies the rollout of LLMOps practices (observability and evaluations) across m
