{"id":74316,"title":"🚅 LiteLLM - Call all LLM APIs using the OpenAI format [Llama2, Anthropic, Huggingface, etc.]","tagline":"Add 100+ new LLMs to your application, with a drop-in replacement for the openai chat completion call.","body":"Hello, I’m Ishaan - one of the maintainers of [LiteLLM.](https://github.com/BerriAI/litellm/)\n\n**TLDR; LiteLLM let’s you call all LLM APIs (Azure, Anthropic, Replicate, etc.) using the OpenAI format. We translate the inputs, standardize exceptions, and guarantee consistent outputs for completion() and embedding() calls**\n\n![uploaded image](/media/?type=post\u0026id=74316\u0026key=user_uploads/1083928/3b30d01e-433a-48e1-9a58-10a34913d45d)\n\n### **Problem ❌: Multiple LLM APIs - Hard to debug**\n\nCalling LLM APIs involved **multiple \\~100 line if/else statements** which made our **debugging problems explode.**\n\nI remember when we added Azure and Cohere to our chatbot. Azure’s API calls would fail so we implemented model fallbacks - (e.g. if Azure fails, try Cohere then OpenAI etc.). However, provider-specific logic meant our code became increasingly complex and hard to debug.\n\n# **Solution 💡**\n\n# 1️⃣ Simplify calling existing LLM APIs\n\nThat’s when we decided to abstract our LLM calls behind a [single package - LiteLLM.](https://github.com/BerriAI/litellm) We needed I/O that just worked, so we could spend time improving other parts of our system (error-handling/model-fallback logic, etc.).\n\n[LiteLLM](https://github.com/BerriAI/litellm) does 3 things really well:\n\n* **Consistent I/O:** It removes the need for multiple if/else statements.\n* **Reliable**: Extensively tested with 50+ cases and used in our production environment.\n* **Observable**: Integrations with Sentry, Posthog, Helicone, etc.\n\n### 2️⃣ Easily add new LLM APIS - [LiteLLM UI](https://docs.litellm.ai/docs/debugging/hosted_debugging)\n\n![uploaded image](/media/?type=post\u0026id=74316\u0026key=user_uploads/1083928/8624880b-20dc-4f39-8b40-1537c91a5380)\n\nThe next big challenge was adding new LLM APIs. Each addition involved 3 changes:\n\n* Update list of available models users can call from\n* Adding key to our secret manager / .env file\n* Mapping the model name - e.g. `replicate/llama2-chat-...` to a user-facing alias `llama2`.\n\nSince LiteLLM integrates with every LLM API - we provide all of this out of the box with zero configuration. With a single environment variable - `LITELLM_EMAIL` you can automatically add 100+ new LLM API integrations into your production server, **without modifying code / redeploying changes 👉** [LiteLLM UI](https://docs.litellm.ai/docs/debugging/hosted_debugging)\n\n## Ask 👀:\n\n* Adding new LLM providers? Contact us at [krrish@berri.ai](mailto:krrish@berri.ai) if you need help!\n* ⭐️ us on [GitHub](https://github.com/BerriAI/litellm) to keep up with releases and news.\n* Join [our Discord](https://discord.com/invite/wuPM9dRgDw)!","slug":"JKe-litellm-call-all-llm-apis-using-the-openai-format-llama2-anthropic-huggingface-etc","created_at":"2023-08-22T19:05:23.350Z","updated_at":"2026-05-25T06:23:35.003Z","total_vote_count":26,"url":"https://www.ycombinator.com/launches/JKe-litellm-call-all-llm-apis-using-the-openai-format-llama2-anthropic-huggingface-etc","share_image_url":"//bookface-static.ycombinator.com/assets/ycdc/yc-og-image-c440a0ad1dacfb86eeeb343717479cc54d256614449b4ef719977a0a451f8bc8.png","company":{"id":28409,"name":"LiteLLM","slug":"litellm","url":"https://www.litellm.ai/","logo":"https://bookface-images.s3.amazonaws.com/small_logos/34f2cd4e65f065f81d234d6cfb21f911860e50cc.png","batch":"Winter 2023","industry":"B2B","tags":["AIOps","Developer Tools","Generative AI","Open Source","AI"],"search_path":"https://bookface.ycombinator.com/company/28409"}}