Model Routing

NotDiamond
Perplexity AI explains Model Routing AI "Model Routing" refers to the process of dynamically directing queries or tasks to the most appropriate AI model based on factors like cost, performance, or task specificity. This is particularly useful in large language model (LLM) workflows, where different models excel at different tasks. For example, simpler queries might be routed to cheaper models, while complex ones go to more advanced models like GPT-4.

Key Tools and Companies for Model Routing

  1. Not Diamond:
    • A commercial AI model router that determines the best LLM for a query, balancing cost and quality.
    • Supports custom routing tailored to user data and integrates with major LLM providers like OpenAI and Anthropic[2][6].
  2. RouteLLM:
    • An open-source framework designed for cost-effective LLM routing.
    • Focuses on improving performance-to-cost ratios by routing tasks to specialized models based on preference data[3][7].
  3. Dagster Integration:
    • Dagster pipelines can use Not Diamond for seamless prompt routing, enabling workflows that combine multiple LLMs[2].
  4. OpenRouter:
    • A platform supporting integrations with various LLMs and customizable routing for specific use cases[4].
  5. UbiOps:
    • Provides multi-model routing for linking specialized models in series or parallel, optimizing efficiency across diverse data types[9].
These tools help businesses optimize AI workflows by leveraging the strengths of multiple models while managing costs and performance effectively.
Sources [1] What is AI Routing? - NICE https://www.nice.com/glossary/what-is-ai-routing [2] Routing LLM prompts with Dagster and Not Diamond https://dagster.io/blog/routing-llm-prompts-with-not-diamond [3] RouteLLM: An Open-Source Framework for Cost-Effective LLM ... https://lmsys.org/blog/2024-07-01-routellm/ [4] OpenRouter Integrations in 2025 - Slashdot https://slashdot.org/software/p/OpenRouter/integrations/ [5] Routing AI Agents - Route Consumers Conversationally https://developers.liveperson.com/conversation-builder-generative-ai-routing-ai-agents-route-consumers-conversationally.html [6] Getting started with Not Diamond — NotDiamond 0.3.35 https://python.notdiamond.ai/source/intro.html [7] Serving Open source LLM router RouteLLM as a production API? https://www.reddit.com/r/LLMDevs/comments/1fqh06e/serving_open_source_llm_router_routellm_as_a/ [8] Leon, Xing Li on LinkedIn: This is super realistic to increase ... https://www.linkedin.com/posts/leonxingli_this-is-super-realistic-to-increase-feasibilities-activity-7229903755377348609-xSYQ [9] What is multi-model routing? - UbiOps https://ubiops.com/what-is-multi-model-routing/