Model Selection Guide
Choose models by quality, latency, cost, context, and tool capabilities using repeatable evaluations.
No single model suits every task. Define quality, speed, cost, and capability constraints, then run a small evaluation using real data.
Main decision factors
| Factor | Question | Validation |
|---|---|---|
| Task quality | Can the model reliably handle reasoning, writing, code, or extraction? | Compare accuracy, human ratings, and format validity on representative samples. |
| Latency | Do time to first token and total generation time meet product requirements? | Measure TTFT and P50, P95, and P99 latency separately. |
| Cost | Are per-request and monthly costs acceptable? | Estimate from input/output token distributions, not unit prices alone. |
| Context | Will prompts, history, and knowledge passages fit the window? | Measure real context-size percentiles and reserve output space. |
| Advanced capabilities | Are tools, vision, structured output, or reasoning required? | Check the catalog and test essential parameters through the API. |
| Stability | Are peak error rates and tail latency manageable? | Load-test success rates and latency with platform routing. |
Recommended selection process
- Build a candidate set. Filter catalog models by text, code, multimodal, or tool requirements.
- Fix evaluation inputs. Use real data covering common requests, edge cases, and failures.
- Align parameters. Compare similar temperatures and output lengths to avoid confounding results.
- Roll out gradually. Start with limited traffic and monitor quality, cost, latency, and errors.
- Re-evaluate regularly. Rerun core evaluations after model version, price, or routing-node changes.
Common model tiers
Low latency
For classification, rewriting, short answers, and real-time interaction, prioritize TTFT and throughput.
High quality
For complex reasoning, important content, or difficult-case fallbacks, allow higher cost and latency.
Structured tasks
Focus on JSON validity, field stability, and tool-argument accuracy.
Code tasks
Use compilation, tests, and static checks as objective metrics rather than subjective review alone.
Use the current model catalog as the source of truth. Availability, identifiers, context limits, prices, and capabilities can change.
qwen-turboanddeepseek-chatillustrate request structure, not fixed recommendations for specific tasks.