Define a good answer
For editing, evaluate natural language. For extraction, check fields. For code, run tests. Use a shared set of representative examples with difficult cases.
Test Georgian explicitly
Evaluate grammar, natural phrasing, consistent terminology, and factual accuracy. Test names, numbers, dates, and longer content.
Measure task cost
For rates per million tokens: input tokens × input rate / 1,000,000 + output tokens × output rate / 1,000,000. Account for retries, caching, group conditions, and dynamic billing.
Measure the actual workflow
Distinguish time to first output from full completion. Test representative input and output lengths. Public rankings cannot replace application-specific evaluation.
Check compatibility and access
- Supported endpoint and response format.
- Sufficient context capacity.
- Access for your account, group, and key.