Run multi-case evaluation suites against datasets using any model available through OpenRouter or the direct Jev adapter. Write an adapter that calls your application and a Scorer that measures the outcomes you care about. Compare models, or track how prompt and parameter changes affect results. Combine metrics such as accuracy, errors, latency, and cost into weighted scores, then review rankings and case-level results in an HTML report. Integrates with Braintrust Evals and several others. Paid integration is not required.
Required Ruby Version
>= 3.4.0
Authors
Konstantin Gredeskoul
Versions
-
0.5.0
-
source
- 0.5.0 source November 02, 2026 (123 KB)
-
-
0.4.0
-
source
- 0.4.0 source November 01, 2026 (79 KB)
-