HOLD-OUT TASKS / REPEATED RUNS / BASELINES / REPORTS

Evaluation suites

Held-out tasks, repeated runs and reports that make one model comparable with the next.

What you get

A versioned task set kept apart from training, a runner that repeats each task across seeds, and a report with spread as well as averages.

Why repeated runs

One run per task hides variance. Reporting intervals across seeds keeps a lucky result from being read as an improvement.

What it plugs into

Your CI or release process, so a regression shows up before a model ships.