Tag:evaluation
All the articles with the tag "evaluation".
One Prompt, Thirty-Two Calls, or Seven
Putting an LLM checklist into fewer model calls forces its retrieval into more, narrower queries. A seeded simulation of three call shapes, the counting limit on what a grouped call can see, and a skipped requirement that used to come back as a severe finding.
Perfect Precision, Zero Recall, and No Curve to Trade Along
A document check cleared every forgery in a labelled set. The threshold was not the problem: the check read the file's structure while the forgeries were in the pixels. A seeded rebuild shows a structure-only detector reaching exactly two operating points, AUC 0.500, and what calibrating on genuine documents alone can and cannot buy.
Forecast Metrics
My notes on forecast metrics for weekly sales forecasts: MAE, bias, MAPE, WAPE, MASE and RMSSE, business KPIs from a split bias, censored demand in backtests, and why RelMAE and RelMSE replaced MASE and RMSSE.