Pechenizkiy, Mykola

AI architectures and models are increasingly evaluated through benchmarks and leaderboards, enabling broad comparison across systems while simultaneously creating a growing disconnect between measured performance and reliable real-world behavior. Systems optimized for benchmark success may still exhibit poor robustness, unexpected failures, or unsafe emergent behavior after deployment. In this keynote, drawing lessons from traditional machine learning, fairness-aware AI, and recent advances in foundation models and generative AI, I examine fundamental methodological challenges in current evaluation paradigms, including Goodhart’s law, limited construct and external validity, and the conflation of narrow competence with reliable behavior. I advocate complementing benchmarks with stress testing, out-of-distribution assessment, adversarial and interactive evaluation, uncertainty and robustness analysis, lifecycle-aware monitoring, socio-technical considerations, and human-centered evaluation methodologies. Ultimately, I argue that advancing safe and trustworthy AI requires not only more capable systems, but also a rigorous science of evaluation capable of distinguishing performed competence from genuinely reliable behavior.