Towards a Practitioner-Led AIED Evaluations Capacity: Lessons Learned from Developing L2-Bench at Oxford University Press

The need for a mature, third-party AI for education (AIED) evaluations capacity grows more acute as AI adoption accelerates. While developing this capacity is necessary to reveal the true frontier of AIED itself, most importantly without such a capability, we have no body of facts to draw on when deciding whether to adopt AIED systems and how to use them.

Evaluations themselves offer a form of productive friction that education stakeholders can apply individually or collectively, and therefore this session demonstrates how practitioners themselves can contribute to third-party AI evaluations capacity. It does so by showcasing Oxford University Press’ AIED evaluations research (specifically OUP’s development of the first large-scale benchmark for AIED, L2-Bench, that measures LLM capabilities in language education) and providing practical guidance to other stakeholders about how they as individuals or within their own institutions can play crucial roles in further developing the emerging third-party AIED evaluations ecosystem. 

---

Register now