Assessing AI Literacy: a framework-based approach

Artificial Intelligence (AI) has the potential to transform various aspects of our lives, but its rapid development is posing several concerns related to its application in increasingly more aspects of daily life. To make informed decisions and fully comprehend AI and its implications, a solid understanding of its underlying mechanisms and awareness of its benefits and drawbacks are crucial. This article presents the development process of a questionnaire designed to assess AI literacy in high-education context. The questionnaire adopts a framework-based approach based upon most recent literature findings. The questionnaire was developed to be administered to doctoral students in the Doctorate Program on Learning Sciences and Digital Technologies. Guided by a comprehensive framework encompassing cognitive, operational, critical, and ethical dimensions, the questionnaire aims at assessing participants’ awareness and understanding of AI. The questionnaire includes over 60 items, covering a wide range of AI topics. Each item aligns with specific dimensions of the framework, assessing participants’ knowledge, application, critical thinking, and ethical considerations about AI. The questionnaire, grounded upon an extensive literature review, and complemented with specific questions related to AI in education, is now undergoing several key steps, to be validated, tested and released as an effective tool to comprehensively assess AI literacy.