Cognitive Assessment and Questionnaires

The International Computer and Information Literacy Study (ICILS) was first conducted in 2013 by the International Association for the Evaluation of Educational Achievement (IEA), an independent, international cooperative of national research agencies, and was conducted again in 2018. ICILS is a computer-based international assessment of 8th-grade students.

What does ICILS assess?
ICILS 2018 assesses two dimensions of an individual's skills with information communications technologies. The first dimension, computer and information literacy, focuses on understanding computer use, gathering information, producing information, and communicating digitally. The second dimension, computational thinking, focuses on conceptualizing problems and operationalizing solutions.

Computer and Information Literacy (CIL) is defined in the ICILS 2018 Assessment Framework as "an individual's ability to use computers to investigate, create, and communicate in order to participate effectively at home, at school, in the workplace, and in society" (Fraillon et al., 2018).

Computational Thinking (CT), new in ICILS 2018, is defined as "an individual’s ability to recognize aspects of real-world problems which are appropriate for computational formulation and to evaluate and develop algorithmic solutions to those problems so that the solutions could be operationalized with a computer" (Fraillon et al., 2018). Computational thinking is the style of thinking used when programming a computer or developing an application for another type of digital device. However, its usefulness extends beyond this. The reasoning strategies that underlie computational thinking can help make sense of complex ideas and solve problems. The assessment of computational thinking through ICILS is an opportunity to gather international comparative data about how students are developing computational problem-solving skills in school.

Computational thinking is an optional component for participating countries. The United States administered the computational thinking component in 2018.

ICILS 2018 was conducted to help answer the following research questions, which apply to both the CIL and CT components:

• What variations exist within and across countries in students’ CIL and/or CT skills?
• What aspects of schools and countries are related to students’ CIL and/or CT skills?
• What are the relationships between students’ levels of access to, familiarity with, and self-reported proficiency in using computers and their CIL and/or CT skills?
• What aspects of students’ personal and social backgrounds (such as their gender and socioeconomic background) are related to their CIL and/or CT skills?
• What is the association between students’ CIL and CT skills?

For more information on the ICILS framework, see the ICILS 2018 Assessment Framework.

What other data are collected in ICILS?
In addition to the student assessment, students complete a questionnaire about their

• background;
• access to ICT;
• use of ICT; and
• familiarity with ICT at home and at school.

Information is also collected from 8th-grade students' teachers and schools. Teachers complete a questionnaire to collect information on their

• background;
• use of ICT;
• attitudes toward using ICT in teaching; and
• professional development related to ICT.

In each school, the ICT coordinator completes a questionnaire about ICT resources in the school and the policies and practices that make use of those resources to support learning. In addition, principals complete a questionnaire to collect information on school characteristics and broad policies, procedures, and priorities for ICT in the school.

Lastly, a national context survey is completed by the national study centers of participating countries to provide information regarding the policies, curricula, resources, and practices related to CIL and CT education from a national perspective.