The first international large-scale study in Information Technology (IT) literacy was conducted in 1987, and a broad range of studies that assess IT literacy and related areas of digital learning have followed. This chapter discusses recent developments in international large-scale studies of IT literacy-related achievement with a focus on their shared necessary attributes and the associated challenges with operationalizing these attributes in the test instruments. Two key attributes addressed are: (i) that the test contents reflect real-world use of Information and Communications Technology (ICT) and (ii) that the tests make use of the dynamic functionality and multimodal opportunities afforded by the computer-based environment. Challenges associated with these attributes include ensuring that the individual tasks within each assessment are independent of each other, maintaining a standardized test-taker experience, providing test-takers with plausible feedback from the computer-based environment, and maintaining construct validity. Examples are discussed of how the common challenges in creating the test instruments are addressed in the design of the instruments with some further discussion of possible future directions in large-scale international studies related to IT literacy.