The study examines the potential of short justifications in clicker activities. A total of 138 students answered individually eight multiple-choice questions in a clicker tool and provided short justifications for their responses, denoting also their confidence that their responses were correct. Next, students received classroom feedback and revised their initial answers and their level of confidence. Results showed that all students increased their performance during the revision phase. However, the group (n = 70) that received as feedback the percentage of students under each question choice along with the respective justifications increased its confidence significantly. Moreover, in this group, students’ final confidence levels and their actual performance were significantly positively correlated, suggesting accuracy between their perceived and actual performance (i.e., better calibration). On the contrary, the same was not observed for the group (n = 68) that received as feedback only the percentage information. This suggests that having access to justifications helps students in self-assessing their level of knowledge more accurately.