Self-assessment serves to improve learning through timely feedback on one's solution and iterative refinement as a way to incrementally improve one's competence. The SelfAssess-plugin developed in Moodle provides unlimited opportunities for students to create and assess solutions to self-assessment questions. A field study examined how 131 students used voluntary self-assessment questions in an online course in B.Sc. Computer Science, how accurate they were able to self-assess their solutions on instructor-defined criteria, and which question-related characteristics influence the ability of self-assessment. Results show the potential for providing scalable learning support. Fine-grained assessment criteria and freedom of solution input are recommended; a limited complexity of the expected solution is still a challenge.