Паметно генерирање на прашања за програмски јазици

Abstract

The process of teaching programming receives significant attention nowadays. Assessment of students’ knowledge in introductory programming courses can (partly) be done by presenting simple (fragments of) source codes. One of the learning goals in these courses is students to become able to understand and comprehend an already written program code. Hence, they should be able to correctly answer the question: “What is the output of the given code?” When preparing the source codes, teachers must be aware of their complexity. Particularly, when preparing many different versions of the same test (to assess a huge number of students), they have to provide same or similar complexity tasks (questions) for all students. A possible solution to this problem is to turn to automatic generation of tasks containing program codes. This doctoral dissertation defines a new model for smart automatic generation of tasks containing program codes. The application of this model will enable achievement of a completely objective and fair assessment on any introductory programming course exam, since all students taking the exam will be asked questions of same or very similar complexity, i.e. questions that require the same level of knowledge and approximately the same effort to provide a correct answer. Thus, the model will contribute to a significant improvement in the quality of the assessment process in the introductory programming courses. In order to achieve complexity consistency in the process of automatic production of questions for programming courses, within this model, the use of a new metric for automatic measurement of complexity of program codes is proposed. The new metric, defined in this dissertation, considers the source code complexity from a perspective of the student’s effort required for manual calculation of the program output, if the values of all the program variables are known. It measures the complexity using cognitive weight values assigned to each of the operators and control flow statements in the code, which should represent their “complexities”, in terms of the effort and time that a particular student needs to spend to manually perform the corresponding operations/statements and calculate the respective results. The dissertation presents arguments which confirm that the metric is suitable for the problem under consideration. Furthermore, in this dissertation, the findings of the research that was conducted in order to determine an appropriate weight values for the arithmetic operators are also described. The main goal of this research was to improve the accuracy of the code complexity calculation for automatically generated tasks that contain source codes, by determining weight values for the basic arithmetic operations with simplest operand types. Here, the results and conclusions from the experiments that were conducted as part of the research are presented. Finally, the dissertation introduces a new software system for smart automatic generation of tasks containing program codes. For a given (initial) source code entered by a teacher, the system calculates the complexity using the proposed metric, and by applying appropriate modification techniques, it generates a desired number of new program codes that have the same or arbitrarily close complexity one to another. This system can be used for automatic creation of appropriate tasks that contain program codes of consistent complexity, so, as such – it represents an implementation of the proposed model for smart automatic task generation. Therefore, it can be used in practice for the purposes of the knowledge assessment process in the introductory programming courses.