Most course delivery systems measure course completions and grades, but we know this is a poor way to measure learning and retention of the content — let alone the quality of the course. Why? Because grades simply measure the success of an individual’s effort at the time of testing, which most likely will not be the time of the need to apply the learning. In fact, most grades and ranking systems simply prompt the learner to do whatever it takes to master the test, not to master the material. Most teachers will tell you that grades are also inflated, due to the need to not harm student success later in life, and to avoid painful conversations with parents. In the workplace, a grade on learning material is seen as simply one more hurdle one has to pass in order to get paid.
The point of any learning endeavor should be mastery of the topic. Mastery by it’s nature involves a type of learning that encourages experimentation and failure, not rote memorization and quiz scores. If you only give a learner one chance to score a passing grade, will they have mastered the material? No. You will know that they scored a percentage accurately, but there is no chance to intervene and teach them what they got wrong, except en masse if the grades for a particular topic are very bad. We use letter grades to artificially divide people into ‘A’ students and ‘B’ students, but this does not help the ‘B’ student turn into the ‘A’ student, it simply tells them that they do not understand this topic very well or are not sufficiently able to cram for the test.
Measuring effort in a mastery environment looks a bit different. You must determine whether someone is using trial-and-error to complete the topic, or whether they are improving by reviewing the material and trying again. In a physical environment, this is highly subjective, although some teachers have managed. In an online environment, we can look at the behavior of the user to determine whether they are randomly clicking, or truly attempting to learn, and score effort accordingly.
Our goal is always that someone is able, regardless of the number of tries, to score 100% on a quiz, or can repeatedly submit an assignment until it is accepted as mastered. This is a technique that can be difficult to manage in the classroom, but is well suited to an online environment where a student can do this regardless of time or place.
Once you have moved to a mastery environment, learning effectiveness can more easily be measured. Learning effectiveness should be defined as the degree to which each activity and, by extension each microcourse, aids in the learner’s mastery of a particular topic. By observing the online behavior of students we are able to link your activities to their progress to mastery, and make recommendations to aid in the improvement of your content over time.