The Educational Feedback Loop

The loop creates an essentially omniscient, infinitely patient, always available “tutor” to guide the student through the entire learning process. For any given concept, there are four stages – overview, instruction, studying and evaluation. The student is led through them according to the following flow chart:

The chart shows the major decision points that create “feedback” that alters a student’s progression according to the results of their assessments. With chart in hand (the link opens in another window by default), let’s flesh out the stages in detail:

Overview

The teaching of any concept should begin with a declaration of purpose and intended use. For example, an introduction to addition should explain the purpose i.e., “to determine the total value of a given set of numbers” as well as why this is useful e.g., “if I give you three quarters and two dimes, addition will tell you how much money you have.”

Why? I would argue that in this connected day and age, knowing what a concept is called and how/why it is used can be just as important as being able to remember how to do it. This is particularly evident in complex, technical fields such as mathematics. For instance, let’s say I give you this seemingly simple function:

f(x) = 3x2 * sin(2x) * sqrt(3x)

What is the sum of f(x) over the interval 0 to 10? If you have ever studied calculus, you should recognize this as an integration problem. You may have long since forgotten some of the complex rules that would enable you to answer this question by hand. You don’t have to though, because you can plug the integral into a tool like Wolfram|Alpha. Simply knowing the problem requires integration is enough to solve it.

An overview is also important to facilitate chunking. Imagine trying to teach proper utilization of the English period when the student doesn’t even know why they should use one in the first place. Retention will be much more difficult because it will just seem like a set of random rules for placing dots on a piece of paper.

Instruction

This is where the actual information to be learned is presented. To accommodate the various idiosyncrasies and needs of each student, multiple methods and sources of instruction should be made available. I define a method of instruction as how information is presented i.e., video, audio, in-person lecture, reading assignment, etc., and sources as the “vendors” of instruction. For example, Khan Academy and PatrickJMT are popular and well known YouTubers of educational math and science videos – each would be considered a source. A teacher or lecturer would also be a source, as would a section from a particular book.

So why are multiple methods of instruction necessary?

• Education can not depend upon a single method. If a student misses a lecture, for example, that should not preclude them from instruction. Other sources must be available (to include, for instance, a video of the lecture!)

• Some students may find certain methods less effective. If reading is difficult, an instructional video may be better than a written passage about the concept. Note that this is not quite the same as the idea of different learning styles. Indeed, the evidence is still not clear that they even exist as commonly believed.

• Some methods are inappropriate at times. For example, a pre-schooler may not be able to read yet, but that should not preclude them from learning basic arithmetic from a teacher. On the other hand, an adult seeking remedial education (say after a brain injury) may be more amenable to learning basic arithmetic unsupervised, perhaps from a video or by reading a book.

• Certain methods do not scale well. If you’ve ever attended a lecture with 300+ students, you know that most of the value of having a live human do the lecture is lost – there is simply not enough time to field every question, too many distractions from other students, lack of proximity further reduces attention, not being able to see, not being able to hear clearly, etc.

Why are multiple sources of instruction necessary?

• You can not equally appeal to every student with the same source. No matter the skill of the lecturer or quality of the video, some students will still find the presentation to be obtuse. This is something well understood by autodidacts – the ability to understand a difficult concept quickly is often dependent upon having multiple sources explain it in different ways.

• Multiple sources enable pedagogical competition. This is an important part of adaptation – you simply don’t know how effective a source will be until you try it and compare the results to other known sources. Without the ability to do so, presentations will not evolve as quickly.

• Multiple effective sources enable repetition of instruction without becoming stale to the student. This is important for maintaining the educational feedback loop, which may have to kick students back to the instructional stage at any time (especially after the running assessment and during the studying stage).

It is important that all instruction be capable of facilitating granularity. For instance, all textbooks should have an electronic equivalent whereby clicking or highlighting any portion of the text has the option to begin teaching the applicable pre-requisite concepts with full rigor, using the same adaptive framework. A fully realized example of this would be handing someone a granularized copy of The Mathematical Theory of General Relativity and, given sufficient time and motivation, them being able to fully understand it by the last page.

How could granularity be facilitated in a lecture? For starters, a dynamic learning system should not be scheduling a student for a lecture until they have demonstrated mastery of the pre-requisite concepts. Of course, some will still fall through the cracks and this is another reason why live lectures can not be the only method of instruction available. At the very least, a student should have access to a recording of the particular lecture they attended. If the recording is annotated with a list of the needed concepts, the system can guide the student through remediation automatically.

Instruction can be improved by applying metrics to the process. For example, if two instructional videos are in use for a concept, and one leads to mastery with half as much studying, it should clearly be favored when presenting instruction to new students. This can also be applied to the next stage, studying, where the ordering of assignments could reduce average study time to mastery.

After instruction is complete, a running assessment should determine if the student is ready to proceed with studying. This is the first feedback loop – if the instruction was ineffective for that student, present them with more instruction. If the student failed to understand the instruction due to lack of necessary pre-requisities, remediate and then perform the assessment again.

Studying

After successful instruction, we enter that stage of learning where we spend time fooling our brain into believing certain information is vital to its survival. Through repetitive recall, we slowly form new pathways and become more “conversant” with the material.

I’m not going to delve too deeply into educational neuroscience here, because this is an area of still considerable debate. What we are pretty sure of though is that spacing studying out over time leads to better long term retention. Thus the system should discourage cramming by limiting the amount of time a student may study a particular subject in one sitting. I don’t claim to know the magic value, but my suggestion is to start with one hour and modify it per student based on biometric feedback and past performance.

The more studying that the system can facilitate, the better it can assure that a student will be ready for the formal assessment. For instance, if a study module for a vocabulary lesson electronically presents flash cards, it can track when the student is approaching perfect recall and then require them to achieve a REM cycle before studying the cards again. Over many students it can gather statistics on the average number of sessions and time needed for mastery.

These statistics can be combined with knowledge of the student’s comparative learning performance in order to determine the natural cutoff point between more studying and entering the evaluation stage. This is not an exact science, the student should be free to override this cutoff and continue studying. There must be a fail-safe though i.e., if a student that normally demonstrates proficiency in the 50th percentile of time is taking more than two standard deviations as long to study, an administrator should be notified.

On the subject of studying, allow me to indulge in a bit of rant…

A personal pet peeve of mine is the nonchalance by which professors decide to assign graded or ungraded homework, and how ill-prepared that homework often leaves students for the exams. I find it particularly ironic how undata-driven science professors are about this. I’ve seen two professors of the same class where one did not require homework to be turned in and the other counted homework for 30% of the final grade. They also chose different problems from the same textbook, even though the final exam was the same. This chaotic approach does not facilitate learning.

If the instruction is particularly obtuse (as is often the case with the core sciences), working problem sets is sometimes the only way to actually learn the material. In that case, graded homework eliminates the ability for students to learn through trial and error for fear of destroying their grade – they either understand and complete the homework in time, or they copy answers from someone else. In addition, students receive little to no feedback on their work until such time as the professor or TA chooses to grade and return it.

My takeaway from this is that homework problem sets should be virtually bottom-less, facilitated electronically and the student should not be penalized for failing to complete any problem properly. Instead, failure or inability to complete a problem in reasonable time should be analyzed and handled immediately, whether that is through further instruction, doing simpler problems first or remediation of core concepts.

Evaluation

When the system is satisfied that the student is ready, they should be scheduled for a formal assessment. Remember, this has one purpose only: to ascertain “mastery” of the concept. Mastery is the critical piece of information required for the system to know when it it is time for the student to proceed to the next concept. My suggestion was for a raw score of 90% to indicate sufficient mastery, but an argument can be made that some concepts should be closer to 100% (you either know it or you don’t) and others much lower (even rudimentary understanding is sufficient to utilize it).

What is important is that the definition of mastery for a particular concept is universal – it must transcend regional and cultural boundaries. The system will not function properly if mastery is defined as 64.2% in Mississippi and 86.2% in Massachusetts. If the definition of a particular concept is too broad, this will be very difficult, which is another reason why it is vital that the knowledge tree be as granular as possible.

The definition of mastery for a concept may change over time; there might be, for instance, a Y2016 and Y2020 version of “Single-variable Calculus.” This is fine, disciplines evolve and it also creates opportunities for continuing education to keep a student fresh (and current) in the material long after they have mastered it.

So what happens if a student does not demonstrate mastery during their evaluation? On to the next page: