The Five Stages of Model Learning

Each time I learn to use a new predictive model I find that I go through five stages of emotion before I truly add that model to my toolbox.

The first stage is wonder: This is when I use the model with the data in the vignettes and everything just works out beautifully and magically. The model runs. The results are awesome. I am just over the moon every time I am in this stage. I think to myself things like "why haven't I used this model before?", "I should use this model all the time?" and "This is the greatest model ever!!!".

The second stage is frustration: Yes, every Data Scientist has experienced this most awful stage. This happens the first time you try to apply the Model to real data. First, there are the errors when the data is not in the right format for the model. Then there are the crazy answers and model crashes. I know I am about to end this stage when I get the first results back, and they totally suck.

The Third Stage is acceptance: This stage happens as you use the model more and more and see that there are some cases where it works well and other where it does not. I usually get to this stage, but there are cases where I abandon the model before I see it do well against a real world data set.

The Fourth Stage is understanding: This is when you start to read more about the model. What makes it work? What makes it work better? What should you look for in a set data to determine if this models is good to try? When I get to this stage I know I am learning and improving my skills

The Fifth Stage is adoption: This happens when you have used the model enough and understand how it works enough to know when and how to use the model. This is the moment the model enters my toolbox as a permanent resident to be used when the data calls for it. I do not always get to this stage, but when I do I know I have achieved something.

Understanding that this is the process of learning a new predictive model will make you more willing to go through the process because the end result is so good. Good luck and keep learning.