There are several different flavors of Machine Learning classes. Many classes are of the ‘zoo’ sort: many different learning algorithms are presented. Others avoid the zoo by not covering the full scope of machine learning.
This is my view of what makes a good machine learning class, along with why. I’d like to specifically invite comment on whether things are missing, misemphasized, or misplaced.
Phase | Subject | Why? |
Introduction | What is a machine learning problem? | A good understanding of the characteristics of machine learning problems seems essential. Characteristics include: a data source, some hope the data is predictive, and a need for generalization. This is probably best taught in a case study manner: lay out the specifics of some problem and then ask “Is this a machine learning problem?” |
Introduction | Machine Learning Problem Identification | Identification and recognition of the type of learning problems is (obviously) a very important step in solving such problems. People need to be familiar witth the concept of ‘regression’, ‘classification’, ‘cost sensitive classification’, ‘reinforcement learning’, etc… A good organization of these things is possible, but not yet well done. |
Introduction | Example algorithm 1 | To really understand machine learning, a couple learning algorithms must be understood in detail. |
Introduction | Example algorithm 2 | Ditto. The reason why the number is “2” and not “1” or “3” is that 2 is the minimum number required to make people naturally aware of the degrees of freedom available in learning algorithm design. |
Analysis | Bias for Learning | The need for a good bias is one of the defining characteristics of learning. This includes discussing the means to specify bias (via Bayesian priors, choice of features, graphical models, etc…). This statement is generic so it will always apply to one degree or another. |
Analysis | Learning can be boosted. | This is the boosting observation: that it is possible to bootstrap predictive ability to create a better overall system. This statement is similarly generic. |
Analysis | Learning can be transformed | This is the reductions observation: that the ability to solve one kind of learning problems implies the ability to solve other kinds of leanring problems. This statement is similarly generic. |
Analysis | Learning can be preserved | This is the online learning with experts observation: that we can have a master algorithm which preserves the best learning performance of subalgorithms. This statement is again generic. |
Analysis | Overfitting | Learning algorithms can easily overfit to existing training data. How to analyze this (with an IID assumption), and how to avoid it are very important for success. |
Analysis | Hardness of Learning | It turns out that there are several different ways in which machine learning can be hard including computational and information theoretic hardness. Some of PAC learning is relevant here. An understanding of how and why learning algorithms can fail seems important to understand the process. |
Applications | Vision | One example of how learning is applied to solve vision problems. |
Applications | Language | Ditto for language problems. |
Applications | Robotics | Ditto for robotics |
Applications | Speech | Ditto for speech |
Applications | Businesses | Ditto for businesses |
Where is machine learning going? | Insert predictions of the future here. It should be understood that the field of machine learning is changing rapidly. |
The emphasis here is on fundamentals: generally applicable mathematical statements and understandings of the learning problem. Given that emphasis, the ‘applications’ section could be cut without harming the integrity of the purpose.