I recently discovered that supervised learning is a controversial term. The two definitions are:
- Known Loss Supervised learning corresponds to the situation where you have unlabeled examples plus knowledge of the loss of each possible predicted choice. This is the definition I’m familiar and comfortable with. One reason to prefer this definition is that the analysis of sample complexity for this class of learning problems are all pretty similar.
- Any kind of signal Supervised learning corresponds to the situation where you have unlabeled examples plus any source of side information about what the right choice is. This notion of supervised learning seems to subsume reinforcement learning, which makes me uncomfortable, because it means there are two words for the same class. This also means there isn’t a convenient word to describe the first definition.
Reviews suggest there are people who are dedicated to the second definition out there, so it can be important to discriminate which you mean.
Isn’t #2 typically referred to as semi-supervised learning?
I can’t tell if it also subsumes semi-supervised learning from the reviewer comments I saw. I’m used to semi-supervised learning meaning “labeled data and unlabeled data”. The context where I saw it used is closer to reinforcement learning, where you learn how good a choice was, but not how good other choices are.
<>
do you mean “labeled examples plus knowledge of the loss of each possible predicted choice”?