Comments on: All Models of Learning have Flaws

By: Tagz | "Machine Learning (Theory) Â» All Models of Learning have Flaws" | Comments

Sat, 16 May 2009 12:15:25 +0000

[...] [upmod] [downmod] Machine Learning (Theory) Â» All Models of Learning have Flaws (hunch.net) 0 points posted 10 months, 1 week ago by jeethu tags ai machine learning reference [...]

By: Study ML or Just Run Some Tools of ML « Abner’s Postgraduate Days

Study ML or Just Run Some Tools of ML « Abner’s Postgraduate Days — Sat, 05 Jul 2008 17:43:34 +0000

[...] Machine Learning (Theory) Ã‚Â» The Meaning of Confidence Machine Learning (Theory) Ã‚Â» All Models of Learning have Flaws [...]

By: [Hunch] All models of learning have flaws « Machine Learning Blog

[Hunch] All models of learning have flaws « Machine Learning Blog — Mon, 10 Mar 2008 05:01:15 +0000

[...] [Hunch] All models of learning have flaws Published March 10, 2008 general http://hunch.net/?p=224 [...]

By: ik

ik — Tue, 13 Nov 2007 19:51:01 +0000

My experience is that if you have a problem with more than a few dozen features and enough examples to build a full tree, the tree gets so large that it ceases to make sense. Moreover, in order to get really competitive performance from DTs, one often has to bag or boost them. Which again makes them uninterpretable.

By: Mark

Mark — Wed, 22 Aug 2007 22:58:32 +0000

To follow up on that da Vinci quote, one of my favourites regarding theory and practice:

In theory, there is no difference between theory and practice. But, in practice, there is.

This has been attributed to Jan Snepschuet, Yogi Berra and Albert Einstein.

By: Jue Wang

Jue Wang — Wed, 18 Apr 2007 20:24:25 +0000

Yan King, you should definitely check out the Markov Logic Networks framework, which is a good combination of logic and statistical learning methods.

By: Yan King Yin

Yan King Yin — Tue, 20 Mar 2007 16:23:08 +0000

You seem to have neglected logic-based learning, which is often neglected nowadays because of other “numerical” learning methods. But logic-based learning (eg inductive logic programming) is also a form of statistical learning. Also, if the substrate is a probabilistic logic then it is related to Bayesian networks. It would be very interesting to find out any deep connection between numerical and logic-based learning paradigms.

By: Chunyu Yang

Chunyu Yang — Mon, 19 Mar 2007 15:09:19 +0000

I am interested in Bayesian Learning which is surely a Generative Model. “specify a prior” is quite boring, but some algorithms have been proposed to tackle this problem in the name of “Probability Estimation”. Maybe the variation of the a prior along with the data sequence is more serious.

By: Leonid Kontorovich

Leonid Kontorovich — Sat, 17 Mar 2007 07:04:00 +0000

I’m glad to see that a recurring objection to various models of
learning is the IID assumption. My recent work in concentration
of measure was originally motivated precisely by this
limitation. While I don’t have a full generalization of VC bounds to
non-IID settings, I believe I’ve made some progress. Check out also
“A Note on Uniform Laws of Averages for Dependent Processes” by Nobel
and Dembo.

By: Aleks

Aleks — Thu, 15 Mar 2007 13:53:27 +0000

Agreed with hal. Igor Kononenko has actually asked physicians if they preferred the conditional probabilities out of naive Bayes or classification trees. They preferred conditional probabilities. They would like them even more had they been shown as a nomogram.