{"id":15,"date":"2005-02-03T10:19:56","date_gmt":"2005-02-03T16:19:56","guid":{"rendered":"\/?p=15"},"modified":"2005-02-03T10:20:29","modified_gmt":"2005-02-03T16:20:29","slug":"learning-theory-by-assumption","status":"publish","type":"post","link":"https:\/\/hunch.net\/?p=15","title":{"rendered":"Learning Theory, by assumption"},"content":{"rendered":"

One way to organize learning theory is by assumption (in the assumption = axiom sense<\/a>), from no assumptions to many assumptions. As you travel down this list, the statements become stronger, but the scope of applicability decreases.<\/p>\n

    \n
  1. No assumptions<\/strong>\n
      \n
    1. Online learning<\/strong> There exist a meta prediction algorithm which compete well with the best element of any set of prediction algorithms.<\/li>\n
    2. Universal Learning<\/strong> Using a “bias” of 2– description length of turing machine<\/sup> in learning is equivalent to all other computable biases up to some constant.<\/li>\n
    3. Reductions<\/strong> The ability to predict well on classification problems is equivalent to the ability to predict well on many other learning problems.\n<\/li>\n<\/ol>\n<\/li>\n
    4. Independent and Identically Distributed (IID) Data<\/strong>\n
        \n
      1. Performance Prediction<\/strong> Based upon past performance, you can predict future performance.<\/li>\n
      2. Uniform Convergence<\/strong> Performance prediction works even after choosing classifiers based on the data from large sets of classifiers.<\/li>\n<\/ol>\n<\/li>\n
      3. IID and partial constraints on the data source<\/strong>\n
          \n
        1. PAC Learning<\/strong> There exists fast algorithms for learning when all examples agree with some function in a function class (such as monomials, decision list, etc…)<\/li>\n
        2. Weak Bayes<\/strong> The Bayes law learning algorithm will eventually reach the right solution as long as the right solution has a positive prior.<\/li>\n<\/ol>\n<\/li>\n
        3. Strong Constraints on the Data Source<\/strong>\n
            \n
          1. Bayes Learning<\/strong> When the data source is drawn from the prior, using Bayes law is optimal<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n

            This doesn’t include all forms of learning theory, because I do not know them all. If there are other bits you know of, please comment.<\/p>\n","protected":false},"excerpt":{"rendered":"

            One way to organize learning theory is by assumption (in the assumption = axiom sense), from no assumptions to many assumptions. As you travel down this list, the statements become stronger, but the scope of applicability decreases. No assumptions Online learning There exist a meta prediction algorithm which compete well with the best element of … <\/p>\n

            Continue reading “Learning Theory, by assumption”<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-15","post","type-post","status-publish","format-standard","hentry","category-organization"],"_links":{"self":[{"href":"https:\/\/hunch.net\/index.php?rest_route=\/wp\/v2\/posts\/15","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/hunch.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/hunch.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/hunch.net\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/hunch.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=15"}],"version-history":[{"count":0,"href":"https:\/\/hunch.net\/index.php?rest_route=\/wp\/v2\/posts\/15\/revisions"}],"wp:attachment":[{"href":"https:\/\/hunch.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=15"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/hunch.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=15"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/hunch.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=15"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}