{"id":90,"date":"2005-06-22T13:08:18","date_gmt":"2005-06-22T19:08:18","guid":{"rendered":"\/?p=90"},"modified":"2005-06-22T13:08:27","modified_gmt":"2005-06-22T19:08:27","slug":"languages-of-learning","status":"publish","type":"post","link":"https:\/\/hunch.net\/?p=90","title":{"rendered":"Languages of Learning"},"content":{"rendered":"

A language is a set of primitives which can be combined to succesfully create complex objects. Languages arise in all sorts of situations: mechanical construction, martial arts, communication, etc… Languages appear to be the key to succesfully creating complex objects—it is difficult to come up with any convincing example of a complex object which is not built using some language. Since languages are so crucial to success, it is interesting to organize various machine learning research programs by language.<\/p>\n

The most common language in machine learning are languages for representing the solution to machine learning. This includes:<\/p>\n

    \n
  1. Bayes Nets and Graphical Models<\/strong> A language for representing probability distributions. The key concept supporting modularity is conditional independence. Michael Kearns<\/a> has been working on extending this to game theory.<\/li>\n
  2. Kernelized Linear Classifiers<\/strong> A language for representing linear separators, possibly in a large space. The key form of modularity here is kernelization.<\/li>\n
  3. Neural Networks<\/strong> A language for representing and learning functions. The key concept supporting modularity is backpropagation. (Yann LeCun<\/a> gave some very impressive demos at the Chicago MLSS<\/a>.)<\/li>\n
  4. Decision Trees<\/strong> Another language for representing and learning functions. The key concept supporting modularity is partitioning the input space.<\/li>\n<\/ol>\n

    Many other learning algorithms can be seen as falling into one of the above families. <\/p>\n

    In addition there are languages related to various aspects of learning.<\/p>\n

      \n
    1. Reductions<\/strong> A language for translating between varying real-world losses and core learning algorithm optimizations.<\/li>\n
    2. Feature Languages<\/strong> Exactly how features are specified varies from on learning algorithm to another. Several people have been working on languages for features that cope with sparsity or the cross-product nature of databases.<\/li>\n
    3. Data interaction languages<\/strong> The statistical query model<\/a> of learning algorithms provides a standardized interface between data and learning algorithm.<\/li>\n<\/ol>\n

      These lists surely miss some languages—feel free to point them out below.<\/p>\n

      With respect to research “interesting” language-related questions include:<\/p>\n

        \n
      1. For what aspects of learning is a language missing? Anytime adhocery is encountered, this suggests that there is room for a language. Finding what is not there is both hard and valuable.<\/li>\n
      2. Are any of these languages fundamentally flawed or fundamentally advantageous with respect to another language?<\/li>\n
      3. What are the most easy to use and effective primitives for these languages?<\/li>\n<\/ol>\n","protected":false},"excerpt":{"rendered":"

        A language is a set of primitives which can be combined to succesfully create complex objects. Languages arise in all sorts of situations: mechanical construction, martial arts, communication, etc… Languages appear to be the key to succesfully creating complex objects—it is difficult to come up with any convincing example of a complex object which is … <\/p>\n

        Continue reading “Languages of Learning”<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-90","post","type-post","status-publish","format-standard","hentry","category-organization"],"_links":{"self":[{"href":"https:\/\/hunch.net\/index.php?rest_route=\/wp\/v2\/posts\/90","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/hunch.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/hunch.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/hunch.net\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/hunch.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=90"}],"version-history":[{"count":0,"href":"https:\/\/hunch.net\/index.php?rest_route=\/wp\/v2\/posts\/90\/revisions"}],"wp:attachment":[{"href":"https:\/\/hunch.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=90"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/hunch.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=90"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/hunch.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=90"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}