{"id":175,"date":"2006-03-17T23:22:43","date_gmt":"2006-03-18T05:22:43","guid":{"rendered":"http:\/\/hunch.net\/?p=175"},"modified":"2006-03-17T23:23:30","modified_gmt":"2006-03-18T05:23:30","slug":"multitask-learning-is-black-boxable","status":"publish","type":"post","link":"https:\/\/hunch.net\/?p=175","title":{"rendered":"Multitask learning is Black-Boxable"},"content":{"rendered":"<p>Multitask learning is the problem of jointly predicting multiple labels simultaneously with one system.   A basic question is <a href=\"https:\/\/hunch.net\/?p=160\">whether or not multitask learning can be decomposed into one (or more) single prediction problems<\/a>.  It seems the answer to this is &#8220;yes&#8221;, in a fairly straightforward manner.<\/p>\n<p>The basic idea is that a controlled input feature is equivalent to an extra output.  Suppose we have some process generating examples: <em>(x,y<sub>1<\/sub>,y<sub>2<\/sub>) in S<\/em> where <em>y<sub>1<\/sub><\/em> and <em>y<sub>2<\/sub><\/em> are labels for two different tasks.  Then, we could reprocess the data to the form <em>S<sub>b<\/sub>(S) = {((x,i),y<sub>i<\/sub>): (x,y<sub>1<\/sub>,y<sub>2<\/sub>) in S, i in {1,2}}<\/em> and then learn a classifier <em>c:X x {1,2} -> Y<\/em>. Note that <em>(x,i)<\/em> is the (composite) input. At testing time, given an input <em>x<\/em>, we can query <em>c<\/em> for the predicted values of y<sub>1<\/sub> and y<sub>2<\/sub> using <em>(x,1)<\/em> and <em>(x,2)<\/em>.<\/p>\n<p>A strong form of equivalence can be stated between these tasks.  In particular, suppose we have a multitask learning algorithm <em>ML<\/em> which learns a multitask predictor <em>m:X -> Y x Y<\/em>.   Then the following theorem can be proved:<\/p>\n<p>For all <em>ML<\/em> for all <em>S<\/em>, there exists an inverse reduction <em>S<sub>m<\/sub><\/em> such that <em>ML(S) = ML(S<sub>m<\/sub>(S<sub>b<\/sub>(S))<\/em>.<\/p>\n<p>In other words, no information is lost in the transformation <em>S<sub>b<\/sub><\/em> which means everything which was learnable previously remains learnable.<\/p>\n<p>This may not be the final answer to the question because there may be some algorithm-dependent (mis)behavior associated with controlled feature <em>i<\/em>.  It may also be the case that single task classification is computationally distinguishable from multitask classification. Certainly, computational concerns are one of the reasons specialized multitask classification algorithms exist.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Multitask learning is the problem of jointly predicting multiple labels simultaneously with one system. A basic question is whether or not multitask learning can be decomposed into one (or more) single prediction problems. It seems the answer to this is &#8220;yes&#8221;, in a fairly straightforward manner. The basic idea is that a controlled input feature &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/hunch.net\/?p=175\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Multitask learning is Black-Boxable&#8221;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[29,12,19],"tags":[],"class_list":["post-175","post","type-post","status-publish","format-standard","hentry","category-machine-learning","category-reductions","category-solutions"],"_links":{"self":[{"href":"https:\/\/hunch.net\/index.php?rest_route=\/wp\/v2\/posts\/175","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/hunch.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/hunch.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/hunch.net\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/hunch.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=175"}],"version-history":[{"count":0,"href":"https:\/\/hunch.net\/index.php?rest_route=\/wp\/v2\/posts\/175\/revisions"}],"wp:attachment":[{"href":"https:\/\/hunch.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=175"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/hunch.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=175"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/hunch.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=175"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}