{"id":62,"date":"2005-04-16T12:50:20","date_gmt":"2005-04-16T18:50:20","guid":{"rendered":"\/?p=62"},"modified":"2005-04-16T12:50:37","modified_gmt":"2005-04-16T18:50:37","slug":"which-assumptions-are-reasonable","status":"publish","type":"post","link":"https:\/\/hunch.net\/?p=62","title":{"rendered":"Which Assumptions are Reasonable?"},"content":{"rendered":"

One of the most confusing things about understanding learning theory is the vast array of differing assumptions. Some critical thought about which of these assumptions are reasonable for real-world problems may be useful.<\/p>\n

Before we even start thinking about assumptions, it’s important to realize that the word has multiple meanings<\/a>. The meaning used here is “assumption = axiom” (i.e. something you can not verify).<\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Assumption<\/td>\nReasonable?<\/td>\nWhich analysis?<\/td>\nExample\/notes<\/td>\n<\/tr>\n
Independent and Identically Distributed Data<\/td>\nSometimes<\/td>\nPAC,ERM,Prediction bounds,statistics<\/td>\nThe KDD cup 2004 physics dataset<\/a> is plausibly IID data. There are a number of situations which are “almost IID” in the sense that IID analysis results in correct intuitions. Unreasonable in adversarial situations (stock market, war, etc…)<\/td>\n<\/tr>\n
Independently Distributed Data<\/td>\nMore than IID, but still only sometimes<\/td>\nonline->batch conversion<\/td>\nLosing “identical” can be helpful in situations where you have a cyclic process generating data.<\/td>\n<\/tr>\n
Finite exchangeability (FEX)<\/td>\nSometimes reasonable<\/td>\nas for IID<\/td>\nThere are a good number of situations where there is a population we wish to classify, pay someone to classify a random subset, and then try to learn.<\/td>\n<\/tr>\n
Input space uniform on a sphere<\/td>\nNo.<\/td>\nPAC, active learning<\/td>\nI’ve never observed this in practice.<\/td>\n<\/tr>\n
Functional form: “or” of variables, decision list, “and” of variables<\/td>\nSometimes reasonable<\/td>\nPAC analysis<\/td>\nThere are often at least OK functions of this form that make good predictions<\/td>\n<\/tr>\n
No Noise<\/td>\nRarely reasonable<\/td>\nPAC, ERM<\/td>\nMost learning problems appear to be of the form where the correct prediction given the inputs is fundamentally ambiguous.<\/td>\n<\/tr>\n
Functional form: Monotonic on variables<\/td>\nOften<\/td>\nPAC-style<\/td>\nMany natural problems seem to have behavior monotonic in their input variables.<\/td>\n<\/tr>\n
Functional form: xor<\/td>\nOccasionally<\/td>\nPAC<\/td>\nI was suprised to observe this<\/a>.<\/td>\n<\/tr>\n
Fast Mixing<\/td>\nSometimes<\/td>\nRL<\/td>\nInteractive processes often fail to mix, ever, because entropy always increases.<\/td>\n<\/tr>\n
Known optimal state distribution<\/td>\nSometimes<\/td>\nRL<\/td>\nSometimes humans know what is going on, and sometimes not.<\/td>\n<\/tr>\n
Small approximation error everywhere<\/td>\nRarely<\/td>\nRL<\/td>\nApproximate policy iteration is known for sometimes behaving oddly.<\/td>\n<\/tr>\n<\/table>\n

If anyone particularly agrees or disagrees with the reasonableness of these assumptions, I’m quite interested.<\/p>\n","protected":false},"excerpt":{"rendered":"

One of the most confusing things about understanding learning theory is the vast array of differing assumptions. Some critical thought about which of these assumptions are reasonable for real-world problems may be useful. Before we even start thinking about assumptions, it’s important to realize that the word has multiple meanings. The meaning used here is … <\/p>\n

Continue reading “Which Assumptions are Reasonable?”<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-62","post","type-post","status-publish","format-standard","hentry","category-general"],"_links":{"self":[{"href":"https:\/\/hunch.net\/index.php?rest_route=\/wp\/v2\/posts\/62","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/hunch.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/hunch.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/hunch.net\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/hunch.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=62"}],"version-history":[{"count":0,"href":"https:\/\/hunch.net\/index.php?rest_route=\/wp\/v2\/posts\/62\/revisions"}],"wp:attachment":[{"href":"https:\/\/hunch.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=62"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/hunch.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=62"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/hunch.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=62"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}