Carla Vicens and Eric Siegel contacted me about Predictive Analytics World in San Francisco February 18&19, which I wasn’t familiar with. A quick look at the agenda reveals several people I know working on applications of machine learning in businesses, covering deployed applications topics. It’s interesting to see a business-focused machine learning conference, as it says that we are succeeding as a field. If you are interested in deployed applications, you might attend.
Eric and I did a quick interview by email.
John >
I’ve mostly published and participated in academic machine learning conferences like ICML, COLT, and NIPS. When I look at the set of speakers and subjects for your conference I think “machine learning for business”. Is that your understanding of things? What I’m trying to ask is: what do you view as the primary goal for this conference?
Eric >
You got it. This is the business event focused on the commercial deployment of technology developed at the research conferences you named. Academics’ term, “machine learning,” is essentially synonymous with the business world’s “predictive modeling”. Predictive Analytics World focuses on business applications of this technology, such as response modeling, churn modeling, email targeting, product recommendations, insurance pricing, and credit scoring. PAW’s goal is to strengthen the business impact delivered by predictive analytics deployment, and establish new opportunities with predictive analytics. The conference delivers case studies, expertise and resources to this end.
The conference program is designed to speak the language of marketing and business professionals using or planning to use predictive analytics to solve business challenges — but for the hands-on practitioner or analytical expert focused on commercial deployment who wishes to speak this same language, it’s an equally valuable event.
John >
People at academic conferences would hope that technology developed there can transfer into business use. In your experience, does this happen? And how fast or difficult is it?
Eric >
The best way to catalyze commercial deployment is to show the people it really works outside “the lab” – which is why PAW’s program is packed primarily with named case studies of commercial deployment. These success stories answer your question with a resounding “yes” that the core technology developed academically is indeed put to use.
But predictive analytics has not yet been broadly adopted across all industries, although success stories show at least initial reach in each vertical. So, sure, as one who previously wore a researcher’s hat, commercial deployment can feel slow; having solved the hardest theoretical, mathematical or statistical problems, the rest should go smoothly, right? Not exactly. The main challenges come in ramping up the business “consumer” on the technology so they see its value, positioning the technology in a way that provides business value, and, on the integration side, in preparing corporate data for predictive modeling (that’s a doozy!) and in integrating predict scores into existing systems and processes. These things take time.
John >
Sometimes people working in the academic world don’t have a good understanding of what the real problems are. Do you have a sense of which areas of research are underemphasized in the academic world?
Eric >
To reach commercial success in deploying predictive analytics for the business applications I listed above, the main challenges are on the process and non-analytical integration side, rather than core machine learning technology; its good enough. So, I don’t consider there to be glaring ommissions in the capabilities of core machine learning (I taught the machine learning graduate course at Columbia University and still consider Tom Mitchell’s textbook to be my bible).
On the other hand, there are always places where “real-world” data is going to bring researchers’ attention to as-yet-unsolved problems. A perfect example is the Netflix Prize, the current leader of which (and winner of the recent Progress Prize) will be speaking at PAW-09 – see here.
John’s last question about “what the real problems are” is the one that we’re always wrestling with as ex-academics in the “real world”.
Eric Siegel’s answer seems inconsistent. It claims that core machine learning technology is good enough, that the real problems are non-analytical integration, but also that real world data brings up unsolved problems.
I think he’s right, because he’s really answering two different interpretations of the same question.
The core tech is good enough for lots of interesting applications. And the major pain in applications isn’t implemented an HMM or logistic regression, but figuring out how to apply it to the problem at hand and then how to do the error analysis to tune it up to acceptable performance.
But for some problems, the core tech’s just not good enough. We probably don’t want auto-diagnosis by machine in the emergency room yet, though we have lots of assistive classification technology making its way into production (e.g. recommending heart attack patients get beta blockers based on reading their chart).
One could argue that Google’s search result ranking was an engineering advance, not a scientific one. Social network researchers had been building the same kind of algorithms Google used to rank pages for decades. It’s just a semantic argument as to what constitutes “science” versus “engineering”, or “discovery” versus “application”.
And what about MapReduce? Dean and Ghemawat’s paper states at the start of its related work section that “MapReduce can be considered a simplification and distillation of some of these [previously known] models based on our experience with large real-world computations.”
Netflix is also an interesting case study. The current results are fairly predictable in that there really wasn’t much (any?) new regression technology introduced. Instead, there was a very careful and thorough application of known techniques (SVD, nearest neighbor, neural nets, committee-based methods, regression, various forms of normalization and regularization, etc.).
But hey, maybe there’s something simpler and more powerful out there to rank pages, do large-scale distributed computation or provide recommendations. That’s why we still do research!