Welcome to ALT Highlights, a series of blog posts spotlighting various happenings at the recent conference ALT 2021, including plenary talks, tutorials, trends in learning theory, and more! To reach a broad audience, the series will be disseminated as guest posts on different blogs in machine learning and theoretical computer science. John has been kind enough to host the first post in the series. This initiative is organized by the Learning Theory Alliance, and overseen by Gautam Kamath. All posts in ALT Highlights are indexed on the official Learning Theory Alliance blog.
The first post is an interview with Joelle Pineau, by Michal Moshkovitz and Keziah Naggita.
We would like you to meet Dr. Joelle Pineau, an astounding leader in AI, based in Montreal, Canada.
Name: Joelle Pineau
Institutions: Joelle Pineau is a faculty member at Mila and an Associate Professor and William Dawson Scholar at the School of Computer Science at McGill University, where she co-directs the Reasoning and Learning Lab. She is a senior fellow of the Canadian Institute for Advanced Research (CIFAR), a co-managing director of Facebook AI Research, and the Montreal, Canada lab director. Learn more information about Joelle here and her talk here.
Reinforcement Learning (RL)
How and why did you choose to work in reinforcement learning? What are the things that inspired you to choose health as a domain of application for your RL work?
I started working in reinforcement learning at the beginning of my PhD in robotics at CMU. Quite honestly, I was delighted by the elegance of the mathematical formulation. It also had some link to topics I studied previously (in supervised learning & in operations search). It was also useful for decision-making, which was complementary to state tracking & prediction, which was the topic studied by many other members of my lab at the time.
I started working on applications to health-care early in my career as a faculty at McGill. I was curious to explore practical applications, and found some colleagues in health-care who had some interesting decision-making problems with the right characteristics.
How would you recommend a newcomer enter the RL field? For RL researchers interested in safety, is there some literature you can recommend as a starting point?
Get familiar with the basic mathematical formalism & algorithm, try your hand at easy simulation cases. For RL and safety, the literature is very small and quite recent, so it’s easy enough to get started. Work on Constrained MDPs (Altman, 1999) is a good starting point. See also the work on Seldonian RL, by Phil Tomas and colleagues.
In your talk you mentioned applications of RL to different domains. What do you think is the main achievement of RL?
The AlphaGo result was very impressive! Recently, the work on using RL to control the flight of the Loon balloons is also quite impressive.
What are the big open problems in RL?
Efficient exploration continues to be a major challenge. Stability of learning, even when the data is non-stationary (e.g. due to policy change), is also very important to address. In my talk I also highlighted the importance of development methods for RL with responsible properties (safety, security, transparency, etc.) as a major open problem.
Collaborations
Based on your work in neurostimulation, it appears that people from different fields of expertise were involved.
Yes, this was a close collaboration between researchers in CS (my own lab) and researchers in neuroscience, with expertise in electrophysiology.
What advice would you give researchers in finding interdisciplinary collaborators?
This collaboration was literally started by me picking up the phone and calling a colleague in neuroscience to propose the project. I then wrote a grant proposal and obtained funding to start the project. More generally, these days it’s actually very easy for researchers in machine learning to find interdisciplinary collaborators. Giving talks, offering office hours, speaking to colleagues you meet in random events – I’ve had literally dozens of projects proposed to me in the last few years, from all sorts of disciplines.
What are some of the best ways to foster successful collaborations tackling work cutting across multiple disciplines?
Spend time understanding the problems from the point of view of your collaborator, and commit to solving *that* problem. Don’t walk in with your own hammer (or pre-selected set of techniques), and expect to find a problem to show-off your techniques. Genuine curiosity about the other field is very valuable! Don’t hesitate to read the literature – don’t expect your collaborator to share all the needed knowledge. Co-supervising a student together is also often an effective way of working closely together.
Academia, industry and everything in between
During the talk, you mentioned variance in freedom of research for theoreticians in industry versus academia. Could you elaborate more about this? Are there certain personality traits or characteristics more likely to make someone more successful in academia versus industry?
For certain more theoretical work, it can be a long time until the impact and value of the work is realized. This is perhaps harder to support in industry, which is better suited to appreciated shorter-term impact. Another big difference is that in Academia, professors work closely with students and junior researchers, and should expect to dedicate a good amount of time and energy to training & developing them (even if it means the work might move along a bit slower). In industry, a researcher will most often work with more senior researchers, and the project is likely to move along faster (also because no one is taking or teaching courses).
How do you balance leadership, for example, at FAIR, with students advising like at McGill, research [CIFAIR, FAIR, McGill, Mila], and personal life?
It’s useful to have clarity about your priorities. Don’t let other people dictate what these are – you should decide for yourself. And then spend your time according to this. I enjoy my work at FAIR a lot, I also really enjoy spending time with my grad students at McGill/Mila, and of course I really enjoy time with my family & friends. So I try to keep a good balance between all of this. I also try to be clear & transparent with other people about my availability & priorities, so they can plan accordingly.
What do you think distinguishes the mindset of an extraordinary researcher?
To be a strong researcher, it helps to be very curious, genuinely want to understand and find out new knowledge. The ability to find new connections between ideas, concepts, is also useful. For scientific research, you also need discipline and good methodology, and a commitment to deep understanding (rather than “proving” whatever hypothesis you hold). Frankly, I also don’t think we need to further cultivate the myth of the “extraordinary researcher”. Research is primarily a collective institution, where many people contribute, in ways small and big, and it is through this collective work that we achieve big discoveries and breakthroughs!
Very Useful and Knowledgeable Tutorial of Machine Learning. I really appreciate you for writing such an informative article.