Martin Pool and I recently discussed the similarities and differences between academia and open source programming.
Similarities:
- Cost profile Research and programming share approximately the same cost profile: A large upfront effort is required to produce something useful, and then “anyone” can use it. (The “anyone” is not quite right for either group because only sufficiently technical people could use it.)
- Wealth profile A “wealthy” academic or open source programmer is someone who has contributed a lot to other people in research or programs. Much of academia is a “gift culture”: whoever gives the most is most respected.
- Problems Both academia and open source programming suffer from similar problems.
- Whether or not (and which) open source program is used are perhaps too-often personality driven rather than driven by capability or usefulness. Similar phenomena can happen in academia with respect to directions of research.
- Funding is often a problem for both groups. Academics often invest many hours in writing grants while open source programmers simply often are not paid.
- Both groups of people work in a mixed competitive/collaborative environment.
- Both groups use conferences as a significant mechanism of communication.
Given the similarities, it is not too surprising that there is significant cooperation between academia and open source programming, and it is relatively common to crossover from one to the other.
The differences are perhaps more interesting to examine because they may point out where one group can learn from the other.
- A few open source projects have achieved significantly larger scales than academia as far as coordination amongst many people over a long time. Big project examples include linux, apache, and mozilla. Groups of people of this scale in academia are typically things like “the ICML community”, or “people working on Bayesian learning”, which are significantly less tightly coupled than any of the above projects. This suggests it may be possible to achieve significantly larger close collaborations in academia.
- Academia has managed to secure significantly more funding than open source programmers. Funding typically comes from a mixture of student tuition and government grants. Part of the reason for better funding in academia is that it has been around longer and so been able to accomplish more. Perhaps governments will start funding open source programming more seriously if they produce an equivalent (with respect to societal impact) of the atom bomb.
- Academia has a relatively standard career path: grade school education, undergraduate education, graduate education, then apply for a job as a professor at a university. In contrast the closest thing to a career path for open source programmers is something like “do a bunch of open source projects and become so wildly succesful that some company hires you to do the same thing”. This is a difficult path but perhaps it is slowly becoming easier and there is still much room for improvement.
- Open source programmers take significantly more advantage of modern tools for communication. As an example of this, Martin mentioned that perhaps half the people working on Ubuntu have blogs. In academia, they are still a rarity.
- Open source programmers have considerably more freedom of location. Academic research is almost always tied to a particular university or lab, while many people who work on open source projects can choose to live esssentially anywhere with reasonable internet access.
The atom bomb? Really? Is that the example you want to stick with? While it is true that government funding is typically easiest to aquire if the research has a military use to it, I doubt that american society would be much different now if the atom bomb hadn’t been invented (japanese society maybe, who knows).
You can really pick almost any technology in use today and it will at least be based on academic research. As they say, we are all standing on the shoulders of giants…
Also I’m not really comfortable with the implication, that the open-source community should produce something equivalent to the atom bomb to secure government funding, but I guess that’s not really what you meant to say.
Regarding difference #1, part of the reason that the academic group is less tightly coupled could be because the members of this group tend to keep their current project a bit more secretive.
Also, in open source, you can contribut a small part to a large project. But in academics, the same situation seems hard to happen. Academics want fame and recognition because this is very important in their career.
I think difference 1, also pointed out by Anonymous, is probably the biggest hindrance to academic collaboration and free sharing, largely due to the already mentioned issues with double-blind reviewing. I used to (i.e. 1 year ago) not think this was a big issue, but as I “grow up,” and try to have conversations with people that get cut short because the person to whom I am speaking doesn’t want to talk about something I haven’t yet published because s/he might have to review it, I being to realize how big a problem this might actually be.
Regarding atom bombs: An atomic bomb is many things. One of those things is a proof to a wide cross section of people that basic research is very important. (I certainly agree that many other pieces of academic-style research have had a large impact.)
Secretiveness is definitely an issue in academia.
For example right now I am working on something that I think is fairly interesting, but I am too scared to talk about for fear of being scooped.
OTOH perhaps publication makes up for this, the only cost being that a lot of “negative results” are never discussed even informally leading people to go down the same blind alleys again and again.
As an open-source, independent-scholar programmer and theorist in artificial intelligence I am generally ignored by academics, but in the wide-open free-for-all of open-source AI I am considered fair game for all manner of vicious invective and slanderous attacks, with only now and then an academic reference or an independent evaluation. The chief advantage of working on open source Artificial Intelligence is the complete and utter freedom to work on what I want, when I want, and at the pace I want. If somebody sat me down in front of a computer and told me to program all day, I would not be able to function as the AI fanatic that I became decades ago.
Interesting topic. In my mind, the biggest difference between academic research and open source is the link to teaching. There is a baseline of funding that’s available simply because there is an accepted purpose to academic research. Not sure if this analogy works: an open source project is to academia as a startup is to a software company. I think “openness” is a property that does apply in large measure to academic work despite the short term tension with secretiveness.