I found these two essays on bad ideas interesting. Neither of these is written from the viewpoint of research, but they are both highly relevant.
- Why smart people have bad ideas by Paul Graham
- Why smart people defend bad ideas by Scott Berkun (which appeared on slashdot)
In my experience, bad ideas are common and over confidence in ideas is common. This overconfidence can take either the form of excessive condemnation or excessive praise. Some of this is necessary to the process of research. For example, some overconfidence in the value of your own research is expected and probably necessary to motivate your own investigation. Since research is a rather risky business, much of it does not pan out. Learning to accept when something does not pan out is a critical skill which is sometimes never acquired.
Excessive condemnation can be a real ill when it’s encountered. This has two effects:
- When the penalty for being wrong is too large, it means people have a great investment in defending “their” idea. Since research is risky, “their” idea is often wrong (or at least in need of amendment).
- A large penalty implies people are hesitant to introduce new ideas.
Both of these effects slow the progress of research. How much, exactly, is unclear and very difficult to imagine measuring.
While it may be difficult to affect the larger community of research, you can and should take these considerations into account when choosing coauthors, advisors, and other people you work with. The ability to say “oops, I was wrong”, have that be accepted without significant penalty, and move on is very valuable for the process of thinking.