Why we need the NEF

Science in the United States is publicly funded and has contributed greatly to the strength of US in both military and economic terms. The business of software development is primarily done in a radically different way today. Although this business has enjoyed some success it has also suffered from various failures. I will argue that software development should be supported in the same way that science is supported in the US. There should be a National Engineering Foundation to which engineers with good project ideas can submit proposals and recieve grants.

The NSF has been a strongly historically validated success. WWII left the US in a strong position with respect to science and economic production because the US did not have significant civilian losses during the war. Capitolizing on this initially strong position, the NSF (and public funding for science in general) have turned the language of science into English - it is spoken everywhere that research is done. This is a strong indication of how succesful the NSF has been in supporting science in the US. I will not go into the details of this success although many others have done so.

Why is public funding for science such a useful methodology? Typically, private companies can not afford to invest in the long range research which comes out of science labs. Without this long range research, companies can not come up with radically new products and create new markets. What companies _are_ good at is refining an existing idea. A competitive environment provides strong incentives to constantly look for ways to refine production.

Another problem also hinders private research. When a company succeeds in new interesting reasearch they patent all results achieved. Leaving aside the issue of whether or not this is desirable, it's important to consider the effect of a patent on a company. A patent is a guaranteed monopoly for about 2 decades. The typical inefficiencies of a monopoly will accrue over this time. The patent system has some worse effects on research. Patents make it undesirable for other researchers to follow up on the patented work. Trade secrets are similarly bad at inspiring follow up research.

Private companies don't do long range research because they must stay focused on short term goals in order to remain competitive. When they do any research, the companies tend to patent or otherwise make the research inaccessible to fellow researchers who might have good ideas for following up on the original work. These combined inefficiences appear to overwhelm the inefficiences of a centrally administered funding agency for much research.

Software is very similar to research in many ways. The cost of copying software is zero just as the cost of copying a research paper is essentially zero. Constructing working software is hard just as constructing a research paper is hard. Once developed, software can potentially benefit everyone just as with science. Thus research and software production have a similar signature for initial and marginal costs. There are some differences of course. There are a few more programmers than researchers and a program is more immediately applicable than a reasearch paper.

This "easier applicability" is what has made companies interested in software production with some success. Large businesses have been founded and exist to build, support and maintain software. Nonetheless, I consider most of these businesses to be clumsy and the use of their products to be dangerous in several ways. Software patents have not yet played a significant role in this market. Instead, the preferred primary means of IP protection is the copyright. Typically, this takes the form of copyright on a compiled program with a trade secret on the actual source code that programmers directly modify. Needless to say, this is a strong disincentive for other (external) programmers who might want to build upon the ideas and code contained in the original program. Other symptoms of the underlying problem also arise. Programing is hard and making programs that work right is difficult. Many companies fail, because of lack of resources or time, to produce bug free programs. In this failure, they pass the cost onto their customers who have to deal with buggy and crashing programs. Since monopolies are typical in the software market many companies can succeed in passing these costs onto their customers without losing the customers.

Darpa and NSF have incidentally fulfilled portions of the NEF role with notable successes. The most striking example of this is the construction of the internet. The internet is a heterogenous collection of computers sharing common communication protocols. It's use has exploded in recent years and appears to be set for a continued explosion. There have been other successes however. Unix was developed partially in a government regulated monopoly and partially in research labs. These successes show that it is possible to achieve succesful software development by governmental grant.

There is a clear need for standardization in the software industry. A standardized format for saving word processing files (for example) would be very beneficial for information exchange in contrast to the hodgepodge of nonstandard formats currently in use. The same applies to spreadsheets and other parts of an "office suite". Standardized "instant messaging" could prove very beneficial to the public at large.

There is another class of applications that which could benefit greatly from the NEF. Many CS researchers write interesting applications of verious sorts. These applications are incidental to their research and typical not of an appropriate quality for external use. In order to develop such an application for external use a domain boundary from grant based research to profit based construction must be passed. This is a significant barrier to entry which kills many of these (fundamentally interesting) research programs. If an NEF existed, then it might be possible for a computer programmer to submit a grant proposal for turning this interesting program into a useful program.

I know little about the practicalities of constructing a bureacracy to administer the NEF. Because of the above scenario, it seems likely that expanding the scope of the NSF is the simplest and most direct way to achieve grants for engineers who want to work on interesting programming projects.

One of the essential ingredients of science is peer review. Peer review is fundamentally what seperates the "good" science from the "bad" science. Unfortunately, this tradition is not nearly as strong in programming. It does exist in some ways. Linux is a reasonably large example of peer reviewed code and is known for stability and robustness - at least in comparison to other non-peer reviewed code.

Since programming has a similar cost profile to research it would be interesting to experiment with an NEF. An NEF might also exist symbiotically with the NSF avoiding funding domain boundaries currently inherent in transitioning from research to commercial use. Arguing that something should be done one way and not another is not easy, even in retrospect, because history only plays out one way. We can't repeat the experiment making the other choice to see how it plays out. Luckily, we don't have to make a binary choice here. Some governmental funds can be allocated to the NEF and we can try to estimate the success of the project after 5 (or so) years time. Based on that more or less funds can be allocated to the NEF. Personally, I expect the NEF to be quite succesful on a scale similar to the way that the NSF has been succesful to science.