In the bad old days, the role of professional societies such as ACM or IEEE was crucial. They appointed the editors, who in turn controlled the limited access to journal space and tried to fill that space with material of the highest feasible quality. Today, the amateur has returned to the world of research publication. Anyone can write anything, put it up on the web, and have it read by far more people than would ever see it in a journal.
What are the implications of this new reality for the way we disseminate and evaluate research? There are powerful changes coming in the role of the professional societies, referees, and editors. There also opportunities to rethink process of tenure and evaluation at research universities.
Today, instead of journal editors, the societies need to lend their prestige to a network of area-editors who would be charged with knowing, evaluating for validity (perhaps with the help of referees), and eventually creating a web page pointing to the most important on-line documents related to a particular subject. These subjects can be organized in a hierarchy, with different degrees of focus, just as today there are broad journals like JACM, narrower journals and conferences, and highly specialized workshops, each playing an important role in the research dissemination picture. The role of area-editor should be similar in prestige and responsibility to that of journal editors or program-committee chairs today.
Whether the majority of institutions will eventually start thinking about impact rather than publications remains to be seen. But whether or not they do, another profound change in the evaluation process is coming, courtesy of the ``web.'' As a supplement to letters measuring impact, it is now possible to count hits on a web document or page. It is time to change ``publish or perish'' to ``get hit or get out.'' While counting papers is next to irrelevant, since almost anything can be published if one finds the right journal or conference, the web allows us to measure something that couldn't before be measured: readership.
It has been argued that it is too easy to stimulate hits on a web page or even generate them yourself, mechanically. Admittedly, telling real hits from bogus is not a trivial problem, and I'm not going to suggest I have an algorithm that cannot be defeated. However, if the practice of measuring web hits becomes important, I suspect that there will be adequate study of the patterns of hits that represent real interest, and we shall be able to glean useful information from observed patterns.