A White Paper for the UNC-Chapel Hill Scholarly Communications Convocation

January 2005



Jeffrey Pomerantz, School of Information and Library Science


Bob Blouin, School of Pharmacy



This whitepaper, as the title suggests, concerns the issue of valuing non-traditional forms of scholarly output. By non-traditional, in the context of this convocation, we mean electronic as opposed to print publication, online as opposed to classroom teaching, the creation of electronic systems for collaboration as opposed to other forms of service to one’s professional community. We mean electronic here, but this discussion can and should be expanded to include scholarly output in other media.


The issue of valuing vehicles of scholarship – whatever the medium – rests on another, more fundamental question: what constitutes scholarship? There is of course no universal standard for evaluating scholarship or scholarly output: the university, the school, the department, the discipline, all are likely to have different standards. This may then lead to Justice Potter Stewart-like evaluations of excellence, where it cannot be defined, but one knows it when one sees it. Still, despite this inconsistency of standards, the scholarly community has been producing excellent scholarship for centuries, so clearly the lack of a universal standard for evaluating traditional scholarly output is not a major stumbling block. The problem is not to arrive at a universal standard for evaluating scholarly output, but to ensure that the different levels of the institution have standards that recognize and reward excellence in scholarship when it is present.


In the evaluation of scholarship, the value is in the ideas, and in the impact that those ideas have, and not in the medium in which those ideas are presented. We must recall that the scholarly journal as we know it today only dates back to the mid-1600s, and it undoubtedly took some time for this new medium to be accepted alongside meetings of learned societies as legitimate vehicles for scholarly communication. The credibility of any new medium must be earned, but once a new medium is established as being valuable, a community of users over time adapts to and embraces that medium. This is as true for scholarly uses as for any other purpose.


One of the goals of this convocation is to begin a university-wide conversation on issues affecting the changing nature of scholarship. This begs the question, why is it in Carolina’s (or any university’s) interest to address this issue? This is not an issue that can be addressed unilaterally, given the inconsistency of standards for evaluating scholarship across the scholarly community. We suggest that this issue is worth addressing for the very reason that Chancellor Moeser said in his State of the University Address in September 2004 was the University’s top priority: faculty recruitment, retention, and development.


As electronic and online forms of scholarly output come to take center stage in all aspects of scholarship, it is increasingly important for the university to be a favorable and nurturing environment for faculty who will be innovators in these areas. At the same time, it is increasingly important for faculty to innovate, to place high demands on the university for infrastructure to support non-traditional vehicles of scholarly output, and for recognition of the importance of this output. Many of the innovators on the faculty are likely to be younger faculty members: this upcoming generation of faculty are not only more comfortable with using and producing electronic materials, they also place more demands on the institution’s infrastructure. As senior faculty approach retirement and new positions are filled with younger faculty members, the demands on the university and the necessity of fostering a nurturing environment for innovation will only increase over time. In order to recruit and retain innovative faculty members, the university must provide such support.


If the next generation of faculty are comfortable with and demand support for electronic materials, then the next generation of students are even more so. Discussions of the Millennials (or GenY, Echo Boomers, etc.) emphasize their early adoption of and adaptability to technology. The education of the Millennials therefore demands the use of innovative instructional technology. These days it is easy and convenient for users to find information online, though this information may be quite poor in quality. Teaching a generation that has grown up accustomed to being able to provide itself with content requires innovative instructional strategies. It will be necessary to move beyond the traditional classroom, the traditional exchange of content, and to move toward more sophisticated methods of integrating technology into the classroom experience.


How, then, should the university change so as to provide support for non-traditional vehicles of scholarship? How can the university create standards for evaluating scholarship that value non-traditional forms of output? This change cannot come by simply demanding it; no one like being dictated to, and academics probably less than most. No, the scholarly community will be swayed by solid intellectual arguments and evidence of the value of new forms of scholarly output. When a solid argument can be made for the value of a new form of scholarly output, scholars will pay attention to output in that form. If excellent scholarship appears in new forms, it will be recognized – perhaps not immediately, but in time. Indeed, these arguments are already being made, evidence is already being presented. Early arguments against electronic scholarly journal publishing were that it lacked intellectual validity and diminished the rigor of the review process; this was in time shown to not be true, and there are now many top-quality electronic journals. Early efforts in online instruction were received with skepticism and the same criticisms of its intellectual validity; now research has demonstrated that online and classroom teaching each are useful for teaching students with different learning styles. The medium is not important; what is important are the ideas presented and the impact that those ideas have on the recipients and on society at large. The demonstration of the excellence of scholarship, in all its forms, is in this impact.


The impact of scholarship is central to the academic review process, and the academic review process is central to the retention and rewarding of faculty. And central to the academic review process is peer review. Peer review is where the rubber hits the road, so to speak, in terms of standards for evaluating scholarship. As scholars are favorably reviewed by their peers for producing non-traditional output, others perhaps less innovative will be emboldened to produce such output. As the Institute for Scientific Information (ISI) computes impact factors for electronic journals, for example, this will provide one standard that may be used in peer review (a simple and expedient one perhaps, but one that is commonly employed). As structured teaching evaluations are developed for online teaching, this will provide another standard. This argument for “critical mass” is circular, but persuasive: when scholars produce non-traditional output, more scholars will produce non-traditional output.


Valuing non-traditional forms of scholarship requires a critical evaluation of non-traditional forms of scholarship. Such a critical evaluation should start within institutions that have the most to lose by failing to recognize the importance. Those institutions may be the university, or schools or departments within the university. To use examples that the authors are familiar with, schools of information science and schools of pharmacy are just such institutions that have much to lose if they fail to recognize the importance of non-traditional forms of scholarship. An information science school that does not recognize the value of electronic access and publishing, or that does not provide support for high-end computing will not be able to recruit or retain faculty. A pharmacy school that does not provide support for systems for managing large data sets for collaborative and distributed research will not be able to recruit or retain faculty. Any school that does not recognize the value of online teaching will not be able to recruit students. This critical evaluation will come in the form of studies and editorials, and convocations like this one. Through critical evaluation, the range of possible approaches will be explored, and the value of non-traditional scholarly output to higher education will be articulated.