I. Technology and Vulnerability

II. An Example Situation

III. Paying Attention

IV. The Electronic Panopticon in Academia

V. The Decentered Classroom

VI. Developing an Approach to Evaluation

VII. Questions and Guidelines

VIII. Other Resources

Steve has already provided us with some excellent guides for asking the right questions about the effectiveness of technology in our classes. I don’t think that there is too much that I can add in the area of student assessment and evaluation, so I would like to turn our attention to a different kind of evaluation and assessment, that which involves the performance of a technology-using faculty member. In the best of worlds, in the ideal school, we would be free to innovate with technology to our heart’s content, confident that our innovations would be recognized at evaluation, tenure, and promotion time as long as they were backed up by sound theory.

Indeed, I am sure that many of us participating in this second Collaboration virtual conference have at least dabbled with computer and Internet-based tools in our classes and scholarship, and many of us have been asked by our schools to try out these tools for distance learning or to enhance our regular classes. Most of us are probably in the amphibious stage, finding our legs in cyberspace with more and more confidence as we become accustomed to the new learning environments of web pages, local area e-mail and chats, and the wider ranges of Internet-based exchanges.

However, many of us have come to realize that we take great risks in innovating with technology in our classes, not only because students can be harsh on teachers who try new approaches when evaluation time comes around, but also because administrators and peers who evaluate our work may not understand the nature of work with technology. They have not done what we do, and often they have not participated in our classes. So, as my colleague Janet Cross is fond of saying "How can they know the dance without dancing?" We are even more vulnerable in that because we are finding our way, charting new territory with these technologies, we may be more apt to make mistakes or rethink our approaches. Further, class work (draft or discussion) on the web may look "messy" or disorganized to the inexperienced eye, resulting in accusations of poor teaching.

Some of the criticisms leveled at online teaching stem from the fact that neither discussion nor draft nor end product look much like traditional college discussions, essays, and tests. Indeed, just today I came across an e-mail message from a colleague who said she had read an article by Dr. W. R. Klemm of Texas A & M "bashing" the use of e-mail discussions for college classes since "navigating these messages necessitated no more than a 'mouse click,' and that in so doing, the student was not 'intellectually driven by context.'" Further, Klemm complained "no real search strategy is required" for students to read such threaded-topic messages. Klemm advocates a hypertext conferencing system "to keep students focused and on task" but fails to take into account the successes many of us teachers have had using simple free tools to engage students in discussions out of class time. Yes, questions and comments may be separated from each other in the flow of online discussion, but our students have shown us that they can ably navigate discussion by making use of such cues as subject line and by copying parts of the messages to which they are responding. True, our discussions may seem messy and disconnected, but the students show that they can engage with others, demonstrate critical thinking skills, and demonstrate their understanding of course fundamentals in writing. The danger is that those evaluating us will not be able to see how learning is taking place in these discussions.

The vulnerability of faculty members who use technology is exemplified in the story of a friend of mine who works for a state university. Until recently she chaired an important national committee, and she is considered a valued contributor to the field of knowledge in computers and writing. Her colleagues across the country had no doubt that she would be tenured, but recently received a shock when they heard that she was denied tenure. It seems that as part of her tenure dossier, she had included the URLs of some of her class pages for evaluators to review. However, certain members of her committee did some investigating of her class materials until they found examples of class web sites (drafts and discussions) in which they felt that the teacher was not participating enough. These they printed and distributed to the whole committee as evidence of lack of involved teaching. This committee may not have been terribly technologically literate, but the one thing they thought they knew about was how to evaluate teaching. Very few of them actually went to the class web sites to examine and try to understand the context for the pages that were printed and brought in; they merely looked at and dismissed the printed pages as evidence of inadequate teaching.

I don’t like to tell the stories of others, and yet when I heard about my friend and colleague’s situation I had to stop and think. On some level, this story clearly demonstrates a failure to communicate excellent academic performance, and illustrates the need for technology-using educators to pay attention to the ways in which the current evaluative structures may fail them when push comes to shove. In an academic world which has up to now been dominated by paper — copies of publications, syllabi, grants, reports, student work, letters of recommendation, etc. — what happens when those of us pushing the envelope into new media try to present evidence of valuable academic work in those very new media? The risk of miscommunication is great, not only because our audience isn’t familiar with new media, but also because we technology-using educators have not specified the criteria for evaluating our work. As "early adopters" of educational technologies, we understand the pedagogical significance of what we do, and each of us knows many colleagues around the country who approve of and applaud our work. And yet our view is rather myopic, since our collegial communities, such as the computers and writing community to which I belong, constitute a rather small percentage of the larger communities of our academic disciplines. Like many issues concerning education and the greater public, evaluation too is about publicity, helping others know and understand not only what we do, but also why we do it.