Strategies for Assessing Student Writing in the Paperless Distance Learning Environment

Terence W. Cavanaugh Ph.D.
tcavanau@unf.edu
Curriculum & Instruction, College of Education and Human Services, University of North Florida

Abstract

In today’s distance learning environments many of the projects and assignments students complete are still being “down-teched” or printed out as hard copy for assessment purposes.  The “paperless” environment can offer advantages for students and teachers in the creation and assessment of work.  This paper will describe practices that have been found to be effective, through action research with web based distance learning classes, on increasing the efficiency for the submission and evaluation processes for student work, such as research papers, within the “paperless” distance learning environment.  These practices related to structuring evaluation and using rubrics, using screen readers, performing online research and referencing, using readability analyzers, word processing collaboration tools and options for submissions.

Strategies for Assessing Student Writing in the Paperless Distance Learning Environment

The basic design of an on-line course is more student centered or constructivist than many face-to-face courses, which leads to online courses being more project based than test based in their assessments, with the most common format of assessment being the “classic” research paper.  This project-based design prevents many online testing concerns.  The distance learning environment can be a “paperless” environment but many instructors depend on printing out the material that students submit by either e-mail or through the distance learning space or by disk.  Many online instructors print out assignments, read them and make comments on the hard copy, and then go back to their computers and rewrite their notes to send to the student by e-mail.  This process often takes more time than is necessary, and may not be the most efficient use of time and materials.  Additionally a student reading a summary message of comments in an e-mail may find it difficult to know which comments match sections of the paper, especially if the paper is more than just a page or two.

Rubrics

From the onset of the course, students need to be informed about how their assignments will be evaluated, even to the extent of providing rubrics that will be used by the course instructor to evaluate the student before he or she begins the assignment.   The students can then use the rubrics to assist in their own self-assessment, checking their material for completeness before the instructor ever sees it.  Components of the rubrics should address items such as style, length, references, and format.  A good plan is to set up a discussion forum topic or thread about paper topics.  This way students can solicit for feedback and suggestions from their peers and the instructor, and while the instructor is providing guidance and topic approval to one, he or she is providing assistance to all.

Be Specific

In the paperless environment it is better to specify lengths as being an approximate number of words versus so many pages.  Word count is very easy to evaluate with a word counter application of a word processor, and removes the ambiguity of line spacing and margins on pages.  A standard rule of thumb for converting from number of pages to words is that a single spaced document with line breaks between paragraphs and a one-inch margin is usually about 500 words per page.  Some professors have complained about the lack of quality or appropriateness of students’ written work.  A suggestion to overcome this problem is to use a readability statistics tool, such as the one that is built into MS Word.  By going into the Tools -- Options menu, Spelling & Grammar tab and checking the box for Readability Statistics, an instructor can run a readability measure on any written work.  Providing these directions to the students will also allow them to analyze their own work.  The process of assessing student writing is much easier if the instructor determines the expected writing grade level for work and provides students with samples of writing at that acceptable level. A good practice with this concept is to find other papers that the instructor likes and to analyze them before setting up a predetermined grade writing level for the students. Understand that MS Word uses the Flesch-Kincaid model and will only report grade level of writing up to grade twelve.  It will also provide information on the Flesch reading ease scale from zero to one hundred percent with zero percent being the most difficult and one hundred percent being the easiest to read. (Microsoft, 2001)

References & Researching

Many students in the online environment of today prefer the use of online references to print based references. A skill that now needs to be taught is that of evaluating web-based information resources.  While this teaching has been done before, it has often happened within the constraints of the library’s collection.  With online teaching, it becomes important for instructors to make sure that students are educated about finding and evaluating resources from the web.  Researching on the web entails additional steps in assessing the quality of the information.  Instruction as to the number and recency of the references should be provided along with information as to how to accurately cite web based references. Depending on the students' abilities, it can be a good plan to provide direction in the appropriateness of web based references, giving information or guidelines as to how to evaluate a website for quality of content and determine if it is a trustworthy source of information.  While we teach our students about tools such as databases and web resources, we must also teach them about deciding if these resources would be appropriate in a scholarly assignment.  We must instruct them to make judgments when doing online research and provide them with information about how to judge the reliability of sources online.  In the author's online class, students were instructed on how to judge web pages for more than the information presented, but also for the authority and content, publisher and purpose, objectivity and accuracy, and whether the information presented could be triangulated with other sources.

Format

It is important to specify, at the onset or even before class begins, the file formats that are acceptable for the submission of student work.  I would suggest that papers be submitted within the e-mail message, or as an attachment if there are diagrams and images, with the text still embedded in the e-mail message.  Use a common format such as MS Word’s document format (DOC) or the cross platform word processing format of Rich Text Format (RTF).

An option for paper submittal, other than standard word processing formats, is to have all papers submitted as web pages, either on or off line.  The papers can still be marked up using the track changes function of a word processor, can be read with a screen reader or audible web browser, and have the additional advantage that they can be posted into the class online learning space without conversion to share among the students.  Even if the papers are not submitted as web documents, it is very easy today to convert any standard document form into a web/html format.  In this way, a distance learning student is able to share with other students, and create a greater sense of classroom community.

Screen Readers

To streamline the evaluation of student writing, an instructor can use a screen reader program (such as CAST’s e-reader) to “read” and hear the paper and at the same time write up a set of comments and evaluations in a second software window.  Screen readers are a form of assistive technology software that converts text into audible words using phonic rules, allowing one to see the text and also hear it.  This “reading” of the paper provides information to the course instructor in multiple intelligence forms of text, sound, and movement (as the text being read is highlighted), likely improving the reading speed, comprehension and retention of the material.  Screen readers range from the simple and free, such as ReadPlease 2000, to the fully interactive and complex such as Dragon’s Naturally Speaking, which not only can read text but also control the entire Windows environment through speech.  It is also suggested that a similar reader be provided to the students so that they can use the screen reader as a personal assistant helping them with their own editing.  While it usually takes about 25% more time to read documents on screen than in print, I feel that through the use of screen readers the time needed to evaluate a “screened paper” over a printed version is actually reduced, especially considering that all comments would have been retyped and sent as an e-mail only response.  It should be understood that unless an instructor is already doing a lot of screen reading, then he or she would be “learning” a new way to read, a new body position, new page shape, and new controls.  It takes time to become comfortable with this new way of reading.  To get more high tech, the instructor could read the paper with a screen reader, make comments into a digital voice recorder, then transfer that recording into the computer and send it as an attachment to the student, very much like having a phone conversation or personal discussion.

Feedback

The instructor has choices for reporting feedback about students’ assessments.  One way is to begin an e-mail to the student while reading the paper, and then make notes and comments that will be sent to the student.  Another way involves either opening the paper file or copying and pasting the paper’s text into MS Word.  In Word the instructor can use the word processing collaboration tools by activating the Track Changes option in the Tools menu.  With the tracking on, the instructor can add comments to the original paper in a way very similar to adding comments to a printed page.  If both parties are using the same form of software such as Word, then the student work can be sent back and forth as attachments with the changes suggested and the student can view the suggestions within the document.  Even if both are not using Word, the comments will be imbedded into the mail sent to the student if the instructor copies the whole paper and then pastes it into an e-mail.   Additionally if both are using Word, then the instructor can use the Insert Comment command to add a more extensive explanation or comment about the written work.  Responses from students using this method have all been positive, including the ease of seeing the location and specifics that needed change, and the ease of understanding inserted typed responses rather than trying to interpret hand-written comments between the lines of the text.

It is a good practice to include the grading rubric in email to students, if the message includes an assignment grade.  This practice provides a consistent tool for the instructor to use while grading the student work and provides a reminder to the student about the expectations concerning his/her work, and encourages him/her to use the rubrics next time if they he/she didn’t this time.

A long-term bonus with paperless submission is ease of archiving all students’ work.  By saving e-mail attachments, an instructor can have good and poor examples (remember to remove names and get permission) for the next class or samples of work for the next program evaluation.

Microsoft. (2001). Readability scores. Available online at: http://www.microsoft.com/Office/techinfo/productdoc/2002/en/word/worefReadabilityformulas.htm. Last accessed March 2001.

CAST eReader http://www.cast.org/products

ReadPlease 2000 http://www.readplease.com