Writing Checklist to Assess Pre-Course Writing Skills
Name: Dudley Reynolds, Carnegie Mellon, Qatar
Scope: Course – Art History
Assessment Tool: Writing Checklist
What factors/data/circumstances initiated the action?
I was acting as a consultant to an Art History professor at a large, urban U.S. university who was concerned about the quality of writing in a large, introductory survey course he was teaching. He had a series of linked writing assignments that began with producing a museum card about an object in a local museum and evolved into a more detailed analysis of the object and its historical context. He had asked the Writing Center to identify students who could benefit from a tutoring session prior to completing the first assignment.
What did you hope to learn from the assessment?
I had 3 goals: 1) to develop a writing profile of the class as a whole prior to instruction; 2) to provide information on each individual student’s writing to the Writing Center so that they could prioritize and target the students’ needs; and 3) to create a rubric for everyone involved (students, the faculty member, tutors) that would highlight characteristics of good writing rather than problems with bad writing.
How was the data collected?
A 13-item writing checklist was developed that decomposed and assessed the writing according to three main components: Ideas, Rhetorical Control, and Language Use. The checklist was used to indicate if the writing did or did not demonstrate a given characteristic and was faster to use than a rubric that would have required the graders to match the students’ performance to levels. Ease of use was important because there were more than 100 students enrolled in the class.
How was the assessment activity carried out?
Students were given a short writing assignment at the beginning of the course. The assignment was similar in content and organization to the first assignment for the course (the museum card) but had the students write about a familiar context (an object in their home).
Who were the participants?
This was a freshmen course. All enrolled students participated.
When was/will the data be collected?
The assignment was written at the beginning of the second class meeting.
What is the current status?
How was the data analyzed/interpreted?
A group of undergraduate students who worked as writing tutors in the Writing Center graded the writing samples. In order to train them, I first gave them a set of five papers that I had evaluated with the checklist. I explained why I had marked each paper the way I did. I then gave them five more papers and asked them to rate them on their own before we discussed them as a group. This process allowed us to form a consensus for how to apply the criteria on the checklist.
For the class profile, I aggregated the number of checks for each trait on the checklist. This provided information on which traits the class as a whole was likely or unlikely to demonstrate in their writing.
How is the data being used?
Individual profiles were shared with the Writing Center to help them target and prioritize their consultations with the students. Because there was not a budget to pay for a large number of tutoring hours, the individual profiles made it much easier for the tutors to target specific issues for discussion with the students.
Creating a diagnostic writing assignment that matches a course assignment in form but not necessarily content is a useful exercise for helping us separate writing demands from content demands. With respect to the checklist rubric, it is important to note, that these are primarily appropriate for low stakes, formative uses. They are relatively quick to use but provide less information than a rubric with more levels or qualitative comments.
CONTACT US to talk with an Eberly colleague in person!