Carnegie Mellon University

Eberly Center

Teaching Excellence & Educational Innovation

Analyzing Data from an Instructor-Administered Early Course Feedback Surveys

We recommend using the protocol provided below for analysing data from an instructor-administered Early Course Feedback Survey.

1.)  Read through all of the student worksheets and identify significant themes or topics (e.g., class discussion, lecture slides, grading, etc.) that students commented on.

Here, “significant themes” refers to categories of feedback that each arise multiple times across students. Write down the conspicuous themes.
  • Try to identify at least three themes for “strengths” and three themes for “suggestions.” These themes should be value-neutral, rather than focused on students’ opinions about them. For instance, if students say “no one speaks in my small breakout room discussions,” the topic would be “Small group discussions” or “breakout rooms”
  • Focus on themes that are within your control. For instance, if many students say “I don’t like the time of the class,” that’s likely something that you cannot change in the middle of the semester. Make a note of it, but look for themes that are within your control.

2.) Review the student worksheets and highlight/mark/color code comments that pertain to each of the themes that you identified.

Comments could be listed under “strengths” or “suggestions”. For example, if you identified “class discussion” as a theme, then student comments like “Zoom breakout rooms are great because I can discuss course concepts with my classmates” and “I’d like more opportunities for classroom discussions” would both get flagged for this theme. The first comment is a strength. The second comment is a suggestion.

3.) In addition to highlighting these comments, record a + or - sign beside the comment and track how many of each sign the theme has.

This step will help to identify if there is some consensus about how effective a particular aspect of your course is. For example, if “class discussion” has 10 plus signs and 1 minus sign, then you can feel good that class discussion is working well. This step can be challenging if students have not worded their feedback in a constructive manner or if they have only listed complaints instead of providing suggestions. Try to focus on the essence of the issue that the student is identifying and not the way the student has expressed it.

4.) Review the number of + and - signs for each theme and dig deeper into the specific strengths or suggestions that students identified.

  • If the “grading” theme has a lot of minus signs, what are the particular, actionable suggestions that students have made (e.g., needing more clarity, needing grades/feedback quicker, etc.).
  • Some themes might have similar numbers of plus and minus signs. In those cases, return to the student comments to see if you can reconcile this difference. For example, maybe students find breakout rooms beneficial for learning, but they are suggesting that they spend more/less time in them. 
  • If students have focused on complaints rather than suggestions, see if you can identify a suggestion that would address the complaints or consider bringing up that theme to the whole class to discuss avenues for solutions (this conversation could be open-ended or you could prepare a few solutions that you would be ready to implement for the rest of the semester and let them decide which one addresses their concerns the best).

5.) Return to unmarked student comments and identify new themes as time permits.

If you only identify the top three themes/topics for “strengths” and the top three for “suggestions”, then you’ll likely still have student comments that you didn’t mark or group together. If time permits, return to these student comments and see if there are additional themes or recommendations that you can glean from them. Sometimes, suggestions from a minority of students can be the most useful for all learners. However, one can also easily get hung up on one or two strongly worded comments. Ask yourself, will implementing this suggestion improve student learning or engagement?

6.) Prepare a plan to address students’ comments during the next class meeting.

How you address the feedback is up to you. At the very least, we recommend thanking students for taking the time to thoughtfully fill out the survey. If you plan on making changes based on student feedback, we encourage you to address both strengths (i.e., things you will continue to do or do more often) and students’ suggestions that you are able/willing to implement. If there are specific changes you are making or considering, we encourage you to name them and link them to student feedback. Otherwise, students may not feel that their feedback was heard or that the process.
  • We recommend that you respond to student feedback verbally, during class, rather than by email, to minimize misunderstandings or misinterpretations.
  • If you feel that some of the suggestions were not specific, or some themes didn’t yield specific ways to improve, or if student feedback was divided evenly (e.g., half the students said they enjoyed small group discussions and the other half said small group discussions do not help with their learning), then consider having a class discussion or administering a more targeted survey to dig deeper into the issue or what potential tweaks could improve the student experience or learning.
  • This discussion with students can also be an opportunity to explain your choices and teaching philosophy. For instance, “We have small group discussions to give you a chance to think about the issues in a lower stakes situation.”
 

If this analysis has identified some areas of your course that you would like to work on, please feel free to email the Eberly Center for assistance (eberly-assist@andrew.cmu.edu).