Using Focus groups to get student feedback
Focus groups (also called group interviews) are a valid and reliable method for collecting data. Faculty have rated focus groups as more accurate, useful, and believable than either student ratings or written comments, although all three methods tend to provide similar impressions of overall quality. (Ory & Braskamp, 1981).
Why use a Focus Group?
Focus groups are particularly effective for identifying agreement across a group and for eliciting suggestions for improvement. They are also much more flexible than surveys or scales because they allow for question clarification and follow-up questions to probe vague or unexpected responses. When conducted by a skilled interviewer, who can use the interaction to motivate students to actively participate, a focus group can generate a wealth of useful information. Contact the Eberly Center [Eberly Center Contact] if you would like one of our consultants to conduct a focus group for you or help you design one.
How to Conduct a Focus Group
The following approach is adapted from Dawson and Caully (1981) and Krueger (1988) and has been used extensively for many years. The following describes a process for conducting a focus group for a course, but the procedure is also applicable for program assessment.
Before the Focus Group
The initial meeting.
The interviewer and the faculty member should meet prior to the focus group so that the interviewer can learn about the goals of the course, any specific concerns regarding the course, and to make arrangements for where and when the interview will take place. It is critical for the interviewer to be someone not directly involved with the course or with the evaluation of the students in another context, such as a teaching consultant or faculty colleague from another department.
Constructing the questions.
Questions should address the concerns of both the instructor and students. If the goal of the focus group is to help the instructor make improvements, it is especially important to address any concerns the instructor might have upfront. Plan on 5 – 7 questions for a sixty-minute session to allow enough time for everyone to speak and for unanticipated answers that lead to new questions. When thinking about the questions, the following guidelines are useful:
- Questions should be Open-ended. Avoid questions that can be answered “yes” or “no”.
- "Why" questions are rarely asked in a focus group. Why questions tend to imply a rational answer whereas you want the participants to openly share their impressions, opinions, and perceptions.
- Questions should be systematically and carefully prepared but have a natural feel and flow. Get feedback on the tone and flow from colleagues beforehand.
- Arrange questions in logical sequence. Usually this means going from the general to more specific about a topic before moving on to another topic.
- Allow for unanticipated questions.
- Pilot test focus group interview.
During the Focus Group (instructions for the interviewer)
Ideally, a focus group is run with an interviewer/moderator and a recorder or note-taker. The note-taker should also record attitudinal or affective characteristics of the respondents if they are particularly strong, (anger, frustration, exuberance, etc). if you are tape recording the session, having a note-taker as well will allow you to quickly summarize and share main points.
Moderate the session so that everyone gets a chance to speak and no one student or small group dominate the discussion. As a general rule, you want to let the students talk with as few interruptions from the interviewer as possible. However, it is sometimes necessary to prompt students to speak or to expand on their answers. Examples of useful probes include:
- “Please tell me (more) about that…”?
- “Could you explain what you mean by…”?
- “Can you tell me something else about…”?
- “Could you give me an example of …”?
Analyzing and Reporting Results
After the session review the notes as soon as possible and fill in any gaps while the session is still clear in your mind. Transcribe the tapes and annotate them with the behavioral or attitudinal observations made by the note-taker.
Usually numbers and percentages are not appropriate for focus group research and should not be included in report. Your report should be descriptive and present the meaning of the data as opposed to a summary of data.
Depending on the needs of the group you are reporting to, you can examine and report your data at different levels:
- Raw data: Present exact statements of the respondents. The data might be ordered or categorized by levels or themes in the topic.
- Descriptive statements: Provide descriptive statements that summarize respondents' comments and provide illustrative quotes using the raw data.
- Interpretation: Building on the descriptive process, provide meaning to the data rather than simply summarizing them. A cautionary note – when interpreting the comments you need to be careful and reflective about your own biases influencing your interpretation.
- Braskamp, L.A. & Ory, J.C. (1994). Assessing faculty work: Enhancing individual and institutional performance. Jossy-Bass: San Francisco.
- Dawson and Caully (1981)
- Krueger, R. A. (1988). Focus groups: A practical guide for applied research. Newbury Park, CA: Sage Publications.
CONTACT US to talk with an Eberly colleague in person!