Surveys to Assess Program Effectiveness

Instructor: Multiple (Example Provided by Mark DeGraef)
Scope: Department - Materials Science and Engineering, School of Engineering
Assessment Tools: Surveys to Assess Program Effectiveness:
– Senior Exit Survey
– Educational Objectives Survey (for faculty & industry employers of graduate students)

Motivation:

The Materials Science and Engineering Department developed six program educational objectives as part of the ABET external assessment process. Two of these objectives are to: 1.) Produce graduates who are successful in a top graduate school in Materials Science and Engineering or in a related discipline; and 2.) To produce graduates who take positions as materials science engineers and are successful in the field or related professional activities. For ABET and ourselves we wanted to know if our program was meeting the needs of our students when they moved on to graduate school or careers.

Goal:

To learn how effectively the MSE curriculum was preparing our students for graduate school and their careers by collecting feedback from the graduates, graduate programs they attended, and their employers. We also wanted to learn the extent to which our program educational objectives are linked to outcomes, and measurable.

Methods/Tools:

We developed three different surveys to collect data from the constituencies that were identified as important to assessing the effectiveness of these two objectives: an MSE senior survey; graduate school supervisor survey; and industry supervisor survey.

Student Survey

Students were asked to fill out an online Senior Exit interview asking them to review the curriculum as a whole in light of the program educational objectives. Surveys were scored on a scale of 1-10. MSE faculty decided as a group in 2004 that a score of 7 or higher on the survey would indicate that a particular program educational objective (PEO) had been achieved.

Educational Objectives Survey (for faculty supervisors of graduate students)

During the spring of 2005 faculty from departments that had hired CMU students as graduate students we invited to evaluate students based on the PEOs.

Educational Objectives Survey (for industry employers of graduate students)

Graduates who work in industry were asked to pass on a survey based on PEOs to their immediate supervisors.

Implementation:

Participants

Three different groups participated in the surveys: graduating seniors, graduate advisors in programs who had admitted CMU graduates; and the immediate supervisors in organizations that had hired a CMU graduate.

What is the current status?

The surveys were carried out during two academic years: 2004-2005; and 2005-2006.

Impact/Results:

Results from the surveys were used to revise and revisit both the curriculum as well as the Program Educational Objectives.

Comments:

Response rates on the surveys varied greatly. The response rate on the industry employer survey was particularly very low.