Surveys - Planning and Project Management Office - Carnegie Mellon University

Surveys


What are your surveys typically used for?

Project-Based Mechanisms:

  • Pre-Assessment

Pre-assessments are usually used to support Delivery or Discovery projects. They can help to scope projects or to evaluate the need for service changes. This project based service can be used to answer the question "What features would you prefer in a new or updated technology or service?"

  • Post-Assessment

Post-assessments are often used to determine a project's success and can be used to answer the question "How well did Computing Services meet your needs with this project?" This type of tool is useful in gathering feedback about what could have been done better, aspects of the project that were well received, and ideas for similar future projects. Post-assessments have also been very helpful in identifying small problems with projects that can easily be remedied.

Ongoing Mechanisms:


Many groups provide ongoing services to customers.  Ongoing survey analysis is a way to evaluate the ebbs and flows of a services' performance from the customer's perspective.  These surveys are generally conducted yearly or bi-annually, with similar questions and can be used to track impacts of service changes to satisfaction ratings. 

Should I use incentives?

The literature is mixed on whether incentives actually work. For most paper surveys, pre-paid incentives work better than post-paid incentives. This may be different for web surveys, however. At Carnegie Mellon, incentives only marginally increase response rates. If you’re going to do it, we’d suggest fewer, higher value prizes over a drawing for several smaller prizes.  Students do love food.

How long should the survey be?

Keep it short. Surveying is easy these days and people are over surveyed. Does the launch period overlap with another major survey? Respondent fatigue is high these days. In fact, response may have dropped over the years due to this fact alone.

Which survey questions are most productive?

For purposes of obtaining quantitative information, closed-ended questions with response options are best. Open-ended questions can be helpful when gathering informal feedback, however there is more error (measurement: getting an unintended response; and non-response: getting no response) associated with these questions and they are difficult to analyze. Generally, we suggest a short, 1-page survey consisting of several closed-ended questions with an open-ended question at the end for informal feedback.

Does the time of week for launch make a difference?

Take a look at response logs for past surveys. On which day of the week are people most likely to respond? This might be a good launch day. At CMU, we try to send faculty surveys out on Friday afternoons and student surveys out on Sunday evenings. The idea here was that faculty are wrapping up their days on Fridays and might like an easy cross off the to-do list. On Sundays, most students are preparing for the coming week and they might enjoy an easy exercise to get their week started.

How much time should the survey be open?

This may sound counter-intuitive, but you might try making the survey window shorter. Folks will feel more urgency around completing the survey if they know it will close sooner than later. They might even fill it out right when they get the invitation instead of letting it fall to the bottom of their inbox. We used to see marginal benefits to keeping a survey open any longer than one week.

What about sending survey reminders?

Set a firm reminder schedule for each survey to remind your non-responders that they haven’t yet replied. We suggest sending a reminder in the middle of the launch period and again on the last day. These email reminders are great for getting little bumps in response.

Information provided on this page courtesy of Leah Lang, Senior IT Metrics and Benchmarking Analyst, EDUCAUSE