At the beginning of the month we shared with you an article by Jim Kirkpatrick about your "training defaults". He defines these as hard-wired behaviors that have become habits, but may not lead to the highest level of results. These are things we do automatically without considering if they are outdated or less effective than other options.

This week we ask you to reset your "survey default", which we see often as a reliance on surveys as the only means of measurement and data collection.

Employee surveys have been the standard evaluation method for decades. It may be due to their ease of implementation, and ability to gather large amounts of quantitative data quickly. Many organizations even refer to their Level 1 reaction sheets as "evals."

Surveys are a critical component in your evaluation toolkit, but they should be combined with other methods that yield richer qualitative information to complement the quantitative data.

Here are some reasons to consider a mixed approach. Research shows that targeted observations with accompanying checklists are statistically more valid and reliable than surveys. In addition, surveys only gather certain type of quantitative data, and little genuinely useful qualitative information.

When Kirkpatrick Partners conducts impact studies, surveys are augmented with a relevant combination of focus groups, direct observation, work reviews, interviews, or other data collection methods. This is a best practice you can duplicate for your own initiatives.

When you do use surveys, here are some recommendations:

• For mission critical initiatives, survey not only actual training participants, but also managers, supervisors, co-workers, peers and customers.

• Ensure that your surveys contain learner-centered questions.

• Use hybrid evaluations; that is, surveys that include questions from two or more of the Kirkpatrick levels.

• Avoid questions that attempt to empirically quantify and isolate the impact of the training event. It is mathematically impossible, contrary to business partnership, and produces measurements that get questioned and doubted by stakeholders. Here are examples of questions to avoid:

o "What percentage of your time on the job relates to the content covered?"

o "What percentage of what you learned do you plan to apply?"

o "What percentage of what you learned in training do you currently apply on the job?"

o "What percent increase in productivity have you seen in your work since the course?"

o "What percentage of that improvement can you directly attribute to the course?"

o "How confident are you in these percentages?"

Read the original article.