Press Release
Handbook of Measuring and Evaluating Training: 2010
Mon Dec 07 2009
Learning professionals have a love-hate relationship with measurement and evaluation. On one hand, most people agree that evaluation and the evolving results can represent important information. On the other hand, the act of evaluation seems daunting and beyond their interest in learning and development. But there is no arguing that the call for accountability of resource expenditures is louder than ever. To support learning professionals as they answer this call, ASTD is launching a new handbook, _Handbook of Measuring and Evaluating Training._****
T his new book addresses the mechanics of evaluation from the perspective of a variety of contributors. It addresses content relevant to the four phases of measurement and evaluation: planning, data collection, data analysis, and reporting. In addition, chapters are included that support implementation of your measurement practice. Each chapter is written to achieve at least three learning objectives. A knowledge check is included at the end of each chapter to ensure readers gain at least one new insight.
Content is presented in four parts:
Evaluation Planning
Data Collection
Data Analysis
Measurement and Evaluation at Work
Evaluation Planning
"Plan your work, work your plan" was a lesson I learned from my dad at a very early age. That explains some of my slow-to-start project executions! The point of this is, however, if you spend time planning, execution is simplified. Part I of the new Handbook describes three critical issues in the planning phase:
Identifying Stakeholder Needs
Developing Powerful Objectives
Planning Your Evaluation Project
Data Collection
Obviously, data collection is an important phase because without data collection, there are no results. The question most often asked when it comes to data collection is, "what is the best technique to collect data?" The answer to this question is simply "it depends." There are a variety of ways in which data are collected. How you decide which technique to use depends on the purpose of the evaluation, type of data, time to collect data, cost, organization culture, and other constraints and conveniences. Part II topics include:
Using Surveys and Questionnaires
Designing Criterion-Referenced Tests
Conducting Interviews
Conducting Focus Groups
Using Action Plans
Using the Success Case Method
Using Performance Records
Data Analysis
Many learning professionals are most concerned with this phase of the evaluation process. Of course without good analysis, it's pretty tough to explain the results. Data analysis may require descriptive statistics, inferential statistics, content analysis, and cause-and-effect analysis, conversion of data to money, cost calculations, and ROI analysis. What you do when depends on the purpose of the evaluation, the type of data, time to analyze, cost, organization culture, and other constraints and conveniences. Part III, Data Analysis presents content on:
Using Statistics
Analyzing Qualitative Data
Isolating the Effects of the Program
Converting Measures to Monetary Value
Identifying Program Costs
Calculating the ROI
Measurement and Evaluation at Work
To make evaluation work, results must be put to use. This begins with reporting results. The results must, however, be relevant to stakeholders. In addition to reporting results, the data evolving from the evaluation process must be put to good use. To make evaluation work for the long-term, systems must be put into place. Part IV, Measurement and Evaluation at Work , includes topics such as:
Reporting Evaluation Results
Giving CEOs the Data They Want
Using Evaluation Results
Implementing and Sustaining a Measurement and Evaluation Practice
Selecting Technology to Support Evaluation
In addition, we've included case studies on evaluating technology-enabled learning, leadership development, global sales training, technical training, and training using a simulation component.
Voices
A final part of the book, Part V, titled Voices , presents summaries of interviews with experts in training measurement and evaluation. Dr. Rebecca Ray, award winning Chief Learning Officer and an expert in talent management, conducted in-depth interviews with the experts who paved the way for training measurement and evaluation. Each interview has been converted into a series of podcasts that will be available for download. In addition, we plan to make the entire transcript of each interview available. You will hear from experts such as Don Kirkpatrick, Jack Phillips, Robert Brinkerhoff, Dana Robinson, Jac Fitz-enz, Bill Rothwell, and more.
How to Get the Most out of This Book
We're excited about this book for many reasons. While the book provides readers with a little information on many aspects of evaluation, it is only the beginning of the learning opportunity. ASTD will be launching a website to support the dissemination of additional content. We're already in the process of collecting case studies, tools, and examples for download. Through the website you will be able to download the Voices podcasts as well as have access to other resources.
And of course, we have the Evaluation and ROI Blog that you are currently viewing. Through this blog we can share thoughts and ideas on measurement and evaluation .
Call to Action
This initial blog announces ASTD's upcoming _Handbook of Measuring and Evaluating Training._** The book will be available in Spring 2010. In weeks to come, I'll post a variety of topics to generate discussion. To get the conversation going, l et us hear how you are using measurement and evaluation in your organization !**