Every part of our daily lives now generates data, from our shopping, to our mobile phone use, to our interaction with the Internet. For good or bad, the collection, analysis, and interpretation of this data is now part of the world we live in. Big data is here to stay.

What effect will this have on the training and development (T&D) profession?

It is clear that the advent of big data presents us with both opportunities and challenges, but my concern here is not how we use that data. It is whether we in T&D are able to. With some exceptions, I am largely skeptical that we have the necessary skills, and I believe this represents a major challenge to the profession.

As the analysis of data and an understanding of its complexities and meaning come to be an essential part of business life, we run the risk of being excluded from serious dialogue if we are unable to be part of the big data conversation.

The need for analytical skills

If we don’t have these skills in large measure already, can they be learned? They can, but not all analytical skills are equally important or equally easy to learn.

There are at least two things required in dealing with big data. The first is to find and query the data. The second is to interpret your findings. The first of these can certainly be learned, and by this I mean the technical skills of running a database query, searching across tables of data to find what you want, and running analysis using a tool such as Excel pivot tables to extract the information you need.

But while it is relatively easy to query and mine data (or at least to learn how to do it), the real trick is to interpret that information and put it to use. This is harder to learn.

The problem with interpreting data is that it requires a good feel for data and numbers as well as a sense of what they mean, usually based on a combination of mathematical knowledge and common sense. In the past, this is not an area we have shone in.

Interpreting data—rather than querying it—goes beyond simply thinking hard about things. It’s important to know what you’ve got when you find it. In T&D, for example, we have to be careful to isolate the cause of any effect. If we carry out some sales training and then sales increase, does that mean the sales training was effective? Not necessarily. Sales could have been on an upward trend anyway, or some external event such as the launch of a new product may have boosted them. The training may have helped, but it may not have contributed all of the effect.

It’s also crucial to be familiar with distributions and averages. Suppose team A’s performance is on average less than team B’s. Does this mean all performers in A are worse than team B? It does not. There could be a single poor player on team A bringing down their average or a super performer on B bringing theirs up. In fact, it is quite possible that most of team A are better than team B, with just a few outliers shifting the averages.

We need to grow

Data is going to play an increasing part in our working lives. The evidence suggests that we do not yet have either enough specialists to really put it to work for T&D, or a broadly spread enough general knowledge that will enable the majority of practitioners to hold their own in any business conversation using data. If we are not to be sidelined, that has to change. The risk of big data is that rather than putting it to good use because we know what we are doing with it, we become the servants of those who do.

Note: This post is excerpted from Big Learning Data (ASTD Press, 2013). Thought leader Elliott Masie and Learning CONSORTIUM Members bring a powerful new book to the T&D profession. They provide a SWOT analysis of big data and implications for learning and development professionals.

• Find out where to start with big learning data.

• Think differently about the data you have.

• Understand the risks that come with big data.

• Solve problems using the new perspectives, thinking, and measurement support that big learning data can provide.