It is well known that the utility of training as a business investment is demonstrated through its impact on performance, and that measurement of this performance is critical to key decision-making. Although some key metrics used to gauge performance are more easily quantifiable (for example, production and sales figures) it is the human capital variables, such as level of skill, motivation, knowledge and satisfaction, that provide a greater measurement challenge to most companies. Training is a key venue for maximizing this human capital, and companies could gain competitive advantage if they could measure human capital as scientifically as they do other performance metrics – in clearly defined, uniformly understood, standardized units.
By now we have all become aware of Steve Jobs’ views on business and innovation. Integral to his approach to innovation was to strike a balance between humanities and science. Measurement is no different in that the same balance is critical to meaningful measurement. Thus, not only do we need to measure training outcomes and the impact of training on human capital, but we also need to capture those using a strictly defined scientific system to produce standardized and replicable human capital metrics.
Take, for example, measures we use on a daily basis, such as time, temperature, and money. The measurement systems of these have been deliberately (and painstakingly) constructed by scientists. The impact of these measurement systems on us as humans is huge. We take for granted how these metrics answer questions such as Am I on time for the meeting? Do I need to bring an umbrella? How is the stock market performing? In fact all measures are a result of deliberate efforts to develop a system of standardized and meaningful units for the purpose of equity, communication, and fair exchange, so that a given number has a common meaning across different groups, settings, and times.
Measures are clearly quantitative in that they are expressed in standardized equal-interval units, yet each measurement unit also possesses a rich qualitative meaning. We all understand what it means to have $100, weigh 125 pounds, or arrive 10 minutes late to a meeting. Each of these measures represents an abstract idea, but these abstractions are replicable and useful! Wouldn’t it be ideal to measure other abstractions such as “performance” or “satisfaction”, using the same principles of measurement, to capture these key human capital variables that result from training? The answer is – we have the tools to do this and it is already routinely done in many high-stakes decision-making. Take, for example, testing companies that develop licensure exams for physicians. These companies routinely convert test responses (including essay responses) into scientifically defensible (and hence legally defensible) human performance measures. They literally cannot afford to license unprepared physicians or to be sued for failing a competent candidate. Can other businesses afford not to do the same?
Measures are developed from a strict scientific system, yet the derived units must capture the human impact of training on individuals. As we know, most business practices attempt to measure human capital with survey data. But no matter how well developed a survey may be, responses are either simply descriptive numbers (“20% agree that…”) or are correlated with one another to examine response patterns. These descriptions and correlations are not measures, and hence need to be converted into measures much the same way as, for example, the statement, “It is unusually warm in this room,” needs to be assigned a temperature if we are to express our discomfort in commonly understood units. In other words, survey responses must be analyzed with a scientific measurement model 1to construct meaningful and replicable numbers. In either their original or correlated form, survey responses do not possess the additive properties required for measurement, but they do contain all the necessary information for constructing a measure, if one additional, yet critical analytic step is taken.
This critical and often-missed step in data analysis is absolutely essential to meet the scientific requirements of measurement and to better understand and replicate the human aspect (the psychological and qualitative meaning). Once human capital is measured scientifically, key training metrics are interpretable across a variety of contexts, ready for use in examining benchmarks, trends, and goals of a training program. With so much business capital invested in training, it is only by adequately measuring its impact on human capital that businesses can assess the value of that investment.
Inference LLC offers comprehensive design and analytical support for academic and corporate settings. Both partners hold a Ph.D. in research design, measurement and statistical analysis, are tenured professors, and have over 30 years of combined consulting experience. You can contact Inference at inferencellc.com.
1 The Rasch family of models is the only approach to date that can use survey data to construct standardized units with common meaning across multiple people, settings, and time.