What is a composite metric?
One of the key features of UDOT's new performance management system is the use of composite metrics to measure the organizations progress towards achieving each of its three strategic objectives. Below, we describe what a composite metric is, how they are generated and how to interpret them in a meaningful way
How do composite metrics work?
A composite metric is a metric whose value is defined by a mathematical formula involving other previously defined metrics. For example, if we gather the speed of every car heading down the highway and record it in a spreadsheet - an interesting composite metric would be the 'average speed' of all cars. In this scenario the 'mathematical formula' is simply to take an average!
Weighting the factors
When creating composite metrics it is fairly typical for analysts to weight the component metrics that are combined to create the composite metric. Using the 'safety rating' from above - if we believe that the condition of the car was twice as important than the speed at which it is traveling, we may want to weight that measurement more heavily. In practice, this simply means multiplying the condition rating by a factor of two and taking the average.
Normalizing the Factors
An important concept to understand when creating composite metrics is normalization - the process of multiplying a series, function or item of data by a factor that makes the norm equal to a desired value. For example, in the scenario above imagine if we not only recorded the speed of the car traveling down the highway but also rated its condition on a scale from 1 to 10. Speed data would typically range from 0 to 100 mph - but the conditions would only range from 1 to 10. If we wanted to create a composite metric called 'safety rating' by combing these two elements, simply taking an average of two measures, the speed data would be over-represented and the condition data would be under-represented. To address this issue, we normalize the data so that all scores are scaled to a norm. In this example, if we wanted our 'safety rating' scores to range from 0-100, we would multiply the condition rating by a factor of ten.
What does this mean for me?
UDOT has taken several performance measures related to each of its three strategic direction objectives and combined them using to create a composite overall performance metric for each that is scaled from 0-100. While an overall score for the 'safety composite' of 40 doesn't have a lot of meaning when considered in a vacuum, by tracking this metric over time and looking at the trend we can see how well the department is performing and determine whether or not the investments made to improve safety have been effective.