World War II was a battle between soldiers, pilots, engineers and scientists. It was a period of incredible technological advances and innovations – unfortunately, mostly directed towards increased efficiency in taking lives.
One group of engineers, however, were tasked with the job of making the British bombers safer for the crew engaging on dangerous attack runs over France and Germany. Their focus was on applying a limited amount of steel armour to provide the maximum protection for the airborne personnel.
It doesn’t take a scientist to know that the lighter the bombers were, the more bombs they could carry and the faster they could get out of conflict. So the engineers got to work, and began where we might too – by looking at the damage sustained by the most recent bombing runs.
When a bomber returned, they would record every bullet hole on the plane. They would then accumulate this data to determine the areas that had the most bullet damage across the squadron, and prioritise these areas for more armour.
So, if the bombers returned with heavily damaged wings and fuselage – this would be where the armour would be placed in future. Simple.
Unfortunately, this was also simply wrong.
And it took one mathematician to correct this. Abraham Wald, a Jew from Hungary, was given the task of reviewing the bomber protection. He spent time with the engineers – and quickly realised that they were getting highly accurate answers to the wrong assumptions.
The planes that made it back to base were the ones being studied. They were the ones that were already well protected – so the bullet holes they carried were showing the areas that didn’t need to be reinforced. Instead, the areas that were intact were the areas that needed to be protected. These were the places where a single bullet could bring the whole bomber down – meaning it would never return to base, and would never be studied.
Wald’s statistical analysis was much more in depth than this simple summary would suggest (a paper on his work is available here), yet it highlights three important lessons for businesses interested in innovation.
1. Data and measurement are critical in innovation.
Most businesses I meet with are stuck in the notion that innovation is a mysterious art, that can only be understood in retrospect. They cite the stories of Apple’s creative genius and the serendipitous innovations of Nike’s waffle-iron shoe – and then resign themselves that if innovation will happen, it will happen in a similarly mysterious way.
This is akin to looking only at the few trophy bombers that return home – and assuming that these heroic planes are the only ones we can learn from.
Instead, countless businesses around the world are measuring their innovation processes, culture, leadership and markets to better understand their innovation potential and to provide accurate benchmarks for their innovation efforts.
Although this may seem less impressive, it is far more effective – and allows for a wide range of verifiable insights and actions to improve innovation – based on the businesses that have gone before.
2. Accurate data measuring an irrelevant (or flawed) assumption is costly.
Unsurprisingly, most businesses measure of their innovation effort is a simple ROI, number of patents filed or a look at their R&D budget. Unfortunately, these measures do not tell the whole story of innovation, and can lead to costly mistakes.
One business I have worked with originally measured their innovation by their investment in innovation training for their employees. That was it.
They had measured effectively how much they were training their employees. They knew this data well – and cited this to show how innovative they were.
They had not measured the effectiveness of the training, the amount of new revenue generated in the past two years by produces worked on by these employees, the cost-value of blockers to their innovation, their average time to market, the effectiveness of their leadership of innovation, their strength in user-centred design – the list goes on. Their accurate data did not highlight the cost the organisation wasted investment in ineffective training, as well as the cost of missed innovation opportunities.
3. Accurate data can reveal the biggest threats.
The engineers in World War II incorrectly thought that the biggest threat to their planes was where the bullet holes were. Abraham Wald recognised it was where the bullet holes weren’t.
Most organisations provide data measuring the risk of innovating – the cost involved, the potential for failure, and a(usually grossly over-estimated) risk of the impact on their brand value.
What most organisations fail to measure is the risk of non-innovation. The cost of losing market-share through slow-movement, the risk of becoming an inflexible incumbent, the risk of missing out on a disruptive possibility, and the risk of greater innovation expense due to their delay.
What do your organisational measurements for innovation reveal about your business? Are you measuring the right assumptions?