By What Measure?

Dec. 17, 2007
Research and development efforts cost plenty. Gauging their effectiveness, though, remains a challenge.

What gets measured gets done. That idea, or a variation of it, frequently is cited in manufacturing by lean proponents and others engaged in continuous improvement, and it typically translates to such efforts as reducing manufacturing cycle times or lowering finished-goods inventory levels. Less clear is what it means when the subject of the scrutiny is a company's research and development efforts. Just how does one measure the effectiveness of R&D?

One thing to get straight is that spending more dollars doesn't guarantee an increase in the effectiveness of R&D efforts or with superior innovation, says Barry Jaruzelski, vice president at Booz Allen Hamilton. The consulting firm conducts an annual analysis of the world's largest 1,000 corporate R&D spenders.

Real innovation is inherently cross-functional, Jaruzelski states, and "is dependent on superlative efforts across marketing, sales, engineering" and other functions.

According to Booz Allen's most recent analysis of its "Global 1,000," the companies that get the greatest return from their R&D investment attribute much of their success to their focus on customer insights. Indeed, companies that emphasize direct customer engagement showed remarkable achievements compared with companies less focused on customer feedback -- three times higher operating income growth and 65% higher shareholder return, for example. Importantly, they also closely align their innovation model with their corporate strategy.

But what about metrics focused solely on research and development? Jaruzelski says some firms look at percentage of revenue from new products. He believes this measure doesn't help a company benchmark because of varying definitions of what constitutes a "new" product. Another measure Booz Allen has seen is return on innovation investment, which should reflect the entire portfolio for the life of the products as well as all investment, including capital expenditures. Few firms embrace that definition.

The Big Ten R&D Spenders (in 2006)
Company R&D spend
1. Toyota $7.7 billion
2. Pfizer $7.6 billion
3. Ford $7.2 billion
4. Johnson & Johnson $7.1 billion
5. DaimlerChrysler $6.7 billion
6. General Motors $6.6 billion
7. Microsoft $6.6 billion
8. GlaxoSmithKline $6.4 billion
9. Siemens $6.3 billion
10. IBM $6.1 billion
Source: Booz Allen Hamilton
Dennis Rheault, a partner with consultancy Bridge Strategy Group, points out a sure sign of trouble. "As soon as any project is labeled as strategic' for lack of profit potential, it should raise suspicion on the R&D effectiveness front," he says. "R&D effectiveness is mostly about aiming at the right targets and limiting the number of times efforts need to be reprioritized, redirected, or worst, cancelled."

Effective innovators are disciplined and regularly measure everything from the time and money spent in product development to the success of the product in the market. Booz Allen says this rigor, combined with strong portfolio management, allows companies to better understand how their innovation efforts contribute to long-term growth.

Jaruzelski says the "holy grail" within the innovation space is to introduce meaningful early warning metrics. For example, look at where you are three months into a two-year plan. If the plan calls for a three-person staff at three months and you have only two people, then one-third of the plan is not getting done and trouble potentially is brewing.

"The key thing around measurement is to think about hit rates and kill rates. If you have 100% hit rates of things being successful, then you're not taking enough risks," Jaruzelski says. "Your innovation process is a pipeline. You should have a wide aperture, but there should be at every step an ability to kill [projects]."

See Also

Popular Sponsored Recommendations

Global Supply Chain Readiness Report: The Pandemic and Beyond

Sept. 23, 2022
Jabil and IndustryWeek look into how manufacturers are responding to supply chain woes.

Empowering the Modern Workforce: The Power of Connected Worker Technologies

March 1, 2024
Explore real-world strategies to boost worker safety, collaboration, training, and productivity in manufacturing. Emphasizing Industry 4.0, we'll discuss digitalization and automation...

How Manufacturers Can Optimize Operations with Weather Intelligence

Nov. 2, 2023
The bad news? Severe weather has emerged as one of the biggest threats to continuity and safety in manufacturing. The good news? The intelligence solutions that build weather ...

How Organizations Connect and Engage with Frontline Workers

June 14, 2023
Nearly 80% of the 2.7 billion workers across manufacturing, construction, healthcare, transportation, agriculture, hospitality, and education are frontline. Learn best practices...

Voice your opinion!

To join the conversation, and become an exclusive member of IndustryWeek, create an account today!