In the late 1980s, Mike Graen of Procter & Gamble Co. moved to Arkansas to work on the company's Walmart team. At that time, Walmart was P&G's fifth largest customer, and data sharing and collaboration between suppliers and retailers was in its infancy. Graen was charged with improving the economics between the two companies using information technology. According to Graen, "Within the first eight months, we made a $50 million swing in profitability [in terms of Walmart's profitability selling our product]."

Before this change, the only thing that P&G knew about its product demand was that another order for a truckload had arrived. Graen called this level of data sharing and collaboration a "whole new world," which he attributed to the ability to see -- for the first time -- inventory levels, store-level sales data, and everything from when P&G shipped an item to Walmart to when it was sold at the register.

The legendary story of how Walmart profited from data sharing and how it improved logistics through better forecasting and inventory management is well understood; however, it has not been replicated to the same level by any other retailer to date. This can be attributed to the genius of Sam Walton, a Big Data analytics pioneer. By taking it to the next level, the information sharing had a network effect, as Walmart expanded its data sharing from P&G to every supplier that wanted in. As Tom Muccio, P&G's president of Global Customer Teams, who began collaborating with Walmart in 1987, said, "We had the ability to invent the future."

An Innovation Magnet

This whole new world of data sharing and collaboration not only improved forecasting and marketing, but also created new points of competition between suppliers within Walmart's growing supply chain. In the early 1990s, Walmart formalized its Retail Link system, which provided sales data -- by item, store and day -- to all of its suppliers. This information translated to lower merchandising cost for Walmart, and also saved suppliers time and expense in planning their production and distribution. The surprising side benefit to Walmart and its customers was that each of its suppliers also competed with each other to make Walmart smarter, allowing Walmart to pass on the savings.

For example, Supplier X might argue that Walmart should dedicate more shelf space to its products because of its high sales volume and high profit margin. Supplier Y would then crunch the numbers and argue that reducing shelf space of its brand might not seem so negative on its face, but that shoppers of its product also often buy additional products that carry high margins for Walmart. While Supplier X and Supplier Y jockeyed back and forth, providing greater insight with each analysis, Walmart gained a wealth of understanding about what was going on in the business. Each brought an alternative approach to forecasting, estimated their own and cross-price elasticity and shelf out-of-stock rates, calculated store and DC fill rates, analyzed assortment decisions, estimated inventory investment and cost analyses, derived return on investment for the shelf space, and illuminated category trends and its drivers.

With so much competition for new and improved insights, new analyses were routinely conceived and birthed. With access to so much data at such a granular level came the need for careful data cleansing and also possible calculations that pushed analytical skills and creativity to their limits.

Walmart itself did not possess the resources to develop focused analyses on a given product because it carried hundreds of thousands of products. But its suppliers did because they had relatively small sets of products and significant vested interest in seeing those products' performances optimized. In addition, the suppliers were the experts on their categories and end consumers. Walmart was an expert on its stores and retail business.

These analyses became increasingly insightful as suppliers placed analytical and creative people on teams working with Walmart because the opportunities for improvements and the strategic value of the competition with other suppliers were greater there than at other retailers who did not provide the information and/or engage in collaboration. Thus, when Walmart began sharing its data, it did more than take the noise out of forecasting for suppliers;it became a magnet attracting innovators to a place where new ideas would be continuously developed and improved to everyone's benefit. (It is well known that, when it comes to demand, forecasting, order and shipment data have significantly more noise than point-of-sale data.)