Our goal in dealing with this new normal of exponential data growth is to manage the relevant data’s Three Vs -- Volume, Variety and Velocity -- to get to the fourth V, Value. The Four Vs allow for new ways of seeing the business.
Mike Newkirk, SAS
The Value of Using All Available Data
So, you can crunch billions of records in a fraction of the time it used to take at a minimal cost. But, if it still takes hours or days to select variables, test and validate models, select and validate a sample, or wrestle the data into a graph, chart or heat map that makes sense to business users, where’s the real value?
The analytical lifecycle has stages, and simply speeding up one part of it doesn’t necessarily mean a shortened cycle overall. Figure 4 illustrates the analytical lifecycle.
Visual analytics in-memory can make the data exploration phase work with all the data and all the variables. More importantly, it can empower business users to independently create reports and see data relationships without involving the IT department or data scientists.
Figure 4: The Analytical Lifecycle
Once the data exploration phase is complete, the next move is to involve data scientists to develop predictive models and deploy them against the data to bring additional value.
At this point, many organizations turn to sampling, or simply dump some of the data so the system doesn’t become bogged down. But sampling can miss nuance – sometimes in important ways. As discussed earlier, temperature sensor technology on factory robots is so advanced and cheap, it can sometimes report temperature readings every 2 milliseconds.
But suppose the data warehouse downloading that information can’t handle that volume, so a data analyst sets the data download to collect readings every five minutes instead. Unfortunately, the rest of the data is tossed. Why not use all the data and all the variables to detect patterns that could indicate impending equipment failure and costly downtime?
A more detailed level of data collection could better refine analytical models that would return a more precise estimate of equipment failure. And that means speeding up the lifecycle to deploy faster overall so the entire cycle timeframe is significantly shortened. Selecting accurate data samples, building models, testing models, deploying models all take time. Speeding up the phases of the lifecycle means a faster decision timeframe and a more competitive organization.
Keep in mind that visual analytics for the masses will not replace the need for advanced analytics in your organization. Advanced analytics are necessary to handle more complex analyses, especially those that involve prediction or optimization. But using visual analytics together with in-memory analytics changes how business users depend on analysts and data scientists for tasks they may be able to do on their own. And, that change can make the entire organization more agile, responsive and profitable.
Mike Newkirk is the Director of Manufacturing and Supply Chain Solutions at SAS.
