Skip navigation
Data Lake or Data Swamp?

Data Lake or Data Swamp?

Every employee whose job relies on data must be coached in new ways for digital transformation to succeed.

Think back to your days as an industrial engineer. You’re on the floor, trying to gather the data you need to understand a line. You’ve got your stopwatch, you’ve got your clipboard and you’ve got 40 minutes to figure things out.

Your conclusion? Well, excluding the odd 22-second outliers you saw here and there, cycle time is 12 seconds.

And so, armed with your cycle time insight, you return to your desk to make some big decisions about how many people you need to hire or how many lines you need to spin up to meet increasing customer demand. You don’t think twice about excluding those outliers because data follows a standard (Gaussian) distribution. The bell curve. Right?

Right. Unless it follows a Weibull distribution. Or a Poisson distribution. Or any number of other distributions. And if it is a Weibull distribution, then those outliers you dismissed were actually key data points—and your big decisions will have big errors.

If you’d had more data, you’d likely have had a better chance of predicting the distribution. But with so many lines and so little time, you’re limited to your small dataset, and basically forced to assume the distribution is standard.

A history of episodic data collection in manufacturing

For any high-volume system, mathematics tells you that you need thousands of data points to understand the shape of your curve—that is, to understand which, if any, given data point is an outlier, or indicative of a pattern.

But even with all of the recent advances in automation, connected factories, robotics, etc., data gathering on the factory floor is still reliant on industrial engineers with stopwatches, clipboards and not enough hours in the day. Data-gathering techniques haven’t evolved since the time of Henry Ford. It’s the paradox of Industry 4.0: More data drives better decisions, but the one place where data would be the most valuable is the one place where it’s hardest to gather—on the floor, from your operators.

And make no mistake, operator performance is the low-hanging fruit. Drishti’s recent study with management consulting firm A.T. Kearney, the State of Human Factory Analytics, confirmed that 72% of tasks on the factory floor are still performed by humans. But because data on these activities is collected manually—with miniscule sample sizes and inherent observation bias—this means nearly three-fourths of what’s happening on the factory floor is virtually invisible to analytics.

For more than 100 years, manufacturers have been forced to make educated guesses about how to make line improvements, increase productivity, decrease cycle time, staff different shifts and more. Today, data is supposedly driving digital transformation in the factory – but digital transformation simply isn’t possible when stopwatches are the méthode du jour.

Lesson in data from the Golden State Warriors

Continuous measurement is key to true digital transformation. It’s only through constant data collection and analysis that manufacturers can be certain the reality on the floor is reflected in the data they use to make decisions.

To understand how important continuous measurement can be, consider a non-industry example from my own backyard, the rise of the Golden State Warriors’ era of domination (at least, that’s what I call it).

In 2010, the Warriors started putting data front-and-center in the team’s coaching strategy, using wearable technology during practices and monitoring heart data, mood, sleep, energy levels and more. That continuously collected data was used to develop high performance and visualization analytics that the Warriors coaching staff, led by head coach Steve Kerr and assistant general manager Kirk Lacob, used to make strategic game decisions, like who should start, plays to run and more. And I don’t have to tell you the arc of the team’s performance from 2010 onward (It was good. Really good).

If it works for the best NBA team in the game today, it can work in your factory, as well. Using data that is continuously produced from new sources – humans – can give you the same edge that, in 2018, propelled the Warriors past LeBron James’ Cavaliers. It requires changes in a number of areas, but in the end, it’s worth it.

The flip side: Now you’re drowning in data

I’ll put it plainly: Most manufacturers will soon be drowning in data.

The vast majority of manufacturers today aren’t set up for, and don’t know how to handle, the amount of data required to drive true digital transformation. As such, manufacturers run the risk of transforming a “data lake” (a massive amount of raw data) into a “data swamp” (a data lake that goes unused).

Fundamentally, most manufacturers aren’t equipped to process the level of data that is required to truly understand your data distribution and draw key insights. It’s as if you’ve been swimming in a single lane of a large pool, and now you’re dropped in the middle of Lake Michigan with no swim lanes or markers to guide you. How many of you would be able to find your way to shore in that scenario?

That’s why shifting from 10 to 10,000 daily data points with continuous measurement requires more than an adjustment in the tools you use to gather and analyze data; it requires a change in your cultural mindset.

As you acclimate to a data lake, expect resistance from multiple fronts:

  • From those who are suddenly being measured for the first time.
  • From those who are no longer gatekeepers of exclusive knowledge.
  • From those whose outdated measurement skills are suddenly obsolete.
  • From those whose good decisions now look bad.
  • From those whose jobs will change, pushing them out of their comfort zone.

The key to managing these challenges lies in complementing data creation with workflow adjustment. Every employee whose job relies on data must be coached in new ways to leverage this sudden influx. Data itself is only valuable if it’s used to drive change.

Data will change not just your decisions, but your standard operating procedures. You’ll hire differently. You’ll invest in various forms of personal technology. You’ll change the job descriptions of everyone in the plant.

Dr. Prasad Akella is the founder and CEO of Drishti, an AI and computer vision company that digitizes human actions on the factory floor.  He led the industry/university team that built the world’s first collaborative robots at GM.

Hide comments

Comments

  • Allowed HTML tags: <em> <strong> <blockquote> <br> <p>

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
Publish