From a constant stream of mobile device releases to the explosion of cloud applications for all areas of business, technology advancements are happening at such an exhausting pace today that it’s easy to get paralyzed by the possibilities. It’s intimidating to think about how any organization – much less one in a production environment dealing with capital budget constraints, global competition, rising maintenance costs, less access to skilled labor, information security and safety concerns – can plan quickly enough to leverage such changes for competitive advantage.
Before a production facility can even consider trying to justify emerging plant-floor technology, it’s generally necessary to maximize the life of its existing automation investments. Managing aging equipment is an often-overlooked component of this: Over 90 percent of process manufacturers acknowledged the use of automation beyond the manufacturer’s obsolescence date, according to a 2010 study from ARC Advisory Group.
In the same study ARC study, 58% of users acknowledge having no formal plan for managing the lifecycle of their equipment. Production facilities that continue to operate aging equipment without a plan for addressing and managing end-of-life technology face a variety of risks that threaten to drastically increase downtime and decrease profitability should legacy systems fail:
- Supply chain fragility. A number of control systems that were designed and launched in the 1970’s and 1980’s are still in use today. Technology shifts, diminishing demand, and economic impacts have made some vital components and subassemblies used in these systems hard to come by – some are in short supply and others are completely extinct.
- Support challenges. Difficulties locating legacy products can be further compounded by maintenance challenges – not only is maintenance expensive, but attrition among subject matter experts brings an added layer of complexity.
- Regulatory restrictions. Safety regulations and rules regarding hazardous substances (e.g. the Restriction of Hazardous Substances directive) have expanded significantly in recent years, leaving many older products at risk for compliance.
Mitigate or Eliminate?
Operating legacy equipment beyond its obsolescence date will always carry a degree of risk. But, by identifying and quantifying this risk, production facilities can determine whether to mitigate the risk until scheduling and/or capital funding becomes available or to eliminate risk through product migration.
There are a few critical questions to ask that will help determine the best path forward:
- How long do you intend to operate equipment?
- Is support readily available?
- Is capital funding available?
- Is there applicable new technology available?
- What are the migration implications related to spare parts, training, support, etc.?
Considerations for Mitigating Obsolescence Risk
In many cases, mitigating obsolescence risk is the best option given budget and resource constraints. Though specific plans for effectively mitigating obsolescence risk may vary from one organization to another, best practice plans contain four basic pillars:
- Conducting consistent preventive maintenance
According to ARC Advisory Group, there is currently more than $65 billion worth of legacy automation assets reaching the end of their useful life. And though some would like to think these systems can run and be serviced indefinitely, it is simply not possible.
After operating a PLC-5® for over 20 years, for example, maintenance might start to feel like the controller is bulletproof – that since it has made it so long without a problem, it will continue to do so. But, as it often does in life, age wins in the end and serviceability may be limited or even non-existent.
As equipment reaches the end of its useful life, age and wear take their toll. Failure rates drastically increase, as do maintenance costs (Fig. 1). Plants are perpetually one major part break or machine failure away from a shut down, and legacy equipment is more susceptible to these sorts of hiccups. As a result, preventive maintenance becomes absolutely essential.
To drive consistent preventive maintenance of discontinued products, maintenance engineers need to be constantly asking questions, such as:
- Are the filters replaced regularly?
- Is the current operating environment within OEM specifications?
- Are cooling fans operational and clear of obstruction?
- Does the equipment possess the latest firmware update?
- Is there an updated logbook documenting inspections of obsolete equipment?
- When is the last time grounding was checked?
After determining the answers, maintenance needs to act to remedy any problems and correct any irregularities. But this is often easier said than done. Most organizations realize the importance of preventive maintenance, but many simply don’t have the tools or the organizational bandwidth to do it. Finding a way to get it done – either by outsourcing or reprioritizing – is paramount to extending the useful life of equipment. Systems that aren’t properly maintained are much more likely to fall victim to extended downtime and lengthy shutdowns. Developing an enhanced preventive maintenance program is vital to maximizing the life of legacy automation.
- Training to support legacy equipment
Preventive maintenance activities will only be productive if performed by personnel with the know-how to handle the machines they are maintaining. It’s not uncommon for a production facility to be running equipment that’s more than 20 years old, and most of the people that designed and installed it have moved on from the department or organization. Oftentimes, these experts have been replaced by younger engineers who simply can’t be expected to hold the same level of knowledge on legacy equipment.
Developing a training program to address these gaps is critical, but 58 percent of companies have faced difficulties in training young engineers and technicians to operate and maintain older control systems, according to a survey conducted by the ARC Advisory Group. Effectively training staff to maintain legacy products can be even more challenging because it requires a great breadth of knowledge. Employees need to know how to maintain all legacy products – this includes installation, configuration, programming, maintenance, diagnosis, troubleshooting and repair.
An undertrained workforce can have significant business impact on performance and metrics like downtime, on-time deliveries, vendor support expenses, overtime, scrap rate, programming costs and maintenance. In fact, the United States Department of Labor estimates that raising the educational level of employees by one year results in eight to 13 percent higher labor productivity. If the in-house staff doesn’t have the time or expertise required to teach new talent, it’s valuable to invest in an outside resource that does.
- Planning for spare replacement and legacy repair support
Just as important as having the right people is having the right parts – an ineffective spares management process can extend downtime by 15 to 35 percent and legacy products can make this statistic even worse.
Maintenance, Repair & Operations (MRO) departments are constantly seeking opportunities to cut carrying costs by reducing inventory. To do so, they examine the turnover rate of each part. During this process, internal financial entities often pressure the department to get rid of parts that aren’t turning over quickly. While this approach makes intuitive sense, it is the opposite of what should be done, especially with legacy automation equipment. As availability of these legacy parts begin to dry up on suppliers’ shelves, firms that rely on them need to carry extra inventory to counter potential shortages.
When they don’t, the result is often less-than-ideal – some production managers even resort to buying unverified spare parts on eBay. Having a machine’s health, and ultimately the company’s profitability, be entirely dependent upon someone’s last purchase on eBay is simply not a sound business model.
A spare part-replacement strategy should involve more than just stockpiling parts, it involves a process of:
- Calculating the optimal amount of spare parts;
- Determining the condition and supply of the spares;
- Identifying spares of legacy equipment ;
- Identifying a trustworthy and timely supplier of legacy spares.
Because of the complexity that can be involved in effectively managing spares for legacy equipment, companies sometimes choose to leverage a third-party to manage the process. Parts management programs can help reduce inventory and carrying costs, and can help provide more immediate spare parts availability. This was the case for the City of College Station when it decided to leverage a vendor-managed parts management agreement after a recent process control system upgrade. Since a vendor owns and manages the spare parts inventory, the City is able to budget a monthly fixed costs. Previously, the City’s control system stocked spare parts inventory was about $250,000, but it was able to reduce it to less than $20,000 while also significantly lowering the less square-footage required for onsite inventory storage. A resource who understands the intricacies of the process can help ensure that the right parts are in the right place, at the right time.
- Managing Obsolescence Status
Further complicating the risk mitigation process is the fact that it’s in a constant state of flux.
Within a year or two of conducting a comprehensive audit of the installed base, production facilities often start to discover products that were once active have become discontinued. If risk isn’t being continually assessed, it’s much more likely status changes and threats associated with them will be missed.
To prevent this, companies need to establish a process for monitoring lifecycle stages of equipment. This should include developing a database and assigning subject matter experts within the organization to collect and maintain all lifecycle information. Vendors also can help provide additional information around lifecycle statuses, parts and service availability, and migration recommendations that align with business goals.
While most facilities agree that better management of automation lifecycles is becoming critical, few programs are in existence and most are yet to get off the ground. One reason behind this is that most facilities don’t have the personnel for it. In most cases, there isn’t a role specifically responsible for creating, monitoring and updating a database, and because of this, these tasks are often overlooked.
Now is the time to develop a lifecycle strategy, before it’s too late. Just because the resources don’t exist to invest in personnel dedicated to managing this process, it doesn’t have to be neglected. Enlisting the help of third-party experts can double the benefits of risk mitigation – saving companies the time and money necessary to train their own personnel and preventing these resources from being squandered on unnecessary downtime.
Creating an Obsolescence Plan
The overarching goal of obsolescence planning is to quantify risk in order to determine whether it makes most business sense to mitigate through maintenance and support, or eliminate it by migrating systems. A phased approach can help ensure risk is addressed effectively and efficiently:
- Identify. Define the goals and scope of the obsolescence plan, and develop a strategy for safely collecting information from all legacy equipment. Dedicating qualified resources to creating the plan generally nets the best results.
- Research. Collect product lifecycle status information by reviewing manufacturer websites and notifications, publications, and distributor and reseller information. Identify inventory gaps and orphans by comparing MRO spares versus the installed base and by comparing repair activity with replace activity.
- Prioritize. Aggregate data to highlight areas of greatest obsolescence concern. Using that information, design a plan to accept, mitigate and eliminate obsolescence risk. Next, develop a process for gathering ongoing lifecycle status changes for all installed products.
ARC Strategy Report, “Electrical Power Systems Lifecycle Management Strategies,” April 2011.