Motorola sees significant efficiency gains by implementing Instrumental on its entire line of mobile phones.

Motorola Mobility has been on the front line of innovation in the new product development process for decades. Designing and manufacturing multiple new phones every year on short timelines has necessitated a new round of innovation. With a focus on accelerating and reducing unnecessary inefficiencies in their development process, Motorola has partnered closely with Instrumental to become more competitive while continuing to deliver high quality products to their customers.

Programs that used Instrumental during development ramped faster than products that did not use it.

Lyon Wang
Director of Engineering and NPI, Motorola Mobility

The Problem

Motorola was an early pioneer in mobile – starting first with radios and the first generation of cellular phones – and has continued to be a leader in quality for their ambitious line of smartphones and tablets. The New Product Introduction (NPI) process developed at Motorola has been replicated and imitated at consumer electronics organizations around the world, and remains core to Motorola’s ability to continually bring great products to market at a rapid pace.

One of the core missions of the development process is to find small but critical issues as fast as possible. For the last two decades, the industry standard for new issue discovery has been to send the design engineering team to the factory during development builds. For hours each day, engineers patrol the hundred meter assembly line trying to be in the right place at the right time to catch new issues. Once an issue is discovered (and a typical consumer electronic product may have tens or hundreds), a serious amount of effort is required to assess and address it. That effort involves late night phone conferences, requests for information via email, trips to the factory to put in days on the ground, extensive experiments, and, in the most extreme cases, daily executive meetings to provide updates and visibility.

One of the core missions of the development process is to find small but critical issues as fast as possible.

Sometimes the diagnosis is elusive. In those cases, companies like Motorola and other leaders in quality resort to building tens or hundreds of additional experimental units for the sole purpose of trying to hone in on a root cause. This methodology is considered a best practice in the industry, but it’s incredibly time consuming and extremely inefficient – taking weeks and costing hundreds of thousands of dollars with no guarantee of resolving the issue.

As a leader in the consumer electronics space who set the foundation for today’s best practices in NPI, Motorola believed Instrumental technology was an opportunity to eliminate the cost and inefficiency of finding and resolving issues in development, while delivering an even better end product to their customers. With Instrumental, they sought to discover issues faster, strengthen quality control on the line, and streamline their issue response to deliver new products on demanding schedule timelines.

The Solution

The first Motorola program that used Instrumental was a new mobile phone with an aggressively short schedule. The Motorola engineering team and Instrumental worked together to identify a handful of states of assembly that would show all of the key components of the product as it was built up. Less than three weeks later, Instrumental team members deployed the array of Instrumental inspection stations – which contained cameras, tunable lighting, and customized fixtures – onto Motorola’s development line in China. The setup, calibration, and training for all stations took only two days.

On the first day of the DVT build, the Instrumental stations collected images of each unit running down the line. These images covered both sides of the PCB assembly, the bare enclosure, two stages of building into the enclosure, and many images of the exterior of the finished product. As soon as these images were collected, they were uploaded to Instrumental’s database, and were immediately available in the Instrumental web application to Motorola engineers around the world. This complete data record is a key differentiator between Instrumental and traditional industrial vision systems, where the applications must be incredibly specific and the data remains trapped on the local machine in the factory, unavailable to the team.

As soon as the first 30 units from the build were available in the application, Motorola engineers could use Instrumental’s machine-learning algorithms to quickly and efficiently find new defects that they weren’t previously aware of. Instead of standing on the assembly line for hours, moving amongst a hundred assembly steps trying to be in the right place at the right time to identify something new, engineers could do a search of all of the units in seconds. Once a defect was found, with a few clicks, an engineer could setup an enduring Monitor, or test, that would

check every subsequent unit for the same failure mode and automatically sort the failures into neat collections where defect rates and trends are calculated in real-time.

The machine-learning methods that drive Instrumental technology also enable each Monitor to learn the difference between a typical and an anomalous unit. This makes it possible (and incredibly easy) to set up tests that can find unforeseen defects automatically – something traditional industrial vision systems cannot do.

Over the ensuing year as new development programs adopted Instrumental, Motorola engineers partnered closely with Instrumental to beta-test several new powerful features. At the end of the first program, Motorola was the first Instrumental customer to use Intercept – a version of Monitor that provides real-time pass or fail judgements directly on the line. Intercept uses edge-computing to enable judgements to be produced in seconds. Making any kind of test, Instrumental or otherwise, go live on the line is a nerve-wracking proposition: what if the test has a lot of false failures? Pre-validation of new tests is super easy with Instrumental. By leveraging the complete data record of every unit built, it’s possible to validate a test against historical data. This lets an engineer not only understand the false failure rate, but the even more elusive false pass rate. Over the next twelve months, the Motorola team evaluated several improved versions of Intercept, as the computation cycle time was reduced from seven seconds in the beta to less than two seconds today.

Instrumental enables you to work smarter. Instead of doing disassembly, you can spend your time solving problems.

Wayne Morrison
Principal Staff Mechanical Engineer, Motorola Mobility

The Motorola team was also a beta-user of Springboard – a program risk assessment dashboard that highlights the top issues and trends from the program, where the raw data is just a click away. These charts are automatically generated based on real-time data, so the latest status is always available. Instrumental charts and images quickly became a mainstay in internal Motorola presentations about program and build status, as well as an easy way to communicate about the issues the team was most focused on.

Motorola now regularly uses an array of Instrumental stations on most of their mobile phone programs to monitor production at key stages of assembly. Leveraging Instrumental’s detection algorithms and the easy-to-use database of all images collected, Motorola can more quickly identify when and where issues arise. They can continually iterate on their process so that known and unknown manufacturing issues are resolved efficiently, and they can monitor those resolutions remotely with access to real-time trend data. With Instrumental’s web application, Motorola engineers are able to easily track failure rate trends and evaluate program-level risks – something that wasn’t possible before. With unprecedented visibility and traceability on their development lines, Motorola engineers spend less time figuring out what to fix, freeing them up to focus on implementing solutions that will delight their customers. As Wayne Morrison, Principal Staff Mechanical Engineer, put it, “Instrumental enables you to work smarter. Instead of doing disassembly, you can spend your time solving problems.”

The Results

Instrumental has inspected every development and pre-production unit on seven Motorola mobile phone products, with more planned. Instrumental’s machine-learning algorithms regularly identify dozens of unique and new issues on each program, and when combined with known issues the Motorola team is tracking, results in tens of live Intercept tests. Since these algorithms are so easy to use and to validate, Motorola usually has its first Intercept test running on a new program within a week of deployment.

Faster iteration cycles across the supply chain enabled Motorola and their broader team of vendor-partners to improve product and process maturity much faster than was possible before – resulting in fewer unplanned experiments, fewer trips to the factory, and, ultimately, significant cost savings.

Key Use Case:

Identifying Unanticipated Defects in PCBs Accelerates Development

The heart of any electronic device is its PCB. During PCB assembly, it’s been the industry standard for decades to use Automatic Optical Inspection (AOI) systems, which compare an image of a circuit board to the digital CAD file to make sure that each part is present. AOI has limitations: it compares a board with the CAD file for part placement. It does not find new defects or damage and cannot analyze PCBs that have undergone additional assembly steps, something that is incredibly common in modern miniaturized devices. These limitations are why PCB inspection is a core use case for Instrumental, and why Motorola has prioritized this inspection in every Instrumental deployment to date. This investment has paid off in every program, and twice has led to significant savings.

The first time, Instrumental algorithms detected dark blotches on empty areas of a subset of the PCBs in an early build. When engineers took a look at what had been found, they immediately diagnosed that the blotches aligned with buried vias that made up the board circuitry itself, and were concerned that the impacted boards would not meet their stringent reliability standards.

Investigating further, they discovered the boards were thicker than specified – which would have wreaked havoc on their critical tolerance stackups and potentially would have been very difficult to track down. These PCBs had gone through AOI and had passed – even though the blotches were clearly visible. Using Instrumental data, it was easy to see that the defective PCBs were all from one vendor, enabling the Motorola team to work with their supplier to correct the issue quickly.

The second time, Motorola engineers used Instrumental images to demonstrate an unacceptable level of variation they were seeing in a component from one of their vendors. One of Motorola’s engineers directed Instrumental algorithms to look at the areas in question and to compile a variety of images showing the different issues. The effect of sharing the report was immediate: Instrumental images enabled both teams to align and to agree on which issues to focus on first. The result was much faster and closer collaboration between Motorola and its vendor and a better overall product for Motorola’s customers. Motorola added an Instrumental Station to their line in order to give their vendor direct access to Instrumental data and algorithms, so that they too could accelerate their iteration cycles.

In each of these cases, Motorola engineers were able to spend less time trying to find defects or to decode the clues of a failure analysis process, and get right down to the work of implementing solutions. Instrumental’s data facilitated better communication across Motorola’s supply chain – amplifying the effect beyond the initial deployment at the assembly line. Faster iteration cycles across the supply chain enabled Motorola and their broader team of vendor-partners to improve product and process maturity much faster than was possible before – resulting in fewer unplanned experiments, fewer trips to the factory, and, ultimately, significant cost savings.

Key Use Case:

Finding Design Issues Early Yields Massive Dividends

Design issues are the most expensive kind of issue to have in a new product. The first reason for this is that sometimes design issues masquerade as part quality, process, or workmanship issues first – so a lot of extra failure analysis work is needed to track down the true root cause. The second reason is that the solution to a design issue is often to modify one or more tools – a costly and time-consuming endeavor that amplifies as the product gets closer to production (when tools have already been replicated). Design issues in first prototype builds are inevitable and common. Those in the industry know that finding and fixing these issues as early as possible is one of the top priorities of any development process. Motorola found that Instrumental can help to identify these design issues earlier in development, when they are cheaper to fix, because Instrumental’s record of every unit made can help to quickly eliminate red herring root causes.

Last year, David Platner, a Motorola mechanical engineer, was trying to puzzle out why some camera modules were failing cosmetic standards. He logged into Instrumental’s web application to look at the images of the defective units compared to units with no defects, and discovered that the lenses on the defective cameras were mis-aligned.

Defect found, case closed, right? David shared his findings via email with his factory team, but he wasn’t able to get their attention on the issue. He suspected a language barrier was getting in the way, so he made a .gif of multiple Instrumental images highlighting the defect mode, and shared it with the team. He got an immediate response – the evidence enabled them to prioritize fixing the design issue so it could be resolved quickly.

This would not have been possible without Instrumental’s visual data – we saved many man hours and time in China.

David Platner
Mechanical Engineer,
Motorola Mobility

By finding this defect early in development, the necessary design change could be rolled into other tooling updates that were already planned and before tools were replicated. This significantly reduced the cost of the actual tool changes and enabled time during development to validate that the changes fixed the issue and didn’t cause new ones. Kevin Zurawski, Senior Manufacturing Engineer explained, “David used Instrumental to find this issue right at the beginning of the build – and within days the supplier had a solution.” That kind of rapid discovery and response cycle was not possible before Instrumental. In David’s words, “This would not have been possible without Instrumental’s visual data – we saved many man hours and time in China.”

Conclusion

MOTOROLA HIGHLIGHTS THE FOLLOWING AS THE TOP BENEFITS THEY RECEIVE FROM INSTRUMENTAL:

Accelerated Failure Analysis

Motorola’s engineering team has praised Instrumental for making the failure analysis process faster and more efficient. Before Instrumental, failure analysis involved flying engineers or even entire teams to the factory or shipping units from the factory back to the engineers, both of which are slow and costly. Instead of tearing down units, which can sometimes obscure or damage the very evidence that engineers are looking for, engineers can use Instrumental to see the images taken of those units as they were built. Motorola shared with Instrumental that occasionally there will be an important issue that is elusive – requiring a special experiment of 100 or even 500 units might be built – which consumes a lot of precious development time as well as expense. The camera lens design issue described in the previous section is an example of a type of issue that may have necessitated a costly experiment process. Instead, the optical engineer was able to quickly discover the root cause of the issue on his own. Dave Konczal, Director of NPI at Motorola said, “Instrumental reduces the number of experimental builds needed to validate our products during development – leading to a more streamlined development process.”

Instrumental reduces the number of experimental builds needed to validate our products during development.

Dave Konczal,
Director of NPI,
Motorola Mobility

Faster Time to Stability in Ramp

While the US-based engineering team is responsible for developing the product, the China-based manufacturing team is responsible for taking the design and producing high quality products for customers. The transition from development to production is a stress test of all of the work the development team has done in the previous months. According to Lyon Wang, Director of Engineering and New Product Introductions at Motorola, “Programs that used Instrumental during development ramped faster than products that did not use it.” By finding issues early in development and relentlessly tracking them from build to build, the products have matured faster, resulting in significant cost savings in the early days of production on each program. He also mentioned that immediate feedback helps operators get up to speed faster because they can learn from their mistakes.