AverNotes: Issue 3 – Are You Over Testing?

A production test that doesn’t raise your confidence in the manufactured good or its manufacturing process is a waste of time and money. Despite that, few people rigorously analyze their production tests to eliminate wasteful tests.

I have found that most people are reticent to make any changes to a test procedure once it is in use. The biggest reason for this seems to be fear. People fear that any change can have unintended consequences. They’ll even justify their excessive testing as being some sort of added safety margin. However, this justification is rarely backed up by a true analysis.

The problem of over-testing is further exacerbated with contract manufacturing. The factory may have little understanding of what they are actually building and the tests they are making. In other words, no one has the knowledge or confidence to recommend changes.

Analyzing your production tests

Here are some important questions one needs to ask when designing a production test:

  1. What is likely to go wrong in production?
    Chances are you are looking for the use of the wrong part, a part misalignment, a failed part, a bad solder joint, an unseated connector or the wrong firmware installed.
  2. What happens to the device’s performance when the problem is present?
    Some problems make themselves readily apparent. Others don’t. The answers to this question help you define what production tests are needed. Simulation can be a big help here.
  3. Does the magnitude of a failure need to be measured?
    The magnitude of a problem might be handy to know if you are evaluating the data for trends. If you don’t need to do this, a simpler test may be appropriate.
  4. Does the problem source need to be isolated on the production line?
    It is easier to detect a problem than to isolate its source. Isolating the problem might be better left for the rework bench.
  5. What savings can be gained through parallel testing?
    Simultaneously testing multiple parts of a product can produce huge time savings. Sometimes, such testing requires a bed-of-nails test fixture.
  6. What is best tested before final assembly?
    Testing before final assembly may afford access points that are not available in finished good form. When such access speeds testing, it should be considered.

Simulation of production problems

Circuit simulators such as the many SPICE derivatives are a great way to simulate production problems.  Monte Carlo analysis lets one consider the impact of standard part variations on the performance distribution. This is helpful for setting the limits or tolerances being tested to. One can also simulate part substitutions such as if parts reels are loaded incorrectly in the pick-and-place machine. You can also test what happens if you get solder bridges in critical areas or have open circuits due to tomb-stoned parts. Such exercises can indicate unnecessary tests.

Example: Potentiometer Testing

Any production test that requires a person to physically set a control is expensive to conduct in production. Such is the case with potentiometers where an operator will typically turn a knob to three positions; fully counter-clock wise, fully clock-wise and the center position.

SPICE simulations of two different types of equalizers circuits yielded some interesting findings. Equalizers will not exhibit many problems when their potentiometers are set to the center position (no boost or cut). Many of the capacitors and resistors could be wrong or even missing and you can still get unity gain through the stage. Testing at full boost or cut will confirm the capacitors and resistors have been installed correctly. However, there is often nothing gained by checking both extremes. And, if the equalizer has a bypass switch, you can probably eliminate testing in the center position too. The net result is reducing the operator involvement to test one part by two thirds.


Be smart about your production test regimen. Simply implementing an engineering test could be very costly. Consider every test and test point. Does each test point confirm that a different part has been correctly used? If not, consider eliminating that test.

About "Jonathan"

Jonathan is an electrical engineer, former VP of the Audio Engineering Society and sits on several audio standards working groups within the Consumer Technology Association. He has worked in the test and measurement industry for over 25 years and has expertise in professional audio, acoustics and RF technology.