Many engineers spend substantial portions of their time in testing. They test materials, processes, parts, and products. They test to evaluate concepts or physical principles, to solve problems, and to improve systems. The amount of resources spent in testing and the importance of this activity for the success of the design effort means that there is great value to any technique that can help engineers become more effective and efficient in these activites. The above figure summarizes an experiment by Swedish engineers that illustrates what may be possible using modern approaches to testing. This experiment showed engineers for SKF, the Swedish ball bearing manufacturer, how to quadruple the life of a ball bearing. An important aspect of this experiment is that the solution might never have been tried by engineers using the traditional "one factor at a time" approach to testing that many engineers have been taught.

To understand this experiment, it helps to know that a ball bearing consists of four basic elements: outer ring, inner ring, balls, and a cage that keeps the balls a constant distance apart. The SKF design team decided to test simultaneously three different modifications to a current design. They tested a modified heat treatment for the inner ring in comparison to the standard -- the top vs. the bottom of the cube. Second, they tested a modified cage design in comparison to the standard -- the back vs. the front faces of the cube. Third, they tested a modification of the outer ring osculation in comparison to the standard -- the right vs. the left hand sides of the cube. (In other contexts, "osculation" means "kissing". Here, it describes whether the ball "kisses" the outer ring in one point or two.) The design team made four inner rings with the modified heat treatment and four with the standard, four modified cages and four standard, and four modified outer rings and four standard. They then assembled them in the eight combinations indicated by the eight points of the cube. The eight resulting ball bearings were then run to failure under several times the maximum rated load. Six of the eight ball bearings failed in the first 26 hours of the test. In less than a week, they had all failed.

The results were dramatic: The combination of modified inner and outer rings quadrupled the lifetime of the ball bearing. The lifetimes in the upper right edge of the cube -- 85 and 128 -- are roughly four times the lifetimes of the other six ball bearings tested. Moreover, the cage design seemed to have little impact on product life. This allowed SKF to save money by using the cheaper cage.

It is important to note that this solution might never have been found without a modern experimental strategy such as the one in the above figure. Many engineers are trained to test "one factor at a time," holding all others constant. An engineer following that advice might never have tested the modified inner ring with the modified outer ring, and therefore might have stopped without finding the dramatic solution presented in the figure. This situation, where the whole is different from the sum of the parts, is called "interaction," and is relatively common in both physical and social processes. The solution to many engineering problems requires getting several things right simultaneously, and the "one factor at a time" approach often becomes an obstacle to solution.

Fortunately, some fairly simple experimental strategies can dramatically increase the chances of solving a problem in a reasonable period of time. They also often increase the quality of the answer obtained. In many cases they require relatively little time and other resources. The modern field of experimental design provides tools that help experimenters (a) explore a wide range of alternatives with minimal effort and (b) organize the results in simple ways that are easy to understand and use. For industrial situations where tests are often relatively inexpensive and quick to perform, it is best to think of a sequence of quick, simple experiments. Later experiments build on the knowledge gained earlier. A sequential approach like this fits well with the way that engineering and scientific knowledge is actually developed. When testing takes longer or is more expensive, good planning using appropriate experimental design techniques can increase the chances that the experiment will resolve the issue it was designed to answer. If reasonable physical theory is available for the phenomena under study, this can be refined with appropriately designed experiments. In many cases unnecessary assumptions are made that only complicate the engineering analysis. Meanwhile, other critical assumptions are overlooked. These issues can be explored with experimentation. In other cases it is easier and quicker to use graphics, such as the above figure, supplemented with simple algebra to summarize the experimental results and decide what to do next. Mathematical summaries of results, whether with simple algebra or higher math, provide substantial advantages for building the proprietary technology base of an organization and improving the quality of future engineering efforts with similar problems.

Designed experiments have a fairly long history. They have helped people solve important problems in agriculture since the 1920s, for example, They have helped in the production of beer (Guiness stout) since the early 1900s, in textiles since at least the 1930s, and in the chemical industry since at least the 1940s, to name only a few. More recently, they helped the US semiconductor industry reverse a slide towards extinction. In 1982, US companies had 54% of the world semiconductor market; Japanese companies had 27%. By 1986, Japanese were in the lead and the US was forecasted to have only 17% of world markets in the year 2000. There was great concern about this in the US. In 1987, the Semiconductor Industry Association founded SEMATECH to sponsor and conduct research aimed at assuring leadership in semiconductor manufacturing technology for the US (www.sematech.org). Shortly thereafter, the ominous trend was reversed. By 1993, the US was back in the lead. The President of SEMATECH proclaimed that statistical methods, including experimental design, had played a major role in this reversal and were "a competitive necessity" [William J. Spencer and Paul A. Tobias, "Statistics in the Semiconductor Industry: A Competitive Necessity", American Statistician, v. 49(2), pp. 245-249]. It is only a slight exaggeration to say that in 1980, almost none of the semiconductor manufacturers were using designed experiments to understand and improve yields. By 1990, virtually all used designed experiments. The companies that were too slow to adopt experimental design techniques had a slower rate of improvement in yield and went out of business. The semiconductor industry may be somewhat unique in its rate of change and therefore its need for a high rate of learning. However, in almost any field, experimental design techniques can help increase the rate of acquisition of new, useful knowledge. If none of your competitors are making serious use of these techniques, then you will not be at a competitive disadvantage by not using them either. On the other hand, anything you do to increase the effectiveness of your engineering team can increase your success in the marketplace.

To introduce experimental design techniques to an organization that currently makes little or no use of them, we recommend experimental design training combined with projects on problems identified as important by a diagonal slice of the organization, using the Quick Start, Quick Results Total Quality model.

For a quick introduction to designed experiments, ask for our free "Do-It-Yourself Experimentation Education Kit." This booklet outlines two experiments with paper helicopters that you can cut out, fold up, and use to understand one common, powerful experimental approach. This is suitable for kids from age 10 to 100. Call (408)294-5779 fax: (408)294-2343, or e-mail: sgraves@prodsyse.com for your free copy.

For more information on Design of Experiments, check out the Center for Quality and Productivity Improvement at the University of Wisconsin-Madison.

Other Productive Systems Engineering programs: Product Development, Reliability Experimentation, Tolerancing, Quick Start Total Quality (Total Quality Control, TQC / Total Quality Management, TQM), Statistical Process Control (SPC); Fiber Optic Communication/Transmission Systems (FOCS or FOTS); Control and Monitoring (RMM); Electronics (Theory & Application); Production Line Assembly for Technicians (Assembly).