Book: DMSMH Appendix II (1950)

Appendix II ¹

The Scientific Method

The Scientific Method is based solidly on definite rules, but is none the less, like the American Way of Life, something that must be lived to be fully understood. The United States has a Constitution, but the American Way of Life is far more than that; so the Scientific Method is, while based on certain readily cited rules, far more than those rules.

For one thing, the Scientific Method implies zestfully, gleefully attacking, with every available weapon of logic, every possible logical loophole in–your own structure of logic and theory. It requires that a man tear into his carefully built theory with the vim, vigor and spite of his worst enemy. It implies that a scientist’s best friend will review his work starting with the premise that it’s all wrong, and do his best to prove it’s wrong.

For the intellectual triumph, the warm glow of victory in science, comes not from producing a new theory–but from producing a new theory that stands up, and is useful, even when the most knowing make deliberate attempts to find a flaw.

The Scientific Method is behind the testing of Navy armor plate. The production of a perfect piece of 16-inch armor plate is routine and gives no special satisfaction. But the production of a slab of 16-inch armor plate with a 16-inch armor-piercing projectile with its nose buried in that armor, a plate bulged, distorted, but unpierced and unbroken–that is triumph and satisfaction. We don’t test the 16-inch plate ,with machine-gun fire, or with 6-inch projectiles. Test it with the heaviest, deadliest weapons you’ve got; then, and only then, do you have something to be proud of.

So with a theory.

There are rules for argument that lead to the building of a theory; they can be condensed to three key, critical points, the sense of which is clear. The problem in application is the subtlety with which violations of those rules can creep in. The critical rules are:

1. Argument by appeal to authority is of no value whatever.

2. The observation, not the observer’s report, are the important data.

3. No theory, however well-established or long-held, can stand in the face of one relevant, contradictory fact.

The first of those rules is the one that is most often violated, usually quite unintentionally and without realizing it. Everybody knows that appeal to authority is no sound way to argue a case, even if the authority happens to be right. Yet so subtle can appeal to authority be that it is exceedingly easy to miss noticing its insertion; the preceding sentence, for instance, deliberately exemplifies one type of very easily missed “appeal to authority,” actually the most common of all such appeals. “Everybody knows,” “of course,” “naturally” and similar phrases are the slipperiest customers in that respect. “Everybody knew” the world was flat for a long, long time, and “of course” the Sun went around the Earth, as any fool could plainly see. And common clay and the precious ruby have nothing in common–nothing, that is, except the same elements in somewhat different proportions.

But even the less subtle appeal-to-authority that is stamped with the Great Name is a source of immense amounts of trouble. It was not Aristotle’s fault that, for nearly a thousand years, science was stopped still by consistent appeal to Aristotle; he didn’t claim he knew all the answers–the scholastic arguers did. Even today, in an age which has some understanding of the scientific method, Great Name arguments show up–except, of course, that the Great Name himself has become a Great Name by most carefully refraining from using that method! The sentence, “Einstein says that nothing is faster than the speed of light; it is theoretically impossible,” contains two arguments by appeal to authority, and sounds so learnedly scientific that anyone might be taken in by it. Saying a thing is “theoretically impossible” is, actually, appeal to the authority of present theories. But a theory is not a fact–it’s an intelligent set of opinions, and no more, as any scientist realizes. So far as the Great Name argument goes, those are easy to spot, and their value comes into focus very quickly if you simply substitute the arbitrary name “Joe Doakes” for the Great Name. The corrected, scientific-method sentence above –so far as argumentative value goes–would read, “Joe Doakes says nothing is faster than the speed of light; in his informed opinion it appears impossible.”

Scientifically, there is no difference whatever between the two statements, so far as evidential value goes. The evidence-statement on the subject would read, “Einstein suggested, and physical experiment appears to prove, that nothing is faster than the speed of light; current physical theory, which seems to fit most of the observed data, indicates it is impossible.”

That is, admittedly, a much less solidly satisfying sort of statement. It sounds weak, uncertain of itself or any ¬thing else. And it is the sort of statement–the sort of thinking–that went from the first small scientific evidence of the atomic theory in 1800 to atomic fission in less than a century and a half. It is the scientist–who operates on the principle that he doesn’t already know all the answers–who is out looking for new and better answers. A man who thinks in terms of “This is the answer. I know this is true. That is impossible, because it disagrees with what I know,” does not have to do re ¬search. He already knows the answers. He is in no danger of making new and disturbing discoveries that might upset his certainty of mind. The scientist, on the other hand, operates with the certain knowledge that he is uncertain; he is never disappointed, for new data is constantly being found–he’s looking for it–that shows that he was, indeed, a bit mistaken.

The non-scientist, who likes to work with Truths and Certainties and think in Absolutes, the method of uncertainties and probabilities seems stifling, an impossible method of operation. It is so impossible that it produces, in a single century, electric light and power, radio, television, atomics, the entire science of organic chemistry ranging from dyes to synthetic drugs, automobiles, air ¬planes–practically an entirely new civilization.

By realizing that no theory is final, complete, or perfect, a new concept is admitted: a theory is good so long as it is useful. It is, naturally, a very pleasant thing if the theory also happens to be true, but that (shocking though the thought may be to the layman) is not at all necessary. The really important question is not, “Is it true?” but “Does it work?” If it works, we can use it and pretend it’s true; if it is true, that’s an added bonus.

This reasoning, which seems to some specious and downright dishonest, is the only method so far found that produces results. Look about you: every product that has been touched by machines in its production is a demonstration of the observed fact that, by provisionally assuming a theory is true, concrete, useful results can be obtained. And that by maintaining a willingness to discard or modify that theory at the first sign of failure, progress is made.

For if a theory is good only when it works, then the first time it fails to work–the first fact it encounters which does not fit–the theory must be discarded, and a new and better one found. Only someone who insists that a theory is Truth would hesitate to discard a theory that didn’t work. And a scientist never insists that a theory is Truth; only that it is useful.

When an apparent contradiction appears, however, the most careful checking must be instituted. First: check the interpretation of the theory. The basic concepts of the theory might be right, and the application of those concepts wrong. The reinterpretation of the theory may explain the new fact. Second, and actually simultaneously, remember that the observation, not the observer’s report, is the datum, and repeat the observations. The observer may have been wrong. Men can’t see beyond the violet or below the red; quinine makes a man’s ears ring, so he hears sounds that aren’t there, and no man can hear sounds above 20,000 cycles when they are there. Under ultraviolet light, the human eyeball glows slightly, so that one sees a mist of light that isn’t there, but since we can’t see ultraviolet light itself, an observer will not see the source of ultraviolet that is there. Always check the observations; the observer may be wrong. But actual observations, facts, are never wrong.

One source of a lot of misunderstanding is the difference between theoretical impossibility and factual impossibility. That is best illustrated, perhaps, by the old story of the man who telephoned his lawyer, explained a legal contretemps, and was told, “Don’t worry about it; they can’t put you in jail for that!” The client replied, “I’m calling from the jail.”

A slight change on that might demonstrate reverse aspect. Make the troubled caller a circus owner; this time we’ll say the lawyer replies, “That’s serious. I’m afraid they can put your elephant in jail for that.”

In each case, theory is in conflict with physical fact; in each case, as it invariably must by the very nature of things, theory, not fact, breaks down.

But all of this is, in essence, a discussion of the scientific method of argument, of thought. There is, at the root of it all, the scientific technique, the final test and proving ground of all scientific thinking. Ideally, the scientific method follows seven steps:

1. Make a series of careful Observations.

A. These observations must be repeated, and are acceptable as observations only if many people following the prescribed techniques can duplicate the results.

B. Variations of the prescribed techniques must be tried to eliminate the possibility that the ob ¬served results might be due to a factor other than that intended. As a gross example, sup ¬pose it is reported that a magnet will attract objects. Demonstrations show it does attract and lift iron balls; that is Step A above. Now variations of the experiment show that the magnet attracts iron but not copper, silver, etc. The observed effect–attraction–is real. Variation of the original experiment is needed to show the actual limits of the effect.

2. Combining all relevant data, from all relevant experiments, formulate a hypothesis.

A. The hypothesis must explain all observed data.

B. It must not demand as a consequence of its logical development, the existence of phenomena that do not, in fact, exist.

C. But it should indicate the existence of real, hitherto unobserved facts.

3. Using the hypothesis, predict new facts.

A. A logical structure broad enough to explain all observed, relevant phenomena will necessarily imply further phenomena that have not yet been observed. Use this mechanism to predict the existence of something which, under previous theories, would not exist.

4. Perform an experiment and make observations on these predictions.

5. As a result of the experiment, discard the hypothesis, or advance it now to the status of “Theory.”

6. Make further predictions, further experiments, and collect more observational evidence until a contradictory relevant fact is found.

7. Discard the old theory, take the new total of observational data, and form a new hypothesis.

8. See Step Three.

This process seems, at first glance, a completely circular, going-nowhere system. It isn’t; the 50-passenger airliner flying by just overhead testifies to that. Notice that each time round that cycle the new hypothesis shows how to get new data, new experimental evidence, new information. The process is not circular; it’s an expanding spiral, and each sweep around it covers a broader and broader field of understanding.

But the most important step of all–the one that took men longest to make once the idea of organized knowledge was started–is Step Seven. “Discard the old theory . . . and start all over again.” It’s hard for men–who are basically conventional, status-quo animals! … to give up the comfortable familiarity, the nice, easy routine, of that Old Time Theory, to embark on a completely new system that calls for a total revision of all their thoughts. It’s so easy and comfortable to believe that the old theory is Truth, and doesn’t and won’t ever need changing, even if it doesn’t work all the time. Like an old pair of shoes, it is comfortable, and familiar, even if the holes are apparent.

The true scientist is in a somewhat different position. He starts off with any theory and finds it useful only so long as it works. If it no longer works, it should be discarded, and a new, better one fashioned.

And that is an old, comfortable familiar theory that you can settle down into, and stick with for life. Expect change; you can be sure you won’t be disappointed.

John W. Campbell, Jr.
Nuclear Physicist,²,³
Author of The Atomic Story

NOTE: Formulation of this Scientific Methodology was contributed in part by the engineers of “Ma Bell”–the Bell Telephone research laboratories–to whom thanks are extended.

Notes