You can see a lot, just by looking.
– Yogi Berra (#data visualization,vision)
Did you ever see such a thing as a drawing of a muchness?
– Lewis Carroll, Alice in Wonderland (#data visualization)
The critical requirement of an effective graphical display is that it stimulate spontaneous perceptions of structure in data.
– S. Smith et al. 1990 (#data visualization)
Like good writing, producing an effective graphical display requires an understanding of purpose—what is to be communicated, and to whom.
– Michael Friendly, Gallery of Data Visualization, 1991 (#data visualization)
Have you ever seen voice mail?
– The Hackers Test (#data visualization,vision)
Graphics is the visual means of resolving logical problems.
– Jacques Bertin, Graphics and Graphic Information Processing, 2011, p. 16. (#data visualization,vision)
The greatest value of a picture is when it forces us to notice what we never expected to see.
– John W. Tukey, Exploratory Data Analysis, 1977 (#data visualization,pictures,eda)
If data analysis is to be well done, much of it must be a matter of judgment, and ‘theory’ whether statistical or non-statistical, will have to guide, not command.
– John W. Tukey, The Future of Data Analysis, Annals of Mathematical Statistics, Vol. 33 (1), 1962.
The physical sciences are used to ‘praying over’ their data, examining the same data from a variety of points of view. This process has been very rewarding, and has led to many extremely valuable insights. Without this sort of flexibility, progress in physical science would have been much slower. Flexibility in analysis is often to be had honestly at the price of a willingness not to demand that what has already been observed shall establish, or prove, what analysis suggests. In physical science generally, the results of praying over the data are thought of as something to be put to further test in another experiment, as indications rather than conclusions.
– John W. Tukey, The Future of Data Analysis, Annals of Mathematical Statistics, Vol. 33 (1), 1962.
If one technique of data analysis were to be exalted above all others for its ability to be revealing to the mind in connection with each of many different models, there is little doubt which one would be chosen. The simple graph has brought more information to the data analyst’s mind than any other device. It specializes in providing indications of unexpected phenomena.
– John W. Tukey, The Future of Data Analysis, The Annals of Mathematical Statistics, Vol. 33, No. 1 (Mar., 1962), pp. 1-67. (#data visualization)
Genius seems to consist merely in trueness of sight.
– Ralph Waldo Emerson, Journals of Ralph Waldo Emerson, Entry dated 1835, May 11 (#data visualization,vision)
The eye obeys exactly the action of the mind.
– Ralph Waldo Emerson, Representative men. English traits. Conduct of life, p.409 (#data visualization)
Vision is the art of seeing things invisible.
– Johnathan Swift, 1711 (#data visualization,vision)
When there is no vision, the people perish.
– Proverbs 29:18 (#data visualization,vision)
If I can’t picture it, I can’t understand it.
– Albert Einstein (#data visualization,pictures)
And those who have insight will shine brightly like the brightness of the expanse of Heaven.
– Daniel 12:3 (#data visualization)
The one thing that marks the true artist is a clear perception and a firm, bold hand, in distinction from that imperfect mental vision and uncertain truth which give up the feeble pictures and the lumpy statues of the mere artisans on canvas or in stone.
– Oliver Wendell Holmes (1860), The Professor at the Breakfast Table Ticknor and Fields, Boston, MA (#data visualization)
I like your motto: One picture is worth 1,000 denials.
– Ronald Reagan to White House News Photographers Assn, 18 May 1983 (#data visualization,pictures)
With brush you paint the possibilities with pens you scribe the probabilities for in pictures we find insight while in numbers find we strength.
– Forrest W. Young (#data visualization,eda,pictures)
A graphic should not only show the leaves it should show the branches as well as the entire tree.
– Jacques Bertin, The Semiology of Graphics, 1983. Translated by W. J. Berg. University of Wisconsin Press : Wisconsin. (#data visualization,eda)
Tables are like cobwebs, like the sieve of Danaides; beautifully reticulated, orderly to look upon, but which will hold no conclusion. Tables are abstractions, and the object a most concrete one, so difficult to read the essence of.
– Thomas Carlyle, Chartism, 1840, Chapter II, Statistics (#data visualization,tables)
A judicious man looks at Statistics, not to get knowledge, but to save himself from having ignorance foisted on him.
– Thomas Carlyle, Chartism, 1840, Chapter II, Statistics (#data visualization)
Although geometrical representations of propositions in the thermodynamics of fluids are in general use and have done good service in disseminating clear notions in this science, yet they have by no means received the extension in respect to variety and generality of which they are capable.
– J. Willard Gibbs, Graphical Methods in the Thermodynamics of Fluids, 1873 (#data visualization,geometry)
Although we often hear that data speak for themselves, their voices can be soft and sly.
– Frederick Mosteller, Stephen Fienberg and Robert E. Rourke, Beginning Statistics with Data Analysis 1983, Reading MA, p. 234 (#data visualization)
Nocturne, of Chopin, so beautiful music. But few people will appreciate the music if I just show them the notes. Most of us need to listen to the music to understand how beautiful it is. But often that’s how we present statistics; we just show the notes, we don’t play the music.
– Hans Rosling, OECD World Forum, Istanbul, June 2007 (#data visualization,statistics)
The greatest possibilities of visual display lie in vividness and inescapability of the intended message. A visual display can stop your mental flow in its tracks and make you think. A visual display can force you to notice what you never expected to see.
– John W. Tukey (#data visualization,vision)
The purpose of [data] display is comparison (recognition of phenomena), not numbers … The phenomena are the main actors, numbers are the supporting cast.
– John W. Tukey (#data visualization)
If an editor should print bad English he would lose his position. Many editors are using and printing bad methods of graphic presentation, but they hold their jobs just the same.
– W. C. Brinton, Graphic methods of presenting facts 1914, p. 3. (#data visualization)
Around the turn of the century, Karl Pearson, an almost elemental force for more and better statistical thought in all areas of life, including with gusto, matters of social policy, was thinking and lecturing about graphical methods. But later in Pearson’s life, and certainly in the careers of R. A. Fisher and the other great statistical minds of the first half of the century, there was a falling away of interest in graphics and an efflorescence of devotion to analytical mathematical methods. Indeed, for many years there was a contagious snobbery against so unpopular, vulgar and elementary a topic as graphics among academic statisticians and their students
– William Kruskal (#data visualization,statistics)
If statistical graphics, although born just yesterday, extends its reach every day, it is because it replaces long tables of numbers and it allows one not only to embrace at glance the series of phenomena, but also to signal the correspondences or anomalies, to find the causes, to identify the laws.
– Emile Cheysson, c. 1877 (#data visualization,tables)
Numbers have an important story to tell. They rely on you to give them a clear and convincing voice.
– Stephen Few (#data visualization)
The purpose of visualization is insight, not pictures.
– Ben Shneiderman, Extreme visualization: squeezing a billion records into a million pixels. In SIGMOD ’08: Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pages 3-12, New York, NY, USA, 2008. ACM. (#data visualization,pictures)
I love taxonomies, categories, ways of dividing people into groups.
– Gretchen Rubin (#data visualization)
Ballerinas are often divided into three categories: jumpers, turners and balancers.
– Robert Gottlieb (#data visualization)
Mr. Funkhouser has made an extremely interesting and valuable contribution to the history of statistical method. I wish, however, that he could have added a warning, supported by horrid examples, of the evils of the graphical method unsupported by tables of figures. Both for accurate understanding, and particularly to facilitate the use of the same material by other people, it is essential that graphs should not be published by themselves, but only when supported by the tables which lead up to them. It would be an exceedingly good rule to forbid in any scientific periodical the publication of graphs unsupported by tables.
– John Maynard Keynes, Review of Funkhouser for The Economic Journal (#data visualization)
Without data you are just another person with an opinion.
– W. Edwards Deming (#data visualization,data)
Without a plot you are just a person missing a convincing argument.
– Di Cook, 2016 (#data visualization)
Whatever relates to extent and quantity may be represented by geometrical figures. Statistical projections which speak to the senses without fatiguing the mind, possess the advantage of fixing the attention on a great number of important facts.
– Alexander von Humboldt (#data visualization,vision)
Segnius irritant animos demissa per aures, Quam quae sunt oculus subjecta fidelibus (Roughly: What we hear excites the mind less than what we see).
– Horace (#data visualization,vision)
You see, but you do not observe. The distinction is clear.
– Sherlock Holmes, The Adventures of Sherlock Holmes (1890), “A Scandal in Bohemia”, p. 162 (#data visualization,vision)
Every picture tells a story.
– Rod Stewart, 1971 (#data visualization,pictures)
A picture is worth a ten thousand words.
– Fred R. Barnard, advertising trade journal Printers Ink, March 10 1927. (#data visualization,pictures)
…But it is not always clear which 1000 words.
– John W. Tukey, 1973 (#data visualization,pictures)
Un croquis vaut mieux qu’un long discours. Tr.: A good sketch is better than a long speech.
– Napoleon Bonaparte (#data visualization,pictures)
A picture is worth a thousand numbers.
– Anon (#data visualization,pictures)
Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won’t usually need your flowcharts; they’ll be obvious.
– Fred Brooks, The Mythical Man-Month (#data visualization,pictures,tables)
Look here, upon this picture, and on this.
– Shakespeare, Hamlet (#data visualization,pictures)
I am only a picture-taster, the way others are wine- or tea-tasters.
– Bernard Berenson, Sunset and Twilight Harcourt, Brace & World, 1963 (#data visualization,pictures)
Getting information from a table is like extracting sunlight from a cucumber.
– Arthur B. Farquhar & Henry Farquhar, Economic and Industrial Delusions , 1891. (#data visualization,pictures)
When a law is contained in figures, it is buried like metal in an ore; it is necessary to extract it. This is the work of graphical representation. It points out the coincidences, the relationships between phenomena, their anomalies, and we have seen what a powerful means of control it puts in the hands of the statistician to verify new data, discover and correct errors with which they have been stained.
– Emile Cheysson, Les methods de la statistique (1890), 34-35. (#data visualization,pictures)
Good design is obvious. Great design is transparent.
– Joe Sparano, graphic designer for Oxide Design Co. (#data visualization,design)
Content precedes design. Design in the absence of content is not design, it’s decoration.
– Jeffrey Zeldman, web designer and entrepreneur (#data visualization,design)
Mankind is not a circle with a single center but an ellipse with two focal points of which facts are one and ideas the other.
– Victor Hugo (#data visualization,ellipses,geometry)
So, Fabricius, I already have this: that the most true path of the planet [Mars] is an ellipse, which Durer also calls an oval, or certainly so close to an ellipse that the difference is insensible.
– Johannes Kepler, 1605 (#data visualization,ellipses)
Programming graphics in X is like finding the square root of pi using Roman numerals.
– Henry Spencer (#computing)
The purpose of computing is insight, not numbers.
– Richard Hamming, Introduction To Applied Numerical Analysis (#computing)
… to be a good theoretical statistician one must also compute, and must therefore have the best computing aids.
– Frank Yates, Sampling Methods for Censuses and Surveys 1949 (#computing)
We [he and Halmos] share a philosophy about linear algebra: we think basis-free, we write basis-free, but when the chips are down we close the office door and compute with matrices like fury.
– Irving Kaplansky, Paul Halmos: Celebrating 50 Years of Mathematics (#computing)
Seek computer programs that allow you to do the thinking.
– George E. P. Box (#computing)
If you only know how to use a hammer, every problem starts to look like a nail. Stay away from that trap.
– Richard B. Johnson (#computing)
[It is] best to confuse only one issue at a time.
– Kernihan & Ritchie (#computing)
The nice thing about standards is that there are so many of them to choose from.
– Andrew Tanenbaum, Computer Networks (#computing)
There are no routine statistical questions, only questionable statistical routines.
– David R. Cox (#computing,statistics)
Be careful the environment you choose for it will shape you be careful the friends you choose for you will become like them.
– W. Clement Stone (#computing,tidy data)
Be careless in your dress if you must, but keep a tidy soul.
– Mark Twain (#computing,tidy data)
I’m a tidy sort of bloke. I don’t like chaos. I kept records in the record rack, tea in the tea caddy, and pot in the pot box.
– George Harrison (#computing,tidy data)
Thou shalt not sit with statisticians nor commit a Social Science.
– W.H. Auden (#statistics)
There are two kinds of statistics, the kind you look up and the kind you make up.
– Rex Stout (#statistics)
Statistics are like alienists – they will testify for either side.
– Fiorello H. La Guardia (#statistics)
You may prove anything by figures.
– Thomas Carlyle (#statistics)
To understand God’s thoughts we must study statistics, for these are the measure of His purpose.
– Florence Nightingale (#statistics)
You cannot feed the hungry on statistics.
– David Lloyd George (#statistics)
A single death is a tragedy, a million deaths is a statistic.
– Kurt Tucholsky, mis-attributed to Joseph Stalin, Franzosischer Witz, 1925 (#statistics)
Statistics are like a bikini. What they reveal is suggestive, but what they conceal is vital.
– Aaron Levenstein (#statistics)
Do not put faith in what statistics say until you have carefully considered what they do not say.
– William W. Watt (#statistics)
Facts are stubborn things, but statistics are more pliable.
– Mark Twain (#statistics)
Statistics are figures used as arguments.
– Leonard L. Levison (#statistics)
Figures won’t lie, but liars will figure.
– Unknown (though often misattributed to Mark Twain) (#statistics)
I always find that statistics are hard to swallow and impossible to digest. The only one I can remember is that if all the people who go to sleep in church were laid end to end they would be a lot more comfortable.
– Mrs Robert A. Taft (#statistics)
Statistician: Delphic figure who lacks the necessary vocabulary to converse with mere mortals.
– Rod Nicolson, Psychology Software News (#statistics)
Get the facts first, and then you can distort them as much as you please.
– Mark Twain (#statistics)
If you want to inspire confidence, give plenty of statistics. It does not matter that they should be accurate, or even intelligible, as long as there is enough of them.
– Lewis Carroll (#statistics)
It is a truth very certain that when it is not in our power to determine what is true we ought to follow what is most probable.
– Rene Descartes (#statistics)
Models are to be used, but not to be believed.
– Henry Theill (#statistics)
The deepest sin of the human mind is to believe things without evidence.
– Thomas H. Huxley (#statistics)
Man must learn to simplify, but not to the point of falsification.
– Aldous Huxley (#statistics)
Since small differences in probability cannot be appreciated by the human mind, there seems little point in being excessively precise about uncertainty.
– George E. P. Box & G. C. Tiao, Bayesian inference in statistical analysis, 1973. Addison-Wesley, Reading, MA, p. 65. (#statistics)
Some people hate the very name of statistics but I find them full of beauty and interest. Whenever they are not brutalized, but delicately handled by the higher methods, and are warily interpreted, their power of dealing with complicated phenomena is extraordinary.
– Francis Galton, Natual Inheritance 1889 p. 62 (#statistics)
[Statistics are] the only tools by which an opening may be cut through the formidable thicket of difficulties that bars the path of those who pursue the Science of Man.
– Francis Galton, Natural Inheritance, 1889. (#statistics)
Data analysis is an aid to thinking and not a replacement for.
– Richard Shillington (#statistics)
Sometimes the only thing you can do with a poorly designed experiment is to try to find out what it died of.
– Ronald A. Fisher (#statistics,experimental design)
The best time to plan an experiment is after you’ve done it.
– Ronald A. Fisher (#statistics,experimental design)
[The War Office kept three sets of figures:] one to mislead the public, another to mislead the Cabinet, and the third to mislead itself.
– Herbert Asquith, Alistair Horne, Price of Glory (#statistics)
Why are you testing your data for normality? For large sample sizes the normality tests often give a meaningful answer to a meaningless question (for small samples they give a meaningless answer to a meaningful question)
– Greg Snow, R-Help, 21 Feb 2014 (#statistics,normality,nhst)
The relevant question is not whether ANOVA assumptions are met exactly, but rather whether the plausible violations of the assumptions have serious consequences on the validity of probability statements based on the standard assumptions
– Gene V. Glass & Percy D. Peckham & James R. Sanders, Consequences of Failure to Meet Assumptions Underlying the Fixed Effects Analyses of Variance and Covariance, Review of Educational Research Vol. 42, No. 3 (Summer, 1972), pp. 237-288 , p. 237. (#statistics,anova)
Exploratory data analysis can never be the whole story, but nothing else can serve as the foundation stone – as the first step.
– John W. Tukey, Exploratory Data Analysis, 1977, p.3. (#statistics,data,data analysis)
The best thing about being a statistician is that you get to play in everyone’s backyard.
– John W. Tukey (#statistics)
Far better an approximate answer to the right question, which is often vague, than an exact answer to the wrong question, which can always be made precise.
– John W. Tukey, The Future of Data Analysis, The Annals of Mathematical Statistics, Vol. 33, No. 1 (Mar., 1962), pp. 1-67. (#statistics)
The worst, i.e., most dangerous, feature of ‘accepting the null hypothesis’ is the giving up of explicit uncertainty … Mathematics can sometimes be put in such black-and-white terms, but our knowledge or belief about the external world never can.
– John W. Tukey, The Philosophy of Multiple Comparisons, Statist. Sci. 6 (1) 100 - 116, February, 1991. (#statistics,nhst)
Better to have an approximate answer to the right question than a precise answer to the wrong question.
– John W. Tukey, Quoted by John Chambers (#statistics)
All models are wrong, but some are useful.
– George E. P. Box (#statistics)
Every model is an approximation.
– George E. P. Box (#statistics)
The business of the statistician is to catalyze the scientific learning process.
– George E. P. Box (#statistics)
Statisticians, like artists, have the bad habit of falling in love with their models.
– George E. P. Box (#statistics)
If there were a probability of only p = 0.04 of finding a crock of gold behind the next tree, wouldn’t you go and look?
– George E. P. Box (#statistics)
When the ratio of the largest to smallest observation is large you should question whether the data are being analyzed in the right metric (transformation).
– George E. P. Box (#statistics)
A useful type of time series model is a recipe for transforming serial data into white noise.
– George E. P. Box (#statistics,time series)
It is the data that are real (they actually happened!) The model is a hypothetical conjecture that might or might not summarize and/or explain important features of the data
– George E. P. Box (#statistics)
It is not unusual for a well-designed experiment to analyze itself.
– George E. P. Box (#statistics)
Discovering the unexpected is more important than confirming the known.
– George E. P. Box (#statistics)
We are drowning in information and starving for knowledge.
– John Naisbitt, Megatrends (#statistics)
`Data! data!’ he cried impatiently. I can’t make bricks without clay.
– Arthur Conan-Doyle, Adventures of Sherlock Holmes “The Copper Beeches” (#data)
I have no data yet. It is a capital mistake to theorize before one has data.
– Arthur Conan-Doyle, Adventures of Sherlock Holmes “A Scandal in Bohemia” (#data)
This was an unexpected piece of luck. My data were coming more quickly than I could have reasonably hoped.
– Arthur Conan-Doyle, Memoirs of Sherlock Holmes, The Musgrave Ritual (#data)
I have not all my facts yet, but I do not think there are any insuperable difficulties. Still, it is an error to argue in front of your data. You find yourself insensibly twisting them round to fit your theories.
– Arthur Conan-Doyle, His Last Bow, Wisteria Lodge (#data)
The only thing we know for sure about a missing data point is that it is not there, and there is nothing that the magic of statistics can do do change that. The best that can be managed is to estimate the extent to which missing data have influenced the inferences we wish to draw.
– Howard Wainer (#data)
Big data can change the way social science is performed, but will not replace statistical common sense.
– Thomas Landsall-Welfare, Nowcasting the mood of the nation, Significance v. 9(4), August 12, 2012, p. 28. (#data)
Baseball is ninety percent mental and the other half is physical.
– Yogi Berra (#data)
Whenever I see an outlier, I never know whether to throw it away or patent it.
– Bert Gunter, R-Help, 9/14/2015 (#data,outliers)
In God we trust. All others must bring data.
– W. Edwards Deming (#data)
The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be extracted from a given body of data.
– John W. Tukey, Sunset Salvo, The American Statistician Vol. 40 (1), 1986. (#data)
How do I love thee? Let me count the ways.
– Elizabeth Barrett Browning, Sonnets from the Portuguese (#data,counts)
Not everything that counts can be counted, and not everything that can be counted counts.
– William Bruce Cameron (#data,counts)
Whenever you can, count.
– Francis Galton (#data,counts)
It is difficult to understand why statisticians commonly limit their inquiries to Averages, and do not revel in more comprehensive views. Their souls seem as dull to the charm of variety as that of the native of our flat English counties, whose retrospect of Switzerland was that, if its mountains could be thrown into its lakes, two nuisances would be got rid of at once.
– Francis Galton, Natural Inheritance (#data,averages)
While the individual man is an insoluble puzzle, in the aggregate he becomes a mathematical certainty. You can, for example, never foretell what any one man will do, but you can say with precision what an average number will be up to. Individuals vary, but percentages remain constant. So says the statistician.
– Arthur Conan-Doyle, Sign of the Four (#data,averages)
If you put a buttock on a hot plate and another one on an ice cube, the average is good, but in reality your bottom is in trouble.
– Grigore Moisil (#data,averages)
The graphical method has considerable superiority for the exposition of statistical facts over the tabular. A heavy bank of figures is grievously wearisome to the eye, and the popular mind is as incapable of drawing any useful lessons from it as of extracting sunbeams from cucumbers.
– Arthur B. Farquhar & Henry Farquhar, Economic and Industrial Delusions, 1891. (#data,tables,vision)
Let it serve for table-talk.
– William Shakespeare, The Merchant of Venice, Act III, Sc. 5. (#data,tables)
I drink to the general joy o’ the whole table.
– William Shakespeare, Macbeth, Act III, Sc. 4. (#data,tables)
Isolated facts, those that can only be obtained by rough estimate and that require development, can only be presented in memoires; but those that can be presented in a body, with details, and on whose accuracy one can rely, may be expounded in tables.
– E. Duvillard, Memoire sur le travail du Bureau de statistique 1806. (#data,tables)
Study without reflection is a waste of time reflection without study is dangerous
– Confuscius, Analects (551-479 BC) (#science)
Things should be made as simple as possible, but not any simpler
– Albert Einstein (#science)
So much has already been written about everything that you can’t find out anything about it.
– James Thurber, 1961 (#science)
The practical power of a statistical test is the product of its’ statistical power and the probability of use.
– John W. Tukey, A Quick, Compact, Two Sample Test to Duckworth’s Specifications (#science,power)
Theory into Practice.
– Mao Tse-Tung, The Little Red Book (#science)
Beauty is truth; truth, beauty. That is all ye know on Earth, and all ye need to know.
– John Keats, Ode on a Grecian urn (#science)
They consider me to have sharp and penetrating vision because I see them through the mesh of a sieve.
– Kahlil Gibran (#science,vision)
The journalistic vision sharpens to the point of maximum impact every event, every individual and social configuration; but the honing is uniform.
– George Steiner (#science,vision)
Some people weave burlap into the fabric of our lives, and some weave gold thread. Both contribute to make the whole picture beautiful and unique.
– Anon. (#science,pictures)
Time extracts various values from a painter’s work. When these values are exhausted the pictures are forgotten, and the more a picture has to give, the greater it is.
– Henri Matisse (#science,pictures)
God is in the details.
– Mies van der Roche, New York Times August 19, 1969 (#science)
The devil is in the details.
– George Schultz (#science)
One has to be able to count if only so that at fifty one doesn’t marry a girl of twenty.
– Maxim Gorky, The Zykovs 1914 (#science,counts)
A man has one hundred dollars and you leave him with two dollars, that’s subtraction.
– Mae West, My Little Chickadee 1940 (#science)
In the fields of observation chance favors only the prepared mind.
– Louis Pasteur (#science,data)
The eye of a human being is a microscope, which makes the world seem bigger than it really is.
– Kahlil Gibran, A Handful of Sand on the Shore (#science,vision)
To the man who only has a hammer in the toolkit, every problem looks like a nail.
– Abraham Maslow (#science)
Four hostile newspapers are more to be feared than a thousand bayonets.
– Napoleon Bonaparte, Maxims (#science)
When I’m working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong.
– Richard Buckminster Fuller (#science)
He who asks a question is a fool for five minutes he who does not ask a question remains a fool forever.
– Chinese Proverb (#science)
The great tragedy of science – the slaying of a beautiful hypothesis by an ugly fact.
– Thomas Huxley (#science,nhst)
Give a man to fish and he will eat for a day. Teach a man to fish and he will eat for the rest of his life.
– Chinese Proverb (#science)
Give a man a fish and he will eat for a day. Teach a man to fish and you lose a consulting job forever.
– Howard Wainer, 2016 (#science)
When you have eliminated the impossible, whatever remains, however improbable, must be the truth.
– Arthur Conan Doyle, The Sign of the Four (1890), Ch. 6 (#science,probability)
If you choose to represent the various parts in life by holes upon a table, of different shapes—some circular, some triangular, some square, some oblong—we shall generally find that the triangular person has got into the square hole, the oblong into the triangular, and a square person has squeezed himself into the round hole.
– Sydney Smith, 1769-1845 (#science,geometry)
I know of scarcely anything so apt to impress the imagination as the wonderful form of cosmic order expressed by the “Law of Frequency of Error.” The law would have been personified by the Greeks and deified, if they had known of it. It reigns with serenity and in complete self-effacement, amidst the wildest confusion. The huger the mob, and the greater the apparent anarchy, the more perfect is its sway. It is the supreme law of Unreason. Whenever a large sample of chaotic elements are taken in hand and marshaled in the order of their magnitude, an unsuspected and most beautiful form of regularity proves to have been latent all along.
– Sir Francis Galton, Natural Inheritance London: Macmillan, 1889. Quoted in J. R. Newman (ed.) The World of Mathematics, New York: Simon and Schuster, 1956. p. 1482. (#science,normality)
In scientific thought we adopt the simplest theory which will explain all the facts under consideration and enable us to predict new facts of the same kind. The catch in this criterion lies in the world “simplest.” It is really an aesthetic canon such as we find implicit in our criticisms of poetry or painting. The layman finds such a law as dx/dt = K(d2x/dy2) much less simple than “it oozes,” of which it is the mathematical statement. The physicist reverses this judgment, and his statement is certainly the more fruitful of the two, so far as prediction is concerned. It is, however, a statement about something very unfamiliar to the plainman, namely, the rate of change of a rate of change.
– John Burdon Sanderson Haldane, Possible Worlds, 1927. (#science)
Oh, what a tangled web we weave, When first we practice to deceive!
– Sir Walter Scott (#science)
Practice is the best of all instructors.
– Publilius Syrus (#science)
We should go to the masses and learn from them, synthesize their experience into better, articulated principles and methods, then do propaganda among the masses, and call upon them to put these principles and methods into practice so as to solve their problems and help them achieve liberation and happiness.
– Chairman Mao Zedong, “Get Organized!” (November 29, 1943), Selected Works, Vol. III, p. 158. (#science)
An elementary demonstration is one that requires no knowledge— just an infinite amount of intelligence.
– Richard Feynman (#science)
Science may be described as the art of systematic over-simplification.
– Karl Popper (#science)
Science is like sex: sometimes something useful comes out, but that is not the reason we are doing it.
– Richard Feynman (#science)
Humanists believe that the world has a fixed number of mysteries, so that when one is solved, our sense of wonder is diminished. Scientists believe that the world has endless mysteries, so that when one is solved, there are always new ones to ponder.
– D. O. Hebb, quoted by Steven Pinker (#science)
Art and science encounter each other when they seek exactitude.
– Etienne-Jules Marey (#science)
Circumstantial evidence is a very tricky thing. It may seem to point very straight to one thing, but if you shift your own point of view a little, you may find it pointing in an equally uncompromising manner to something entirely different.
– Sherlock Holmes, The Adventures of Sherlock Holmes (1892) “The Boscombe Valley Mystery” (#science)
To find out what happens when you change something, it is necessary to change it.
– Box, Hunter, and Hunter, Statistics for Experimenters (1978) (#science)
He who loves practice without theory is like the sailor who boards ship without a rudder and compass and never knows where he may be cast.
– Leonardo da Vinci (#science)
Science is built up of facts, as a house is with stones. But a collection of facts is no more a science than a heap of stones is a house.
– Henri Poincare (#science)
The museum spreads its surfaces everywhere, and becomes an untitled collection of generalizations that mobilize the eye.
– Robert Smithson (#science,generalizations)
In one word, to draw the rule from experience, one must generalize; this is a necessity that imposes itself on the most circumspect observer.
– Henri Poincare, The Value of Science: Essential Writings of Henri Poincare (#science,generalizations)
If I have seen further, it is by standing on the shoulders of giants.
– Sir Isaac Newton, Letter to Robert Hooke, Feb. 5, 1676 (#science,generalizations)
Few intellectual pleasures are more keen than those enjoyed by a person who, while he is occupied in some special inquiry, suddenly perceives that it admits of a wide generalization, and that his results hold good in previously-unsuspected directions.
– Francis Galton, North American Review, 150 419-431 (1890) (#science,generalizations)
The elegance of a theorem is directly proportional to the number of ideas you can see in it and inversely proportional to the effort it takes to see them.
– George Polya (#science,generalizations)
A mathematician, like a painter or a poet, is a master of pattern. The mathematician’s patterns, like the painter’s or the poet’s, must be beautiful; the ideas, like the colors or the words, must fit together in a harmonious way. … There is no permanent place in the world for ugly mathematics.
– G. H. Hardy (#science,generalizations)
An idea which can be used once is a trick. If it can be used more than once it becomes a method.
– George Polya and Gabor Szego (#science,generalizations)
When the time is ripe for certain things, these things appear in different places in the manner of violets coming to light in early spring.
– Farkas Bolyai, To his son Janos Bolyai, urging him to claim the invention of non-Euclidean geometry without delay, quoted in Ming Li and Paul Vitanyi, An introduction to Kolmogorov Complexity and Its Applications, 1st ed., 1993, p. 83. (#science,generalizations)
The only new thing in the world is the history you don’t know.
– Harry S. Truman, Quoted by David McCulloch (#history)
So we beat on, boats against the current, borne back ceaselessly into the past.
– F. Scott Fitzgerald, The Great Gatsby (1925) (#history,time)
Euclid alone has looked on beauty bare.
– Edna St Vincent Millay (#history,geometry)
The past only exists insofar as it is present in the records of today. And what those records are is determined by what questions we ask. There is no other history than that.
– Wheeler, 1982:24 (#history)
A generation which ignores history has no past and no future
– Robert Heinlein (#history)
For my part, I consider that it will be found much better by all parties to leave the past to history, especially as I propose to write that history myself.
– Winston Churchill (#history)
If you would understand anything, observe its beginning and its development.
– Aristotle (#history)
God alone knows the future, but only an historian can alter the past.
– Ambrose Bierce (#history)
Since God himself cannot change the past, he is obliged to tolerate the existence of historians.
– Attributed to Samuel Butler (#history)
At the heart of good history is a naughty little secret: good storytelling.
– Stephen Schiff (#history)
It has been said that though God cannot alter the past, historians can; it is perhaps because they can be useful to Him in this respect that He tolerates their existence.
– Samuel Butler, Erewhon Revisted (#history)
History is moving statistics and statistics is frozen history.
– A. L. Schlozer, Theorie der Statistik 1804, p. 86 (#history)
Time flies like an arrow fruit flies like a banana.
– Anthony G. Oettinger, Often mis-attributed to Groucho Marx (#history,time)
Time is the longest distance between two places.
– Tennessee Williams, The Glass Menagerie (#history,time)
Those who make the worst use of their time are the first to complain of its brevity.
– Jean de La Bruyere, Les Caracteres (#history,time)
The past is a foreign country: they do things differently there.
– L. P. Hartley, The Go-Between (#history,time)
I never think of the future - it comes soon enough.
– Albert Einstein (#history,time)
The best way to predict the future is to invent it
– Alan Kay (#history,time)
The future ain’t what it used to be
– Yogi Berra (#history,time)
Look not mournfully into the past. It comes not back again. Wisely improve the present. It is thine. Go forth to meet the shadowy future, without fear.
– Henry Wadsworth Longfellow (#history,time)
Let him who would enjoy a good future waste none of his present.
– Roger Babson (#history,time)
When in doubt, predict that the present trend will continue.
– Merkins Maxim (#history,time)
The only use of a knowledge of the past is to equip us for the present. The present contains all that there is. It is holy ground; for it is the past, and it is the future.
– Alfred North Whitehead (#history,time)
My past is my wisdom to use today. … my future is my wisdom yet to experience. Be in the present because that is where life resides.
– Gene Oliver, Life and the Artistry of Change (#history,time)
I have realized that the past and future are real illusions, that they exist in the present, which is what there is and all there is.
– Alan Watts (#history,time)
The future is uncertain but the end is always near.
– Jim Morrison (#history,time)
Time has no divisions to mark its passage, there is never a thunder-storm of blare of trumpets to announce the beginning of a new month or year. Even when a new century begins, it is only we mortals who ring bells and fire off pistols.
– Thomas Mann, The Magic Mountain (1924) (#history,time)
Direction is more important than speed. We are so busy looking at our speedometers that we forget the milestone.
– Anonymous (#history,milestones)
Only sixteen players have hit fifty or more homers in a season. To me, that’s a very special milestone.
– Mark McGwire (#history,milestones)
As life runs on, the road grows strange with faces new – and near the end. The milestones into headstones change, Neath every one a friend.
– James Russell Lowell (#history,milestones)
This paper contains much that is new and much that is true. Unfortunately, that which is true is not new and that which is new is not true.
– attributed to Wolfgang Pauli (#reviews)
This book fills a much-needed gap.
– Attributed to Moses Hadas (#reviews)
Russell left the vast darkness of the topic unobscured
– Alfred North Whitehead, Referring to Bertrand Russell (#reviews)
Mathematicians have always been rather of a jealous nature, and undoubtedly jealousy was a family characteristic of the Bernoullis. There is some excuse for mathematicians, for their reputation stands for posterity largely not on what they did, but on what their contemporaries attributed to them.
– Karl Pearson, The History of Statistics in the 17th and 18th Centuries. (#history)
When choosing between two evils, I always like to take the one I’ve never tried before.
– Mae West, 1941 (#ethics)
To err is human—but it feels divine!
– Mae West (paraphrase of Alexander Pope, “To err is human, to forgive devine”) (#data,averages)
Good judgment comes from experience experience comes from bad judgment.
– Fred Brooks (#history)
One must learn by doing the thing; for though you think you know it, you have no certainty until you try.
– Sophocles
There is a magic in graphs. The profile of a curve reveals in a flash a whole situation—the life history of an epidemic, a panic, or an era of prosperity. The curve informs the mind, awakens the imagination, convinces.
– Henry D. Hubbard, Foreword to Brinton (1939), Graphic Presentation (#data visualization,pictures)
Graphs carry the message home. A universal language, graphs convey information directly to the mind. Without complexity there is imaged to the eye a magnitude to be remembered. Words have wings, but graphs interpret. Graphs are pure quantity, stripped of verbal sham, reduced to dimension, vivid, unescapable.
– Henry D. Hubbard, Foreword to Brinton (1939), Graphic Presentation (#data visualization,pictures)
Graphs are all inclusive. No fact is too slight or too great to plot to a scale suited to the eye. Graphs may record the path of an ion or the orbit of the sun, the rise of a civilization, or the acceleration of a bullet, the climate of a century or the varying pressure of a heart beat, the growth of a business, or the nerve reactions of a child.
– Henry D. Hubbard, Foreword to Brinton (1939), Graphic Presentation (#data visualization,pictures)
The graphic art depicts magnitudes to the eye. It does more. It compels the seeing of relations. We may portray by simple graphic methods whole masses of intricate routine, the organization of an enterprise, or the plan of a campaign. Graphs serve as storm signals for the manager, statesman, engineer; as potent narratives for the actuary, statist, naturalist; and as forceful engines of research for science, technology and industry. They display results. They disclose new facts and laws. They reveal discoveries as the bud unfolds the flower.
– Henry D. Hubbard, Foreword to Brinton (1939), Graphic Presentation (#data visualization,pictures,vision)
The graphic language is modern. We are learning its alphabet. That it will develop a lexicon and a literature marvelous for its vividness and the variety of application is inevitable. Graphs are dynamic, dramatic. They may epitomize an epoch, each dot a fact, each slope an event, each curve a history. Wherever there are data to record, inferences to draw, or facts to tell, graphs furnish the unrivalled means whose power we are just beginning to realize and to apply.
– Henry D. Hubbard, Foreword to Brinton (1939), Graphic Presentation (#data visualization,pictures)
In One Dimensions, did not a moving Point produce a Line with two terminal points? In two Dimensions, did not a moving Line produce a Square wit four terminal points? In Three Dimensions, did not a moving Square produce - did not the eyes of mine behold it - that blessed being, a Cube, with eight terminal points? And in Four Dimensions, shall not a moving Cube - alas, for Analogy, and alas for the Progress of Truth if it be not so - shall not, I say the motion of a divine Cube result in a still more divine organization with sixteen terminal points?
– Edwin A. Abbott, Flatland: A Romance of Many Dimensions (#data visualization,geometry)
To comport oneself with perfect propriety in Polygonal society, one ought to be a Polygon oneself.
– Edwin A. Abbott, Flatland: A Romance of Many Dimensions (#data visualization,geometry)
True, said the Sphere; it appears to you a Plane, because you are not accustomed to light and shade and perspective; just as in Flatland a Hexagon would appear a Straight Line to one who has not the Art of Sight Recognition. But in reality it is a Solid, as you shall learn by the sense of Feeling.
– Edwin A. Abbott, Flatland: A Romance of Many Dimensions (#data visualization,geometry)
There are twenty one mystical dimensions of consciousness. Enlightenment is abiding in the highest three dimensions of consciousness.
– Amit Ray, Enlightenment Step by Step (#data visualization,geometry)
I would say time is definitely one of my top three favorite dimensions.
– Randall Munroe, xkcd (#history,time)
Poetry is when an emotion has found its thought and the thought has found its words.
– Robert Frost (#science)
I don’t like to commit myself about heaven and hell - you see, I have friends in both places.
– Mark Twain
Errors using inadequate data are much less than those using no data at all.
– Charles Babbage, Circa 1850 (#data)
From carefully compiled statistical facts more may be learned [about] the moral nature of Man than can be gathered from all the accumulated experiences of the preceding ages.
– Henry Thomas Buckle, A History of Civilization in England, 1857/1898, p. 17 (#statistics)
Statistical thinking will one day be as necessary for efficient citizenship as the ability to read and write.
– H.G. Wells, Mankind in Making, 1903 (#statistics)
Statistical accounts are to be referred to as a dictionary by men of riper years, and by young men as a grammar, to teach them the relations and proportions of different statistical subjects, and to imprint them on the mind at a time when the memory is capable of being impressed in a lasting and durable manner, thereby laying the foundation for accurate and valuable knowledge.
– William Playfair, The Statistical Breviary (1801), 5-6. (#statistics)
Geography is only a branch of statistics, a knowledge of which is necessary to the well-understanding of the history of nations, as well as their situations relative to each other.
– William Playfair, The Commercial and Political Atlas, p. 29 (#statistics)
No study is less alluring or more dry and tedious than statistics, unless the mind and imagination are set to work, or that the person studying is particularly interested in the subject; which last can seldom be the case with young men in any rank of life.
– William Playfair, The Statistical Breviary (1801), p. 16 (#statistics)
DIAGRAMS are of great utility for illustrating certain questions of vital statistics by conveying ideas on the subject through the eye, which cannot be so readily grasped when contained in figures.
– Florence Nightingale, Mortality of the British Army, 1857 (#data visualization,pictures,vision)
To give insight to statistical information it occurred to me, that making an appeal to the eye when proportion and magnitude are concerned, is the best and readiest method of conveying a distinct idea.
– William Playfair, The Statistical Breviary (1801), p. 2 (#data visualization,pictures,vision)
Regarding numbers and proportions, the best way to catch the imagination is to speak to the eyes.
– William Playfair, Elemens de statistique, Paris, 1802, p. XX. (#data visualization,pictures,vision)
The aim of my carte figurative is to convey promptly to the eye the relation not given quickly by numbers requiring mental calculation.
– Charles Joseph Minard (#data visualization,pictures)
Information that is imperfectly acquired, is generally as imperfectly retained; and a man who has carefully investigated a printed table, finds, when done, that he has only a very faint and partial idea of what he has read; and that like a figure imprinted on sand, is soon totally erased and defaced.
– William Playfair, The Commercial and Political Atlas (p. 3), 1786 (#data visualization,pictures,tables)
Since the aim of exploratory data analysis is to learn what seems to be, it should be no surprise that pictures play a vital role in doing it well.
– John W. Tukey, John W. Tukey’s Works on Interactive Graphics. The Annals of Statistics Vol. 30, No. 6 (Dec., 2002), pp. 1629-1639 (#data visualization,pictures,eda)
There is nothing better than a picture for making you think of questions you had forgotten to ask (even mentally).
– John W. Tukey & Paul Tukey, John W. Tukey’s Works on Interactive Graphics. The Annals of Statistics Vol. 30, No. 6 (Dec., 2002), pp. 1629-1639 (#data visualization,pictures)
Functional visualizations are more than innovative statistical analyses and computational algorithms. They must make sense to the user and require a visual language system that uses colour, shape, line, hierarchy and composition to communicate clearly and appropriately, much like the alphabetic and character-based languages used worldwide between humans.
– Matt Woolman, Digital Information Graphics (#data visualization,pictures)
A people without the knowledge of their past history, origin and culture is like a tree without roots
– Marcus Garvey (#history)
The bushels of rings taken from the fingers of the slain at the battle of Cannae, above two thousand years ago, are recorded; … but the bushels of corn produced in England at this day, or the number of the inhabitants of the country, are unknown, at the very time that we are debating that most important question, whether or not there is sufficient substance for those who live in the kingdom.
– William Playfair, The Statistical Breviary (1801), p. 7-8 (#history)
How can the past and future be, when the past no longer is, and the future is not yet? As for the present, if it were always present and never moved on to become the past, it would not be time, but eternity.
– St. Augustine of Hippo, Confessions (#history,time)
Every measurable thing, except numbers, is imagined in the manner of continuous quantity. Therefore, for the mensuration of such a thing, it is necessary that points, lines and surfaces, or their properties be imagined. For in them, as the Philosopher has it, measure or ratio is initially found, while in other things it is recognized by similarity as they are being referred to by the intellect to the geometrical entities.
– Nicole Oresme, The Latitude of Forms (#data visualization,geometry)
Any one who considers arithmetical methods of producing random digits is, of course, in a state of sin.
– John von Neumann, Various techniques used in connection with random digits, Applied Mathematics Series, 1951, no 12, 36-38. (#computing,random numbers)
With four parameters I can fit an elephant, and with five I can make him wiggle his trunk.
– John von Neumann, Quoted by Freeman Dyson (#science)
The punishment of every disordered mind is its own disorder.
– St. Augustine of Hippo, Confessions (#science)
It is strange that only extraordinary men make the discoveries, which later appear so easy and simple.
– Georg C. Lichtenberg (#science)
Two things are infinite: the universe and human stupidity; and I’m not sure about the universe.
– Albert Einstein (#science)
Facts, however numerous, do not constitute a science. Like innumerable grains of sand on the sea shore, single facts appear isolated, useless, shapeless; it is only when compared, when arranged in their natural relations, when crystallised by the intellect, that they constitute the eternal truths of science
– William Farr, “Observation,” Br. Ann. Med. 1 (1837): 693 (#science)
Tidy datasets are all alike, but every messy dataset is messy in its own way
– Hadley Wickham (#computing,tidy data)
We cannot understand the world without numbers, and we cannot understand it with numbers alone.
– Hans Rosling (#data)
Bad data makes bad models. Bad models instruct people to make ineffective or harmful interventions. Those bad interventions produce more bad data, which is fed into more bad models.
– Cory Doctorow, Machine Learning’s Crumbling Foundations, Aug 2021. (#data)
Whenever I am infuriated, I revenge myself with a new Diagram
– Florence Nightingale, Letter 1857.8.19 to Sidney Herbert (#data visualization,pictures)
Again I must repeat my objections to intermingling causation with statistics. It might be to a certain extent admissible if you had no sanitary head. But you have one, & his report should be quite separate. The statistician has nothing to do with causation: he is almost certain in the present state of knowledge to err.
– Florence Nightingale, Letter, March 1861 (#statistics)
A Bayesian is one who, vaguely expecting a horse, and catching a glimpse of a donkey, strongly believes he has seen a mule.
– Stephen John Senn (#statistics)
What nature hath joined together, multiple regression cannot put asunder.
– Richard Nisbett (#statistics)
Stepwise regression is probably the most abused computerized statistical technique ever devised. If you think you need stepwise regression to solve a particular problem you have, it is almost certain that you do not. Professional statisticians rarely use automated stepwise regression.
– Leland Wilkinson, SYSTAT (1984). P. 196. (#computing)
It would help if the standard statistical programs did not generate t statistics in such profusion. The programs might be written to ask, “Do you really have a probability sample?”, “By what standard would you judge a fitted coefficient large or small?” Or perhaps they could merely say, printed in bold capitals beside each equation, “So What Else Is New?”
– Donald M. McCloskey, The Loss Function Has Been Mislaid: The Rhetoric of Significance Tests, American Economic Review, Vol 75, #2. (#computing)
The documentation level of R is already much higher than average for open source software and even than some commercial packages (esp. SPSS is notorious for its attitude of “You want to do one of these things. If you don’t understand what the output means, click help and we’ll pop up five lines of mumbo-jumbo that you’re not going to understand either.”)
– Peter Dalgaard, R-help mailing list 4.2.2002 (#computing)
S has forever altered the way people analyze, visualize, and manipulate data …. S is an elegant, widely accepted, and enduring software system, with conceptual integrity, thanks to the insight, taste, and effort of John Chambers.
– Association for Computing Machinery Software System Award (#computing)
Tradition among experienced S programmers has always been that loops (typically ‘for’ loops) are intrinsically inefficient: expressing computations without loops has provided a measure of entry into the inner circle of S programming.
– John Chambers, Programming With Data, p. 173. (#computing)
While the distribution and publication of Version 2 [of S] was still evolving, parallel research work was starting to shape the next major version. At first this seemed to be a move away from S altogether: something called the “Quantitative Programming Environment” was initially a separate research project, aimed more explicitly at programming and trying to emphasize that users need not be statistically sophisticated. By 1986, however, the decision was made to merge this work with the facilities (especially the graphics) underlying S, to produce a new version of S. (This explains, by the way, why the main program for S is called Sqpe, another one of those little puzzles for users.)
– Unknown (#computing)
This computationally intensive operation [bootstrapping] is not one calculated to endear a user to a database administrator.
– Leland Wilkinson, The Grammar of Graphics, p. 49. (#computing)
Most computer software is not yet intelligent enough to stop the user doing something stupid. The old adage ‘Garbage In -> Garbage Out’ still hold good, and it must be realized that careful thought and close inspection of the data are vital preliminaries to complicated computer analysis.
– Christopher Chatfield, Problem solving : a statistician’s guide, 1988, p. 59. (#computing)
Today…we have high-speed computers and prepackaged statistical routines to perform the necessary calculations…statistical software will no more make one a statistician than would a scalpel turn one into a neurosurgeon. Allowing these tools to do our thinking for us is a sure recipe for disaster.
– Good & Hardin, Common Errors in Statistics and How to Avoid Them, p. ix (#computing)
The generation of random numbers is too important to be left to chance.
– Robert R. Coveyou, Oak Ridge National Laboratory, 1969 (#computing,random numbers)
Landon Noll…has been tinkering with random number generators for nearly a decade–an exercise in bringing order to chaos. “There’s a lot of beauty in chaos,” Noll says. “The Grand Canyon wouldn’t be so popular if it was just a uniform trench. The trick is controlling and managing chaos and turning it into something useful.”
– Tom McNichol, Wired, August, 2003, page 088. (#computing,random numbers)
Someone has characterized the user of stepwise regression as a person who checks his or her brain at the entrance of the computer center.
– D. R. Wittink, The application of regression analysis. Needham Heights, MA: Allyn and Bacon. p. 259. (#computing)
The idea of optimization transfer is very appealing to me, especially since I have never succeeded in fully understanding the EM algorithm.
– Andrew Gellman, Discussion, Journal of Computational and Graphical Statistics, vol 9, p 49. (#computing)
This reminds me of the duality in statistics between computation and model fit: better-fitting models tend to be easier to compute, and computational problems often signal modeling problems. and ‘The Black Swan’. Law, Probability and Risk (2008) 7, 151-163.
– Andrew Gelman, Thoughts inspired by Nassim Taleb’s ‘Fooled by Randomness’ (#computing)
It is a nontrivial exercise to correctly program even the simplest split-plot model using PROC MIXED.
– Jeremy Aldworth & Wherly P Hoffman, Split-Plot Model With Covariate: A Cautionary Tale, The American Statistician, 56, 284–289. (#computing)
Sometimes the most important fit statistic you can get is ‘convergence not met’–it can tell you something is wrong with your model.
– Oliver Schabenberger, 2006 Applied Statistics in Agriculture Conference. (#computing)
It is obviously pointless to report or quote results to more digits than is warranted. In fact, it is misleading or at the very least unhelpful, because it fails to communicate to the reader another important aspect of the result–namely its reliability! A good rule (sometimes known as Ehrenberg’s rule) is to quote all digits up to and including the first two variable digits.
– Philipp K. Janert, Data Analysis with Open Source Tools, O’Reilly, 2010. (#computing)
Doubt is not a pleasant mental state, but certainty is a ridiculous one.
– Voltaire (1694-1778) (#probability)
It is a part of probability that many improbable things will happen.
– Agathon, 445 - 400 BC, Chance News 7.02 (#probability)
Statistics make it clear the fact that one’s chances of being hurt by a bear are far, far fewer than being struck by an auto almost anywhere, or being mugged on a city street, for that matter. We pursue our automotive, urban lives undaunted, often indifferent amid the police and ambulance sirens, but in the Alaskan wilderness we lie awake worrying about bears.
– John Kauffmann, Alaska’s Brooks Range (#probability)
Cougars can be dangerous, especially to unsupervised children, but the chances of becoming a cougar victim are far less than becoming a victim of lightning, honeybees, moose, deer, pit bulls, football, snow-shoveling, or crossing the street in front of your house. For some reason, we fear the true risks of being killed far less than the remote risk of becoming prey.
– Dennis L Olsen, Cougars, page 46. (#probability)
Things which ought to be expected can seem quite extraordinary if you’ve got the wrong model.
– David Hand, Significance, 2014, 11, 36-39. (#data,models)
In many applications, the data analyst has a dilemma: Should an effect be classified as [fixed] and a BLUE obtained, or as [random] and a BLUP obtained? The traditional distinction between fixed and random effects is not helpful; it may, in fact, lead the data analyst to choose the less efficient route.
– Walter Stroup and D K Mulitze, Nearest Neighbor Adjusted Best Linear Unbiased Prediction, 1991, The American Statistician, 45, 194–200. (#data,models)
What should be the distribution of random effects in a mixed model? I think Gaussian is just fine, unless you are trying to write a journal paper.
– Terry Therneau, Speaking at useR 2007. (#data,models)
Competent scientists do not believe their own models or theories, but rather treat them as convenient fictions. …The issue to a scientist is not whether a model is true, but rather whether there is another whose predictive power is enough better to justify movement from today’s fiction to a new one.
– Steve Vardeman, Comment, 1987, Journal of the American Statistical Association, 82 : 130-131. (#data,models)
If you just rely on one model, you tend to amputate reality to make it fit your model.
– David Brooks (#data,models)
Statistical models are sometimes misunderstood in epidemiology. Statistical models for data are never true. The question whether a model is true is irrelevant. A more appropriate question is whether we obtain the correct scientific conclusion if we pretend that the process under study behaves according to a particular statistical model.
– Scott Zeger, Statistical reasoning in epidemiology, American Journal of Epidemiology, 1991 (#data,models)
It is not always convenient to remember that the right model for a population can fit a sample of data worse than a wrong model – even a wrong model with fewer parameters. We cannot rely on statistical diagnostics to save us, especially with small samples. We must think about what our models mean, regardless of fit, or we will promulgate nonsense.
– Leland Wilkinson, The Grammar of Graphics, p. 335. (#data,models)
Fitting models to data is a bit like designing shirts to fit people. If you fit a shirt too closely to one particular person, it will fit other people poorly. Likewise, a model that fits a particular data set too well might not fit other data sets well.
– Rahul Parsa, Speaking to the Iowa SAS User’s Group (#data,models)
You might say that there’s no reason to bother with model checking since all models are false anyway. I do believe that all models are false, but for me the purpose of model checking is not to accept or reject a model, but to reveal aspects of the data that are not captured by the fitted model.
– Andrew Gelman, Some thoughts on the sociology of statistics, 2007. (#data,models)
When evaluating a model, at least two broad standards are relevant. One is whether the model is consistent with the data. The other is whether the model is consistent with the ‘real world.’
– Kenneth Bollen, Structural Equations with Latent Variable (#data,models)
The point of a model is to get useful information about the relation between the response and predictor variables. Interpretability is a way of getting information. But a model does not have to be simple to provide reliable information about the relation between predictor and response variables; neither does it have to be a data model. The goal is not interpretability, but accurate information.
– Leo Breiman, Statistical Modeling: The Two Cultures, Statistical Science, Vol 16, p. 210. (#data,models)
The goals in statistics are to use data to predict and to get information about the underlying data mechanism. Nowhere is it written on a stone tablet what kind of model should be used to solve problems involving data. To make my position clear, I am not against models per se. In some situations they are the most appropriate way to solve the problem. But the emphasis needs to be on the problem and on the data. Unfortunately, our field has a vested interest in models, come hell or high water.
– Leo Breiman, Statistical Modeling: The Two Cultures, Statistical Science, Vol 16, p. 214. (#data,models,statistics)
Unless exploratory data analysis uncovers indications, usually quantitative ones, there is likely to nothing for confirmatory data analysis to consider.
– John Tukey, Exploratory Data Analysis, p. 3. (#data,data analysis)
One thing the data analyst has to learn is how to expose himself to what his data are willing–or even anxious–to tell him. Finding clues requires looking in the right places and with the right magnifying glass.
– John Tukey, Exploratory Data Analysis, p. 21. (#data,data analysis)
In data analysis, a plot of y against x may help us when we know nothing about the logical connection from x to y–even when we do not know whether or not there is one–even when we know that such a connection is impossible.
– John Tukey, Exploratory Data Analysis, p. 131. (#data,data analysis)
Whatever the data, we can try to gain understanding by straightening or by flattening. When we succeed in doing one or both, we almost always see more clearly what is going on.
– John Tukey, Exploratory Data Analysis, p. 148. (#data,data analysis)
When nearest neighbor effects exist, the randomized complete block analysis [can be] so poor as to deserver to be called catastrophic. It [can not] even be considered a serious form of analysis. It is extremely important to make this clear to the vast number of researchers who have near religious faith in the randomized complete block design.
– Walt Stroup & D Mulitze, Nearest Neighbor Adjusted Best Linear Unbiased Prediction, The American Statistician, 45, 194–200. (#data,data analysis)
There are two books devoted solely to principal components, Jackson (1991) and Jolliffe (1986), which we think overstates its value as a technique.
– Venables & Ripley, Modern Applied Statistics with S, 4th ed., page 305. (#data,data analysis)
Understanding the split-plot isn’t everything. It’s the only thing.
– Oliver Schabenberger, Speaking at JSM 2008. (#data,data analysis)
Residual analysis is similarly unreliable. In a discussion after a presentation of residual analysis in a seminar at Berkeley in 1993, William Cleveland, one of the fathers of residual analysis, admitted that it could not uncover lack of fit in more than four to five dimensions. The papers I have read on using residual analysis to check lack of fit are confined to data sets with two or three variables. With higher dimensions, the interactions between the variables can produce passable residual plots for a variety of models. A residual plot is a goodness-of-fit test, and lacks power in more than a few dimensions. An acceptable residual plot does not imply that the model is a good fit to the data.
– Leo Breiman, Statistical Modeling: The Two Cultures, Statistical Science, Vol 16, p. 203. (#data,data analysis)
I was profoundly disappointed when I saw that S-PLUS 4.5 now provides Type III sums of squares as a routine option for the summary method for aov objects. I note that it is not yet available for multistratum models, although this has all the hallmarks of an oversight (that is, a bug) rather than common sense seeing the light of day. When the decision was being taken of whether to include this feature, because the FDA requires it a few of my colleagues and I were consulted and our reply was unhesitatingly a clear and unequivocal No, but it seems the FDA and SAS speak louder and we were clearly outvoted.
– Bill Venables, Exegeses on Linear Models (#data,data analysis,lsmeans)
Some of us feel that type III sum of squares and so-called LS-means are statistical nonsense which should have been left in SAS.
– Brian Ripley, Discussing features of S-Plus, S-news 5.29.1999 (#data,data analysis,lsmeans)
I think it would be interesting to ask people who use the results from LSMEANS to explain what the results represent. My guess is that less than 1% of the people who use LSMEANS know what they in fact are.
– Doug Bates, R-help mailing list, 16 Oct 2003 (#data,data analysis,lsmeans)
Doing applied statistics is never easy, especially if you want to get it right.
– Xiao-Li Meng, 2005 Joint Statistical Meetings (#data,data analysis)
I agree with the general message: “The right variables make a big difference for accuracy. Complex statistical methods, not so much.” This is similar to something Hal Stern told me once: the most important aspect of a statistical analysis is not what you do with the data, it’s what data you use.
– Andrew Gelman, The most important aspect of a statistical analysis is not what you do with the data, it’s what data you use, 2018. (#data,data analysis)
Once upon a time, the phrase ‘statistical reduction of data’ was used as a synonym for statistical analysis; it implied refining and concentrating the data so as eventually to express the main features in a much smaller number of means, indices, coefficients. … Today some statisticians and some computer programs seem more disposed to undertake ‘statistical expansion of data’, perhaps with an original 96 observations leading to 25 pages of output.
– D. J. Finney, Was This In Your Statistics Textbook: 1. Agricultural Scientist And Statistician, Experimental Agriculture, 24, 153-161. (#data,data analysis)
Data analysis is a tricky business – a trickier business than even tricky data analysts sometimes think.
– Bert Gunter, S-news mailing list, 6 Jun 2002 (#data,data analysis)
A first analysis of experimental results should, I believe, invariably be conducted using flexible data analytical techniques–looking at graphs and simple statistics–that so far as possible allow the data to ‘speak for themselves’. The unexpected phenomena that such a approach often uncovers can be of the greatest importance in shaping and sometimes redirecting the course of an ongoing investigation.
– George Box, Signal to Noise Ratios, Performance Criteria, and Transformations, Technometrics, 30, 1–17 (#data,data analysis,eda)
When I was in graduate school, a fellow student who was writing his dissertation with the late William G. Cochran passed along some of Cochran’s advice: You make a bigger contribution to statistics if you find a workable solution to an important unsolved problem than if you find an optimal solution where a good one already exists.
– Fred L. Ramsey and Daniel W. Schafer, The American Statistician, 54, 78. (#data,data analysis)
The six degrees of freedom for error provided by the 4x4 Latin square have long been recognized as inadequate, at least by Fisher. Something of the order of 12 error degrees of freedom would appear desirable…unless the effects under investigation are large in comparison with their experimental errors.
– Frank Yates, Complex Experiments, Supplement to the Journal of the Royal Statistical Society, 1935, Vol 2, No. 1. (#data,data analysis)
The old rule of trusting the Central Limit Theorem if the sample size is larger than 30 is just that–old. Bootstrap and permutation testing let us more easily do inferences for a wider variety of statistics.
– Tim Hesterberg (#data,data analysis)
A competent data analysis of an even moderately complex set of data is a thing of trials and retreats, of dead ends and branches.
– John Tukey, Computer Science and Statistics: Proceedings of the 14th Symposium on the Interface, p. 4. (#data,data analysis)
Scrutiny [of data] should take in the names of variates. Analysis of variates y1 to y5 is not statistics; analysis of plant height in centimeters, root weight in grams, etc., may be.
– D. A. Preece, In discussion of C. Chatfield, “The initial examination of data”, Journal of the Royal Statistical Society. Series A (1985), p. 234. (#data,data analysis)
To call in the statistician after the experiment is done may be no more than asking him to perform a postmortem examination: he may be able to say what the experiment died of.
– R.A. Fisher, Sankya, Indian Statistical Congress, Vol 4, p. 17. (#data,data analysis)
It is clear that a statistician who is involved at the start of an investigation, advises on data collection, and who knows the background and objectives, will generally make a better job of the analysis than a statistician who was called in later on.
– Christopher Chatfield, Problem solving : a statistician’s guide, 1988, p. 12. (#data,data analysis)
We really haven’t got any great amount of data on the subject, and without data how can we reach any definite conclusions?
– Thomas Alva Edison (1847-1931) (#data)
Small data…fits in memory on a laptop: <10 GB. Medium data…fits in memory on a server: 10 GB-1 TB. Big data…can’t fit in memory on one computer: >1 TB.
– Hadley Wickham, Big Data Pipelines, 2015. (#data)
A massive data set is one for which the size, heterogeneity, and general complexity cause serious pain for the analyst(s).
– J. Kettenring, Massive data sets…reflections on a workshop, Computing Science and Statistics, Proceedings of the 33rd Symposium on the Interface, Vol 33, 2001. (#data)
The Dirty Data Theorem states that “real world” data tends to come from bizarre and unspecifiable distributions of highly correlated variables and have unequal sample sizes, missing data points, non-independent observations, and an indeterminate number of inaccurately recorded values.
– Unknown, Statistically Speaking, p. 282. (#data)
The Titanic survival data seem to become to categorical data analysis what Fisher’s Iris data are to discriminant analysis.
– Andreas Buja, A Word from the Editor of JCGS, Statistical Computing & Graphics Newsletter, V10, N1, p 32. (#data)
Consideration needs to be given to the most appropriate data to be collected. Often the temptation is to collect too much data and not give appropriate attention to the most important. Filing cabinets and computer files world-wide are filled with data that have been collected because they may be of interest to someone in future. Most is never of interest to anyone and if it is, its existence is unknown to those seeking the information, who will set out to collect the data again, probably in a trial better designed for the purpose. In general, it is best to collect only the data required to answer the questions posed, when setting up the trial, and plan another trial for other data in the future, if necessary.
– P. Portmann & H. Ketata, Statistical Methods for Plant Variety Evaluation, p. 15. (#data)
We have found that some of the hardest errors to detect by traditional methods are unsuspected gaps in the data collection (we usually discovered them serendipitously in the course of graphical checking).
– Peter Huber, Huge data sets, Compstat ’94: Proceedings, 1994. (#data)
Every messy data is messy in its own way - it’s easy to define the characteristics of a clean dataset (rows are observations, columns are variables, columns contain values of consistent types). If you start to look at real life data you’ll see every way you can imagine data being messy (and many that you can’t)!
– Hadley Wickham, R-help mailing list, 17 Jan 2008 (#data)
What all practicing data analysts agree on is that the proportion of project time spent on data cleaning is huge. Estimates of 75-90 percent have been suggested.
– Unknown, Graphics of Large Datasets, p. 20. (#data)
That the ten digits do not occur with equal frequency must be evident to any one making much use of logarithmic tables, and noticing how much faster the first pages wear out than the last ones.
– Simon Newcomb, Note on the frequencies of the different digits in natural numbers, Amer. J. Math, 4, 39-40, 1881. (#data)
For a hundred years or so, mathematical statisticians have been in love with the fact that the probability distribution of the sum of a very large number of very small random deviations always converges to a normal distribution. This infatuation tended to focus interest away from the fact that, for real data, the normal distribution is often rather poorly realized, if it is realized at all.
– Unknown, Numerical Recipes in C, p 520. (#data,normality)
I abhor averages. I like the individual case. A man may have six meals one day and none the next, making an average of three meals per day, but that is not a good way to live.
– Louis Brandeis (#data,averages)
The per capita gross national product of a nation…as a measure of the comfort of individual lives is about as apt, say, as deciding how to dress in the morning according to the mean annual temperature of the region in which one lives. If one lives in the tropics this would work well. But if one lives in Minnesota, where the temperature might be thirty degrees below zero one morning and one hundred degrees above zero another morning, one would be in danger of dying of exposure or of prostration most of the time. The problem with aggregate statistics is that they obscure both the extremes and patterns of distribution.
– Paul Gruchow, Grass Roots, p. 44. (#data,averages)
In former times, when the hazards of sea voyages were much more serious than they are today, when ships buffeted by storms threw a portion of their cargo overboard, it was recognized that those whose goods were sacrificed had a claim in equity to indemnification at the expense of those whose goods were safely delivered. The value of the lost goods was paid for by agreement between all of those whose merchandise had been in the same ship. This sea damage to cargo in transit was known as ‘havaria’ and the word came naturally to be applied to the compensation money which each individual was called upon to pay. From this Latin word derives our modern word ‘average’.
– M. J. Moroney, Facts from Figures, p. 34. (#data,averages)
Some people hate the very name of statistics, but I find them full of beauty and interest. Whenever they are not brutalised, but delicately handled by the higher methods, and are warily interpreted, their power of dealing with complicated phenomena is extraordinary. They are the only tools by which an opening can be cut through the formidable thicket of difficulties that bars the path of those who pursue the Science of man.
– Frances Galton, Natural Inheritance. (#statistics,science)
The central limit theorem is often used to justify the assumption of normality when using the sample mean and the the sample standard deviation. But it is inevitable that real data contain gross errors. Five to ten percent unusual values in a dataset seem to be the rule rather than the exception (Hampel 1973). The distribution of such data is no longer Normal.
– A. S. Hedayat and Guoqin Su, Robustness of the Simultaneous Estimators of Location and Scale From Approximating a Histogram by a Normal Density Curve, The American Statistician, 2012, 66, p. 25. (#data,outliers,normality)
Why is a particular record or measurement classed as an outlier? Among all who handle and interpret statistical data, the word has long been in common use as an epithet for any item among a dataset of N that departs markedly from the broad pattern of the set.
– David Finney, Calibration Guidelines Challenge Outlier Practices, The American Statistician, 2006, Vol 60, No 4, p. 310. (#data,outliers)
Dodge (2003) provided a definition of ‘outlier’ that is helpful but far from complete: In a sample of N observations, it is possible for a limited number to be so far separated in value from the remainder that they give rise to the question whether they are not from a different population, or that the sampling technique is at fault. Such values are called outliers.
– David Finney, Calibration Guidelines Challenge Outlier Practices, The American Statistician, 2006, Vol 60, No 4, p. 310. (#data,outliers)
The finding of an outlier is not necessarily a discovery of a bad or misleading datum that may contaminate the data, but it may amount to a comment on the validity of distributional assumptions inherent in the form of analysis that is contemplated.
– David Finney, Calibration Guidelines Challenge Outlier Practices, The American Statistician, 2006, Vol 60, No 4, p. 312. (#data,outliers)
If any observation has been classed as an outlier, the next step should be if possible to infer the cause…attention should be given to the possibility that laboratory and data management techniques have been imperfect: improvements and safeguards for the future should be considered.
– David Finney, Calibration Guidelines Challenge Outlier Practices, The American Statistician, 2006, Vol 60, No 4, p. 312. (#data,outliers)
The motivation for any action on outliers must be to improve interpretation of data without ignoring unwelcome truth. To remove bad and untrustworthy data is a laudable ambition, but naive and untested rules may bring harm rather than benefit.
– David Finney, Calibration Guidelines Challenge Outlier Practices, The American Statistician, 2006, Vol 60, No 4, p. 312. (#data,outliers)
One cautious approach is represented by Bernoulli’s more conservative outlook. If there are very strong reasons for believing that an observation has suffered an accident that made the value in the data-file thoroughly untrustworthy, then reject it; in the absence of clear evidence that an observation, identified by formal rule as an outlier, is unacceptable then retain it unless there is lack of trust that the laboratory obtaining it is conscientiously operated by able persons who have “…taken every care.”
– David Finney, Calibration Guidelines Challenge Outlier Practices, The American Statistician, 2006, Vol 60, No 4, p. 313. (#data,outliers)
Treat outliers like children. Correct them when necessary, but never throw them out.
– Unknown., Top 12 Tip #2. Practical Stats’ Applied Environmental Statistics course (#data,outliers)
There are a lot of statistical methods looking at whether an outlier should be deleted…I don’t endorse any of them.
– Barry Nussbaum, Significance, Apr 2017. (#data,outliers)
All this discussion of deleting the outliers is completely backwards. In my work, I usually throw away all the good data, and just analyze the outliers.
– Unknown pharmaceutical statistician, The American Statistician, Vol 61, No 3, page 193. (#data,outliers)
I have often thought that outliers contain more information than the model.
– Arnold Goodman, 2005 Joint Statistical Meetings (#data,outliers)
Whatever actually happened, outliers need to be investigated not omitted. Try to understand what caused some observations to be different from the bulk of the observations. If you understand the reasons, you are then in a better position to judge whether the points can legitimately removed from the data set, or whether you’ve just discovered something new and interesting. Never remove a point just because it is weird.
– Rob J. Hyndman, Omitting outliers, 2016 (#data,outliers)
Scholars feel the need to present tables of model parameters in academic articles (perhaps just as evidence that they ran the analysis they claimed to have run), but these tables are rarely interpreted other than for their sign and statistical significance. Most of the numbers in these tables are never even discussed in the text. From the perspective of the applied data analyst, R packages without procedures to compute quantities of scientific interest are woefully incomplete. A better approach focuses on quantities of direct scientific interest rather than uninterpretable model parameters. … For each quantity of interest, the user needs some summary that includes a point estimate and a measure of uncertainty such as a standard error, confidence interval, or a distribution. The methods of calculating these differ greatly across theories of inference and methods of analysis. However, from the user’s perspective, the result is almost always the same: the point estimate and uncertainty of some quantity of interest.
– Kousuke Imai, Gary King, Oliva Lau, Toward a Common Framework for Statistical Analysis and Development, Journal of Computational and Graphical Statistics, 2008, v 17. (#data,tables,uncertainty)
The purpose of plotting is to convey phenomena to the viewer’s cortex, not to provide a place to lookup observed numbers.
– Kaye Basford, John Tukey, Graphical Analysis of Multi-Response Data, p. 373. (#data visualization)
Had we started with this [quantile] plot, noticed that it looks straight and not looked further, we would have missed the important features of the data. The general lesson is important. Theoretical quantile-quantile plots are not a panacea and must be used in conjunction with other displays and analyses to get a full picture of the behavior of the data.
– John M. Chambers, William S. Cleveland, Beat Kleiner, Paul A. Tukey, Graphical Methods for Data Analysis, p. 212. (#data visualization)
Visualization for large data is an oxymoron–the art is to reduce size before one visualizes. The contradiction (and challenge) is that we may need to visualize first in order to find out how to reduce size.
– Peter Huber, Massive datasets workshop: Four years after, Journal of Computational and Graphical Statistics, Vol 8, 635–652. (#data visualization)
Pie charts have severe perceptual problems… If you want to display some data, and perceiving the information is not so important, then a pie chart is fine.
– Unknown, S-Plus 2000 Programmer’s Guide, p. 349. (#data visualization)
Merely drawing a plot does not constitute visualization. Visualization is about conveying important information to the reader accurately. It should reveal information that is in the data and should not impose structure on the data.
– W. Huber, X. Li, and R. Gentleman, Bioinformatics and Computational Biology Solutions using R and Bioconductor, p. 162. (#data visualization)
While the dendrogram has been widely used to represent distances between objects, it cannot really be considered to be a visualization method. Dendrograms do not necessarily expose structure that exists in the data. In many cases they impose structure on the data, and when that is the case it is dangerous to interpret the observed structure.
– W. Huber, X. Li, and R. Gentleman, Bioinformatics and Computational Biology Solutions using R and Bioconductor, p. 170. (#data visualization)
[When] you see excellent graphics, find out how they were done. Borrow strength from demonstrated excellence. The idea for information design is: Don’t get it original, get it right.
– Edward Tufte (#data visualization,design)
Graphical excellence is that which gives to the viewer the greatest number of ideas in the shortest time with the least ink in the smallest space.
– Edward Tufte, The Visual Display of Quantitative Information, 1983. (#data visualization)
Chartjunk does not achieve the goals of its propagators. The overwhelming fact of data graphics is that they stand or fall on their content, gracefully displayed. Graphics do not become attractive and interesting through the addition of ornamental hatching and false perspective to a few bars.
– Edward Tufte, The Visual Display of Quantitative Information, 1983, p. 121. (#data visualization)
A table is nearly always better than a dumb pie chart; the only worse design than a pie chart is several of them, for then the viewer is asked to compare quantities located in spatial disarray both within and between pies…Given their low data-density and failure to order numbers along a visual dimension, pie charts should never be used. Above all else show the data.
– Edward Tufte, The Visual Display of Quantitative Information, 1983, p. 178. (#data visualization)
In medical research, too often the first published study testing a new treatment provides the strongest evidence that will ever be found for that treatment. As better controlled studies–less vulnerable to the enthusiasms of researchers and their sponsors–are then conducted, the treatment’s reported efficacy declines. Years after the initial study […] sometimes the only remaining issue is whether the treatment is in fact harmful.
– Edward Tufte, Beautiful Evidence, p. 144. (#statistics,significance)
The preliminary examination of most data is facilitated by the use of diagrams. Diagrams prove nothing, but bring outstanding features readily to the eye; they are therefore no substitutes for such critical tests as may be applied to the data, but are valuable in suggesting such tests, and in explaining the conclusions founded upon them.
– Ronald A Fisher, Statistical Methods for Research Workers, p. 27. (#data visualization)
Our statistical puritanism may incline us not to use shadows, but we confess that a little bit of shadow is fun.
– Dan Carr, Using Layering and Perceptual Grouping in Statistical Graphics, Statistical Computing & Graphics Newsletter, V. 10, N. 1, p. 25. (#data visualization)
We are not saying that the primary purpose of a graph is to convey numbers with as many decimal places as possible. We agree with Ehrenberg (1975) that if this were the only goal, tables would be better. The power of a graph is its ability to enable one to take in the quantitative information, organize it, and see patterns and structure not readily revealed by other means of studying the data.
– William Cleveland & Robert McGill, Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Models, Journal of the American Statistical Association, 79, 531-554, 1984. (#data visualization)
There was a controversy [in the 1920s]…about whether the divided bar chart or the pie chart was superior for portraying the parts of a whole. The contest appears to have ended in a draw. We conclude that neither graphical form should be used because other methods are demonstrably better.
– William Cleveland & Robert McGill, Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Models, Journal of the American Statistical Association, 79, 531-554, 1984. (#data visualization)
Our conclusion about [choropleth] patch maps agrees with Tukey’s (1979), who left little doubt about his opinions by stating, ‘I am coming to be less and less satisfied with the set of maps that some dignify by the name statistical map and that I would gladly revile with the name patch map’.
– William Cleveland & Robert McGill, Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Models, Journal of the American Statistical Association, 79, 531–554, 1984. (#data visualization)
There is no more reason to expect one graph to ‘tell all’ than to expect one number to do the same.
– John Tukey, Exploratory Data Analysis. (#data visualization)
There is no excuse for failing to plot and look.
– John Tukey, Exploratory Data Analysis (#data visualization)
It’s generally considered bad practice to use more than six colors in a single display.
– Ross Ihaka, R-help mailing list, 2004 (#data visualization)
The mere multiplicity of the attempts to deal with more than three continuous dimensions by encoding additional variables into glyphs, Chernoff faces, stars, Kleiner-Hartigan trees, and so on indicates that each of them has met only with rather limited success.
– Peter Huber, Statistical graphics: history and overview, Proceedings of the Fourth Annual Conference and Exposition, p. 674. (#data visualization)
Spatial patterns may be due to many sources of variation. In the context of seeking explanations, John Tukey said that, “the unadjusted plot should not be made.” In other words, our perceptual/cognitive abilities are poor in terms of adjusting for known source of variations and envisioning the resulting map. A better strategy is to control for known sources of variation and/or adjust the estimates before making the map.
– Dan Carr, Survey Research Methods Section newsletter, July 2002. (#data visualization)
It’s not easy to select more than a few clearly distinct colors. Also, “distinct” is context-dependent, because: What will be the spatial relationships of the different colors in your output? You can successfully have fairly similar colors adjacent to each other, since the contrast is more obvious when they’re adjacent. However, if you want to use colors to track identity and difference across scattered points or patches, then you need bigger separations between colors, since you want to be able to see easily that patch “A” here is of the same kind as patch “A” there and different from patch “B” somewhere else, when mingled with patches of other kinds. And size matters. Big patches of similar color (as on a map) can look quite distinct, while the same colors used to plot filled circular blobs on a graph might be barely distinguishable, and totally indistinguishable if used to plot colored “.”s or “+”s. … It’s all very psycho-visual and success usually requires experimentation!
– Ted Harding, R-help mailing list, 2004 (#data visualization)
The concept of randomness arises partly from games of chance. The word ‘chance’ derives from the Latin cadentia signifying the fall of a die. The word ‘random’ itself comes from the French randir meaning to run fast or gallop.
– G. Spencer Brown, Probability and Scientific Inference, Chapter VII, p. 35. (#history)
Statistics derives from a German term, ‘Statistik’, first used as a substantive by the Gottingen professor Gottfried Achenwall in 1749.
– Theodore M. Porter, The Rise of Statistical Thinking 1820-1900. (#history)
Strangely, the motto chosen by the founders of the Statistical Society in 1834 was ‘Aliis exterendum’, which means ‘Let other thrash it out.’ William Cochran confessed that ‘it is a little embarrassing that statisticians started out by proclaiming what they will not do’.
– Edmund A. Gehan and Noreen A. Lemak, Statistics in Medical Research: Developments in Clinical Trials (#history)
What accounts for the success of the [Iowa State] Stat Lab? I believe that it is because it was not driven by the mathematics, but by actual problems in biology, genetics, demography, economics, psychology, and so on. To be sure, a real problems give rise to abstract problems in statistical inference which have a fascination of their own. However, for statistics to remain viable, statistical problems should have their genesis in real, data-related problems.
– Oscar Kempthorne, A conversation with Oscar Kempthorne, Statistical Science, 1995, V 10, p. 335. (#history)
You prepare yourself to win. You prepare yourself for the possibility that you won’t win. You don’t really prepare yourself for the possibility that you flip the coin in the air and it lands on its edge and you get neither outcome.
– Al Gore, On the 2004 presidential election, Chance News 10.01. (#history)
The invalid assumption that correlation implies cause is probably among the two or three most serious and common errors of human reasoning.
– Stephen Jay Gould, The Mismeasure of Man (#statistics)
When noise is correlated it becomes music.
– Anindya Roy, Personal communication (#statistics)
As I left consulting to go back to the university, these were the perceptions I had about working with data to find answers to problems: (a) Focus on finding a good solution–that’s what consultants get paid for. (b) Live with the data before you plunge into modelling. (c) Search for a model that gives a good solution, either algorithmic or data. (d) Predictive accuracy on test sets is the criterion for how good the model is. (e) Computers are an indispensable partner.
– Leo Breiman, Statistical Modeling: The Two Cultures, Statistical Science, Vol. 16, p. 201. (#statistics)
I have always thought that statistical design and sampling from populations should be the first courses taught, but all elementary courses I know of start with statistical methods or probability. To me, this is putting the cart before the horse!
– Walter Federer, A Conversation with Walter T Federer, Statistical Science, 2005, Vol 20, p. 312. (#statistics)
Bill Hunter told me that their editor wanted a title for their book with sex appeal. Thus, “Statistics for Experimenters”, which is pretty subliminal but it’s there.
– Robert Easterling, The American Statistician, v 58, p 248. (#statistics)
The only useful function of a statistician is to make predictions, and thus to provide a basis for action.
– W. Edwards Deming, W. A. Wallis, 1980. The Statistical Research Group. Journal of the American Statistical Association, 75, 321. (#statistics)
We must watch our own language. For example, “Type I error” and “Type II error” are meaningless and misleading terms. Instead, try “chance of a false alarm” and “a missed opportunity”.
– Deborah J. Rumsey, Assessing Student Retention of Essential Statistical Ideas: Perspectives, Priorities, and Possibilities, American Statistician, Vol 62, No 1, p. 58. (#statistics)
Statistics prove near & far that folks who drive like crazy…are.
– Anon, Burma Shave sign in the Advertising Museum in Portland (#statistics)
It’s a random pattern. That’s the pattern.
– Ed Chigliak, TV series “Northern Exposure” (#statistics,random numbers)
Randomness is NOT the absence of a pattern.
– Bill Venables, 1999 S-Plus User’s Conference (#statistics,random numbers)
The statistics on sanity are that one out of every four Americans is suffering from some form of mental illness. Think of your three best friends. If they are okay, then it’s you.
– Rita Mae Brown
It is proven that the celebration of birthdays is healthy. Statistics show that those people who celebrate the most birthdays become the oldest.
– S. den Hartog, Ph D. Thesis University of Groningen (#statistics)
Because no one becomes statistically self-sufficient after one semester of study, I try to prepare students to become intelligent consumers of the assistance that they will inevitably seek. Service courses train future clients, not future statisticians.
– Michael W. Tosset, “Statistical Science”, Feb 98, p. 24. (#statistics)
If there was ever an idea in statistics which evokes the reaction, “Why the hell didn’t I think of that,” it has to be the bootstrap.
– James R. Thompson, 1997 Interface Proceedings (#statistics)
The sensible statistician should be wary of other people’s statistics. In particular, it is unwise to believe all official statistics, such as government statements that the probability of a nuclear meltdown is only one in 10,000 years (remember Chernobyl!).
– Christopher Chatfield, Problem solving: a statistician’s guide, 1988, p 73. (#statistics)
The government are very keen on amassing statistics. They collect them, add them, raise them to the n-th power, take the cube root and prepare wonderful diagrams. But you must never forget that every one of these figures comes in the first instance from the village watchman, who just puts down what he damn pleases.
– English judge on the subject of Indian statistics, Quoted in Sir Josiah Stamp in Some Economic Matters in Modern Life, London: King and Sons, 1929, pp. 258-259. (#statistics)
It was always important for the biometrician to take part in the field-work for most kinds of trials. A willingness to get our hands dirty did much to dispel the distrust of the theoretician from Head Office, as well as giving us an appreciation of the practical problems. Trials were always carried out to simulate farming conditions as much as possible. We had once advocated a change in wheat plot lengths from 2 chains to 3, based on results from uniformity trials. It seemed a very good idea until a biometrician went to help harvest a very good crop and found he had to lift and carry bags of over 100 lb, this being the yield from each plot. But the local agriculturist in charge of the trial would never have reported this; he would only have gone on grumbling about those ‘theory guys’ in Head Office forever.
– Jean Heywood (nee Miller), A History of Statistics in New Zealand, edited by H.S.Roberts. p. 23-24. (#statistics, biometry, expt design)
Econometrics has successfully predicted 14 of the last 3 economic depressions.
– David Hand, Speaking at Interface 2000. (#statistics)
We feel that nothing can replace the value to a [corn] breeder of careful study and understanding of his plants…More and more, we feel that grave danger exists of statistics being used as a substitute for critical observation and thought…Statistics have their place, a very important one, but they can never serve as a substitute for close association with plants. Their real value, it seems to us, is in measuring precisely what we already know in a general way. Statistics tends to be an office art based on machines and figures rather than a field art based on living things.
– Henry A. Wallace and William L. Brown, Corn and Its Early Fathers, 1956, p. 123. (#statistics)
The great scientific weakness of America today is that she tends to emphasize quantity at the expense of quality–statistics instead of genuine insight–immediate utilitarian application instead of genuine thought about fundamentals.
– Henry A. Wallace and William L. Brown, Corn and Its Early Fathers, 1956, p. 124. (#statistics)
It is easy to lie with statistics, but it is easier to lie without them.
– Frederick Mosteller (#statistics)
There are aspects of statistics other than it being intellectually difficult that are barriers to learning. For one thing, statistics does not benefit from a glamorous image that motivates students to persist through tedious and frustrating lessons…there are no TV dramas with a good-looking statistician playing the lead, and few mothers’ chests swell with pride as they introduce their son or daughter as “the statistician.”
– Chap T. Le and James R. Boen, Health and Numbers: Basic Statistical Methods (#statistics)
At its core statistics is not about cleverness and technique, but rather about honesty. Its real contribution to society is primarily moral, not technical. It is about doing the right thing when interpreting empirical information. Statisticians are not the world’s best computer scientists, mathematicians, or scientific subject matter specialists. We are (potentially, at least) the best at the principled collection, summarization, and analysis of data.
– Stephen B. Vardeman and Max D. Morris, Statistics and Ethics: Some Advice for Young Statisticians, The American Statistician, vol 57, p. 21. (#statistics,ethics)
Statistical analysis of data can only be performed within the context of selected assumptions, models, and/or prior distributions. A statistical analysis is actually the extraction of substantive information from data and assumptions. And herein lies the rub, understood well by Disraeli and others skeptical of our work: For given data, an analysis can usually be selected which will result in “information” more favorable to the owner of the analysis then is objectively warranted.
– Stephen B. Vardeman and Max D. Morris, Statistics and Ethics: Some Advice for Young Statisticians, The American Statistician, vol 57, p. 25. (#statistics,ethics)
Too much of what all statisticians do … is blatantly subjective for any of us to kid ourselves or the users of our technology into believing that we have operated ‘impartially’ in any true sense. … We can do what seems to us most appropriate, but we can not be objective and would do well to avoid language that hints to the contrary.
– Steve Vardeman, Comment, 1987, Journal of the American Statistical Association, 82, 130-131. (#statistics)
The standard error of most statistics is proportional to 1 over the square root of the sample size. God did this, and there is nothing we can do to change it.
– Howard Wainer, Improving Tabular Displays, With NAEP Tables as Examples and Inspirations, Journal of Educational and Behavioral Statistics, Vol 22, No. 1, pp. 1-30. (#statistics)
Suppose that Sir R. A. Fisher–a master of public relations–had not taken over from ordinary English such evocative words as “sufficient”, “efficient”, and “consistent” and made them into precisely defines terms of statistical theory. He might, after all, have used utterly dull terms for those properties of estimators, calling them characteristics A, B, and C. … Would his work have had the same smashing influence that it did? I think not, or at least not as rapidly.
– William H. Kruskal, Formulas, Numbers, Words: Statistics in Prose, The American Scholar, 1978 (#statistics)
Statistics state the status of the state.
– Leland Wilkinson, The Grammar of Graphics, p. 165. (#statistics)
My philosophy on lotteries is that while you actually have to buy a ticket in order to win the lottery, buying a ticket does not significantly increase your odds of winning.
– Howie Smith, Prsonal communication (#statistics)
If you show your friends your confidence interval for the standard error of the estimated length of the confidence interval of your confidence about yourself, I guess one nice thing to ask to freak them out is: “Can you construct a confidence interval for the confidence level of my confidence?
– Tony Baiching, Personal communication (#statistics)
To make the preliminary test on variances is rather like putting to sea in a rowing boat to find out whether conditions are sufficiently calm for an ocean liner to leave port.
– George E. P. Box, Non-normality and Tests on Variances, Biometrika, 40, 318-335. (#statistics,nhst)
Statistics is, or should be, about scientific investigation and how to do it better, but many statisticians believe it is a branch of mathematics.
– George Box, AmStat News, Oct 2000, page 11. (#statistics,Box quotes)
These days the statistician is often asked such questions as “Are you a Bayesian?” “Are you a frequentist?” “Are you a data analyst?” “Are you a designer of experiments?”. I will argue that the appropriate answer to ALL of these questions can be (and preferably should be) “yes”, and that we can see why this is so if we consider the scientific context for what statisticians do.
– George E.P. Box (#statistics)
One is so much less than two. [John Tukey’s eulogy of his wife.]
– John Tukey, The life and professional contributions of John W. Tukey, The Annals of Statistics, 2001, Vol 30, p. 46. (#statistics)
Statisticians classically asked the wrong question–and were willing to answer with a lie, one that was often a downright lie. They asked “Are the effects of A and B different?” and they were willing to answer “no”. All we know about the world teaches us that the effects of A and B are always different–in some decimal place–for every A and B. Thus asking “Are the effects different?” is foolish. What we should be answering first is “Can we tell the direction in which the effects of A differ from the effects of B?” In other words, can we be confident about the direction from A to B? Is it “up”, “down” or “uncertain”?
– John Tukey, The Philosophy of Multiple Comparisons, Statistical Science, 6, 100-116. (#statistics)
No one has ever shown that he or she had a free lunch. Here, of course, “free lunch” means “usefulness of a model that is locally easy to make inferences from”.
– John Tukey, Issues relevant to an honest account of data-based inference, partially in the light of Laurie Davies’ paper. (#statistics)
If asymptotics are of any real value, it must be because they teach us something useful in finite samples. I wish I knew how to be sure when this happens.
– John Tukey, Issues relevant to an honest account of data-based inference, partially in the light of Laurie Davies’ paper. (#statistics)
George Box: We don’t need robust methods. A good statistician (particularly a Bayesian one) will model the data well and find the outliers. John Tukey: They ran over 2000 statistical analyses at Rothamsted last week and nobody noticed anything. A red light warning would be most helpful.
– George Box vs. John Tukey, Douglas Martin, 1999 S-Plus Conference Proceedings. (#statistics)
Statistics is a science in my opinion, and it is no more a branch of mathematics than are physics, chemistry, and economics; for if its methods fail the test of experience–not the test of logic–they will be discarded.
– John Tukey, The life and professional contributions of John W. Tukey, by David Brillinger, The Annals of Statistics, 2001, Vol 30. (#statistics)
One Christmas Tukey gave his students books of crossword puzzles as presents. Upon examining the books the students found that Tukey had removed the puzzle answers and had replaced them with words of the sense: “Doing statistics is like doing crosswords except that one cannot know for sure whether one has found the solution.”
– John Tukey, The life and professional contributions of John W. Tukey, by David Brillinger, The Annals of Statistics, 2001, Vol 30, p. 22. (#statistics)
A sort of question that is inevitable is: “Someone taught my students exploratory, and now (boo hoo) they want me to tell them how to assess significance or confidence for all these unusual functions of the data. Oh, what can we do?” To this there is an easy answer: TEACH them the JACKKNIFE.
– John Tukey, We Need Both Exploratory and Confirmatory, The American Statistician, Vol 34, No 1, p. 25. (#statistics)
John Tukey’s eye for detail was amazing. When we were preparing some of the material for our book (which was published last year), it was most disconcerting to have him glance at the data and question one value out of several thousand points. Of course, he was correct and I had missed identifying this anomaly.
– Kaye Basford (#statistics)
Many students are curious about the ‘1.5 x IQR Rule’;, i.e. why do we use Q1 - 1.5 x IQR (or Q3 + 1.5 x IQR) as the value for deciding if a data value is classified as an outlier? Paul Velleman, a statistician at Cornell University, was a student of John Tukey, who invented the boxplot and the 1.5 x IQR Rule. When he asked Tukey, ‘Why 1.5?’, Tukey answered, ‘Because 1 is too small and 2 is too large.’ [Assuming a Gaussian distribution, about 1 value in 100 would be an outlier. Using 2 x IQR would lead to 1 value in 1000 being an outlier.]
– Unknown (#statistics)
It is a rare thing that a specific body of data tells us as clearly as we would wish how it itself should be analyzed.
– John Tukey, Exploratory Data Analysis, p. 397. (#statistics)
Just which robust/resistant methods you use is not important–what is important is that you use some. It is perfectly proper to use both classical and robust/resistant methods routinely, and only worry when they differ enough to matter. But, when they differ, you should think hard.
– John Tukey, Quoted by Doug Martin (#statistics)
We thus echo the classical Bayesian literature in concluding that ‘noninformative prior information’ is a contradiction in terms. The flat prior carries information just like any other; it represents the assumption that the effect is likely to be large. This is often not true. Indeed, the signal-to-noise ratio s is often very low and then it is necessary to shrink the unbiased estimate. Failure to do so by inappropriately using the flat prior causes overestimation of effects and subsequent failure to replicate them.
– Erik van Zwet & Andrew Gelman, A proposal for informative default priors scaled by the standard error of estimates, The American Statistician, 76, p. 7. (#statistics,bayesian)
Another reason for the applied statistician to care about Bayesian inference is that consumers of statistical answers, at least interval estimates, commonly interpret them as probability statements about the possible values of parameters. Consequently, the answers statisticians provide to consumers should be capable of being interpreted as approximate Bayesian statements.
– Donald B. Rubin, Bayesianly justifiable and relevant frequency calculations for the applied statistician. Annals of Statistics, 12(4):1151-1172, 1984. (#statistics,bayesian)
In most cases the frequentist adopts numerical values because they are convenient in that the calculations can be easily performed. For instance, a reliability engineer will use an exponential distribution or, if that is too gross, a Weibull. In the majority of frequentist analyses there is little justification for the assumed likelihood, and it is as subjective as any prior.
– D. V. Lindley, Discussion, The American Statistician, August 1997, Vol. 51, page 265. (#statistics,bayesian)
In contrast to the logical development and intuitive interpretations of the Bayesian approach, frequentist methods are nearly impossible to understand, even for the best students. Consider confidence intervals. Many instructors err in describing confidence intervals and even some texts err. But whether texts or instructors err in explaining them, students do not understand them. And they carry this misunderstanding with them into later life. Calculating a confidence interval is easy. But everyone except the cognoscenti believes that when one calculates 95% confidence limits of 2.6 and 7.9, say, the probability is 95% that the parameter in question lies in the interval from 2.6 to 7.9. P values are nearly as obscure as confidence intervals. … Students in frequentist courses may learn very well how to calculate confidence intervals and P values, but they cannot give them correct interpretations. I stopped teaching frequentist methods when I decided that they could not be learned.
– Donald A. Berry, Teaching Elementary Bayesian Statistics with Real Applications in Science, The American Statistician, 51, p 242. (#statistics,bayesian)
Uniform priors on probabilities are ubiquitous. I agree that they can be useful. However, if the probability in question is the prevalence of HIV in California, it is ridiculous to assert that a prevalence of 100% is equally plausible with 0%, or that the chance that it is above 50% is the same as the chance that it is below 50%. It is particularly noxious to call such a prior noninformative. Instead, it is disinformative. The likelihood notwith standing, positing a uniform prior for the probability that the sun will rise tomorrow is equally ridiculous, given that we are all confident that it has risen all the days of our lives.
– Wesley O. Johnson, Comment: Bayesian Statistics in the Twenty First Century, The American Statistician, Feb 2013, 67, p 10. (#statistics,bayesian)
The best way to convey to the experimenter what the data tell him about theta is to show him a picture of the posterior distribution.
– George E. P. Box & G. C. Tiao, Bayesian Inference in Statistical Analysis (1973) (#statistics,bayesian)
If one could get some rational basis for obtaining the prior, then there would be no problem. But people have seminars these days about something where someone says, ‘I am going to use such and such a prior’. Where does he get the prior? It is not data based. It is a mathematical convenience or something like that. It is not even obtained by using Bayes’ theorem. Why one should believe the outcome of using this seems to be a very moot point.
– Oscar Kempthorne, A conversation with Oscar Kempthorne, Statistical Science, 1995, V 10, p. 333. (#statistics,bayesian)
In the design of experiments, one has to use some informal prior knowledge. How does one construct blocks in a block design problem for instance? It is stupid to think that use is not made of a prior. But knowing that this prior is utterly casual, it seems ludicrous to go through a lot of integration, etc., to obtain ‘exact’ posterior probabilities resulting from this prior. So, I believe the situation with respect to Bayesian inference and with respect to inference, in general, has not made progress. Well, Bayesian statistics has led to a great deal of theoretical research. But I don’t see any real utilizations in applications, you know. Now no one, as far as I know, has examined the question of whether the inferences that are obtained are, in fact, realized in the predictions that they are used to make.
– Oscar Kempthorne, “A conversation with Oscar Kempthorne”, Statistical Science, 1995, V 10, p. 334. (#statistics,bayesian)
I sometimes think that the only real difference between Bayesian and non-Bayesian hierarchical modelling is whether random effects are labeled with Greek or Roman letters.
– Peter Diggle, Comment on Bayesian analysis of agricultural field experiments, 1999, J. Royal Statistical Society B, 61, 691–746. (#statistics,bayesian)
The practicing Bayesian is well advised to become friends with as many numerical analysts as possible.
– James Berger, Statistical Decision Theory and Bayesian Analysis, p. 202. (#statistics,bayesian)
You just say “Bayesian,” and people think you are some kind of genius.
– Gary Churchill, Bayes offers a new way to make sense of numbers, Science, 19 Nov 1999. (#statistics,bayesian)
Bayesian computations give you a straightforward answer you can understand and use. It says there is an X% probability that your hypothesis is true-not that there is some convoluted chance that if you assume the null hypothesis is true, you’ll get a similar or more extreme result if you repeated your experiment thousands of times. How does one interpret THAT!
– Steven Goodman, Bayes offers a new way to make sense of numbers, Science, 19 Nov 1999. (#statistics,bayesian)
Bayesian methods are complicated enough, that giving researchers user-friendly software could be like handing a loaded gun to a toddler; if the data is crap, you won’t get anything out of it regardless of your political bent.
– Brad Carlin, Bayes offers a new way to make sense of numbers, Science, 19 Nov 1999. (#statistics,bayesian)
If a study, even a statistically significant one, suggests that pigs can fly, Bayes’s theorem allows researchers to combine the study’s results mathematically with hundreds of years of knowledge about the travel habits of swine.
– David Leonhardt, New York Times, April 28, 2001 (#statistics,bayesian)
If the prior distribution, at which I am frankly guessing, has little or no effect on the result, then why bother; and if it has a large effect, then since I do not know what I am doing how would I dare act on the conclusions drawn?
– Richard W Hamming, The Art of Probability for Scientists and Engineers, 1991, p. 298. (#statistics,bayesian)
I believe that there are many classes of problems where Bayesian analyses are reasonable, mainly classes with which I have little acquaintance.
– John Tukey, The life and professional contributions of John W. Tukey, The Annals of Statistics, 2001, Vol 30, p. 45. (#statistics,bayesian)
If you read Bayesian polemics from the 1970s and 1980s, including my own, it’s usually arrogant and even insulting. Some of the terms were excessively pointed. For example, Bayesians identified which frequentist methods were “incoherent”, or more accurately, lamented that none seemed to be coherent. On the other hand, Bayesians were accused of being “biased”. The rhetoric was not all that different from that of the Fisher/Pearson duels. But we Bayesians have stopped saying derogatory things, partly because we have changed and partly because frequentists have been listening. When you’re walking beside someone you tend to be cordial; when you’re trying to catch up to tell them something and they are ignoring what you say, you sometimes yell.
– Don Berry, Celebrating 70: An Interview with Don Berry, Statistical Science, 2012, Vol. 27, No. 1, 144-159. (#statistics,bayesian)
The traditional methods design of experiments are taught and/or discussed in textbooks are not the ways design of experiments are or should be used for real-world applications.
– George Milliken, Applied Statistics in Agriculture Conference, 2009 (#statistics,experimental design)
The statistician who supposes that his main contribution to the planning of an experiment will involve statistical theory, finds repeatedly that he makes his most valuable contribution simply by persuading the investigator to explain why he wishes to do the experiment, by persuading him to justify the experimental treatments, and to explain why it is that the experiment, when completed, will assist him in his research.
– Gertrude Cox, Lecture in Washington 11 January 1951 (#statistics,experimental design)
An important distinction needs to be made between experimental designs using complete blocks and those using incomplete blocks as regards to three functions: 1. reducing the error mean square 2. adjusting estimates closer to true values, and 3. refining rankings. Complete blocks include all treatments whereas incomplete blocks include a subset of the treatments. Both can reduce the residual error, but only incomplete blocks can also adjust estimates of treatment effects closer to the true values and thereby refine rankings among the treatments. These adjusted estimates are usually more accurate than the raw averages over replicates, but not always (exactly as is the case for accuracy gain through more replication). Likewise, these adjusted rankings are more likely to identify correctly the best treatments. By declaring a smaller error but doing nothing to sharpen estimates or refine rankings, complete block designs are rather impotent. It is ironic that scientists rarely understand this huge difference between getting one benefit or three from blocking.
– Hugh G Gauch, Three Strategies for Gaining Accuracy, American Scientist. (#statistics,experimental design)
On a final note, we would like to stress the importance of design, which often does not receive the attention it deserves. Sometimes, the large number of modeling options for spatial analysis may raise the false impression that design does not matter, and that a sophisticated analysis takes care of everything. Nothing could be further from the truth.
– Hans-Peter Piepho, Martin P. Boer, Emlyn R. Williams, Two-dimensional P-spline smoothing for spatial analysis of plant breeding trials, “Biometrical Journal”, Feb 2022. (#statistics,experimental design)
At this meeting for the hybrid corn industry, we have no reservation about recommending a design which consists of a single replicate of treatments at a given location. For some audiences, such a statement can severely damage the reputation of the person making the statement. University experiment station personnel in particular regard replications within an environment as a necessary part of good research. They are not. Sprague (1955) and many others have shown most researchers otherwise.
– R. E. Stucker & D. R. Hicks, Experimental Design and Plot Size Considerations for On-Farm Research, Proceedings of the 46th Annual Corn and Sorghum Industry Research Conference , 1991, p. 60. (#statistics,experimental design)
The message from a statistician’s point of view is very clear. Replicate over environments, do not replicate within environments. This is not news. At North Carolina State in the early 60s, any graduate student interested in quantitative aspects of plant breeding and genetics had a standard answer for the number of replicates needed in an experiment: use one replicate if you’re estimating means, and use two replicates if you’re estimating variances. Implicit in the answer was, “the experiment will be evaluated in more than one environment”.
– R. E. Stucker & D. R. Hicks, Experimental Design and Plot Size Considerations for On-Farm Research, Proceedings of the 46th Annual Corn and Sorghum Industry Research Conference , 1991, p. 62. (#statistics,experimental design)
Which I would like to stress are: (1) A significant effect is not necessarily the same thing as an interesting effect. (2) A non-significant effect is not necessarily the same thing as no difference.
– Christopher Chatfield, Problem solving : a statistician’s guide, p. 51. (#statistics,significance)
Rejection of a true null hypothesis at the 0.05 level will occur only one in 20 times. The overwhelming majority of these false rejections will be based on test statistics close to the borderline value. If the null hypothesis is false, the inter-ocular traumatic test [“hit between the eyes”] will often suffice to reject it; calculation will serve only to verify clear intuition.
– W. Edwards, Harold Lindman, Leonard J. Savage, Bayesian Statistical Inference for Psychological Research, University of Michigan (#statistics,significance,nhst)
When statistical inferences, such as p-values, follow extensive looks at the data, they no longer have their usual interpretation. Ignoring this reality is dishonest: it is like painting a bull’s eye around the landing spot of your arrow. This is known in some circles as p-hacking, and much has been written about its perils and pitfalls.
– Robert E Kass, Brian S. Caffo, Marie Davidian, Xiao-Li Meng, Bin Yu, Nancy Reid., Ten Simple Rules for Effective Statistical Practice, PLoS Comput Biol 12(6):e1004961. (#statistics,significance,nhst)
The difference between “statistically significant” and “not statistically significant” is not in itself necessarily statistically significant. By this, I mean more than the obvious point about arbitrary divisions, that there is essentially no difference between something significant at the 0.049 level or the 0.051 level. I have a bigger point to make. It is common in applied research–in the last couple of weeks, I have seen this mistake made in a talk by a leading political scientist and a paper by a psychologist–to compare two effects, from two different analyses, one of which is statistically significant and one which is not, and then to try to interpret/explain the difference. Without any recognition that the difference itself was not statistically significant.
– Andrew Gelman, The difference between ‘statistically significant’ and ‘not statistically significant’ is not in itself necessarily statistically significant, 2005 (#statistics,significance)
The p-value is a concept so misaligned with intuition that no civilian can hold it firmly in mind. Nor can many statisticians.
– Matt Briggs, Why do statisticians answer silly questions that no one ever asks?, Significance, Vol 9, No 1, p. 30. (#statistics,significance,nhst)
A quotation of a p-value is part of the ritual of science, a sprinkling of the holy waters in an effort to sanctify the data analysis and turn consumers of the results into true believers.
– William Cleveland, Visualizing Data, p. 177. (#statistics,significance,nhst)
We should push for de-emphasizing some topics, such as statistical significance tests–an unfortunate carry-over from the traditional elementary statistics course. We would suggest a greater focus on confidence intervals—these achieve the aim of formal hypothesis testing, often provide additional useful information, and are not as easily misinterpreted.
– Gerry Hahn et. al, The Impact of Six Sigma Improvement–A Glimpse Into the Future of Statistics, The American Statistician, August 1999. (#statistics,significance,nhst)
We statisticians must accept much of the blame for cavalier attitudes toward Type I errors. When we teach practitioners in other scientific fields that multiplicity is not important, they believe us, and feel free to thrash their data set mercilessly, until it finally screams “uncle” and relinquishes significance. The recent conversion of the term “data mining” to mean a statistical good rather than a statistical evil also contributes to the problem.
– Peter Westfall, Applied Statistics in Agriculture (Proceedings of the 13th annual conference), page 5. (#statistics,significance,nhst)
While the main emphasis in the development of power analysis has been to provide methods for assessing and increasing power, it should also be noted that it is possible to have too much power. If your sample is too large, nearly any difference, no matter how small or meaningless from a practical standpoint, will be ‘statistically significant’.
– Clay Helberg (#statistics,significance,power,nhst)
Remember that a p-value merely indicates the probability of a particular set of data being generated by the null model–it has little to say about the size of a deviation from that model (especially in the tails of the distribution, where large changes in effect size cause only small changes in p-values).
– Clay Helberg (#statistics,significance,nhst)
Given what I know about data, models, and assumptions, I find more than 2 significant digits of printout for a p-value to be indefensible. (I actually think 1 digit is about the max).
– Terry Therneau, S-news mailing list, 8 Nov 2000 (#statistics,significance)
In the calculus of real statistical inference, and by that I mean actual data problems (which S was designed for), all p-values < 10^-6 or so are identical. This is one of the few areas in fact where I like SAS better: the creators of their PROCs are smart enough to print these numbers as zero and leave it at that. There are no Gaussian distributions in the real world, and the central limit theorem has failed long, long before 10^-17.
– Terry Therneau, S-news mailing list, 4 Apr 2002 (#statistics,significance,computing)
It’s a commonplace among statisticians that a chi-squared test (and, really, any p-value) can be viewed as a crude measure of sample size: When sample size is small, it’s very difficult to get a rejection (that is, a p-value below 0.05), whereas when sample size is huge, just about anything will bag you a rejection. With large n, a smaller signal can be found amid the noise. In general: small n, unlikely to get small p-values. Large n, likely to find something. Huge n, almost certain to find lots of small p-values.
– Andrew Gelman, The sample size is huge, so a p-value of 0.007 is not that impressive, 2009. (#statistics,significance,nhst)
Work by Bickel, Ritov, and Stoker (2001) shows that goodness-of-fit tests have very little power unless the direction of the alternative is precisely specified. The implication is that omnibus goodness-of-fit tests, which test in many directions simultaneously, have little power, and will not reject until the lack of fit is extreme.
– Leo Breiman, Statistical Modeling: The Two Cultures, Statistical Science, Vol 16, p. 203. (#statistics,significance,nhst)
Visualizations act as a campfire around which we gather to tell stories.
– Al Shalloway, 2011 (#eda,data visualization)
If students have students have no experience with hands-on [telescope] observing, they may take all data as ‘truth’ without having an understanding of how the data are obtained and what could potentially go wrong in that process, so I think it becomes crucially important to give a glimpse of what’s happening behind the scenes at telescopes, so they can be appropriately skeptical users of data in the future.
– Colette Salyk, Sky & Telescope, Apr 2022 p. 31. (#data)
Normality is a myth; there never was, and never will be, a normal distribution. This is an over-statement from the practical point of view, but it represents a safer initial mental attitude than any in fashion during the past two decades.
– R. C. Geary, Testing for normality, 1947. Biometrika 34 : 209-242. (#nhst,normality)
Furthermore, the mere declaration that the interaction is or is not significant is far too coarse a result to give agronomists or plant breeders effective insight into their research material.
– Hugh G. Gauch Jr., Model selection and validation for yield trials with interaction, 1988. Biometrics 44 : 705-715.
Analysis of variance … stems from a hypothesis-testing formulation that is difficult to take seriously and would be of limited value for making final conclusions.
– Herman Chernoff, Comment, 1986. The American Statistician 40(1) : 5-6. (#nhst)
The peculiarity of … statistical hypotheses is that they are not conclusively refutable by any experience.
– Richard B. Braithwaite, Scientific Explanation. A Study of the Function of Theory, Probability and Law in Science (p. 151), 1953. Cambridge University Press. (#nhst)
…“no batch of observations, however large, either definitively rejects or definitively fails to reject the hypothesis H0.
– Richard B. Braithwaite, Scientific Explanation. A Study of the Function of Theory, Probability and Law in Science (p. 160), 1953. Cambridge University Press. (#nhst)
what John Dewey called ‘the quest for certainty’ is, in the case of empirical knowledge, a snare and a delusion.
– Richard B. Braithwaite, Scientific Explanation. A Study of the Function of Theory, Probability and Law in Science (p. 163), 1953. Cambridge University Press. (#knowledge,uncertainty)
The ultimate justification for any scientific belief will depend upon the main purpose for which we think scientifically–that of predicting and thereby controlling the future.
– Richard B. Braithwaite, Scientific Explanation. A Study of the Function of Theory, Probability and Law in Science (p. 174), 1953. Cambridge University Press. (#science)
Most readers of The American Statistician will recognize the limited value of hypothesis testing in the science of statistics. I am not sure that they all realize the extent to which it has become the primary tool in the religion of Statistics.
– David Salsburg, The Religion of Statistics as Practiced in Medical Journals, 1985. The American Statistician, 39, 220-223. (#nhst)
We are better off abandoning the use of hypothesis tests entirely and concentrating on developing continuous measures of toxicity which can be used for estimation.
– David Salsburg, Statistics for Toxicologists, 1986. New York, Marcel Dekker, Inc. (#nhst)
I do not think that significance testing should be completely abandoned … and I don’t expect that it will be. But I urge researchers to provide estimates, with confidence intervals: scientific advance requires parameters with known reliability estimates. Classical confidence intervals are formally equivalent to a significance test, but they convey more information.
– Nigel G. Yoccoz, Use, Overuse, and Misuse of Significance Tests in Evolutionary Biology and Ecology. Bulletin of the Ecological Society of America, Vol. 72, No. 2 (Jun., 1991), pp. 106-111. (#nhst,significance,uncertainty)
In marked contrast to what is advocated by most statisticians, most evolutionary biologists and ecologists overemphasize the potential role of significance testing in their scientific practice. Biological significance should be emphasized rather than statistical significance. Furthermore, a survey of papers showed that the literature is infiltrated by an array of misconceptions about the use and interpretation of significance tests. … By far the most common error is to confound statistical significance with biological, scientific significance… Statements like ‘the two populations are significantly different relative to parameter X (P=.004)’ are found with no mention of the estimated difference. … Most biologists and other users of statistical methods still seem to be unaware that significance testing by itself sheds little light on the questions they are posing.
– Nigel G. Yoccoz, Use, Overuse, and Misuse of Significance Tests in Evolutionary Biology and Ecology. Bulletin of the Ecological Society of America, Vol. 72, No. 2 (Jun., 1991), pp. 106-111. (#nhst,significance)
Tests appear to many users to be a simple way to discharge the obligation to provide some statistical treatment of the data.
– H. V. Roberts, For what use are tests of hypotheses and tests of significance, 1976. Communications in Statistics, Series A, 5:753-761. (#nhst)
We shall marshal arguments against [significance] testing, leading to the conclusion that it be abandoned by all substantive science and not just by educational research and other social sciences which have begun to raise voices against the virtual tyranny of this branch of inference in the academic world.
– Louis Guttman, The illogic of statistical inference for cumulative science, 1985. Applied Stochastic Models and Data Analysis 1:3-9. (#nhst)
In practice, of course, tests of significance are not taken seriously.
– Louis Guttman, The illogic of statistical inference for cumulative science, 1985. Applied Stochastic Models and Data Analysis 1:3-9. (#nhst)
Since a point hypothesis is not to be expected in practice to be exactly true, but only approximate, a proper test of significance should almost always show significance for large enough samples. So the whole game of testing point hypotheses, power analysis notwithstanding, is but a mathematical game without empirical importance.
– Louis Guttman, The illogic of statistical inference for cumulative science, 1985. Applied Stochastic Models and Data Analysis 1:3-9. (#nhst)
…lack of interaction in analysis of variance and … lack of correlation in bivariate distributions–such nullities would be quite surprising phenomena in the usual interactive complexities of social life.
– Louis Guttman, What is not what in statistics, 1977. The Statistician, 26:81-107. (#nhst,correlation)
Estimation and approximation may be more fruitful than significance in developing science, never forgetting replication.
– Louis Guttman, What is not what in statistics, 1977. The Statistician, 26:81-107. (#nhst)
[the normal distribution] is seldom, if ever, observed in nature.
– Louis Guttman, What is not what in statistics, 1977. The Statistician, 26:81-107. (#normality)
The test of statistical significance in psychological research may be taken as an instance of a kind of essential mindlessness in the conduct of research.
– D. Bakan, The test of significance in psychological research, 1966. Psychological Bulletin 66: 423-437. (#nhst)
…the test of significance has been carrying too much of the burden of scientific inference. It may well be the case that wise and ingenious investigators can find their way to reasonable conclusions from data because and in spite of their procedures. Too often, however, even wise and ingenious investigators…tend to credit the test of significance with properties it does not have.
– D. Bakan, The test of significance in psychological research, 1966. Psychological Bulletin 66: 423-437. (#nhst)
…a priori reasons for believing that the null hypothesis is generally false anyway. One of the common experiences of research workers is the very high frequency with which significant results are obtained with large samples.
– D. Bakan, The test of significance in psychological research, 1966. Psychological Bulletin 66: 423-437. (#nhst)
…there is really no good reason to expect the null hypothesis to be true in any population … Why should any correlation coefficient be exactly .00 in the population? … why should different drugs have exactly the same effect on any population parameter?
– D. Bakan, The test of significance in psychological research, 1966. Psychological Bulletin 66: 423-437. (#nhst)
…we need to get on with the business of generating … hypotheses and proceed to do investigations and make inferences which bear on them, instead of … testing the statistical null hypothesis in any number of contexts in which we have every reason to suppose that it is false in the first place.
– D. Bakan, The test of significance in psychological research, 1966. Psychological Bulletin 66: 423-437. (#nhst)
the tests of null hypotheses of zero differences, of no relationships, are frequently weak, perhaps trivial statements of the researcher’s aims … in many cases, instead of the tests of significance it would be more to the point to measure the magnitudes of the relationships, attaching proper statements of their sampling variation. The magnitudes of relationships cannot be measured in terms of levels of significance.
– Leslie Kish, Some statistical problems in research design, 1959. American Sociological Review 24: 328-338. (#nhst)
There are instances of research results presented in terms of probability values of ‘statistical significance’ alone, without noting the magnitude and importance of the relationships found. These attempts to use the probability levels of significance tests as measures of the strengths of relationships are very common and very mistaken.
– Leslie Kish, Some statistical problems in research design, 1959. American Sociological Review 24: 328-338. (#nhst)
One reason for preferring to present a confidence interval statement (where possible) is that the confidence interval, by its width, tells more about the reliance that can be placed on the results of the experiment than does a YES-NO test of significance.
– Mary G. Natrella, The relation between confidence intervals and tests of significance, 1960. American Statistician 14 : 20-22, 33. (#nhst)
Confidence intervals give a feeling of the uncertainty of experimental evidence, and (very important) give it in the same units … as the original observations.
– Mary G. Natrella, The relation between confidence intervals and tests of significance, 1960. American Statistician 14 : 20-22, 33. (#nhst)
The current obsession with .05 … has the consequence of differentiating significant research findings and those best forgotten, published studies from unpublished ones, and renewal of grants from termination. It would not be difficult to document the joy experienced by a social scientist when his F ratio or t value yields significance at .05, nor his horror when the table reads ‘only’ .10 or .06. One comes to internalize the difference between .05 and .06 as ‘right’ vs. ‘wrong,’ ‘creditable’ vs. ‘embarrassing,’ ‘success’ vs. ‘failure’.
– James K. Skipper Jr., Anthony L. Guenther and Gilbert Nass, The sacredness of .05: A note concerning the uses of statistical levels of significance in social science. The American Sociologist 2 : 16-18. (#nhst)
…blind adherence to the .05 level denies any consideration of alternative strategies, and it is a serious impediment to the interpretation of data”
– James K. Skipper Jr., Anthony L. Guenther and Gilbert Nass, The sacredness of .05: A note concerning the uses of statistical levels of significance in social science. The American Sociologist 2 : 16-18. (#nhst)
… surely, God loves the .06 nearly as much as the .05.
– R. L. Rosnow and R. Rosenthal, Statistical procedures and the justification of knowledge and psychological science, 1989. American Psychologist 44: 1276-1284. (#nhst)
How has the virtually barren technique of hypothesis testing come to assume such importance in the process by which we arrive at our conclusions from our data?
– G. R. Loftus, On the tyranny of hypothesis testing in the social sciences, 1991. Contemporary Psychology 36: 102-105. (#nhst)
Despite the stranglehold that hypothesis testing has on experimental psychology, I find it difficult to imagine a less insightful means of transitting from data to conclusions.
– G. R. Loftus, On the tyranny of hypothesis testing in the social sciences. Contemporary Psychology 36: 102-105. (#nhst)
Whereas hypothesis testing emphasizes a very narrow question (‘Do the population means fail to conform to a specific pattern?’), the use of confidence intervals emphasizes a much broader question (‘What are the population means?’). Knowing what the means are, of course, implies knowing whether they fail to conform to a specific pattern, although the reverse is not true. In this sense, use of confidence intervals subsumes the process of hypothesis testing.
– G. R. Loftus, On the tyranny of hypothesis testing in the social sciences. Contemporary Psychology 36: 102-105. (#nhst)
This remarkable state of affairs [overuse of significance testing] is analogous to engineers’ teaching (and believing) that light consists only of waves while ignoring its particle characteristics—and losing in the process, of course, any motivation to pursue the most interesting puzzles and paradoxes in the field.
– G. R. Loftus, On the tyranny of hypothesis testing in the social sciences, 1991. Contemporary Psychology 36: 102-105. (#nhst)
The result is that non-statisticians tend to place undue reliance on single ‘cookbook’ techniques, and it has for example become impossible to get results published in some medical, psychological and biological journals without reporting significance values even if of doubtful validity. It is sad that students may actually be more confused and less numerate at the end of a ‘service course’ than they were at the beginning, and more likely to overlook a descriptive approach in favour of some inferential method which may be inappropriate or incorrectly executed.
– C. Chatfield, The initial examination of data, 1985. Journal of the Royal Statistical Society, Series A 148: 214-253. (#nhst)
‘Common sense’ is not common but needs to learnt systematically… A ‘simple analysis’ can be harder than it looks…. All statistical techniques, however sophisticated, should be subordinate to subjective judgement.
– C. Chatfield, The initial examination of data, 1985. Journal of the Royal Statistical Society, Series A 148: 214-253. (#nhst)
Thus statistics should generally be taught more as a practical subject with analyses of real data. Of course some theory and an appropriate range of statistical tools need to be learnt, but students should be taught that Statistics is much more than a collection of standard prescriptions.
– C. Chatfield, The initial examination of data, 1985. Journal of the Royal Statistical Society, Series A 148: 214-253. (#data)
More fundamentally students should be taught that instead of asking ‘What techniques shall I use here?,’ they should ask ‘How can I summarize and understand the main features of this set of data?’
– C. Chatfield, The initial examination of data, 1985. Journal of the Royal Statistical Society, Series A 148: 214-253. (#data)
All statistical techniques, however sophisticated, should be subordinate to subjective judgement.
– C. Chatfield, The initial examination of data, 1985. Journal of the Royal Statistical Society, Series A 148: 214-253.
…it has … become impossible to get results published in some medical, psychological and biological journals without reporting significance values even when of doubtful validity.
– C. Chatfield, The initial examination of data, 1985. Journal of the Royal Statistical Society, Series A 148: 214-253. (#nhst)
…to make measurements and then ignore their magnitude would ordinarily be pointless. Exclusive reliance on tests of significance obscures the fact that statistical significance does not imply substantive significance.
– I. R. Savage, Nonparametric Statistics. Journal of the American Statistical Association, 52, 331-344. (#nhst)
Null hypotheses of no difference are usually known to be false before the data are collected … when they are, their rejection or acceptance simply reflects the size of the sample and the power of the test, and is not a contribution to science”
– I. R. Savage, Nonparametric Statistics. Journal of the American Statistical Association, 52, 331-344. (#nhst)
too many users of the analysis of variance seem to regard the reaching of a mediocre level of significance as more important than any descriptive specification of the underlying averages Our thesis is that people have strong intuitions about random sampling; that these intuitions are wrong in fundamental respects; that these intuitions are shared by naive subjects and by trained scientists; and that they are applied with unfortunate consequences in the course of scientific inquiry. We submit that people view a sample randomly drawn from a population as highly representative, that is, similar to the population in all essential characteristics. Consequently, they expect any two samples drawn from a particular population to be more similar to one another and to the population than sampling theory predicts, at least for small samples.
– Amos Tversky & Daniel Kahneman, Belief in the law of small numbers. Psychological Bulletin, 76(2), 105-110. (#sampling)
People have erroneous intuitions about the laws of chance. In particular, they regard a sample randomly drawn from a population as highly representative, that is, similar to the population in all essential characteristics. The prevalence of the belief and its unfortunate consequences for psychological research are illustrated by the responses of professional psychologists to a questionnaire concerning research decisions
– Amos Tversky & Daniel Kahneman, Belief in the law of small numbers. Psychological Bulletin, 76(2), 105-110. (#sampling)
the statistical power of many psychological studies is ridiculously low. This is a self-defeating practice: it makes for frustrated scientists and inefficient research. The investigator who tests a valid hypothesis but fails to obtain significant results cannot help but regard nature as untrustworthy or even hostile.
– Amos Tversky & Daniel Kahneman, Belief in the law of small numbers. Psychological Bulletin, 76(2), 105-110. (#nhst,power)
Significance levels are usually computed and reported, but power and confidence limits are not. Perhaps they should be.
– Amos Tversky & Daniel Kahneman, Belief in the law of small numbers. Psychological Bulletin, 76(2), 105-110. (#nhst)
The emphasis on significance levels tends to obscure a fundamental distinction between the size of an effect and its statistical significance.
– Amos Tversky & Daniel Kahneman, Belief in the law of small numbers. Psychological Bulletin, 76(2), 105-110. (#nhst)
Statistical hypothesis testing is commonly used inappropriately to analyze data, determine causality, and make decisions about significance in ecological risk assessment,… It discourages good toxicity testing and field studies, it provides less protection to ecosystems or their components that are difficult to sample or replicate, and it provides less protection when more treatments or responses are used. It provides a poor basis for decision-making because it does not generate a conclusion of no effect, it does not indicate the nature or magnitude of effects, it does address effects at untested exposure levels, and it confounds effects and uncertainty…. Risk assessors should focus on analyzing the relationship between exposure and effects….
– Glenn W. Suter, Abuse of hypothesis testing statistics in ecological risk assessment, 1996. Human and Ecological Risk Assessment 2: 331-347. (#nhst)
I argued that hypothesis testing is fundamentally inappropriate for ecological risk assessment, that its use has undesirable consequences for environmental protection, and that preferable alternatives exist for statistical analysis of data in ecological risk assessment. The conclusion of this paper is that ecological risk assessors should estimate risks rather than test hypothesis
– Glenn W. Suter, Abuse of hypothesis testing statistics in ecological risk assessment, 1996. Human and Ecological Risk Assessment 2: 331-347. (#nhst)
The purpose of an experiment is to answer questions. The truth of this seems so obvious, that it would not be worth emphasizing were it not for the fact that the results of many experiments are interpreted and presented with little or no reference to the questions that were asked in the first place.
– T. M. Little, Interpretation and presentation of results, 1981. Hortscience 16: 637-640.
The idea that one should proceed no further with an analysis, once a non-significant F-value for treatments is found, has led many experimenters to overlook important information in the interpretation of their data.
– T. M. Little, Interpretation and presentation of results, 1981. Hortscience 16: 637-640. (#nhst,anova)
the null-hypothesis models … share a crippling flaw: in the real world the null hypothesis is almost never true, and it is usually nonsensical to perform an experiment with the sole aim of rejecting the null hypothesis.
– Jum Nunnally, The place of statistics in psychology, 1960. Educational and Psychological Measurement 20 : 641-650. (#nhst)
If rejection of the null hypothesis were the real intention in psychological experiments, there usually would be no need to gather data.
– Jum Nunnally, The place of statistics in psychology, 1960. Educational and Psychological Measurement 20 : 641-650. (#nhst)
Closely related to the null hypothesis is the notion that only enough subjects need be used in psychological experiments to obtain ‘significant’ results. This often encourages experimenters to be content with very imprecise estimates of effects.
– Jum Nunnally, The place of statistics in psychology, 1960. Educational and Psychological Measurement 20 : 641-650. (#nhst)
We should not feel proud when we see the psychologist smile and say ‘the correlation is significant beyond the .01 level.’ Perhaps that is the most that he can say, but he has no reason to smile.
– Jum Nunnally, The place of statistics in psychology, 1960. Educational and Psychological Measurement 20 : 641-650. (#nhst)
the finding of statistical significance is perhaps the least important attribute of a good experiment.
– D. T. Lykken, Statistical significance in psychological research, 1968. Psychological Bulletin 70 : 151-159. (#nhst)
Editors must be bold enough to take responsibility for deciding which studies are good and which are not, without resorting to letting the p value of the significance tests determine this decision.
– D. T. Lykken, Statistical significance in psychological research, 1968. Psychological Bulletin 70 : 151-159. (#nhst)
Statistical significance testing has involved more fantasy than fact. The emphasis on statistical significance over scientific significance in educational research represents a corrupt form of the scientific method. Educational research would be better off if it stopped testing its results for statistical significance.
– R. P. Carver, The case against statistical testing. Harvard Educational Review 48: 378-399. (#nhst)
Statistical significance ordinarily depends upon how many subjects are used in the research. the more subjects the researcher uses, the more likely the researcher will be to get statistically significant results.
– R. P. Carver, The case against statistical testing. Harvard Educational Review 48: 378-399. (#nhst)
What is the probability of obtaining a dead person (D) given that the person was hanged (H); that is, in symbol form, what is p(D|H)? Obviously, it will be very high, perhaps .97 or higher. Now, let us reverse the question: What is the probability that a person has been hanged (H) given that the person is dead (D); that is, what is p(H|D)? This time the probability will undoubtedly be very low, perhaps .01 or lower. No one would be likely to make the mistake of substituting the first estimate (.97) for the second (.01); that is, to accept .97 as the probability that a person has been hanged given that the person is dead. Even thought this seems to be an unlikely mistake, it is exactly the kind of mistake that is made with the interpretation of statistical significance testing—by analogy, calculated estimates of p(H|D) are interpreted as if they were estimates of p(D|H), when they are clearly not the same.
– R. P. Carver, The case against statistical testing. Harvard Educational Review 48: 378-399. (#nhst,probability)
The author recommends abandoning all statistical significance testing and suggests other ways of evaluating research results.” … “Another reason for the popularity of statistical significance testing is probably the complicated mathematical procedures lend an error of scientific objectivity to conclusions.” … “Given that statistical significance testing usually involves a corrupt form of the scientific method and, at best, is of trivial scientific importance, journal editors should not require it as a necessary part of a publishable research article.
– R. P. Carver, The case against statistical testing. Harvard Educational Review 48: 378-399. (#nhst)
Pencil and paper for construction of distributions, scatter diagrams, and run-charts to compare small groups and to detect trends, are more efficient methods of estimation than statistical inference that depends on variances and standard errors, as the simple techniques preserve the information in the original data.
– W. Edwards Deming, On probability as a basis for action, 1975. American Statistician 29: 146-152. (#eda)
We admit with Sir Winston Churchill that it sometimes pays to admit the obvious: we do not perform an experiment to find out if two varieties of wheat or two drugs are equal. We know in advance without spending a dollar on an experiment that they are not equal. The difference between two treatments or between two areas or two groups of people, will show up as ‘significantly different’ if the experiment be conducted through a sufficient number of trials, even thought the difference be so small that it is of no scientific or economic consequence. Likewise tests of whether the data of a survey or an experiment fit some particular curve is of no scientific or economic consequence…. With enough data no curve will fit the results of an experiment. The question that one faces in using any curve or any relationship is this: how robust are the conclusions? Would some other curve make safer predictions? Statistical significance of B/A thus conveys no knowledge, no basis for action.
– W. Edwards Deming, On probability as a basis for action, 1975. American Statistician 29: 146-152. (#significance)
Under the usual teaching, the trusting student, to pass the course must forsake all the scientific sense that he has accumulated so far, and learn the book, mistakes and all.
– W. Edwards Deming, On probability as a basis for action, 1975. American Statistician 29: 146-152. (#science)
While [Edward C. Bryant] was at the University of Wyoming, someone came in from the Department of Animal Husbandry to announce to him an astounding scientific discovery—the fibres on the left side of the sheep and those on the right side are of different diameter. Dr. Bryant asked him how many fibres he had in the sample: answer, 50,000. This was a number big enough to establish significance. But what of it? Anyone would know in advance, without spending a dollar, that there is a difference between fibres of the left side and the right side of any sheep, or of n sheep combined. The question is whether the difference is of scientific importance.
– W. Edwards Deming, On probability as a basis for action, 1975. American Statistician 29: 146-152. (#nhst,science)
Small wonder that students have trouble [with statistical hypothesis testing]. They may be trying to think.
– W. Edwards Deming, On probability as a basis for action, 1975. American Statistician 29: 146-152. (#nhst,significance)
Data analysis methods in psychology still emphasize statistical significance testing, despite numerous articles demonstrating its severe deficiencies. It is now possible to use meta-analysis to show that reliance on significance testing retards the development of cumulative knowledge. The reform of teaching and practice will also require that researchers learn that the benefits that they believe flow from use of significance testing are illusory. Teachers must re-vamp their courses to bring students to understand that a) reliance on significance testing retards the growth of cumulative research knowledge; b) benefits widely believed to flow from significance testing do not in fact exist; c) significance testing methods must be replaced with point estimates and confidence intervals in individual studies and with meta-analyses and the integration of multiple studies. This reform is essential to the future progress of cumulative knowledge and psychological research.
– Frank L. Schmidt, Statistical significance testing and cumulative knowledge in psychology: implications for training of researchers. Psychological Methods 1(2), Jun 1996, 115-129. (#nhst,significance,knowledge)
If the null hypothesis is not rejected, Fisher’s position was that nothing could be concluded. But researchers find it hard to go to all the trouble of conducting a study only to conclude that nothing can be concluded.
– Frank L. Schmidt, Statistical significance testing and cumulative knowledge in psychology: implications for training of researchers. Psychological Methods 1(2), Jun 1996, 115-129. (#nhst,significance)
Many researchers believe that statistical significance testing confers important benefits that are in fact completely imaginary.
– Frank L. Schmidt, Statistical significance testing and cumulative knowledge in psychology: implications for training of researchers. Psychological Methods 1(2), Jun 1996, 115-129. (#nhst,significance)
An important part of the explanation [of continued use of significance testing] is that researchers hold false beliefs about significance testing, beliefs that tell them that significance testing offers important benefits to researchers that it in fact does not. Three of these beliefs are particularly important. The first is the false belief that the significance level of a study indicates the probability of successful replications of the study…. A second false belief widely held by researchers is that statistical significance level provides an index of the importance or size of a difference or relation…. The third false belief held by many researchers is the most devastating of all to the research enterprise. This is the belief that if a difference or relation is not statistically significant, then it is zero, or at least so small that it can safely be considered to be zero. This is the belief that if the null hypothesis is not rejected then it is to be accepted. This is the belief that a major benefit from significance tests is that they tell us whether a difference or affect is real or ‘probably just occurred by chance’.
– Frank L. Schmidt, Statistical significance testing and cumulative knowledge in psychology: Implications for training of researchers. Psychological Methods 1(2), Jun 1996, 115-129. (#nhst,significance)
We can no longer tolerate a situation in which our upcoming generation of researchers are being trained to use discredited data analysis methods while the broader research enterprise of which they are to become a part has moved toward improved methods.
– Frank L. Schmidt, Statistical significance testing and cumulative knowledge in psychology: implications for training of researchers. Psychological Methods 1(2), Jun 1996, 115-129. (#nhst)
I believe … that hypothesis testing has been greatly overemphasized in psychology and in the other disciplines that use it. It has diverted our attention from crucial issues. Mesmerized by a single all-purpose, mechanized, ‘objective’ ritual in which we convert numbers into other numbers and get a yes-no answer, we have come to neglect close scrutiny of where the numbers come from.
– Jacob Cohen, Things I have learned (so far), 1990. American Psychologist 45: 1304-1312. (#nhst,significance)
… the primary product of a research inquiry is one or more measures of effect size, not p values.
– Jacob Cohen, Things I have learned (so far), 1990. American Psychologist 45: 1304-1312. (#nhst,significance)
The prevailing yes-no decision at the magic .05 level from a single research is a far cry from the use of informed judgment. Science simply doesn’t work that way. A successful piece of research doesn’t conclusively settle an issue, it just makes some theoretical proposition to some degree more [or less] likely.
– Jacob Cohen, Things I have learned (so far), 1990. American Psychologist 45: 1304-1312. (#nhst)
One of the things I learned early on was that some things you learn aren’t so.
– Jacob Cohen, Things I have learned (so far), 1990. American Psychologist 45: 1304-1312.
When a Fisherian null hypothesis is rejected with an associated probability of, for example, .026, it is not the case that the probability that the null hypothesis is true is .026 (or less than .05, or any other value we can specify). Given our framework of probability as long-run relative frequency–as much as we might wish it to be otherwise–this result does not tell us about the truth of the null hypothesis, given the data. (For this we have to go to Bayesian or likelihood statistics, in which probability is not relative frequency but degree of belief.
– Jacob Cohen, Things I have learned (so far), 1990. American Psychologist 45: 1304-1312. (#nhst,significance)
Despite widespread misconceptions to the contrary, the rejection of a given null hypothesis gives us no basis for estimating the probability that a replication of the research will again result in rejecting that null hypothesis.
– Jacob Cohen, Things I have learned (so far), 1990. American Psychologist 45: 1304-1312. (#nhst,significance)
Of course, everyone knows that failure to reject the Fisherian null hypothesis does not warrant the conclusion that it is true. Fisher certainly knew and emphasized it, and our textbooks duly so instruct us. Yet how often do we read in the discussion and conclusions of articles now appearing in our most prestigious journals that ‘there is no difference’ or ‘no relationship’.
– Jacob Cohen, Things I have learned (so far), 1990. American Psychologist 45: 1304-1312. (#nhst)
A little thought reveals a fact widely understood among statisticians: The null hypothesis, taken literally (and that’s the only way you can take it in formal hypothesis testing), is always false in the real world…. If it is false, even to a tiny degree, it must be the case that a large enough sample will produce a significant result and lead to its rejection. So if the null hypothesis is always false, what’s the big deal about rejecting it.
– Jacob Cohen, Things I have learned (so far), 1990. American Psychologist 45: 1304-1312. (#nhst)
I am, however, appalled by the fact that some publishers of statistics packages successfully hawk their wares with the pitch that it isn’t necessary to understand statistics to use them.
– Jacob Cohen, Things I have learned (so far), 1990. American Psychologist 45: 1304-1312. (#computing)
I argue herein that NHST [null hypothesis significance testing] has not only failed to support the advance of psychology as a science but also has seriously impeded it.
– Jacob Cohen, The earth is round (p<.05). 1994. American Psychologist 49: 997-1003. (#nhst)
they [confidence limits] are rarely to be found in the literature. I suspect that the main reason they are not reported is that they are so embarrassingly large!
– Jacob Cohen, The earth is round (p<.05). 1994. American Psychologist 49: 997-1003. (#nhst)
After four decades of severe criticism, the ritual of null hypothesis significance testing—mechanical dichotomous decisions around a sacred .05 criterion—still persist. This article reviews the problems with this practice…” … “What’s wrong with [null hypothesis significance testing]? Well, among many other things, it does not tell us what we want to know, and we so much want to know what we want to know that, out of desperation, we nevertheless believe that it does!
– Jacob Cohen, The earth is round (p<.05). 1994. American Psychologist 49: 997-1003. (#nhst)
Tests of the null hypothesis that there is no difference between certain treatments are often made in the analysis of agricultural or industrial experiments in which alternative methods or processes are compared. Such tests are … totally irrelevant. What are needed are estimates of magnitudes of effects, with standard errors.
– F. J. Anscombe, Discussion on Dr. David’s and Dr. Johnson’s Paper. 1956. Journal of the Royal Statistical Society B 18 : 24-27. (#nhst)
statistical significance is not the same as scientific significance.
– Norman S. Matloff, Statistical hypothesis testing: problems and alternatives. 1991. Environmental Entomology 20 : 1246-1250. (#nhst)
the number of stars by itself is relevant only to the question of whether H0 is exactly true–a question which is almost always not of interest to us, especially because we usually know a priori that H0 cannot be exactly true.
– Norman S. Matloff, Statistical hypothesis testing: problems and alternatives. 1991. Environmental Entomology 20 : 1246-1250. (#nhst,anova)
no population has an exact normal distribution, nor are variances exactly homogeneous, and independence assumptions are often violated to at least some degree.
– Norman S. Matloff, Statistical hypothesis testing: problems and alternatives. 1991. Environmental Entomology 20 : 1246-1250. (#normality)
Exact truth of a null hypothesis is very unlikely except in a genuine uniformity trial.
– David R. Cox, Some problems connected with statistical inference. 1958. Annals of Mathematical Statistics 29 : 357-372. (#nhst)
Assumptions that we make, such as those concerning the form of the population sampled, are always untrue.
– David R. Cox, Some problems connected with statistical inference. 1958. Annals of Mathematical Statistics 29 : 357-372. (#sampling)
Overemphasis on tests of significance at the expense especially of interval estimation has long been condemned.
– David R. Cox, The role of significance tests. 1977. Scandanavian Journal of Statistics 4: 49-70. (#nhst)
…There are considerable dangers in overemphasizing the role of significance tests in the interpretation of data.
– David R. Cox, The role of significance tests. 1977. Scandanavian Journal of Statistics 4: 49-70. (#nhst)
In any particular application, graphical or other informal analysis may show that consistency or inconsistency with H0 is so clear cut that explicit calculation of p is unnecessary.
– David R. Cox, The role of significance tests. 1977. Scandanavian Journal of Statistics 4: 49-70. (#nhst)
The central point is that statistical significance is quite different from scientific significance and that therefore estimation …of the magnitude of effects is in general essential regardless of whether statistically significant departure from the null hypothesis is achieved.
– David R. Cox, The role of significance tests. 1977. Scandanavian Journal of Statistics 4: 49-70. (#nhst)
It is very bad practice to summarise an important investigation solely by a value of P.
– David R. Cox, Statistical significance tests. 1982. British Journal of Clinical Pharmacology 14 : 325-331. (#nhst)
The criterion for publication should be the achievement of reasonable precision and not whether a significant effect has been found.
– David R. Cox, Statistical significance tests. 1982. British Journal of Clinical Pharmacology 14 : 325-331. (#nhst)
The continued very extensive use of significance tests is alarming.
– David R. Cox, Some general aspects of the theory of statistics. 1986. International Statistical Review 54: 117-126. (#nhst)
It has been widely felt, probably for thirty years and more, that significance tests are overemphasized and often misused and that more emphasis should be put on estimation and prediction. While such a shift of emphasis does seem to be occurring, for example in medical statistics, the continued very extensive use of significance tests is on the one hand alarming and on the other evidence that they are aimed, even if imperfectly, at some widely felt need.
– David R. Cox, Some general aspects of the theory of statistics. 1986. International Statistical Review 54: 117-126. (#nhst)
the emphasis given to formal tests of significance … has resulted in … an undue concentration of effort by mathematical statisticians on investigations of tests of significance applicable to problems which are of little or no practical importance … and … it has caused scientific research workers to pay undue attention to the results of the tests of significance … and too little to the estimates of the magnitude of the effects they are investigating.
– Frank Yates, The influence of Statistical Methods for Research Workers on the development of the science of statistics. 1951. Journal of the American Statistical Association 46: 19-34. (#nhst)
…the unfortunate consequence that scientific workers have often regarded the execution of a test of significance on an experiment as the ultimate objective.
– Frank Yates, The influence of Statistical Methods for Research Workers on the development of the science of statistics. 1951. Journal of the American Statistical Association 46: 19-34. (#nhst)
[Researchers] pay undue attention to the results of tests of significance they perform on their data, particularly data derived from experiments, and too little to the estimates of the magnitude of the effects which they are investigating…. The emphasis on tests of significance, and the consideration of the results of each experiment in isolation, have had the unfortunate consequence that scientific workers have often regarded the execution of a test of significance on an experiment as the ultimate objective. Results are significant or not and that is the end to it.
– Frank Yates, The influence of Statistical Methods for Research Workers on the development of the science of statistics. 1951. Journal of the American Statistical Association 46: 19-34. (#nhst)
The most commonly occurring weakness … is … undue emphasis on tests of significance, and failure to recognise that in many types of experimental work estimates of treatment effects, together with estimates of the errors to which they are subject, are the quantities of primary interest.
– Frank Yates, Sir Ronald Fisher and the design of experiments. 1964. Biometrics 20: 307-321. (#nhst)
In many experiments … it is known that the null hypothesis … is certainly untrue.
– Frank Yates, Sir Ronald Fisher and the design of experiments. 1964. Biometrics 20: 307-321. (#nhst)
A common misconception is that an effect exists only if it is statistically significant and that it does not exist if it is not [statistically significant].
– Jonas Ranstam, A common misconception about p-value and its consequences. 1996. Acta Orthopaedica Scandinavica 67 : 505-507. (#nhst)
I contend that the general acceptance of statistical hypothesis testing is one of the most unfortunate aspects of 20th century applied science. Tests for the identity of population distributions, for equality of treatment means, for presence of interactions, for the nullity of a correlation coefficient, and so on, have been responsible for much bad science, much lazy science, and much silly science. A good scientist can manage with, and will not be misled by, parameter estimates and their associated standard errors or confidence limits.
– Marks Nester, A Myopic View and History of Hypothesis Testing. (#nhst)
The scientist must always give due thought to the statistical analysis, but must never let statistical analysis be a substitute for thinking!
– Marks Nester, A Myopic View and History of Hypothesis Testing. (#science,significance,statistics)
The purpose of this paper is severalfold. First, we attempt to convince the reader that at its worst, the results of statistical hypothesis testing can be seriously misleading, and at its best it offers no informational advantage over its alternatives; in fact it offers less.
– D. Jones and N. Matloff, Statistical hypothesis testing in biology: a contradiction in terms. 1986. Journal of Economic Entomology 79: 1156-1160. (#nhst,significance)
In view of our long-term strategy of improving our theories, our statistical tactics can be greatly improved by shifting emphasis away from over-all hypothesis testing in the direction of statistical estimation. This always holds true when we are concerned with the actual size of one or more differences rather than simply in the existence of differences.
– David A. Grant, Testing the null hypothesis and the strategy and tactics of investigating theoretical models. 1962. Psychological Review 69 : 54-61. (#nhst)
The null hypothesis of no difference has been judged to be no longer a sound or fruitful basis for statistical investigation… Significance tests do not provide the information that scientists need, and, furthermore, they are not the most effective method for analyzing and summarizing data.
– Cherry Ann Clark, Hypothesis testing in relation to statistical methodology. 1963. Review of Educational Research 33: 455-473. (#nhst,significance)
There is nothing wrong with the t-test; it has merely been used to give an answer that was never asked for. The Student t-test answers the question: ‘Is there any real difference between the means of the measurement by the old and the new method, or could the apparent difference have arisen from random variation?’ We already know that there is a real difference, so the question is pointless. The question we should have answered is: ‘How big is the difference between the two sets of measurements, and how precisely have we determined it?’
– L. Sayn-Wittgenstein, Statistics - salvation or slavery? 1965. Forestry Chronicle 41 : 103-105. (#nhst,significance)
Somehow there has developed a widespread belief that statistical analysis is legitimate only if it includes significance testing. This belief leads to, and is fostered by, numerous introductory statistics texts that are little more than catalogues of techniques for performing significance tests.
– D. G. Altman, Discussion of Dr Chatfield’s paper. 1985. Journal of the Royal Statistical Society A 148 : 242. (#nhst,significance)
Testing the equality of 2 true treatment means is ridiculous. They will always be different, at least beyond the hundredth decimal place.
– V. Chew, Statistical hypothesis testing: an academic exercise in futility. 1977. Proceedings of the Florida State Horticultural Society 90 : 214-215. (#nhst)
It is surely apparent that anyone who wants to obtain a significant difference badly enough can obtain one … choose a sample size large enough.
– A. Binder, Further considerations on testing the null hypothesis and the strategy and tactics of investigating theoretical models. 1963. Psychological Review 70 : 107-115. (#nhst,significance,sampling)
As Confucius might have said, if the difference isn’t different enough to make a difference, what’s the difference?
– V. Chew, Testing differences among means: correct interpretation and some alternatives. 1980. HortScience 15(4) : 467-470. (#nhst,significance)
Some hesitation about the unthinking use of significance tests is a sign of statistical maturity.
– D. S. Moore and G. P. McCabe, Introduction to the Practice of Statistics. 1989. W. H. Freeman and Company (New York). (#nhst)
It is usually wise to give a confidence interval for the parameter in which you are interested.
– D. S. Moore and G. P. McCabe, Introduction to the Practice of Statistics. 1989. W. H. Freeman and Company (New York). (#significance)
Unfortunately, when applied in a cook-book fashion, such significance tests do not extract the maximum amount of information available from the data. Worse still, misleading conclusions can be drawn. There are at least three problems: (1) a conclusion that there is a significant difference can often be reached merely by collecting enough samples; (2) a statistically significant result is not necessarily practically significant; and (3) reports of the presence or absence of significant differences for multiple tests are not comparable unless identical sample sizes are used.
– G. B. McBride, J. C. Loftis, & N. C. Adkins, What do significance tests really tell us about the environment?. 1993. Environmental Management 17, 423-432 (1993). (#nhst,significance,sampling)
In many experiments it seems obvious that the different treatments must have produced some difference, however small, in effect. Thus the hypothesis that there is no difference is unrealistic: the real problem is to obtain estimates of the sizes of the differences.
– William G. Cochran, and George M. Cox, Experimental Designs. 2nd ed. 1957. John Wiley & Sons, Inc. (#nhst,significance)
I suggest to you that Sir Ronald has befuddled us, mesmerized us, and led us down the primrose path. I believe that the almost universal reliance on merely refuting the null hypothesis as the standard method for corroborating substantive theories in the soft areas is a terrible mistake, is basically unsound, poor scientific strategy, and one of the worst things that ever happened in the history of psychology.
– P. E. Meehl, Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. 1978. Journal of Consulting and Clinical Psychology 46 : 806-834. (#nhst)
Probably all theories are false in the eyes of God.
– P. E. Meehl, Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. 1978. Journal of Consulting and Clinical Psychology 46 : 806-834. (#science,significance)
The grotesque emphasis on significance tests in statistics courses of all kinds … is taught to people, who if they come away with no other notion, will remember that statistics is about tests for significant differences. … The apparatus on which their statistics course has been constructed is often worse than irrelevant, it is misleading about what is important in examining data and making inferences.
– John A. Nelder, Discussion of Dr Chatfield’s paper. 1985. Journal of the Royal Statistical Society A 148 : 238. (#nhst)
Statistics is intimately connected with science and technology, and few mathematicians have experience or understanding of the methods of either.
– John A. Nelder, Discussion of Dr Chatfield’s paper. 1985. Journal of the Royal Statistical Society A, 148, p. 238. (#nhst)
if experimenters realized how little is the chance of their experiments discovering what they are intended to discover, then a very substantial proportion of the experiments that are now in progress would have been abandoned in favour of an increase in size of the remaining experiments, judged more important.
– Jerzy Neyman, The use of the concept of power in agricultural experimentation. 1958. Journal of the Indian Society of Agricultural Statistics 9 : 9-17. (#power)
What was the probability (power) of detecting interactions … in the experiment performed? … The probability in question is frequently relatively low … in cases of this kind the fact that the test failed to detect the existence of interactions does not mean very much. In fact, they may exist and have gone undetected.
– Jerzy Neyman, The use of the concept of power in agricultural experimentation. 1958. Journal of the Indian Society of Agricultural Statistics 9 : 9-17. (#power)
In addition to important technical errors, fundamental errors in the philosophy of science are frequently involved in this indiscriminate use of the tests [of significance].
– Denton E. Morrison & Ramon E Henkel, Significance tests reconsidered. 1969. The American Sociologist 4 : 131-140. (#nhst,science,significance)
Researchers have long recognized the unfortunate connotations and consequences of the term ‘significance,’ and we propose it is time for a change.
– Denton E. Morrison & Ramon E Henkel, Significance tests reconsidered. 1969. The American Sociologist 4 : 131-140. (#nhst,significance)
there is evidence that significance tests have been a genuine block to achieving … knowledge.
– Denton E. Morrison & Ramon E Henkel, Significance tests reconsidered. 1969. The American Sociologist 4 : 131-140. (#nhst,significance,knowledge)
The twin assumptions of normality of distribution and homogeneity of variance are not ever exactly fulfilled in practice, and often they do not even hold to a good approximation.
– John W. Tukey, The problem of multiple comparisons. 1973. Unpublished manuscript, Dept. of Statistics, Princeton University. (#normality)
[A]sking ‘Are the effects different?’ is foolish.
– John W. Tukey, The philosophy of multiple comparisons. 1991. Statistical Science 6 : 100-116. (#nhst,statistics,statistician)
Empirical knowledge is always fuzzy! And theoretical knowledge, like all the laws of physics, as of today’s date, is always wrong-in detail, though possibly providing some very good approximations indeed.
– John W. Tukey, The philosophy of multiple comparisons. 1991. Statistical Science 6 : 100-116.
scientists care about whether a result is statistically significant, but they should care much more about whether it is meaningful.
– Deirdre N. McCloskey, The insignificance of statistical significance. 1995. Scientific American 272(4) : 104-105. (#nhst,science,significance)
The statistician should not always remain in his or her own office: not only is relevant information more likely to be on hand in the experimenter’s department, but in the longer term the statistician stands to gain immeasurably in understanding of agricultural problems by often visiting other departments and their laboratories and fields.
– David J. Finney, Was this in your statistics textbook? I. Agricultural Scientist and Statistician. 1988. Experimental Agriculture 24 : 153-161. (#statistician)
Rigid dependence upon significance tests in single experiments is to be deplored.
– David J. Finney, Was this in your statistics textbook? III. Design and analysis. 1988. Experimental Agriculture 24 : 421-432. (#nhst,significance)
A null hypothesis that yields under two different treatments have identical expectations is scarcely very plausible, and its rejection by a significance test is more dependent upon the size of an experiment than upon its untruth.
– David J. Finney, Was this in your statistics textbook? III. Design and analysis. 1988. Experimental Agriculture 24 : 421-432. (#nhst,significance)
I have failed to find a single instance in which the Duncan test was helpful, and I doubt whether any of the alternative tests [multiple range significance tests] would please me better.
– David J. Finney, Was this in your statistics textbook? III. Design and analysis. 1988. Experimental Agriculture 24 : 421-432. (#nhst,significance)
Is it ever worth basing analysis and interpretation of an experiment on the inherently implausible null hypothesis that two (or more) recognizably distinct cultivars have identical yield capacities?
– David J. Finney, Was this in your statistics textbook? III. Design and analysis. 1988. Experimental Agriculture 24 : 421-432. (#nhst)
Prediction is very difficult, especially of the future.
– Niels Henrick David Bohr (#time,science,history)
Standard errors of variance components are dumb because the distribution of a variance component is not symmetric, but Chi-squared and highly skewed.
– Doug Bates, Presentation at useR 2007 (#skewness,models)
All data are wrong, but some are useful.
– Jim Kloet (after George Box), RStudio::Conf 2022 (#data)
A lot of data science and analytics is just counting things and labeling them.
– Hamdan Azhar, 2022 New York R Conference (#science,counts,data analysis)
An observation is judged significant, if it would rarely have been produced, in the absence of a real cause of the kind we are seeking. It is a common practice to judge a result significant, if it is of such a magnitude that it would have been produced by chance not more frequently than once in twenty trials. This is an arbitrary, but convenient, level of significance for the practical investigator, but it does not mean that he allows himself to be deceived once in every twenty experiments. The test of significance only tells him what to ignore, namely all experiments in which significant results are not obtained. He should only claim that a phenomenon is experimentally demonstrable when he knows how to design an experiment so that it will rarely fail to give a significant result. Consequently, isolated significant results which he does not know how to reproduce are left in suspense pending further investigation.
– Ronald Fisher, The Statistical Method in Psychical Research, Proceedings of the Society for Psychical Research, 39: 189-192 (1929). (#significance)
The statistician has no magic touch by which he may come in at the stage of tabulation and make something of nothing. Neither will his advice, however wise in the early stages of a study, ensure successful execution and conclusion. Many a study, launched on the ways of elegant statistical design, later boggled in execution, ends up with results to which the theory of probability can contribute little.
– W. Edwards Deming, Principles of Professional Statistical Practice. Annals of Mathematical Statistics, 36(6), 1883. (1965) (#statistician)
Evaluation of the statistical reliability of a set of results is not mere calculation of standard errors and confidence limits. The statistician must go far beyond the statistical methods in textbooks. He must evaluate uncertainty in terms of possible uses of the data. Some of this writing is not statistical but draws on assistance from the expert in the subject-matter.
– W. Edwards Deming, Principles of Professional Statistical Practice. Annals of Mathematical Statistics, 36(6), 1883. (1965) (#data,statistician)
An inference, if it is to have scientific value, must constitute a prediction concerning future data. If the inference is to be made purely with the help of the distribution theory of statistics, the experiments that constitute evidence for the inference must arise from a state of statistical control; until that state is reached, there is no universe, normal or otherwise, and the statistician’s calculations by themselves are an illusion if not a delusion.
– W. Edwards Deming, Statistical Method from the Viewpoint of Quality Control, 1939. (#statistics,science)
Data visualization is part art and part science. The challenge is to get the art right without getting the science wrong and vice versa.
– Claus O. Wilke, Fundamentals of Data Visualization (#data visualization,science)
In other words, the model is terrific in all ways other than the fact that it is totally useless. So why did we create it? In short, because we could: we have a data set, and a statistical package, and add the former to the latter, hit a few buttons and voila, we have another paper.
– Andew J. Vickers & Angel M. Cronin, Everything you always wanted to know about evaluating prediction models (but were too afraid to ask). Urology. 2010;76(6):1298-1301. (#models)
The definition of a medical statistician is one who will not accept that Columbus discovered America because he said he was looking for India in the trial plan.
– Stephen J. Senn, Power is indeed irrelevant in interpreting completed studies. BMJ. 2002;325(7375):1304. (#statistician)
Inept graphics also flourish because many graphic artists believe that statistics are boring and tedious. It then follows that decorated graphics must pep up, animate, and all too often exaggerate what evidence there is in the data. … If the statistics are boring, then you’ve got the wrong numbers.
– Edward R Tufte, The Visual Display of Quantitative Information, 1983. (#data visualization,statistics)
Excellence in statistical graphics consists of complex ideas communicated with clarity, precision, and efficiency. Graphical displays should show the data, induce the viewer to think about the substance rather that about the methodology, graphic design, the technology of graphic production, or something else, avoid distorting what the data have to say, present many numbers in a small space make large data sets coherent, encourage the eye to compare different pieces of data, reveal the data at several levels of detail, from a broad overview to the fine structure, serve a reasonable clear purpose: description, exploration, tabulation, or decoration [should] be closely integrated with the statistical and verbal descriptions of a data set.
– Edward R Tufte, The Visual Display of Quantitative Information, 1983 (#data visualization)
If you can’t have an experiment, do the best you can with whatever data you can gather, but do be very skeptical of historical data and subject them to all the logical tests you can think of.
– Robert Hooke, Statistics, Sports, and Some Other Things. In: Statistics: A Guide to the Unknown, Judith M. Tanur
The purely random sample is the only kind that can be examined with entire confidence by means of statistical theory, but there is one thing wrong with it. It is so difficult and expensive to obtain for many uses that sheer cost eliminates it.
– Darell Huff, How to Lie with Statistics, 1954. (#sampling)
Probability is the most important concept in modern science, especially as nobody has the slightest notion what it means.
– Bertrand Russell, 1929 Lecture (cited in Bell 1945, The Development of Mathematics, p. 587) (#probability)
It is now proved beyond doubt that smoking is one of the leading causes of statistics.
– Fletcher Knebel, 1961 (#statistics)
Statistics show that of those who contract the habit of eating, very few survive.
– William W Irwin
We are hardwired to make sense of the world around us - to notice patterns and invent theories to explain these patterns. We underestimate how easily patterns can be created by inexplicable random events - by good luck and bad luck.
– Gary Smith, Standard Deviations, 2014
A very different - and very incorrect - argument is that successes must be balanced by failures (and failures by successes) so that things average out. Every coin flip that lands heads makes tails more likely. Every red at roulette makes black more likely. … These beliefs are all incorrect. Good luck will certainly not continue indefinitely, but do not assume that good luck makes bad luck more likely, or vice versa.
– Gary Smith, Standard Deviations, 2014 (#probability)
Remember that even random coin flips can yield striking, even stunning, patterns that mean nothing at all. When someone shows you a pattern, no matter how impressive the person’s credentials, consider the possibility that the pattern is just a coincidence. Ask why, not what. No matter what the pattern, the question is: Why should we expect to find this pattern?
– Gary Smith, Standard Deviations, 2014 (#probability)
We are seduced by patterns and we want explanations for these patterns. When we see a string of successes, we think that a hot hand has made success more likely. If we see a string of failures, we think a cold hand has made failure more likely. It is easy to dismiss such theories when they involve coin flips, but it is not so easy with humans. We surely have emotions and ailments that can cause our abilities to go up and down. The question is whether these fluctuations are important or trivial.
– Gary Smith, Standard Deviations, 2014
[In statistics] you have the fact that the concepts are not very clean. The idea of probability, of randomness, is not a clean mathematical idea. You cannot produce random numbers mathematically. They can only be produced by things like tossing dice or spinning a roulette wheel. With a formula, any formula, the number you get would be predictable and therefore not random. So as a statistician you have to rely on some conception of a world where things happen in some way at random, a conception which mathematicians don’t have.
– Lucien LeCam, Interview, 1988 (#probability)
Flip a coin 100 times. Assume that 99 heads are obtained. If you ask a statistician, the response is likely to be: ‘It is a biased coin’. But if you ask a probabilist, he may say: ‘Wooow, what a rare event’.
– Chamont Wang, Sense and Nonsense of Statistical Inference, 1993 (#probability)
It is seen that continued shuffling may reasonably be expected to produce perfect ‘randomness’ and to eliminate all traces of the original order. It should be noted, however, that the number of operations required for this purpose is extremely large.
– William Feller, An Introduction To Probability Theory And Its Applications, 1950
Figures may not lie, but statistics compiled unscientifically and analyzed incompetently are almost sure to be misleading, and when this condition is unnecessarily chronic the so-called statisticians may be called liars.
– Edwin B Wilson, Bulletin of the American Mathematical Society, Vol 18, 1912 (#statisticians)
The statistician’s job is to draw general conclusions from fragmentary data. Too often the data supplied to him for analysis are not only fragmentary but positively incoherent, so that he can do next to nothing with them. Even the most kindly statistician swears heartily under his breath whenever this happens.
– M J Moroney, Facts from Figures, 1927 (#statisticians)
Just as by ‘literacy’, in this context, we mean much more than its dictionary sense of the ability to read and write, so by ‘numeracy’ we mean more than mere ability to manipulate the rule of three. When we say that a scientist is ‘illiterate’, we mean that he is not well enough read to be able to communicate effectively with those who have had a literary education. When we say that a historian or a linguist is ‘innumerate’ we mean that he cannot even begin to understand what scientists and mathematicians are talking about.
– Sir Geoffrey Crowther, A Report of the Central Advisory Committee for Education, 1959, p. 270. (#numeracy)
Numeracy has come to be an indispensable tool to the understanding and mastery of all phenomena, and not only of those in the relatively close field of the traditional natural sciences.
– Sir Geoffrey Crowther, A Report of the Central Advisory Committee for Education, 1959, p. 271. (#numeracy)
Numeracy has two facets–reading and writing, or extracting numerical information and presenting it. The skills of data presentation may at first seem ad hoc and judgmental, a matter of style rather than of technology, but certain aspects can be formalized into explicit rules, the equivalent of elementary syntax.
– Andrew Ehrenberg, Rudiments of Numeracy, Journal of Royal Statistical Society, 140, 277-297, 1977. (#numeracy)
People often feel inept when faced with numerical data. Many of us think that we lack numeracy, the ability to cope with numbers. … The fault is not in ourselves, but in our data. Most data are badly presented and so the cure lies with the producers of the data. To draw an analogy with literacy, we do not need to learn to read better, but writers need to be taught to write better.
– Andrew Ehrenberg, The problem of numeracy, American Statistician 35, 67-71, 1981. (#numeracy)
To be numerate means to be competent, confident, and comfortable with one’s judgements on whether to use mathematics in a particular situation and if so, what mathematics to use, how to do it, what degree of accuracy is appropriate, and what the answer means in relation to the context.
– Diana Coben, Numeracy, mathematics and adult learning, 2000 (#numeracy)
Numeracy is the ability to process, interpret and communicate numerical, quantitative, spatial, statistical, even mathematical information, in ways that are appropriate for a variety of contexts, and that will enable a typical member of the culture or subculture to participate effectively in activities that they value.
– Jeff Evans, Adults’ Mathematical Thinking and Emotion, 2000 (#numeracy)
Statistics is the art of stating in precise terms that which one does not know.
– William Kruskal, Statistics, Moliere, and Henry Adams, American Scientist, 55, 416-428, 1967. (#statistics)
If significance tests are required for still larger samples, graphical accuracy is insufficient, and arithmetical methods are advised. A word to the wise is in order here, however. Almost never does it make sense to use exact binomial significance tests on such data - for the inevitable small deviations from the mathematical model of independence and constant split have piled up to such an extent that the binomial variability is deeply buried and unnoticeable. Graphical treatment of such large samples may still be worthwhile because it brings the results more vividly to the eye.
– Frederick Mosteller & John W Tukey, The Uses and Usefulness of Binomial Probability Paper, Journal of the American Statistical Association 44, 1949. (#significance,data visualization)
Sequences of random numbers also inevitably display certain regularities. … The trouble is, just as no real die, coin, or roulette wheel is ever likely to be perfectly fair, no numerical recipe produces truly random numbers. The mere existence of a formula suggests some sort of predictability or pattern.
– Ivars Peterson, The Jungles of Randomness: A Mathematical Safari, 1998. (#random numbers)
It is very easy to devise different tests which, on the average, have similar properties, … they behave satisfactorily when the null hypothesis is true and have approximately the same power of detecting departures from that hypothesis. Two such tests may, however, give very different results when applied to a given set of data. The situation leads to a good deal of contention amongst statisticians and much discredit of the science of statistics. The appalling position can easily arise in which one can get any answer one wants if only one goes around to a large enough number of statisticians.
– Frances Yates, Discussion on the Paper by Dr. Box and Dr. Andersen, Journal of the Royal Statistical Society B Vol. 17, 1955
Beware of the problem of testing too many hypotheses; the more you torture the data, the more likely they are to confess, but confessions obtained under duress may not be admissible in the court of scientific opinion.
– Stephen M Stigler, Neutral Models in Biology, 1987, p. 148. (#significance)
Statistics may be regarded as (i) the study of populations, (ii) as the study of variation, and (iii) as the study of methods of the reduction of data.
– Sir Ronald A Fisher, Statistical Methods for Research Workers, 1925 (#statistics)
The primes have tantalized mathematicians since the Greeks, because they appear to be somewhat randomly distributed but not completely so. … Although the prime numbers are rigidly determined, they somehow feel like experimental data.
– Timothy Gowers, Mathematics: A Very Short Introduction, 2002 (#random numbers)
Frequentist statistics assumes that there is a ‘true’ state of the world (e.g. the difference between species in predation probability) which gives rise to a distribution of possible experimental outcomes. The Bayesian framework says instead that the experimental outcome - what we actually saw happen - is the truth, while the parameter values or hypotheses have probability distributions. The Bayesian framework solves many of the conceptual problems of frequentist statistics: answers depend on what we actually saw and not on a range of hypothetical outcomes, and we can legitimately make statements about the probability of different hypotheses or parameter values.
– Ben Bolker, Ecological Models and Data in R, 2007 (#bayesian)
A statistical estimate may be good or bad, accurate or the reverse; but in almost all cases it is likely to be more accurate than a casual observer’s impression, and the nature of things can only be disproved by statistical methods.
– Sir Arthur L Bowley, Elements of Statistics, 1901 (#statistics)
An extremely odd demand is often set forth but never met, even by those who make it; i.e., that empirical data should be presented without any theoretical context, leaving the reader, the student, to his own devices in judging it. This demand seems odd because it is useless simply to look at something. Every act of looking turns into observation, every act of observation into reflection, every act of reflection into the making of associations; thus it is evident that we theorize every time we look carefully at the world.
– Johann Wolfgang von Goethe (#data)
From the moment we first roll a die in a children’s board game, or pick a card (any card), we start to learn what probability is. But even as adults, it is not easy to tell what it is, in the general way.
– David Stirzaker, Probability and Random Variables: A Beginner’s Guide, 1999 (#probability)
We cannot really have a perfectly shuffled pack of perfect cards; this ‘collection of equally likely hands’ is actually a fiction. We create the idea, and then use the rules of arithmetic to calculate the required chances. This is characteristic of all mathematics, which concerns itself only with rules defining the behaviour of entities which are themselves undefined (such as ‘numbers’ or ‘points’).
– David Stirzaker, Probability and Random Variables: A Beginner’s Guide, 1999
The whole point of probability is to discuss uncertain eventualities before they occur. After this event, things are completely different.
– David Stirzaker, Probability and Random Variables: A Beginner’s Guide, 1999 (#probability)
There is no such thing as randomness. No one who could detect every force operating on a pair of dice would ever play dice games, because there would never be any doubt about the outcome. The randomness, such as it is, applies to our ignorance of the possible outcomes. It doesn’t apply to the outcomes themselves. They are 100% determined and are not random in the slightest. Scientists have become so confused by this that they now imagine that things really do happen randomly, i.e. for no reason at all.
– Thomas Stark, God Is Mathematics: The Proofs of the Eternal Existence of Mathematics, 2018
Why is the human need to be in control relevant to a discussion of random patterns? Because if events are random, we are not in control, and if we are in control of events, they are not random. There is therefore a fundamental clash between our need to feel we are in control and our ability to recognize randomness. That clash is one of the principal reasons we misinterpret random events.
– Leonard Mlodinow, The Drunkard’s Walk: How Randomness Rules Our Lives, 2008
An experiment is a failure only when it also fails adequately to test the hypothesis in question, when the data it produces don’t prove anything one way or the other.
– Robert M Pirsig, Zen and the Art of Motorcycle Maintenance, 1974
A hypothesis is empirical or scientific only if it can be tested by experience. […] A hypothesis or theory which cannot be, at least in principle, falsified by empirical observations and experiments does not belong to the realm of science.
– Francisco J Ayala, Biological Evolution: Natural Selection or Random Walk, American Scientist, 1974
…no one believes an hypothesis except its originator but everyone believes an experiment except the experimenter.
– William I B Beveridge, The Art of Scientific Investigation, 1950
The hypothesis is the principal intellectual instrument in research. Its function is to indicate new experiments and observations and it therefore sometimes leads to discoveries even when not correct itself. We must resist the temptation to become too attached to our hypothesis, and strive to judge it objectively and modify it or discard it as soon as contrary evidence is brought to light. Vigilance is needed to prevent our observations and interpretations being biased in favor of the hypothesis. Suppositions can be used without being believed.
– William I B Beveridge, The Art of Scientific Investigation, 1950 (#science)
Experiments are like cross-questioning a witness who will tell the truth but not the whole truth.
– Alan Gregg, The Furtherance of Medical Research, 1941
A random sequence is a vague notion embodying the idea of a sequence in which each term is unpredictable to the uninitiated and whose digits pass a certain number of tests traditional with statisticians and depending somewhat on the uses to which the sequence is to be put.
– Derrick H Lehmer, 1951 (#random numbers)
The moment you forecast you know you’re going to be wrong, you just don’t know when and in which direction.
– Edgar R Fiedler, “Across the Board”, 1977 (#time series)
Statistics at its best provides methodology for dealing empirically with complicated and uncertain information, in a way that is both useful and scientifically valid.
– John M Chambers, 1993 (#statistics)
…it does not seem helpful just to say that all models are wrong. The very word model implies simplification and idealization. The idea that complex physical, biological or sociological systems can be exactly described by a few formulae is patently absurd. The construction of idealized representations that capture important stable aspects of such systems is, however, a vital part of general scientific analysis and statistical models, especially substantive ones, do not seem essentially different from other kinds of model.
– Sir David Cox, Comment on ‘Model uncertainty, data mining and statistical inference’, Journal of the Royal Statistical Society, Series A 158, 1995. (#models,uncertainty)
The science of statistics may be described as exploring, analyzing and summarizing data; designing or choosing appropriate ways of collecting data and extracting information from them; and communicating that information. Statistics also involves constructing and testing models for describing chance phenomena. These models can be used as a basis for making inferences and drawing conclusions and, finally, perhaps for making decisions.
– Fergus Daly et al, Elements of Statistics, 1995 (#statistics,knowledge)
If a man stands with his left foot on a hot stove and his right foot in a refrigerator, the statistician would say that, on the average, he’s comfortable.
– Walter Heller (#statistician)
Things are changing. Statisticians now recognize that computer scientists are making novel contributions while computer scientists now recognize the generality of statistical theory and methodology. Clever data mining algorithms are more scalable than statisticians ever thought possible. Formal statistical theory is more pervasive than computer scientists had realized.
– Larry A Wasserman, All of Statistics: A concise course in statistical inference, 2004 (#statistician)
One feature […] which requires much more justification than is usually given, is the setting up of unplausible null hypotheses. For example, a statistician may set out a test to see whether two drugs have exactly the same effect, or whether a regression line is exactly straight. These hypotheses can scarcely be taken literally.
– Cedric A B Smith, Book review of Norman T. J. Bailey: Statistical Methods in Biology, Applied Statistics 9, 1960 (#statistician)
In general, it is necessary to have some data on which to calculate probabilities. […] Statisticians do not evolve probabilities out of their inner consciousness, they merely calculate them.
– Leonard C Tippett (#data,probability)
Even properly done statistics can’t be trusted. The plethora of available statistical techniques and analyses grants researchers an enormous amount of freedom when analyzing their data, and it is trivially easy to ‘torture the data until it confesses’.
– Alex Reinhart, Statistics Done Wrong: The Woefully Complete Guide, 2015
Using data from the population as it stands is a dangerous substitute for testing.
– Frederick Mosteller & Gale Mosteller, “New Statistical Methods in Public Policy. Part I: Experimentation”, Journal of Contemporary Business 8, 1979
The closer that sample-selection procedures approach the gold standard of random selection - for which the definition is that every individual in the population has an equal chance of appearing in the sample - the more we should trust them. If we don’t know whether a sample is random, any statistical measure we conduct may be biased in some unknown way.
– Richard E Nisbett, “Mindware: Tools for Smart Thinking”, 2015
A popular misconception holds that the era of Big Data means the end of a need for sampling. In fact, the proliferation of data of varying quality and relevance reinforces the need for sampling as a tool to work efficiently with a variety of data, and minimize bias. Even in a Big Data project, predictive models are typically developed and piloted with samples.
– Peter C Bruce & Andrew G Bruce, “Statistics for Data Scientists: 50 Essential Concepts”, 2016 (#sampling,data)
All predictions are statistical, but some predictions have such a high probability that one tends to regard them as certain.
– Marshall J Walker, The Nature of Scientific Thought, 1963 (#uncertainty)
Statistics is a scientific discipline concerned with collection, analysis, and interpretation of data obtained from observation or experiment. The subject has a coherent structure based on the theory of Probability and includes many different procedures which contribute to research and development throughout the whole of Science and Technology.
– Egon Pearson, 1936 (#statistics,probability,science)
[Statistics] is both a science and an art. It is a science in that its methods are basically systematic and have general application; and an art in that their successful application depends to a considerable degree on the skill and special experience of the statistician, and on his knowledge of the field of application, e.g. economics.
– Leonard H C Tippett, Statistics, 1943 (#statistics)
The fact must be expressed as data, but there is a problem in that the correct data is difficult to catch. So that I always say ‘When you see the data, doubt it!’ ‘When you see the measurement instrument, doubt it!’ […]For example, if the methods such as sampling, measurement, testing and chemical analysis methods were incorrect, data. […] to measure true characteristics and in an unavoidable case, using statistical sensory test and express them as data.
– Kaoru Ishikawa, Annual Quality Congress Transactions, 1981 (#data,sampling)
There is a tendency to mistake data for wisdom, just as there has always been a tendency to confuse logic with values, intelligence with insight. Unobstructed access to facts can produce unlimited good only if it is matched by the desire and ability to find out what they mean and where they lead.
– Norman Cousins, “Human Options : An Autobiographical Notebook”, 1981 (#data,knowledge)
Data in isolation are meaningless, a collection of numbers. Only in context of a theory do they assume significance…
– George Greenstein, “Frozen Star”, 1983 (#data)
Intuition becomes increasingly valuable in the new information society precisely because there is so much data.
– John Naisbitt, “Re-Inventing the Corporation”, 1985 (#data)
No matter what the laws of chance might tell us, we search for patterns among random events wherever they might occur–not only in the stock market but even in interpreting sporting phenomena.
– Burton G. Malkiel, “A Random Walk Down Wall Street: The Time-Tested Strategy For Successful Investing”, 2011, p. 149. (#random numbers)
One can be highly functionally numerate without being a mathematician or a quantitative analyst. It is not the mathematical manipulation of numbers (or symbols representing numbers) that is central to the notion of numeracy. Rather, it is the ability to draw correct meaning from a logical argument couched in numbers. When such a logical argument relates to events in our uncertain real world, the element of uncertainty makes it, in fact, a statistical argument.
– Eric R Sowey, The Getting of Wisdom: Educating Statisticians to Enhance Their Clients’ Numeracy, The American Statistician 57(2), 2003 (#numeracy,uncertainty)
We would wish ‘numerate’ to imply the possession of two attributes. The first of these is an ‘at-homeness’ with numbers and an ability to make use of mathematical skills which enable an individual to cope with the practical mathematical demands of his everyday life. The second is ability to have some appreciation and understanding of information which is presented in mathematical terms, for instance in graphs, charts or tables or by reference to percentage increase or decrease.
– Cockcroft Committee, Mathematics Counts: A Report into the Teaching of Mathematics in Schools, 1982 (#numeracy,data visualization)
In all scientific fields, theory is frequently more important than experimental data. Scientists are generally reluctant to accept the existence of a phenomenon when they do not know how to explain it. On the other hand, they will often accept a theory that is especially plausible before there exists any data to support it.
– Richard Morris, 1983 (#science,data)
To find out what happens to a system when you interfere with it you have to interfere with it (not just passively observe it).
– George E P Box, “Use and Abuse of Regression”, 1966 (#Box quotes)
Since all models are wrong the scientist cannot obtain a ‘correct’ one by excessive elaboration. On the contrary following William of Occam he should seek an economical description of natural phenomena. Just as the ability to devise simple but evocative models is the signature of the great scientist so overelaboration and overparameterization is often the mark of mediocrity.
– George E P Box, Science and Statistics, Journal of the American Statistical Association 71, 1976 (#Box quotes,models)
Since all models are wrong the scientist must be alert to what is importantly wrong. It is inappropriate to be concerned about mice when there are tigers abroad.
– George E P Box, Science and Statistics, Journal of the American Statistical Association 71, 1976 (#models,science,Box quotes)
The fact that [the model] is an approximation does not necessarily detract from its usefulness because models are approximations. All models are wrong, but some are useful.
– George E P Box, 1987 (#models,science,Box quotes)
The central limit theorem says that, under conditions almost always satisfied in the real world of experimentation, the distribution of such a linear function of errors will tend to normality as the number of its components becomes large. The tendency to normality occurs almost regardless of the individual distributions of the component errors. An important proviso is that several sources of error must make important contributions to the overall error and that no particular source of error dominate the rest.
– George E P Box et al, “Statistics for Experimenters: Design, discovery, and innovation” 2nd Ed., 2005 (#normality)
The postulate of randomness thus resolves itself into the question, ‘of what population is this a random sample?’ which must frequently be asked by every practical statistician.
– Ronald Fisher, “On the Mathematical Foundation of Theoretical Statistics”, Philosophical Transactions of the Royal Society of London Vol. A222, 1922 (#random numbers,sampling,statistics)
Statistics has been likened to a telescope. The latter enables one to see further and to make clear objects which were diminished or obscured by distance. The former enables one to discern structure and relationships which were distorted by other factors or obscured by random variation.
– David J Hand, “The Role of Statistics in Psychiatry”, Psychological Medicine Vol. 15, 1985 (#statistics)
When looking at the end result of any statistical analysis, one must be very cautious not to over interpret the data. Care must be taken to know the size of the sample, and to be certain the method for gathering information is consistent with other samples gathered. […] No one should ever base conclusions without knowing the size of the sample and how random a sample it was. But all too often such data is not mentioned when the statistics are given - perhaps it is overlooked or even intentionally omitted.
– Theoni Pappas, “More Joy of Mathematics: Exploring mathematical insights & concepts”, 1994 (#sampling)
BREAKING: The Supreme Court just ruled 6-3 that according to the US constitution logistic regression IS machine learning.
– Kareem Carr, @Kareem_Carr, Twitter 7/1/21 (#models)
In questions of science the authority of a thousand is not worth the humble reasoning of a single individual
– Galileo Galilei, 1632, Dialog concerning the Two Chief World Systems (#science)
Science is organized knowledge. Wisdom is organized life.
– Immanuel Kant (#science,knowledge)
When you steal from one author, it’s plagiarism; if you steal from many, it’s research.
– Wilson Mizner (American playwright and entrepreneur) (#research,ethics)
You can’t always get what you want, but if you try, sometimes, well you just might find, you get what you need.
– Mick Jagger (#data,science)
He who gives up code safety for code speed deserves neither.
– Hadley Wickham (#computing)
Any fool can write code that a computer can understand. Good programmers write code that humans can understand.
– Martin Fowler, Refactoring: Improving the Design of Existing Code (#computing)
Thank you for sending me a copy of your book. I’ll waste no time reading it.
– Moses Hadas (#reviews)
I will let the data speak for itself when it cleans itself.
– Allison Reichel (#data)
To the untrained eye, randomness appears as regularity or tendency to cluster.
– W. Feller, An Introduction to Probability Theory and its Applications (1950) (#probability,data visualization)
Your assumptions are your windows on the world. Scrub them off every once in a while, or the light won’t come in.
– Alan Alda, 1936 (#assumptions)
Little experience is sufficient to show that the traditional machinery of statistical processes is wholly unsuited to the needs of practical research. Not only does it take a cannon to shoot a sparrow, but it misses the sparrow! The elaborate mechanism built on the theory of infinitely large samples is not accurate enough for simple laboratory data. Only by systematically tackling small sample problems on their metrics does it seem possible to apply accurate tests to practical data.
– Ronald Fisher, Statistical Methods for Research Workers (1925) (#sample size)
All generative models are wrong, but some are useful.
– Jared Lander, Copilot for R, 56:25 (#models)
Everyone who has carried out experiments in the field or farmyard must be well aware that the result of a single experiment is very often entirely misleading. Yet it is still common practice to publish single results and to base practical advice upon them.
– T. B. Wood and F. J. M. Stratton, The interpretation of experimental results. The Journal of Agricultural Science, 3, 417-440. (#science)
The preparation of clear and simple plans, and a convenient system of numbering the [treatments] that are to be applied, will lighten the work of the man in the field, who is usually operating under averse conditions, is frequently in a hurry, and is sometimes not very certain of the points at issue.
– F. Yates, The Design and Analysis of Factorial Experiments (1937). Harpenden Imperial Bureau of Soil Science. (#biometry, expt design)
I do not always agree with Sir Ronald Fisher, but it is due to him that the standard of presentation of results in agriculture is better than in any of the so-called exact sciencees; and this is a state of affairs that physicists should cease to tolerate.
– Sir Harald Jeffreys, Half a Century in Geophysics: An original article from the Report of the British Association for the Advancement of Science, 1953. (#science)
Data are not just numbers, they are numbers with a context. … In data analysis, context provides meaning.
– George W. Cobb & David S. Moore, Mathematics, Statistics, and Teaching. American Mathematical Monthly, 801-23. (#numeracy)
When [the profession of] statistics becomes clearly embedded in people’s minds as being concerned with investigation rather than simply calculation (or worse still, mere cataloguing of data), there can be no room for doubts about the relevance of the subject.
– C. J. Wild, Embracing the ‘Wider view’ of Statistics, The American Statistician, 48, 163-171. p. 165.
We have to teach non-statisticians to recognize where statistical expertise is required. No one else will. We teach students how to solve simple statistical problems, but how often do we make any serious effort to teach them to recognize situations that call for statistical expertise that is beyond the technical content of the course.=?
– C. J. Wild, Embracing the ‘Wider view’ of Statistics, The American Statistician, 48, 163-171. p. 166. (#teaching)
A careful and sophisticated analysis of the data is often quite useless if the statistician cannot communicate the essential features of the data to a client for whom statistics is an entirely foreign language.
– C. J. Wild, Embracing the ‘Wider view’ of Statistics, The American Statistician, 48, 163-171. p. 170.
The dominant feature of our 129 barley trials was the large differences…overy the five years of the study, even for the same field. To give sound advice on choice of plot size and shape, statisticians will require results from many trials conducted over many years, not to mention crops.
– Dorothy Robinson, Discussion of the paper by Dr Brewer and Professor Mead, Journal of the Royal Statistical Society. Series A, 1986, 149, pp. 314-348. Quote on p. 341.
If we need a short suggestion of what exploratory data analysis is, I would suggest that: 1. it is an attitude, AND 2. a flexibility, AND 3. some graph paper (or transparencies, or both).
– John W. Tukey, Jones, L. V. (Ed.). (1986). The collected works of John W. Tukey: Philosophy and principles of data analysis 1949-1964 (Vols. III & IV). London: Chapman & Hall. (#eda,data visualization)
Three of the main strategies of data analysis are: 1. graphical presentation. 2. provision of flexibility in viewpoint and in facilities, 3. intensive search for parsimony and simplicity.
– John W. Tukey, Jones, L. V. (Ed.). (1986). The collected works of John W. Tukey: Philosophy and principles of data analysis 1949-1964 (Vols. III & IV). London: Chapman & Hall. (#data visualization,data analysis)
If you torture the data enough, nature will always confess.
– Ronald Coase, quoted from Coase, R. H. (1982). How should economists chose? American Enterprise Institute, Washington, D. C. (#data analysis)
A big computer, a complex algorithm and a long time does not equal science.
– Robert Gentleman (#science,computing)
Absence of evidence is not evidence of absence.
– Martin Rees, Project Cyclops: A Design Study of a System for Detecting Extraterrestrial Intelligent Life (1971) by Bernard M. Oliver, and John Billingham (#science)
Statistics - A subject which most statisticians find difficult but which many physicians are experts on.
– Stephen Senn, Senn, S. (2007). Statistical Issues in Drug Development (2nd Edition). Chichester: John Wiley & Sons (#statistics)
The statistician cannot evade the responsibility for understanding the process he applies or recommends.
– Ronald A Fisher, Fisher, R. A. (1971 [1935]). The Design of Experiments (9th ed.). Macmillan. (#statistics,science)
Taking a model too seriously is really just another way of not taking it seriously at all.
– Andrew Gelman (#models)
What the use of a p-value implies, therefore, is that a hypothesis that may be true may be rejected because it has not predicted observable results that have not occurred.
– Harold Jeffreys, Jeffreys, H. (1939). Theory of Probability. Oxford, England: Clarendon Press. (#p-values)
Statistics are no substitution for judgment.
– Henry Clay, Evening Sentinel (Staffordshire Sentinel), 1930 October 13, Production Prices and Depression: Professor Clay on the Trade Outlook, Quote Page 5, Column 5, Staffordshire, England. (British Newspaper Archive)
The most effective debugging tool is still careful thought, coupled with judiciously placed print statements.
– Brian Kernighan, Unix for Beginners (1979) (#computing)
We live on an island of knowledge surrounded by a sea of ignorance. As our island of knowledge grows, so does the shore of our ignorance.
– John A. Wheeler, Scientific American, 1992 (#science,knowledge)
It’s not just about visualizing information, but making information visible.
– Jose Duarte (#data visualization)
I don’t know what the definition of “philosopher” is. But I know for a fact that a “statistician” is someone who has written an R package.
– Richard McElrath, Twitter, 4-29-2023 (#computing,statistics)
In every project you have at least one other collaborator; future-you. You don’t want future-you to curse past-you.
– Solomon Kurz (#time)
The whole point of science is that most of it is uncertain. That’s why science is exciting–because we don’t know. Science is all about things we don’t understand. The public, of course, imagines science is just a set of facts. But it’s not. Science is a process of exploring, which is always partial. We explore, and we find out things that we understand. We find out things we thought we understood were wrong. That’s how it makes progress.
– Freeman Dyson, mentioned in a 2014 interview (#science,knowledge,uncertainty)