Also called Biased Statistics, Loaded Sample, Prejudiced Statistics, Prejudiced Sample, Loaded Statistics, Biased Induction, Biased Generalization.
This fallacy is committed when a person draws a conclusion about a population based on a sample that is biased or prejudiced in some manner. It has the following form:
The person committing the fallacy is misusing the following type of reasoning, which is known variously as Inductive Generalization, Generalization, and Statistical Generalization:
The fallacy is committed when the sample of a's is likely to be biased in some manner. A sample is biased or loaded when the method used to take the sample is likely to result in a sample that does not adequately represent the population from which it is drawn.
Biased samples are generally not very reliable. As a blatant case, imagine that a person is taking a sample from a truckload of small colored balls, some of which are metal and some of which are plastic. If he used a magnet to select his sample, then his sample would include a disproportionate number of metal balls (after all, the sample will probably be made up entirely of the metal balls). In this case, any conclusions he might draw about the whole population of balls would be unreliable since he would have few or no plastic balls in the sample.
The general idea is that biased samples are less likely to contain numbers proportional to the whole population. For example, if a person wants to find out what most Americans thought about gun control, a poll taken at an NRA meeting would be a biased sample.
Since the Biased Sample fallacy is committed when the sample (the observed instances) is biased or loaded, it is important to have samples that are not biased making a generalization. The best way to do this is to take samples in ways that avoid bias. There are, in general, three types of samples that are aimed at avoiding bias. The general idea is that these methods (when used properly) will result in a sample that matches the whole population fairly closely. The three types of samples are as follows:
People often commit Biased Sample because of bias or prejudice. For example, a person might intentionally or unintentionally seek out people or events that support his bias. As an example, a person who is pushing a particular scientific theory might tend to gather samples that are biased in favor of that theory.
People also commonly commit this fallacy because of laziness or sloppiness. It is very easy to simply take a sample from what happens to be easily available rather than taking the time and effort to generate an adequate sample and draw a justified conclusion.
It is important to keep in mind that bias is relative to the purpose of the sample. For example, if Bill wanted to know what NRA members thought about a gun control law, then taking a sample at a NRA meeting would not be biased. However, if Bill wanted to determine what Americans in general thought about the law, then a sample taken at an NRA meeting would be biased.
Bill is assigned by his editor to determine what most Americans think about a new law that will place a federal tax on all modems and computers purchased. The revenues from the tax will be used to enforce new online decency laws. Bill, being technically inclined, decides to use an email poll. In his poll, 95% of those surveyed opposed the tax. Bill was quite surprised when 65% of all Americans voted for the taxes.
The United Pacifists of America decide to run a poll to determine what Americans think about guns and gun control. Jane is assigned the task of setting up the study. To save mailing costs, she includes the survey form in the group's newsletter mailing. She is very pleased to find out that 95% of those surveyed favor gun control laws and she tells her friends that the vast majority of Americans favor gun control laws.
Large scale polls were taken in Florida, California, and Maine and it was found that an average of 55% of those polled spent at least fourteen days a year near the ocean. So, it can be safely concluded that 55% of all Americans spend at least fourteen days near the ocean each year.
To see how Canadians will vote in the next election we polled a hundred people in Calgary.
This shows conclusively that the Reform Party will sweep the polls.
People in Calgary tend to be more conservative, and hence more likely to vote Reform, than people in the rest of the country.
The apples on the top of the box look good. The entire box of apples must be good.
Of course, the rotten apples are hidden beneath the surface.
Show how the sample is relevantly different from the population as a whole, then show that because the sample is different, the conclusion is probably different.