DEFILIPPIS: Well, this number comes from a study in 1992 conducted by professors at Florida State University, Gary Kleck and Marc Gertz. What happened was that these researchers called a random sub-sample of 5,000 Americans. 66 of them reported that they had used a gun in self-defense, so that’s a little bit more than 1 percent. And then from that number, the researchers simply scale it up to the entire American population to get around 2.5 million defensive gun uses a year.
BOB: Now it can be legitimate to take a sample of data and project it based on larger populations, but there are statistical protocols for doing that. Was the sample size too small?
DEFILIPPIS: Well the real problem wasn’t so much the sample size as much as it was the fact that defensive gun uses is an inherently rare phenomenon. And the real concern is that there just needs to be a small handful of people that are making defensive gun use claims when there isn’t, and it completely biases this entire statistic.
BOB: You cite an example of another dubious survey…I’m speaking of course of alien abductions.
DEFILIPPIS: That’s right. This is actually one of my favorite demonstrations of how absurd it is to scale up rare events in the way that Kleck and Gertz did. And this example comes from a paper done by David Hemingway at the Harvard School of Public Health. He mentions an ABC poll that was conducted in 1994 that surveyed 1500 adults. Well the survey asked whether respondents had ever seen an alien spacecraft and 150 people reported that they had in fact seen an alien spacecraft. Now if we were to use precisely the same methodology employed by Kleck and Gertz, we would be forced to conclude that something like 20 million americans have seen a spacecraft
BOB: And some of them are in Congress
DEFILIPPIS: [Laughs] I mean this is why respected social scientists aren’t using this poll as definitive evidence of the existence of aliens, because they recognize that some people are simply confused about what it is that they’re observing. And if you examine responses from surveys that inquire into defensive gun uses, for example, you have a situation where someone hears some sort of spooky sound outside. They fire a warning shot into the air, and then they report that as an effective defensive gun use. There’s one respondent who claimed that there was a group of kids surrounding his car and he brandished his weapon and they scattered. And he reported he had deterred some sort of potential crime. So we have all of these potential issues of simply misinterpretation.
BOB: Beyond the statistical risk, this subject of questioning in a telephone survey, you say will inevitably yield false positives — why?
DEFILIPPIS: There was this article written by Dr. David Hemenway where he mentions that there’s a host of potential biases. One of them is known as the social desirability bias. People will either lie or more probably embellish stories in order to appear better in front of an interviewer. Gun owners often feel that they need to justify ownership through the use of these defensive gun use examples, real or not. And another large problem is known as telescoping: a respondent may be reporting a legitimate self defense use that is outside the timeframe of the question asked in the survey. We know that this is an enormous problem because the National Crime Victimization survey which is considered the gold standard in criminal victimization surveys, they find that between 30-40% of defensive gun use incidents are simply an artifact of this telescoping bias.
BOB: So there’s misremembering there’s telescoping, and there’s another not insignificant fact that the defensive gun uses reported by the survey takers are often, you know, crimes themselves.
DEFILIPPIS: That’s right, and this is the part of the story that gun advocates actually don’t like to talk about. Because even Gary Clerk admits that between 36-64% of defensive gun uses in his own survey were likely illegal. And Hemenway attempted to substantiate this claim. He did 2 random digit dial surveys in 1996 and 1999 where he asked open ended questions about defensive gun use incidents to respondents. He then took their detailed responses and gave them to 5 criminal court judges. And the judges determined that the majority of defensive gun uses were illegal, and dangerous to society. If this 2.5 million number has any credibility at all it would show an epidemic of massive proportions.
