Jim Berkland seemed to predict a large earthquake in mid- to late- March 2011 somewhere in North America. Watch the footage here. (The Fox commentator is pretty funny. At one point he says to pay attention because “he is a pretty good geologist.”)

There was no large earthquake during that time, but we can’t really know if Berkland was technically wrong, because what he actually predicted was a “high probability” of a large earthquake in North America. If you want to know how accurate a predictor who uses language like this is, you have to track outcomes of a whole bunch of their predictions, not just one. This is what Philip Tetlock does in his research on prediction accuracy–track the outcomes of hundreds of predictions of political experts. He also had to force the experts make specific enough predictions that they would either be true or false, not ambiguous–not always an easy task. Berkland, while casting a wide net, was fairly precise with “large earthquake” and “North America,” though we must wonder whether he would have claimed success if there had been a large earthquake, say, in the northern Pacific.

I’m not sure how many earthquake predictions Berkland has made, but if there have been enough, we could judge his rough accuracy: When he predicts a high probability of an earthquake, does it happen most of the time? When he predicts a low probability of an earthquake does it usually not happen? How about a medium probability?

The point is, if your prediction is of a probability, rather than a certainty of an event, we need to do some statistics to figure out if you’re a good predictor. And this is the form that careful people make their predictions. If, on the other hand, you tend to make predictions about certainties–100% or 0% probability events, it’s quite a bit easier to check your accuracy–as long as you make sufficiently specific, falsifiable predictions. Most prediction by ideologues, for example, set up what Tetlock calls an “outcome-irrelevant learning situation,” a situation in which the predictor can claim they were right no matter what actually happens. Every ideologue, therefore, is in the position to explain what happened, using their own ideology.

An example of that may be the Mayan-calendar predictions. Here is Graham Hancock on Art Bell’s radio show, seeming to predict something happening on December 21, 2012. It is full of talk of cataclysms, the end of the world, tumult, a ball of fire hitting the earth, etc. (And lots of talk about how accurate the Mayan calendar was, as if having a really accurate way to measure time lends credence to your predictions. Better ask the guy who invented the atomic clock!) I bet these guys will be patting themselves on the back on 12/21/2012 if a ball of fire does hit the earth. But if nothing particularly tumultuous happens, will they be wrong about anything? No. They are not precise at all, and they attach no probability to their “prediction.” There are plenty of “just mights” and “maybes” and “a window of about 40 years.” They even say that if humanity gets their act together in some vague way, we might avert what may or may not have been coming. This is a perfect setup for an outcome-irrelevant learning situation.

Tetlock says that when predictors are wrong, they generally either claim to be right in some way, based on the fuzziness of their prediction, or they use one of several “belief system defenses.” The most common of these is “Just off on timing.” The other two major defenses are the upward counterfactual defense, or “you think this is bad?” and the downward counterfactual defense, or “you think this is good?”

If nothing particularly tumultuous happens on 12/21/2011, and we ask Bell and his guest about it, how will they respond? They might use “just off on timing,” and blame our modern, inaccurate calendars. More likely they would claim to have been right, something like, “All the war and bad stuff happening on the earth–this is what we were talking about. It’s just a lot more slow and drawn out than we thought.” There is some, small, chance that they might cop to being wrong. I haven’t listened to Bell in over a decade, and I can’t remember how he handles his predictors being wrong, or if he even addresses it.

Berkland could also claim to be right: “Well, there was a high probability of a large earthquake, but not everything with a high probability happens every time.” A “just off on timing” defense would be pretty weak for him, since timing is everything in earthquake prediction.

The third predictor I’ve been thinking about, though, has given himself very little wiggle room. It takes guts  to make a prediction like this. According to Harold Camping, next Saturday, May 21, 2011:

“A great earthquake will occur the Bible describes it as “such as was not since men were upon the earth, so mighty an earthquake, and so great.” This earthquake will be so powerful it will throw open all graves. The remains of the all the believers who have ever lived will be instantly transformed into glorified spiritual bodies to be forever with God.

“On the other hand the bodies of all unsaved people will be thrown out upon the ground to be shamed.

“The inhabitants who survive this terrible earthquake will exist in a world of horror and chaos beyond description. Each day people will die until October 21, 2011 when God will completely destroy this earth and its surviving inhabitants.”

That’s from his website, which you can see here. I have also heard Camping say that millions of people are certain to die on May 21, 2011, and every day thereafter until the very end, October 21, 2011. I have heard him say “It is going to happen.” I have heard him say “It is absolutely certain.” I was disappointed when I heard him back down from that, recently, saying he can’t be absolutely certain, but he has stuck with “going to happen” and “there is no doubt.”

I wonder how Camping will react if his predictions are wrong. The counterfactual defenses won’t apply at all. It will be very difficult to argue that he was right in some way if there is not at least the largest earthquake ever recorded (that would be at least a 9.6), that all buried bodies are somehow exposed (ideally as the result of the earthquake), that millions of people will die on May 21, and that approximately 7 billion people will die by October 21.

So my prediction is that he will use “Just off on timing” and go back to calculating the real day of judgment. Based on social psychology research, I will also predict that in general, this event will increase believers conviction, rather than decrease it. And if I am wrong, I will do my best to just admit it.