Wednesday, September 28, 2011

How Good Are We at Predicting the Future (like the 2012 Presidential Election)?

As many of you know, every four years I throw out my prediction for who is going to win the Presidential election. So far, I've only been "wrong" once when in 2004, I "punted" and didn't declare a winner (however, I did try to place a bet with "Intrade" on "W", but I couldn't get the site to take my credit card). It's a bit early to predict next year's election
(however, if the economy doesn't pick up between now and next summer, Republicans will control the House, Senate, and Mitt Romney will be sitting in the Oval Office come January 2013),
but evidently I'm not alone in my propensity to predict as a recent Freakonomics podcast noted ("The Folly of Prediction"). Specifically, the podcast noted that
  • Human beings love to predict the future
  • We're not very good at it, and
  • Because bad predictions are rarely punished, this situation is unlikely to change
This podcast explores why humans are so bad at predicting. One individual interviewed, Philip Tetlock, documents just how bad we are. Tetlock is a psychology professor and author of Expert Political Judgment. He conducted an experiment that asked nearly 300 political "experts" to make a variety of forecasts about dozens of countries around the world. After tracking the accuracy of about 80,000 predictions over the course of 20 years, he found that they weren't a whole lot better than the average person on the street. So much for being an expert.

The podcast also notes how the predictions we remember tend to be the ones which were wildly unexpected and then came true. Steven Levitt, one of the Freakonomics co-authors, notes how every year his mother predicted that there would be a stock market crash. She has only been right twice, but she conveniently forgets the numerous times she's been wrong and only talks about the two times she's been right.

Levitt's mother is not the only one. The podcast notes that if you look at all the people, the economists, who talked about the financial crisis ahead of time, those guys constantly remind anyone who will listen: “I was right, I was right, I was right.” But when people are wrong, there’s no person on the other side of the transaction who draws any real benefit from embarrassing you by highlighting the bad predictions. In other words, because there is no market mechanism or incentive for keeping the prediction makers honest, there’s lots of incentive to go out and make wild predictions.

Imagine, however, if every time a pundit appeared on TV, his or her "batting average" appeared next to their name, letting viewers know just how good they are at predicting. That would be an incentive not to make bad predictions, or at least not opining in areas in which you have little or no expertise.

There is at least one arena where bad predictions are punished and are aptly called "prediction markets." These are speculative markets where people "bet" on various outcomes (e.g., who will win the next Presidential election, whether the economy will go into a recession by December 2012, whether there will be a "successful" WMD attack by December 2013, whether the Yankees will win the World Series, etc.) where people are rewarded for making accurate predictions and punished for making inaccurate ones, which is why advocates of prediction markets argue that they are better at predicting outcomes than most experts.

The most popular prediction market is probably "Intrade" although the Iowa Electronic Market is the oldest (and from my limited experience less volatile and more accurate than Intrade). The way it works is that if you buy 10 shares of, say, President Obama winning the 2012 election for $4.74 per share (the current price on Intrade), then if he wins, you will receive $10.00 per share or $100.00 and have made $52.26. The current price of a share is interpreted as the probability that the outcome will occur. That's right, currently Intrade is predicting that there is a 47.4% chance that President Obama will be reelected (the Iowa Electronic Markets currently have Obama at a 47.6% chance of winning), which largely reflects the current state of the world economy. If it picks up, then so will Obama's chances. If it doesn't, then he will probably be a one-term President. We will see what we will see.

If you want to learn more about "The Folly of Prediction" listen to the Freakonomics podcast. Or, if you don't want to listen, you can also access the transcript of the podcast: "The Folly of Prediction: Full Transcript."

Monday, September 26, 2011

Who Would Have Thought? Atheists and Agnostics Live Shorter and Unhappier Lives

It turns out, being a so-called free thinker isn't necessarily good for your health, at least if by "free thinker" you mean someone who is an agnostic or atheist. This isn't actually new news. Social scientists who study religion have known about it for awhile, but in a recent article in the Daily Caller ("What Better Health? Go to Church"), Patrick Chisholm highlights the findings of a lot of these studies. He notes:
"This isn't about what happens to you in the hereafter. It’s about what happens to you in the here and now. Atheists and agnostics suffer, on average, higher rates of physical ailments, depression, suicide, alcohol use and drug addiction. They have greater marital instability, weaker parent-child relationships, lower lifetime earnings, lower educational attainment and higher rates of criminal activity.” 
"These aren’t some trumped-up claims made by people with a religious ax to grind. These are the conclusions of many scholars in the sciences and social sciences whose work appears in numerous non-religious scholarly journals including Psychological Bulletin, Journal of Personality and Clinical Studies, Social Science Research, Preventive Medicine, Demography and many more.”
I would add that several articles on the topic (many authored by Chris Ellison from UT Austin) have appeared in the Journal for the Scientific Study of Religion, the Review of Religious Research, and the Sociology of Religion. Why are the faithful healthier?
“On the physical side, religious belief often prompts one to view one’s body as sacred and a gift from God, which reduces the likelihood of such factors as smoking, drinking, unhealthy eating, unsafe driving, physical inactivity and substance abuse. Religious persons also tend to have a greater support network of family and friends, which encourages healthier lifestyles.”
Of course, you could argue that there is a bit of self-selection going on here. People living more conventional and healthier lifestyles are more likely to attend or join a faith community. There is undoubtedly some of that going on, but researchers have been able to disentangle self-selection effects from religious ones and have found that religious beliefs and practices increase the probability that you will enjoy a longer and happier life.

So there you have it. Want better health? Regularly attend church (or synagogue, mosque, temple, etc.), make lots of friends, invest time in those relationships, and follow their lead on how to live your life.

Tuesday, September 20, 2011

Is it Time to End the War on Terror?

On September 7th, Intelligence Squared US held a debate on the motion "It's Time to End the War on Terror," which addressed the issue of whether the threat of terrorism should the organizing principle behind our foreign policy (amongst other things). Arguing on behalf of the motion were Peter Bergen and Juliette Kayyem;  arguing against it were Richard Falkenrath and Michael Hayden.

If you recall those attending an Intelligence Squared US debate vote prior to and after the debate, and the winning debate team is decided by how many minds were changed and in what direction. As always not only can you listen to the debate at the Intelligence Squared website ("It's Time to End the War on Terror"), but you can access transcripts of the debate as well. The debates can also be downloaded from iTunes.

Here's a brief description of the participants (from the Intelligence Squared website):

Peter Bergen is one of few Americans to have interviewed Osama bin Laden face-to-face, Peter Bergen is one of today’s foremost commentators on America’s national security and the War on Terror. Author of two New York Times bestsellers, Bergen’s recent release The Longest War: The Enduring Conflict Between America and Al-Qaeda has been called “one of the most important accounts on the subject” by the newspaper. He is Editor of the AfPak Channel, a premiere clearinghouse of news covering Afghanistan, Pakistan and issues of transnational terrorism. Bergen is also the Director of the National Security Program at the New America Foundation and a Research Fellow at NYU’s Center on Law and Security.

Juliette Kayyem formerly served under the Obama Administration as Assistant Secretary for Intergovernmental Affairs at the Department of Homeland Security. With nearly 15 years of experience in counterterrorism and homeland security, she was Massachusetts’ first Undersecretary for Homeland Security, a member of the National Commission on Terrorism, and a legal advisor to Attorney General Janet Reno. Kayyem, named a CNN/Fortune Magazine’s People to Watch, co-wrote the critically acclaimed Preserving Liberty in an Age of Terror. The highest ranking Arab-American woman in federal government, she now is a professor at Harvard’s Kennedy School of Government and a foreign affairs columnist for the Boston Globe.

Richard Falkenrath, who was the Deputy Assistant to President Bush and Deputy Homeland Security Advisor, is no stranger to the complexities involved with the United States’ large-scale effort to combat terrorism. The principal author of the National Strategy for Homeland Security, Falkenrath also served as Senior Director of Policy and Plans within the Office of Homeland Security after 9/11. As the Deputy Commissioner for Counterterrorism at the New York Police Department from 2006 to 2010, Falkenrath strengthened the city’s overall effort to prevent, prepare for and respond to terrorist attacks. Falkenrath is now Principal at The Chertoff Group, a global security and risk-management advisory firm, an Adjunct Sr. Fellow for Counterterrorism and Homeland Security at the Council on Foreign Relations, and a Contributing Editor at Bloomberg News.

Michael Hayden, with a prolific career in national security – from serving 39 years in the U.S. Air Force to directing the NSA for six years – General Michael Hayden has overseen nearly every branch of the intelligence community. Once the highest ranking military intelligence officer in the country, Hayden later became the Director of the CIA in 2006, gaining unprecedented access to the collection of information concerning the plans, intentions and capabilities of America’s adversaries. His remarkable list of senior positions includes Commander of the Air Intelligence Agency, Director of the Joint Command and Control Warfare Center and Chief of the Central Security Service. He is currently a Principal at The Chertoff Group focusing on global political and terrorist risk analysis.

Monday, September 19, 2011

How Well Does the Jesus Seminar Follow Its Own Rules? Part I

At the end of October, I'll be presenting a paper at the annual meeting of the Society for the Scientific Study of Religion (SSSR) that examines how well the Jesus Seminar follows its own rules. In this post I present some of the preliminary results of my analysis of the Jesus' Seminar's voting, focusing in particular on the role that various sources play in determining the authenticity (or inauthenticity) of a particular saying of Jesus. While I don't want to rehash all of what I wrote in earlier posts, it's probably helpful to briefly recall what the Jesus Seminar is, its method of voting, and how they (and other New Testament scholars) use the sources of Jesus' sayings in making their decisions.

Jesus Seminar Redux

The Jesus Seminar is a group of biblical scholars founded by Robert Funk and John Dominic Crossan under the umbrella of the Westar Institute with the stated purpose to develop a scholarly consensus on the sayings attributed to Jesus in the New Testament and other early Christian writings.

The Fellows met twice a year, and each meeting focused on a particular collection of sayings. Members would write and circulate papers on the collection of sayings that was being discussed at the upcoming meeting, so that at the meetings themselves the Fellows would discuss the sayings rather than listening to people presenting their papers. Each saying was discussed until no one had any more to say, and then they would move on to discuss the next one. Once they completed their discussion, the Fellows would then vote by secret ballot, dropping one of four differently colored beads into a ballot box.
  • A red bead meant that a Fellow believed that the words were the authentic words of Jesus
  • A pink bead meant that they believed that the words closely approximated what Jesus said
  • A gray bead meant they believed the words weren’t Jesus’ but they may reflect his ideas
  • A black bead meant Jesus definitely did not speak them
Someone else proposed that the meanings could be understood as follows:
  • Red = “That’s Jesus”
  • Pink = “Sure sounds like him”
  • Gray = “Well, maybe.”
  • Black = “There’s been some mistake.”
In all they voted on 1,544 sayings of Jesus. Often these were multiple instances of the same saying found in different sources (sometimes with slight variations in the wording). They then tallied the votes and summarized there results in “The Five Gospels” as well as providing detail voting records in the group's academic journal, "Forum," which is from where I gathered the data for the preliminary results presented below. Before turning to those, however, a quick word on the criterion of multiple sources (or attestation). 

The Criterion of Multiple Attestation: More Sources are Better!

Most biblical scholars, including the members of the Jesus Seminar, hold that sayings and acts that are found in two or more independent sources are more likely to have originated with the historical Jesus. Identifying the source of a particular saying involves more than simply listing what gospel it came from, however. For example, with regards to the synoptic gospels (i.e., Matthew, Mark, and Luke), most New Testament believe that Mark is the earliest Gospel and Matthew and Luke used Mark as the narrative basis for their own Gospels. They marshaled considerable amount of evidence in support of this theory:
  • Sometimes the same wording & sequence of material is found in Matthew, Luke & Mark
  • Sometimes Matthew & Mark agree in sequence and/or wording of passages, while Luke differs
  • Sometimes Luke & Mark have the same sequence and/or wording of passages, while Matthew differs
  • But the texts of Matthew & Luke almost never agree in both wording & sequence except for material found also in Mark
  • In passage after passage Mark is demonstrably the middle term in any narrative agreement between the synoptic gospels. Thus, the first premise of the two source hypothesis is that Matthew & Luke each followed the text of Mark as their primary narrative sourceA Gospel synopsis, in which all three synoptic Gospels are printed in parallel columns, permits scholars and students to observe how Matthew and Luke used (and changed) Mark to create their own Gospels.
New Testament critics also note that there appears to be a second source on which Matthew and Luke draw. In some places in the Gospels there is considerable verbal agreement between Matthew and Luke where there is no parallel passage in Mark. Although scholars have yet to find this mysterious source, they have given it a name: “Q,” from the German word “Quelle,” which means, “source.” Together, this led scholars to argue for what they call the “two-source” hypothesis, which argues that Matthew and Luke made use of two written sources – Mark and Q -- in constructing their Gospels:


However, scholars couldn’t help noticing that Matthew had material that was neither in Mark or Luke, and Luke contained material that was neither in Mark or Matthew. This led later scholars to expand the two-source hypothesis to a four-source hypothesis, which argues that the author of Matthew used Mark, Q and his own special source (“M”), while Luke used Mark, Q and his own special source (“L”). It is likely that "M" and "L" are drawn from multiple sources, possibly even the same sources, including Q, but for convenience scholars treat "M" and "L" as single independent sources:


In other words, the sayings and events that we find in the synoptic Gospels come from one of four separate sources: Mark, "Q," "M" or "L." What this means is that if scholars are adhere to the multiple sources criterion, they have to look beyond these four sources to find a second independent source in order for a saying that appears in Matthew, Mark or Luke to be deemed authentic. In other words, because any saying or event that appears in Matthew, Mark or Luke can only be traced back to either M, Mark, Q or L, scholars need to look to other sources, such as the Gospel of John, the Gospel of Thomas, the letters of Paul (they include a few sayings of Jesus), the Didache, and so on for additional independent attestations of the event or saying.

Take, for example, the beatitude, "Blessed are the poor, for yours is the kingdom of God" (Luke 6:20). A parallel saying occurs in Matthew (5:3), and scholars trace this saying back to Q (because it appears in Luke and Matthew but not in Mark). In other words, while the saying occurs twice in the Gospels, it is only seen as having one independent source. Scholars do believe there's a second independent source for this saying, however (the Gospel of Thomas), which provides empirical support that the saying goes back to the historical Jesus (at least in theory).

Results

In what follows, I present the voting of the Jesus Seminar fellows in three ways: by gospel, by number of sources, and by the sources themselves. Table 1 breaks down the Jesus Seminar voting (by color) by the four New Testament Gospels plus the Gospel of Thomas. As you can see Matthew contains more sayings of Jesus than any of the other gospels although Luke is not far behind. Mark and John contain surprisingly few (although Mark is pretty short), fewer than Thomas in fact. One thing that is striking is how few sayings the Jesus Seminar voted to be red (31 out of 1,544 or 2.01%). A higher percentage were voted pink (13.67%), but most of the sayings were voted either black or gray by the Jesus Seminar fellows. Of the five gospels, John has the fewest sayings that fellows believed to have originated with the historical Jesus (i.e., those voted either pink or red). In fact almost all of John's sayings were considered to contain no traces of the historical Jesus (95.74%), while a few (3.55%) may reflect his ideas but not necessarily his words. Somewhat surprising (but then again, maybe not), a higher perecentage of Thomas's sayings were voted either pink or red than any of the other Gospels. Luke and Matthew follow, and then comes Mark. Mark's second to last place finish is somewhat surprising given that it is the earliest of the gospels, which makes one wonder how seriously the Jesus Seminar fellows took the dates of sayings into account when casting their votes. Of course, the results presented in this post don't take into account the dates of the sayings; that will be topic for a later post, and may alter the preliminary conclusions presented here.

Table 1: Results by Gospel
Gospel
Color
Black
Gray
Pink
Red
Total
Matthew
234
114
61
11
420
55.71%
27.14%
14.52%
2.62%
100.00%
Mark
92
66
18
1
177
51.98%
37.29%
10.17%
0.56%
100.00%
Luke
185
129
65
15
394
46.95%
32.74%
16.50%
3.81%
100.00%
John
135
5
1
0
141
95.74%
3.55%
0.71%
0.00%
100.00%
Thomas
92
67
40
3
202
45.54%
33.17%
19.80%
1.49%
100.00%
Other
148
35
26
1
210
70.48%
16.67%
12.38%
0.48%
100.00%
Total
886
416
211
31
1,544
57.38%
26.94%
13.67%
2.01%
100.00%

Table 2 breaks the results down by number of sources for each saying and voting color. As it indicates most sayings of Jesus could be traced to only a single source (i.e., 946 out of 1,544 or 61.27%), and most of these (69.03%) were believed to contain absolutely no trace of the historical Jesus. Interestingly, though, more (and higher percentage) single source sayings were voted red than were two and three-source sayings. Only nine two source-sayings and only one (!) three-source saying were voted red. That said, two and three source sayings are much less likely to be voted black and much more likely to be voted gray or pink. In short, the effect that having multiple sources seems to have on a saying's "authenticity" is pushing it out of the black category and into either the gray or pink categories.

Table 2: Results by Number of Sources
Number of Sources
Color
Black
Gray
Pink
Red
Total
1
653
197
75
21
946
69.03%
20.82%
7.93%
2.22%
100.00%
2
175
168
96
9
448
39.06%
37.50%
21.43%
2.01%
100.00%
3
58
51
40
1
150
38.67%
34.00%
26.67%
0.67%
100.00%
Total
886
416
211
31
1,544
57.38%
26.94%
13.67%
2.01%
100.00%

While it appears that the number of sources is important, one can't help but wonder (or at least I can't) whether the source itself matters more than the count. That's where the results presented in the next table (Table 3) come into play. It breaks down the results by source (rather than gospel). You can see that 136 sayings of Jesus can be traced back to sources unique to Matthew, 639 to those unique to Mark, 90 to Luke, 201 to John, 527 to Thomas, 609 to "Q," and 78 to other sources. Note that the numbers for "Lukan" do not match those for Luke in Table 1. That is because in Table 1, the sayings are broken down by gospel. Here they are broken down by source. Remember, the source of sayings that appear in Luke's Gospel can be Mark, Q, or sources unique to Luke (i.e., Lukan sources). Also, because a particular saying can have more than one source, the amounts listed in each column for each source do not add to the total at the bottom.

Table 3: Results by Source
Source
Vote (Color)
Black
Gray
Pink
Red
Total
Matthean
97
21
17
1
136
71.32%
15.44%
12.50%
.74%
100.00%
Markan
330
223
82
4
639
51.64%
34.90%
12.83%
0.63%
100.00%
Lukan
61
12
14
3
90
67.78%
13.33%
15.56%
3.33%
100.00%
Johannine
178
15
8
0
201
88.56%
7.46%
3.98%
0.00%
100.00%
Thomas
201
190
126
10
527
38.14%
36.05%
23.91%
1.90%
100.00%
Quelle
230
216
139
24
609
37.77%
35.47%
22.82%
3.94%
100.00%
Other
69
8
1
0
78
88.46%
10.26%
1.28%
0.00%
100.00%
Total
886
416
211
31
1,544
57.38%
26.94%
13.67%
2.01%
100.00%

The Jesus Seminar fellows apparently preferred Thomas and Q over New Testament sources. They concluded that approximately 25% of the sayings found in the Thomas and Q sources go back to the historical Jesus, compared to 19% of those found in Lukan sources, just over 13% found in the Markan and Matthean sources, and only 4% of those in the Johannine sources.

These results do suggest that perhaps it is possible that the source(s) of a saying matters more than the number of sources. In fact, I estimated two types of multivariate regression models (both an ordinary least squares and an ordered logit model) that regressed the Jesus Seminar's voting results onto a series of independent variables that included the gospel from which a saying came, its source, and how many sources each saying had (as noted earlier, the date of the saying has yet to be factored in). For now, I'll spare you the details of multivariate models; all I'll note here is that they allow analysts to separate genuine causes from spurious ones. In other words, they can help determine questions such as whether what really matters is the actual source of a saying or the number of independent sources lying behind it. What both models found was that the former matters more than the latter with "Q" and "Thomas" being the two most important sources. In fact, after "controlling" for all the different sources, the number of sources actually has a negative effect on the voting outcome. That is, more sources meant a lower (not a higher) score, which, of course, calls into question how seriously members of the seminar took this criterion into account.

Of course, these results are preliminary. I still have to control for other criteria (e.g, date, dissimilarity, length of a saying, etc.). Still, preferring non-canonical sources over canonical ones has been one of the charges leveled by critics at the Jesus Seminar (see e.g., Philip Jenkins, "Hidden Gospels: How the Search for Jesus Lost Its Way"). Even though most mainline New Testament (NT) scholars believe that many of the sayings of Jesus found in the NT Gospels don't originate with the historical Jesus, they still consider the Gospels to the primary source of reconstructing who Jesus was, what he did, and what he taught. This doesn't appear to be the case for members of the Jesus Seminar.