Mechanical Turk (MT) is an online labor marketplace run by Amazon in which large groups of online workers ("Turkers") can be recruited to carry out routine tasks that for whatever reason can't be automated -- tagging content in online videos for instance. (The service is named after a famous 18th-century fraud in which people were fooled into believing they were playing chess against a life-size supposedly automated "Turk" who was in fact controlled by a skilled human player.)
In recent years, MT has become a popular way for social scientists, particularly psychologists, to recruit test subjects. Turkers come cheap after all -- they're often paid as little as $1.50 an hour -- and are available in abundance. A number of the papers I've discussed on this site have relied in whole or in part on MT samples.
In a two-part blog post, Dan Kahan -- a professor of law and psychology at Yale University -- takes researchers to task for relying on Turk subjects in their research. Kahan discusses three primary flaws with Turk samples:
1. Selection bias. Given the types of tasks performed by MT workers, there's good reason to suspect subjects recruited via MT differ in material ways from the people in the world whose dispositions we are interested in measuring, particularly conservative males.
2. Prior, repeated exposure to study measures. Many MT workers have participated multiple times in studies that use performance-based measures of cognition and have discussed among themselves what the answers are. Their scores are thus not valid.
3. MT subjects misrepresent their nationality. Some fraction of the MT work force participating in studies that are limited to "U.S. residents only" aren't in fact U.S. residents, thereby defeating inferences about how psychological dynamics distinctive of U.S. citizens of diverse ideologies operate.
On the first point, Kahan notes that MT workers are 62 percent female, only 5 percent African-American (the general U.S. population is 12 percent), and 53 percent self-identified liberals (only 20 percent of the general population.)
The problem of representativeness in test samples isn't limited to MT of course. A 2011 University of British Columbia Study noted that "in the top international journals in six fields of psychology from 2003 to 2007, 68 percent of subjects came from the United States -- and a whopping 96 percent from Western, industrialized countries. In one journal, 67 percent of American subjects and 80 percent of non-American subjects were undergraduates in psychology courses." This means that a good deal of what we know about the human brain comes from research on so-called WEIRD -- Western, educated, industrialized, rich, and democratic societies -- test subjects.
Still, if Turk-recruited subjects are essentially becoming professional test-takers, misrepresenting their qualifications and even their nationality in order to be eligible for studies, that's a bigger problem than making generalizations based on the responses of UC undergrads.
And the bad press continues for psych research.
One of the few things Egyptians and Americans seem able to agree on during the events of the past few weeks is that they don't like how CNN has covered them. In Egypt, as my colleague David Kenner explains, protesters were upset by the network's decision to call his ouster a "coup" as well as its mislabeling of an anti-Morsy demonstration.
Here in the States, the Most Trusted Name in News has taken flack for downplaying the events in Cairo in favor of its wall-to-wall coverage of George Zimmerman's trial. As New York University's Jay Rosen put it, CNN President Jeff Zucker "wants everyone in his company to know what the priorities are: Mini-series in the center, world events off to the side." (This critique presumably applies to domestic CNN rather than CNN International.)
On the other side, defenders of the network such as Politico's Dylan Byers and Reuters's Jack Shafer point out that high-profile trials are a ratings boon for the network and it's not as if people interested in Egypt don't have other options in the modern media landscape.
"In today’s media environment, the media critic who insists that the cable networks follow Egypt and drop Zimmerman is like the nudging dining companion who wants to order both his meal and yours, lest you embarrass him by mistakenly ordering the burger and fries," Shafer writes. "He finds the burger and fries déclassé and bad for you and would rather you add something more tofu-and-wheatgrassy to your media diet."
(Update: Rosen has a follow-up post addressing these objections in greater detail.)
I'm obviously always in favor of major media outlets spending more time on world events, but I feel as if the way this debate is being framed -- serious important Egypt coverage vs. tawdry courtroom drama -- to be too simplistic on both sides.
First of all, as Aaron David Miller wrote last week, foreign-policy wonks have an irritating habit of assuming that global politics take place in some exalted Olympian realm while there's something distasteful about domestic issues. Putting aside the question of whether it warrants 24-hour wall-to-wall coverage, the Trayvon Martin story isn't just another celebrity trial -- it initially garnered public interest because it involves issues of racial profiling and gun violence that affect many Americans.
The issue is less that CNN is covering a trial that has excited major public interest, than how it's covering that trial. As former CNN producer Sid Bedingfield writes on Gawker, "Is this the sort of trial that should receive the full-on Reality-TV show treatment? Probably not.… The goal of the criminal-trial-as-entertainment genre is simple: Hook the viewer into the narrative. Get them emotionally invested in the characters. And, most importantly, persuade them to choose sides. Team Zimmerman vs. Team Trayvon."
It's one thing when world events are pushed aside for a big domestic story; it's another when they're pushed aside for a 15-minute debate on "N Word vs. 'Cracker': Which Is Worse?"
There also seems to be an assumption here that reporting world news has to be the equivalent of getting viewers to eat their vegetables before you transition to the stuff they actually want. Even the network's own John King seems resigned to this way of thinking, saying, "The American people don't often pay attention to what's going on in the world until they have to."
Not only is this a sad statement from the network that made it's reputation on the fall of the Berlin Wall and the first Gulf War, but isn't it the network's job to tell viewers why they have to pay attention to a story and to make it interesting to them?
I'm also not quite sure I buy that U.S. television viewers have no interest in the world beyond their borders. CNN's own most high-profile programming addition this year has been a documentary series that drops Anthony Bourdain into places like Burma, Libya, and Congo. Granted, Bourdain is a celebrity chef, not a reporter, but it's still a sign the network thinks there are prime-time viewers interested in spending some time learning about the M-23. HBO has made a similar bet on the thrill-seeking gonzo hipsters of Vice.
Again, I'm not necessarily holding up either of these programs as journalistic paragons, but they suggest that an organization with the resources of CNN should be able to find a way to interest viewers in an important story, even if it happens in another country. If viewers will watch a documentary about international issues, why not breaking news? From a pure storytelling point of view, it certainly seems like there's more human drama in Mohamed Morsy's downfall than in the testimony of Zimmerman's personal trainer.
The issue shouldn't be whether CNN covers Trayvon or Egypt. It's how it -- and the rest of the media -- could do a better job covering both of them.
It's generally taken as a given in contemporary political debates -- and sometimes expressed not all that tactfully -- that there's a negative correlation between economic well-being and fertility. People in rich countries, and richer people within countries, tend to have fewer kids.
But a statistical analysis by Princeton University's Tom Vogl, published by the National Bureau of Economic Research, suggests this correlation is relatively recent in much of the world. Vogl used fertility data to create "cross-sections of families from 20 countries in the 1986-1994 and 2006-2011 periods." Here's what he found:
"Between these periods, the relationship between parental economic status (measured by durable goods ownership) and the number of surviving children flipped from positive to negative in the African countries in my sample, as well as in the rural parts of Asia in my sample. The relationship was negative throughout in Latin America, leading one to wonder whether these data capture the tail end of a global transition.…
A wide range of data from 48 developing countries reveals that both associations were indeed positive well into the 20th century. They became negative only recently: first in Latin America, then in Asia, and finally in Africa.
Vogl also finds that in his sample, "increases in the parent generation's education were by far the most important predictor of the reversal," as opposed to more commonly cited factors like GDP, child mortality, urbanization, and women's labor force participation, countering the stereotype that poor families have more children because they're needed for farm labor or because it's assumed that some won't make it to adulthood.
This is interesting to consider in light of other research making the case that gains in "effective labor supply"-- healthier, more educated people -- can make up for losses in productivity as fertility rates fall in wealthier countries.
BEHROUZ MEHRI/AFP/Getty Images
I've written a few posts now on GDELT -- Global Data on Events, Location, and Tone -- the recently-created auotmatically updating database of more than 200 million political events culled from news accounts.
Over at the Monkey Cage, computer scientist David Masad has a guest post testing out GDELT observations on violent incidents in Syria against volunteered reports of fatalities. He found that "the two trends initially move together: an increase in violent events accompanies an increase in reported deaths, and vice versa," but "the correlation between the two data sources seems to weaken in 2012."
This would seem to support Jay Ulfelder's argument that GDELT is vulnerable to the problem of "media fatigue": " press coverage of a sustained and intense conflicts is often high when hostilities first break out but then declines steadily thereafter." As the level of coverage drops, there are fewer reported events for GDELT to record, even though real-world events are still happening.
I spoke with Kalev Leetaru, one of GDELT's developers, about this issue back in April. "As quality journalism is under attack from all sectors, whether that's government stepping up efforts to squelch it or the collapsing economics of it, we're starting to look at all the citizen journalism that's out there," he told me. On the other hand, social media and participant-reported news creates more even quality-control problems.
GDELT's reliance on the media doing it's job well is clearly going to continue to be a problem going forward, particularly with regions that don't get as much attention. Even with falling coverage, GDELT does a remarkably good job monitoring events in a well-covered conflict like Syria. I'm a little more curious to see how it does on, say, Nagorno-Karabakh.
Comedies about narcissistic yuppies facing the apocalypse seem to be the hottest genre in Hollywood right now, with This is the End, It's a Disaster, The World's End, and Rapture-Palooza hitting theaters this year alone. So it seems as good a time as any to take a look at what a real world catastrophe would mean for the value of your home.
We estimate spatially disaggregated impacts on median sales price of residential housing within the Seattle metro area following an attack on the central business district (CBD). Using a combination of longitudinal panel regression and GIS analysis, we find that the median sales price in the CBD could decline by as much as $280,000, and by nearly $100,000 in nearby communities. These results indicate that total residential property values could decrease by over $50 billion for Seattle, or a 33% overall decline. We combine these estimates with HUD's 2009 American Housing Survey (AHS) to further predict 70,000 foreclosures in Seattle spatial zones following the terrorism event.
Sean Gallup/Getty Images
University of Alabama criminologist Adam Lankford has done a lot of interesting and provocative work on terrorists and mass killers -- arguing for instance that suicide bombers should be viewed not as rational political actors but as disturbed suicidal individuals motivated by similar factors to school shooters like those and Columbine or Sandy Hook.
A recent study by Lankford looks at data on mass shootings in the United States between 1966 and 2010 in order to look at the differences between shooters who survive their attacks and those that die with their victims. Using an NYPD database of "active shootings" -- those engaged in killing people in a populated area, excluding gang shootings, domestic violence, robberies, drive-bys and other more conventional crimes -- the study isolates a sample of 185 attacks, with the number of casualties ranging from two to the 32 killed by Seung Hui-cho at Virginia Tech in 2007. The shooters were 96 percent male with a mean age of 34.03.
38 percent of perpetrators of mass shootings commit suicide by their own hand and 48 percent die in the attacks. (Some of those extra 10 percent could probably also be categorized as "suicide by cop".)
Lankford also found that for each additional victim killed in the attack, the shooters' likelihood of dying was 1.2 times higher. Shooters are also far more likely to die when they bring multiple weapons to the scene and when attacking factories or warehouses rather than schools and office buildings.
The data in the paper is just from U.S. shootings and it would be interesting to know if the same patterns hold in other countries. Europe's most notable mass killer in recent years, for instance, is still very much alive.
Another day, another round of government secrecy revelations. Jennifer Valentino-Devries and Siobhan Gorman of the Wall Street Journal report that many of the NSA surveillance activities revealed in Edward Snowden's recent leaks were justified by Foreign Intelligence Surveillance Court rulings from the mid-2000s, which used a broad interpretation of how the word "relevant" is phrased in the Patriot Act to permit a vast range of previously restricted data gathering.
As Eric Lichblau detailed in another article on the court over the weekend, the FISC, or FISA court as it's sometimes known, has become a kind of "parallel Supreme Court," issuing important decisions that are almost never made public or subjected to review by other courts.
In a seperate development, the AP reports that, Adm. William McRaven, head of the U.S. Special Operations Command, has "ordered military files about the Navy SEAL raid on Osama bin Laden's hideout to be purged from Defense Department computers and sent to the CIA" in a possible violation of the Freedom of Information Act. Reporter Richard Lardner speculates that this move "could represent a new strategy for the U.S. government to shield even its most sensitive activities from public scrutiny."
Stories like these contribute to the public perception that secrecy -- both decisions made in secret and information being indefinitely kept from the public -- is increasingly become the norm rather than the exception in the U.S. national security community. Much of the secrecy surrounding counterterrorism operations in particular may be necessary -- or at least officials genuinely feel that revealing certain information would be damaging to national security. The less generous interpretation is that a great deal of information is being kept classified to shield it from scrutiny. As others have pointed out, it's a bit disingenuous for the president to "welcome debate" on policies his government has kept out of the public eye for years.
But there's also another way to look at the incentives behind government secrecy, one explored in a recent paper published in the journal Political Psychology. The authors, Mark Travers, Leaf Van Boven and, Charles Judd of the University of Colorado, suggest that readers tend to believe information and recommendations are of higher quality when they're "secret."
In one experiment, subjects read two documents from 1995 about whether the U.S. should intervene to prevent the sale of fighter jets from Belarus to Peru -- a real debate from this era -- one from the State Department supporting intervention and one from the National Security Council opposing it. Some participants were told that the State document was "classified" at the time the decision was made, others were told that it was the NSC that was secret. "On average, the judgment of information quality when it was secret was significantly greater than the judgment of information quality when it was public," they write.
In another test, one document -- a 1978 NSC memo about the sale of military jets to Taiwan -- was judged to be of much higher quality by participants when they were told that it was a "secret document" that had been declassified on the Freedom of Information Act rather than a "non-classified, public document."
The authors posit that there's a "secrecy heuristic" that people use when evaluating the weight of information. Specifically, "(a) people weigh secret information more heavily than public information when making decisions, (b) people perceive the same information as being of higher quality when it is portrayed as secret rather than public, and (c) people evaluate otehrs' decisions more favorable when those decisions are based on secret information rather than on public information."
The authors suggest this effect was very much in evidence following the original Wikileaks disclosures, when people who are normally skeptical about the judgments of U.S. officials were suddenly taking as gospel documents written privately by those very same officials. As Dan Drezner wrote at the time, there was a "natural inclination to think that any Wikileaks document will endow it with the totemic value of Absolute Truth. "If it was secret, then it must be true," goes this logic." In fact, it's quite possible for diplomats or military commanders to be as wrong in private as they are in public.
It's also not hard to imagine the kind of incentive structure that this bias creates. If classified information and secret recommendations are judged to be more credible, there's going to be a natural tendency for the officials crafting and passing along this information to keep as much of it secret as possible.
The New York Times article on the FISA court notes that the judges issued nearly 1,800 surveillance orders last year and didn't deny a single request from intelligence agencies. We obviously can't judge the information these decisions were based on, but assuming that the 11 judges who visit the capital in shifts to meet in a "secure, windowless courtroom" have brains that are wired like the rest of ours, the very secrecy of these requests and the setting in which they're made could be a factor in why they're basically always approved.
We tend to assume that secrets are important. Otherwise, why would they be secret? Therefore the people who have access to secrets are important and worth listening to. It's not long before you have an incentive for more and more of the government's business to be conducted out of public view.
Events are developing quickly in Egypt but it seems that the Egyptian military is currently intervening to remove President Mohamed Morsy from power following a week of massive protests against the Muslim Brotherhood-led government.
Morsy's national security advisor has described what's happening as a "military coup"and by the traditional definition of that term -- "when the military or a section of the military, turns its coercive power against the apex of the state, establishes itself there, and the rest of the state takes its orders from the new regime" -- this certainly seems to fit the bill. [Update: It's official. The Army "told President Morsi at 1700 GMT he was no longer president". SCAF leader Abdul Fatah Khalil al-Sisi announced on television that the Constitution is being suspended and a "technocratic" government will be installed led by the chief justice of the Supreme Court.]
So how then are supporters of democracy in Egypt -- both the crowds in Tahrir and foreign observers -- to think about these events? Traditionally, military coups are thought of as the antithesis of the democratic process -- raw political power being weilded through the barrel of a gun rather than a ballot box. In fact according to U.S. law --albeit a frequently skirted law -- foreign aid cannot be provided to governments that took power in military coups. (However they respond to today's events, don't expect Obama administration officials to be throwing around the word "coup" this afternoon.)
But are there cases when a coup can advance democracy? In a 2012 article for the Harvard International Law Journal, Ozan Varol, now a professor at Lewis & Clark Law School, argues that while the vast majority of military coups are undemocratic in nature, and lead to less democratic political regimes, there are significant examples of "democratic coups d'etat."
If the concept seems ridiculous, consider the fact that tomorrow Americans will celebrate an armed uprising to overthrow an autocratic government. Why are bloody insurgencies sometimes considered legitimate but not actions taken by established militaries acting on behalf of disenfranchised citizens?
Varol cites three case studies: the 1960 Turkish Coup in which the military overthrew the ruling Democratic Party, which had gradually consolidated political power and cracked down on political opposition and the press; the 1974 Portuguese Coup, also known as the Carnation Revolution, in which the authoritarian "Estado Novo" was overthrown by the military after tanking the country's economy and embroiling it in a series of unpopular wars in its African colonies; and -- interesting in this context -- the 2011 ouster of Hosni Mubarak.
Varol argues that there are seven characteristics a coup must generally meet in order to be considered democratic:
(1) the coup is staged against an authoritarian or totalitarian regime; (2) the military responds to persistent popular opposition against that regime; (3) the authoritarian or totalitarian regime refuses to step down in response to the popular uprising; (4) the coup is staged by a military that is highly respected within the nation, ordinarily because of mandatory conscription; (5) the military stages the coup to overthrow the authoritarian or totalitarian regime; (6) the military facilitates free and fair elections within a short span of time; and (7) the coup ends with the transfer of power to democratically elected leaders
So does what's happening in Egypt right now fit the bill? For the last two criteria, it remains to be seen. Two through five are arguably a good fit. But the first and most important one is a tough sell. Morsy was democratically elected just a year ago. To the extent that that election was marred by political interference, it was to the detriment of the Brotherhood.
On the other hand, Morsy's opponents would probably argue that the Brotherhood was itself engaged in what's sometimes called a self-coup or autogolpe, in which a democratically elected goverment gradually erodes the country's political institutions in order to keep itself in power -- in Morsy's case, increasing the power of the executive through a series of presidential decrees.
The military will argue that its actions were necessary to prevent the emergence of a new authoritarian strongman. The good news is that around the world, coups now more frequently result in a quick return to the normal democratic process than in the bad old days of the Cold War, but it's also possible Egypt may be in for something like the old-fashioned Turkish model in which the government was nominally democratic but the military would step in periodically to make "corrections". There's some evidence to suggest the Egyptian military has been interested in such a model since Mubarak's ouster.
The Egyptian military's actions over the next few weeks will largely determine how history views today's events and the danger of admitting the existence of "democratic coups" is that coup plotters almost always describe what they're doing as safeguarding democracy even as they accumulate power for themselves. Whether something is a "coup" or a revolution, and whether or not that coup is democratic, is generally in the eye of the beholder.
Update: Jay Ulfelder has more on why this meets the definition of a coup.
KHALED DESOUKI/AFP/Getty Images
War of Ideas is a blog on the theory behind the practice of global politics. Foreign Policy associate editor Joshua E. Keating brings you the latest research, data, and intellectual debates from around the world.