Messing with Montana: Get-out-the-Vote Experiment Raises Ethics Questions

by Melissa R. Michelson

Today, the Internet exploded with news about and reactions to a get-out-the-vote field experiment fielded by three political science professors that may have broken Montana state law and, at a minimum, called into question the ethics of conducting experiments that might impact election results. No stranger to such controversy myself—a postage experiment I conducted with some colleagues in November 2010 (Michelson et al 2012) raised the ire of one losing candidate and generated some unpleasant media coverage—I am sympathetic to the idea that the researchers meant no harm. But harm has indeed been done, to the experiments subfield and to the discipline more broadly, if not to the people of Montana.

First, let’s review the facts. Adam Bonica and Jonathan Rodden, Political Science professors and fellows at Stanford’s Hoover Institution, along with Dartmouth Political Scientist Kyle Dropp, obtained $250,000 from the William and Flora Hewlett Foundation and a matching $100,000 grant from Stanford University to conduct the experiment. According to Stanford spokeswoman Lisa Lapin, the project was approved by the Dartmouth Institutional Review Board (IRB).

The experiment was conducted in three states, including Montana (100,000 mailers), New Hampshire (66,000 mailers) and California (143,000 mailers). The mailers sent to Montana voters, where the controversy is raging, included the state seal and information about the ideological placement (compared to President Barack Obama and former Republican Presidential Candidate Mitt Romney) of candidates for two non-partisan elections to the Montana Supreme Court. The mailer was labeled “2014 Montana General Election Voter Information Guide.” The ideological placement of the judicial candidates is based on a methodology developed for Bonica’s dissertation research, using a large-scale database of campaign contributions. One of those judicial races, between incumbent Justice Mike Wheat and challenger Lawrence VanDyke, is hotly contested, and has seen significant outside spending.

Using the state seal without permission may have violated Montana election law, which states that it is a misdemeanor to “knowingly or purposely disseminate to any elector information about election procedures that is incorrect or misleading or gives the impression that the information has been officially disseminated by an election administrator.”

In addition to a charge of violating that law, Secretary of State Linda McCulloch has filed a legal complaint charging that the mailers also violate three campaign laws: 1) A ban on “fraudulent contrivance” that could cause a person to vote a certain way; 2) A prohibition on the dissemination of information that gives incorrect or misleading election procedures; and 3) A requirement that a person or group engaging in political activity register with the state.

U.S. Senator Jon Tester sent a strongly worded letter to the presidents of Stanford and Dartmouth on October 24 that began, “As I am sure you are now aware, Montanans recently received a misleading campaign flier in their mailboxes that sought to inject partisanship into non-partisan Montana Supreme Court elections.” Tester goes on to denounce the “so-called research project,” and adds, “Efforts to undermine elections in Montana – whether by fraud or merely by poorly-designed experiments – must not be tolerated.”

To me, this is more important than the possible violations of election and campaign law, which perhaps will result in the Bonica team paying some fines. The grouping together of fraud and experiments, and the interpretation of the mailers as an effort to undermine the Montana elections, illustrates the true price that is being paid in terms of public perceptions of political science experiments, if not the broader discipline.

Montanans are sensitive to outside groups coming to their state trying to influence elections. One of the top stories of the year in 2012 was the “dark money”—money whose sources and donors don’t have to be disclosed, thanks to Citizens United—that flooded the state with television ads and mailers. This followed scandals in the previous two election cycles, 2008 and 2010, of illegal coordination between funders and candidates by Western Tradition Partnership.

The sordid details of those previous dark-money scandals set the backdrop to Montanans’ outrage at the idea that an outside source was coming in to try to influence their election; the Bonica group blundered in not anticipating how that context might color reaction to their mailers.

Was the experiment ethical? The Bonica team chose to experiment with non-partisan Montana judicial races, as opposed to legislative races where ideology and partisanship often play a visible role, inserting partisanship (by placing the judicial candidates on a continuum with Obama and Romney) in a place where many think it does not belong. The tradition of a non-partisan, independent judiciary has deep roots in the West, stemming from the Progressive Era; regardless of whether or not laws were broken, the mailers blatantly violated that tradition. An interesting research question? Perhaps. But just because something is interesting doesn’t mean you should do the experiment.

Another ethics issue arises from the closeness of the election, and the possibility that the content of the mailers might have influenced the outcome. To maximize external validity, experimenters logically want to conduct their research in real elections. But should such experiments, particularly persuasion experiments, be conducted in contexts where they might determine outcomes? This is precisely the sort of experimentation that raises eyebrows about the ethics of doing work outside of laboratories or without informed consent. Dawn Teele claims in Field Experiments and Their Critics (2012: 119) that ethical field research requires that the research itself leave no trace.

Just last year, the Coburn Amendment struck a major blow to political science National Science Foundation grants. Political science was dismissed as wasteful and not in the national interest. Experiments that strike the public as unethical, possibly illegal, and deceptive, are harmful to our image as a discipline and will make it more difficult for future studies to obtain funding and IRB approval. Intentionally or not, these researchers are guilty of damaging the field.

Melissa R. Michelson is a professor of political science at Menlo College. She holds a Ph.D. from Yale University. Her publications Living the Dream: New Immigration Policies and the Lives of Undocumented Latino Youth (Paradigm Publishers, 2014) and Mobilizing Inclusion: Transforming the Electorate Through Get-Out-the-Vote Campaigns (Yale University Press, 2012).

33 thoughts on “Messing with Montana: Get-out-the-Vote Experiment Raises Ethics Questions

  1. This is really interesting stuff. I don’t have a substantive contribution per se, other than when I was in grad school my advisor and I talked about running an experiment where we ran a fake candidate for an election – and our independent variable would be the density/location of yard signs. That would be the *only* campaigning our “Alan Smithee” would do. We found our pretty quickly this could be construed as fraud. I guess if there’s any take home, it’s that a great deal of care needs to be taken when experimented around elections – though I totally understand the benefits/interest in doing so.


  2. A nice framing of the dilemma: How do you conduct a political experiment without interference or the appearance of interference?

    I’d be interested to know if there was any partisan campaigning for the campaign, and what the materials supplied by the candidates and their supporters and detractors looked like. Was the experiment the only injection of partisanship?


    • It was not the only injection of partisanship. The VanDyke campaign sued to attempt to invalidate the non-partisan designation as a violation of free speech. The suit was thrown out, but the filing of it identified him as an extreme Republican conservative. In a year that saw a lot of support for Republicans in Montana his campaign went down in flames. Montana’s in general hold strong views about the value of a non-partisan judiciary and I don’t think they liked the sneaky methods of VanDyke’s campaign, of which there were many.


  3. Pingback: Powerful blog posting on the Montana field experiment

  4. Good commentary Melissa. Hard to understand how the Dartmouth IRB would approve this, but maybe they have a different definition of research ethics up there. On the bright side, it gives me some good material for my senior seminar discussion on research this week!


  5. Pingback: The Worm Turns in Mailergate – Here’s the Latest |

  6. Pingback: Well, In a Worst Case Scenario, Your Treatment Works… | The Math Of Politics

  7. Pingback: More on GlacierGate: If non-partisan judicial elections are not a democratic good, can we conduct research intended to actively undermine them?

  8. What surprises me about the IRB is that the falsification of the document presents a legal liability for all of those institutions to be sued by the various states or the parties. IRBs really aren’t ethics boards but rather legal liability boards. My guess is that the state seal wasn’t part of the original plan.


  9. If Adam Bomica wishes to change the Montana Constitution there’s a legal way to do that, though I doubt we Montana voters would go along.
    We revolted 100 years ago against the Copper Kings’ corporate control of our state and made out highest court a non-partisan office.
    Instead, 10 days before the election, Bornica and his Hoover Institute fellows violated four state laws, possibly federal mail fraud statutes plus its contactual obligations under California law when it ignored the Hewlett Foundation’s strict ethical standards while taking Hewlett’s $250,000.
    Welcome to dark money…
    But then maybe it wasn’t a political science experiment at all.
    Maybe it was a test of the Heisenberg Uncertainty Principle.


  10. I don’t agree with the conclusions here. This was a proof-of-concept run for DIME’s methods, and while I squirm that Crowdpac might commercialize this, the reality is, DIME seems sound and robust.
    I am pretty up on Montana politics and funny money, and my first reaction was, some GOP operatives had gotten ahold of DIME data and tried the same nuclear stunt as the Democrats/League of Conservation Voters/Montana Hunters and Anglers last minute Dan Cox mailing of 2012.
    As for the mailing, the information is not biased. Comparative information can be critical to voters if an objective and robust means of comparing “knowns” to “unknowns” can be made available.
    Finally, what a lot of you might not know is DIME also found that Montana’s supreme court is the sixth most liberal in America — but the electorate most certainly is not the same as Vermont or Washington state. In fact, the courts seem to be a last bastion for Democratic politics in Montana. At least two of the justices, including Wheat, are former Democratic politicians, and ever more were Democratic donors and were supported in their races by, yep, donors who support Democrats almost exclusively.
    No wonder the people screaming the loudest are, yep, Democrats.


  11. Nice read. Now the anticipated wait for those to chime in with I hate this or that surely coming from someone who doesn’t vote. If that doesn’t put the pee in your Cheerios I don’t know what will.


  12. The observer effect comes through again. You know, though, if you seriously have a problem with this research on the basis that private funding may have massively effected voters going with one candidate or the other, you should probably consider how any election even happens – private money is the lion’s share of all campaign donations. And each one of those could be considered an experiment in politics by people who are not even experts .

    Should it need to be doled out, I prefer the money going to affect elections (experimental or not) at least be doled out by people interested in delivering more easily-digestible, accurate academic information.


  13. Thank you for this post. This is something I never would have heard about if you hadn’t been freshly pressed. There are some interesting questions here, and I enjoyed your commentary.


  14. Melissa, why such a large sample in Montana? I mean, percentage-wise, 143,000 in California is a small number compared to 100,000 in Montana. Seems like Montana was targeted for some reason.


  15. Pingback: Ethics and research in comparative politics - The Washington Post

  16. Pingback: Monkey Cage: Ethics and research in comparative politics | IGV News Online


    For decades I refused to believe it, but it is now incontrovertibly established. The Colorado Supreme Court is indisputably a political actor. Our Colorado Supreme Court exists to serve Colorado political parties. At present, the Colorado Supreme Court is more rightly considered an adjunct of the Colorado Legislative Branch, than a check on the Colorado Legislative Branch. Rather than “truth-seeking,” the Colorado Supreme Court now sees its role as “political-outcome seeking.” Litigants successfully use the Colorado Supreme Court to achieve political purposes. In the Ralph Carr Justice Center, rather than meeting impartial guardians of the law, litigants meet their political allies on the bench.

    “I think there are many who think of judges as politicians in robes. In many states, that’s what they are.” “They seem to think judges should be a reflex of the popular will.”

    Sandra Day O’Connor

    In this article, I provide an example of the political and partisan role of the Colorado Supreme Court. I describe a case in which the Colorado Supreme Court summarily erases billions of dollars of debt owed by Colorado state and local governments. That is, one branch of Colorado state government relieves another branch of Colorado government of its legal debts.

    The case involves Colorado statutory contracts that create financial obligations on the part of Colorado governments. Over decades, political considerations induced the Colorado Legislature to mismanage those financial obligations. In recent years, the terms of those statutory contracts were deemed politically inconvenient and politically unpopular. The Legislative Branch asked the Colorado Supreme Court to discard the contracts.

    In 2010, the Colorado Legislative Branch requested that the Colorado Supreme Court grant this political favor by ignoring the Contract Clause of the US Constitution, ignoring the history of legislative mismanagement of these state financial obligations, and relieving Colorado governments of their accrued legal debts.

    In this article, I address the Colorado Supreme Court’s lack of independence, integrity, and impartiality. I provide a brief history of the efforts of the Colorado Legislature and the Colorado Supreme Court to escape Colorado governmental financial obligations. I comment on the recent (October, 2014) Colorado Supreme Court Decision itself, which summarily erased these billions of dollars of Colorado public sector debt. I highlight some of the numerous factual and logical errors that exist in the Colorado Supreme Court’s Decision in the case. I express incredulity at the Colorado Supreme Court’s willful ignorance of public pension administration, knowledge that was necessary to any court claiming to “seek truth” in the case.

    My intent in writing this article is to enhance the public record of, and further document, what I consider to be one of the greatest “crimes” in Colorado history.
    Visit the following link for the complete article:


  18. Pingback: Encountering your IRB: What political scientists need to know | The Political Methodologist

  19. Pingback: Why IRB Matters: A Real-Life Example from a Political Science Experiment Gone Wrong | PSA Newsletter

  20. Melissa, I’ve just read your follow-up from December in the Pi Sigma Alpha newsletter, “Why IRB matters….” I appreciate your emphasis there on the possible harm these experimenters caused political science research, by undertaking illegal [in re. use of 2 states’ seals without permission] and possibly unethical experimentation, and also by flaunting the 2 campus IRBs. It’s the latter that prompts me to write. Do you — or does anyone reading this — know what the final disposition of the IRB matter was? E.g., whether, in fact, Dartmouth’s IRB had reviewed the research proposal before it was launched?

    In answer to Andy Aoiki’s and other questions/comments posted above:

    1. when research involves people at different campuses, universities can delegate the review of the proposal to other campuses’ IRBs; so, Stanford might have indicated that they would accept Dartmouth’s review. But research is not supposed to begin before that review is received.

    2. yes, different universities — that is, their IRBs — interpret the federal policy differently, but all researchers working with human participants AND who receive NIH funding AND who do what the IRB policy defines as “research” are supposed to submit their proposals for review. That’s the de minimus position. Many campuses extend this policy to include all funded research, regardless of the source of that funding, and others simply include all research, funded or otherwise. The question then is, Is this “research”? The policy defines that as intending to contribute to general knowledge. This is what enables many universities to exempt course-related exercises from review [think: research methods courses, which include, e.g., an exercise on interviewing].

    For more on this, see the link above to the condensed version of Peri Schwarz-Shea and my essay “Encountering your IRB.”

    I’m still curious to know, though, whether Dartmouth in fact reviewed the proposal.


  21. Pingback: What does informed consent mean when conducting a field experiment? - The Washington Post

  22. Pingback: Research Ethics and Public Opinion – Political Violence at a Glance

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s