University of Chicago MAPSS Program MAC Prime Talk

Presentation delivered on March 14, 2014.

I wanted to study political theory. Before I arrived in Chicago I had blithely dismissed data and quantitative analysis as tools for understanding the social and political structures that really fascinated me. Yet within the first few weeks I found myself completely absorbed in a quantitative methods class that used data to understand why people vote. At a basic level, taking Eric Oliver’s Introduction to Data Analysis class parallel to Perspectives helped me understand how thoughtful analysis of data is just another epistemological approach to categorizing social phenomena. In the winter I worked so hard on Causal Inference and American Political Behavior with Betsy Sinclair. They taught me the mechanics of coding data, proper statistical techniques, and how to think and write clearly about quantitative analysis. My thesis, with Prof. Sinclair, used national survey data to identify factors that correlated with misidentifying President Obama’s religion as non-Christian. Two years later I was working for the President’s reelection campaign as a data analyst. I used skills from MAPSS to build a career in progressive political data analysis, an entire field dedicated to using smart data-driven research to answer important questions and build a better country through political change.

#StateOfEnrollment: Make It Personal

Originally published April 21st, 2014 on Enroll America's State of Enrollment blog series.

Strategy: Motivate consumers by giving them personalized information

Where it was used: Our entire digital and field outreach

Notable metrics: The right message can boost intent to enroll by six percent relative to other good messages.

Best Practices to Replicate: Making sure consumers get personalized information about how the Affordable Care Act (ACA) impacts their lives motivates them and makes them feel more informed.

At Enroll America, we believe in making data-driven decisions wherever possible. One of the areas that might seem a little difficult to measure is figuring out what kinds of messages are most motivating to consumers. However, this past December and January we conducted research that helped us quantify the best messages for motivating consumers. We found that, despite testing many ideas that seemed like they would work, the best way to motivate consumers was to get them personalized information about how much financial assistance they might be able to get, and what this meant for how much they would pay for health insurance.

Picking the right message is challenging. If you talk to 10 different people, you’ll often get 10 different ideas about what to say. We thought quantitative research might be able to shed some light on which messages were making a measurable difference. So, last winter we recruited thousands of uninsured people who were eligible for the health insurance marketplaces to take an online survey. During the survey, we randomly exposed people to several different messages that were developed in consultation with experts in behavioral science and health care outreach.

Here are the messages that were tested:

• Using the Get Covered America calculator. Understanding the effect of using the calculator was the highest priority of the test. The calculator (available here) gives consumers specific, personalized information about what form(s) of coverage they might be eligible for under the ACA, and how those insurance options affect them financially. Respondents exposed to the calculator were given the opportunity to input their own information and view their customized results.
• Savings message frame. The savings message frame presented insurance purchased through the marketplace as a “good deal” financially. We wanted to test the effect of showing consumers how much money they could expect to save, relative to what they would pay without financial assistance.
• Fine message frame. The fine (penalty) message frame informed consumers of the specific details of the fine they would have to pay if they didn’t get insurance. The message made a strong case by mentioning specific dollar amounts and making explicit the trade-off between paying the penalty and purchasing health insurance.
• Risk message frame. The risk message frame explained that tens of millions of Americans will spend at least $5,000 in health care expenses every year. The idea was to motivate uninsured people to get health insurance by both informing them of their risk of having to pay, and their potential liability if they got sick or injured.

After exposing survey respondents to different message frames, we then asked them if they would be willing to pay for an insurance plan through the marketplace. As you can see in the graph below, there was a clear difference between people who used the calculator and people who saw the other message frames.



On average 63.1 percent of people seeing the other three messages said they would buy insurance, but that number increased to 66.8 percent among people who used the calculator. That increase of 3.7 percentage points relative to a baseline of 63.1 percent is equivalent to about a six percent increase in intent. In other words, for every 100 consumers that use the calculator, four consumers will be motivated to get insurance that wouldn’t have if they had been exposed to some other message instead. Over the course of months of outreach, that can really add up!

We think one of the reasons the calculator is so effective is because it provides consumers with the information that is most relevant to them: cost. How do we know that cost is one of the most relevant factors for many consumers? In our survey, we also asked people whether they felt like they had enough information about how the ACA affects them and their families. As you can see in the graph below, people who used the calculator were more likely to report that they had enough information, relative to people exposed to the other message frames.



We believe that this 3.6 percentage point difference shows that consumers aren’t that interested in broad messages or aggregate statistics represented in the other frames. What they really care about is finding out exactly what they’re eligible for and how much it may cost them. We believe the calculator is the most effective way to get that information out to consumers, which is why we made it one of the centerpieces of our outreach efforts. After completing this research, Enroll America began promoting the calculator more consistently through digital outreach, and encouraged field staff and volunteers to use the calculator when talking with consumers.

NOI's Tip of the Day

Originally published February 25th, 2014 on the New Organizing Institute's Tip of the Day

As data-driven organizers, we all get why we have hard goals. Goals help us build out a plan, set expectations for what we want to achieve, hold people accountable, and ensure that everyone is working towards the same vision in a big organization.

But what do we really know about getting goals right? It’s like Goldilocks. If a goal is too high, people will have trouble getting motivated to take the first step. If a goal is too low, we lose productivity because people aren’t motivated to do more once they achieve the goal.

At Enroll America, we had this problem when we realized early on that there wasn’t an obvious way to set our goals. There are 41 million uninsured people out there and, no matter how good we were, we weren’t going to talk to all of them in our first year. So we reached out to academic experts in behavioral economics and psychology and asked what they think. Here are some of the takeaways we found:

• Define the type of goal. Are 500 calls a week too much to ask from your organizer? It depends on whether you’re dealing with hard goals or motivational goals. A political campaign has hard goals like getting to 51% of the vote by Election Day. Other types of organizations don’t have a win number. Then the question becomes how you set goals to best motivate your staff on a weekly or monthly basis. 
• Set goals that people will achieve about 50% of the time. Studies suggest that people are most likely to stick with a challenging activity when they are successful about half the time. A 50-50 ratio of hitting goals strikes a nice balance where people get used to stretching themselves without getting burned out. 
• Set goals over a short timeframe. Weekly goals are more motivating than monthly goals. Shorter goals make it easier for people to see positive movement and feel like the goal is reachable. 
• Make goals social. People are motivated to keep up with their peers. Find ways to let people know how they’re doing relative to others. Make sure to share best practices and problem solving tips, both to motivate the innovators by giving them credit and to improve how everyone works. 

Have more tips on setting goals? Share in the comments!

Peter Backof is Senior Data Analyst at Enroll America and a member of the NOI community