First Polling Results on Roll-Out of Exchanges

Based on news reports since the October 1 opening of the online health-insurance exchanges, two dominant -- and likely inter-related -- themes have been the high volume of response and the technical glitches plaguing the federal exchange site, healthcare.gov. Where governors opted to have their respective states create their own exchanges rather than having their citizens use the federal site as a fallback, there seems to be variation in how smoothly the websites are working (compare, for example, New York to Maryland).

Here at the Health Care Polls blog, we're interested in public opinion toward, and reaction to, the Affordable Care Act and its component parts, such as the exchanges (also known as marketplaces). And we now have what is, to my knowledge, the first major survey on the topic, from AP-GfK. The poll was conducted from October 3-7, thus gauging immediate reaction to the unveiling of the exchanges. A Huffington Post article on the poll is available here, whereas the AP-GfK technical report is here.

AP-GfK surveyed 1,227 adults, for which the margin of error (MoE) is said to be 3.4 percentage points for the 95% confidence level. According to the Huffington Post article:

Overall, the poll found, 40 percent of Americans said the launch of the insurance markets hasn't gone well, 20 percent said it's gone somewhat well and 30 percent didn't know what to say. Just 7 percent said the launch had gone "very well" or "[extremely] well."

(The article had a typo about the final 7 percent, which I corrected based on the technical report.)

Seventy-six individuals from the original 1,227 respondents, or 7%, reported that they or someone in their household had "tried to sign up for health insurance coverage through this market." With the stated 3.4% MoE around the 7% estimate, we could say (assuming accurate responses) that between 3.6 and 10.4 percent of the U.S. public as a whole had tried the exchanges.

However, the MoE varies slightly depending on whether the sample is relatively evenly divided on an issue (e.g., 50/50, 55/45) or relatively unanimous (e.g., 90/10), with the margin being smaller (more precise) the closer to unanimity. My impression is that pollsters usually report the MoE assuming a 50/50 breakdown on the survey items, so if AP-GfK did so, then the MoE around the 7% who claimed to have used the exchanges would be tighter than 3.6 to 10.4 percent.*

Those 76 exchange users were asked a variety of questions. Quoting from the above-linked Huffington Post article, "Among those who've actually tested out the system, three-quarters of those polled said they've experienced problems trying to sign up. Only about 1 in 10 succeeded in buying health insurance." As the same article notes, however, the MoE for responses of just these 76 people is a whopping +/- 13.5%. Now that AP-GfK has provided an initial estimate of the proportion of Americans using the exchanges, they and other pollsters might consider obtaining oversamples of exchange users in future surveys, so that estimates from this group can have greater precision.

One area in which the AP-GfK poll appears to be somewhat out-of-whack is the 37% job-approval rating for President Obama, compared to other polls from around the same time that peg the President's approval between 40-51%. Assuming that attitudes toward the Affordable Care Act and the exchanges are correlated with support for Obama, then evaluations of the ACA and exchanges may be biased somewhat in a negative direction by the AP-GfK poll.

---
*The margin of error with 95% confidence is calculated from:

+/- 1.96 X square root (pq/n)

where p = the percentage of respondents who answered in one way (e.g., "yes" on a yes/no question), q = the percentage who answered with the other choice, and n = sample size. (Readers with some statistical training may recognize +/- 1.96 as the values that cut off the upper and lower 2.5% of the "bell curve" distribution, thus leaving 95% of the distribution.)

Using the AP-GfK's reported sample size of 1,227, a 50/50 division on a question (i.e., p and q each equal .50), would yield a margin of error of:

1.96 X square root (.25/1,227) = .028 or 2.8%

(I do not know why there is the slight discrepancy between AP-GfK's reported MoE of 3.4% and my calculation of 2.8%; if AP-GfK has a cluster-sampling component to its work, that would raise the MoE.)

Anyway, using the conventional MoE formula with a breakdown of .07 and .93 for usage vs. non-usage of the health-insurance exchanges, the calculation is...

1.96 X square root (.065/1,227) = .014 or 1.4%

An online margin-of-error calculator, which corrects for population size, is available here. You might enter 200,000,000 for the population size, which is roughly how many adults there are in the U.S. With any large population, the middle term in the larger equation tends toward 1, so has no effect when used to multiply the other elements of the equation.