Issues when conducting political polls (Part 3)

The classic application of confidence intervals is political polling: the science of sampling relatively few people to predict the opinions of a large population. However, in the 2010s, the art of political polling — constructing representative samples from a large population — has become more and more difficult. FiveThirtyEight.com had a nice feature about problems that pollsters face today that were not issues a generation ago. A sampling:

The problem is simple but daunting. The foundation of opinion research has historically been the ability to draw a random sample of the population. That’s become much harder to do, at least in the United States. Response rates to telephone surveys have been declining for years and are often in the single digits, even for the highest-quality polls. The relatively few people who respond to polls may not be representative of the majority who don’t. Last week, the Federal Communications Commission proposed new guidelines that could make telephone polling even harder by enabling phone companies to block calls placed by automated dialers, a tool used in almost all surveys.

What about Internet-based surveys? They’ll almost certainly be a big part of polling’s future. But there’s not a lot of agreement on the best practices for online surveys. It’s fundamentally challenging to “ping” a random voter on the Internet in the same way that you might by giving her an unsolicited call on her phone. Many pollsters that do Internet surveys eschew the concept of the random sample, instead recruiting panels that they claim are representative of the population.

Previous posts in this series: Part 1 and Part 2.

Wason Selection Task: Part 3

I recently read about a simple but clever logic puzzle, known as the “Wason selection task,” which is often claimed to be “the single most investigated experimental paradigm in the psychology of reasoning.” More than 90% of Wason’s subjects got the answer wrong when Wason first studied this problem back in the 1960s, and this result has been repeated time over time by psychologists ever since.

Here’s the puzzle: You are shown four different cards, showing a 5, an 8, a blue card, and a green card. You are told that each card has a number on one side and a color on the other side. You are asked to test the truth of the following statement:

If a card has an even number on one side, then its opposite side is blue.

Question: Which card (or cards) must you turn over to test the truth of this statement?

Interestingly, in the 1980s, a pair of psychologists slightly reworded the Wason selection puzzle in a form that’s logically equivalent, but this rewording caused a much higher rate of correct responses. Here was the rewording:

On this task imagine you are a police officer on duty. It is your job to make sure that people conform to certain rules. The cards in front of you have information about four people sitting at a table. On one side of the card is a person’s age and on the other side of the card is what the person is drinking. Here is a rule: “If a person is drinking beer, then the person must be over 19 years of age.” Select the card or cards that you definitely must turn over to determine whether or not the people are violating the rule.

Four cards are presented:

Drinking a beer

Drinking a Coke

16 years of age

22 years of age

In this experiment, 29 out of 40 respondents answered correctly. However, when presented with the same task using more abstract language, none of the 40 respondents answered correctly… even though the two puzzles are logically equivalent. Quoting from the above article:

Seventy-five percent of subjects nailed the puzzle when it was presented in this way—revealing what researchers now call a “content effect.” How you dress up the task, in other words, determines its difficulty, despite the fact that it involves the same basic challenge: to see if a rule—if P then Q—has been violated. But why should words matter when it’s the same logical structure that’s always underlying them?

This little study has harrowing implications for those of us that teach mathematical proofs and propositional logic. It’s very easy for people to get some logic questions correct but other logic questions incorrect, even if the puzzles look identical to the mathematician/logician who is posing the questions. Pedagogically, this means that it’s a good idea to use familiar contexts (like rules for underage drinking) to introduce propositional logic. But this comes with a warning, since students who answer questions arising from a familiar context correctly may not really understand propositional logic at all when the question is posed more abstract (like in a mathematical proof).

Wason Selection Task: Part 2

If a card has an even number on one side, then its opposite side is blue.

Question: Which card (or cards) must you turn over to test the truth of this statement?

The answer is: You must turn over the 8 card and the green card. The following video explains why:

Briefly:

Clearly, you must turn over the 8 card. If the opposite side is not blue, then the proposition is false.
Clearly, the 5 card is not helpful. The statement only tells us something if the card shows an even number.
More subtly, the blue card is not helpful either. The statement claim is “If even, then blue,” not “If blue, then even.” This is the converse of the statement, and converses are not necessarily equivalent to the original statement.
Finally, the contrapositive of “If even, then blue” is “If not blue, then not even.” Therefore, any card that is not blue (like the green one) should be turned over.

If you got this wrong, you’re in good company. More than 90% of Wason’s subjects got the answer wrong when Wason first studied this problem back in the 1960s, and this result has been repeated time over time by psychologists ever since.

Speaking for myself, I must admit that I blew it too when I first came across this problem. In the haze of the early morning when I first read this article, I erroneously thought that the 8 card and the blue card had to be turned.

Wason Selection Task: Part 1

Here’s the puzzle: You are shown four different cards, showing a 5, an 8, a blue card, and a green card. You are asked to test the truth of the following statement:

If a card has an even number on one side, then its opposite side is blue.

Question: Which card (or cards) must you turn over to test the truth of this statement?

I’ll start discussing the answer to this puzzle in tomorrow’s post. If you’re impatient, you can click through the interactive video above or else read the article where I first learned about this puzzle: http://m.nautil.us/blog/the-simple-logical-puzzle-that-shows-how-illogical-people-are (I got the opening sentence of this post from this article).

io9: “I Fooled Millions Into Thinking Chocolate Helps Weight Loss. Here’s How.”

Peer-reviewed publications is the best way that we’ve figured out for vetting scientific experiments and disseminating scientific knowledge. But that doesn’t mean that the system can’t be abused, either consciously or unconsciously.

The eye-opening article http://io9.com/i-fooled-millions-into-thinking-chocolate-helps-weight-1707251800 describes how the author published flimsy data that any discerning statistician should have seen through and even managed to get his “results” spread in the popular press. Some money quotes:

Here’s a dirty little science secret: If you measure a large number of things about a small number of people, you are almost guaranteed to get a “statistically significant” result. Our study included 18 different measurements—weight, cholesterol, sodium, blood protein levels, sleep quality, well-being, etc.—from 15 people. (One subject was dropped.) That study design is a recipe for false positives.

Think of the measurements as lottery tickets. Each one has a small chance of paying off in the form of a “significant” result that we can spin a story around and sell to the media. The more tickets you buy, the more likely you are to win. We didn’t know exactly what would pan out—the headline could have been that chocolate improves sleep or lowers blood pressure—but we knew our chances of getting at least one “statistically significant” result were pretty good.

And:

With the paper out, it was time to make some noise. I called a friend of a friend who works in scientific PR. She walked me through some of the dirty tricks for grabbing headlines. It was eerie to hear the other side of something I experience every day.

The key is to exploit journalists’ incredible laziness. If you lay out the information just right, you can shape the story that emerges in the media almost like you were writing those stories yourself. In fact, that’s literally what you’re doing, since many reporters just copied and pasted our text.

And:

The only problem with the diet science beat is that it’s science. You have to know how to read a scientific paper—and actually bother to do it. For far too long, the people who cover this beat have treated it like gossip, echoing whatever they find in press releases. Hopefully our little experiment will make reporters and readers alike more skeptical.

If a study doesn’t even list how many people took part in it, or makes a bold diet claim that’s “statistically significant” but doesn’t say how big the effect size is, you should wonder why. But for the most part, we don’t. Which is a pity, because journalists are becoming the de facto peer review system. And when we fail, the world is awash in junk science.

Who was kissing in the famous VJ Day picture?

We are approaching the 70th anniversary of VJ Day (August 14, 1945), which marked the end of World War II. And perhaps the iconic photograph of that day is the picture of two anonymous strangers kissing in New York City’s Times Square celebrating the end of the war.

And a question that is still unresolved after 70 years is: Who are they?

The short answer is, Nobody knows for certain. But in a clever bit of geometric and astronomical forensics, physicists at Texas State University (Donald Olson and Russell Doescher) and Iowa State University (Steven D. Kawaler) recently pinpointed the exact time that the photograph was taken: 5:51 pm, or about an hour before President Truman formally announced that the war was over. From the press release:

Overlooked in the right hand background of the photo is the Bond Clothes clock. The minute hand of this clock is clear, but the oblique angle of view and the clock’s unusually short hour hand makes a definitive reading of the time difficult. The clock might show a time near 4:50, 5:50, or 6:50 p.m. A prominent shadow falls across the Loew’s Building just beyond the clock, however, and this shadow could potentially give just as accurate a time reading as the clock.

Every tall building in Manhattan acts as a sundial, its cast shadow moving predictably as the sun traverses the sky. In this case, the Texas State team studied hundreds of photographs and maps from the 1940s to identify the source of the shadow, considering, in turn, the Paramount Building, the Hotel Lincoln and the Times Building. The breakthrough came when a photograph of the Astor Hotel revealed a large sign shaped like an inverted L that advertised the Astor Roof garden.

Calculations showed that only the Astor Roof sign could have cast the shadow, but to be certain, Olson and Doescher built a scale model of the Times Square buildings with a mirror to project the sun’s rays. The location, size and shape of the shadow on the model exactly matched the shadow in Eisenstaedt’s kiss photographs.

So who are the kissers? Again from the press release:

Over the years, dozens of men and women have come forward claiming to be the persons in the photograph. All have different stories, but the one thing they share in common is kissing a stranger in Times Square that fateful day.

“All those people have said they were there and identify themselves in the photograph,” Olson said. “Who’s telling the truth? They all could be telling the truth about kissing someone. They were probably all there, and kisses were common in Times Square on VJ Day.

“I can tell you some things about the picture, and I can rule some people out based on the time of day,” he said. “We can show that some of the accounts are entirely inconsistent with the astronomical evidence”…

“Astronomy alone can’t positively identify the participants, but we can tell you the precise moment of the photograph,” Olson said. “Some of the accounts are inconsistent with the astronomical evidence, and we can rule people out based on the position of the sun. The shadows were the key to unlocking some of the secrets of the iconic VJ Day images–we know when the famous kiss happened, and that gives us some idea of who might or might not have been in the picture.”

From a news report:

“There are probably 50 or 60 sailors who have come forward and say, ‘That’s me! I’m the guy in the photograph.’ Fewer women, maybe five or six women, have said they’re the woman in white. There are articles all over the internet advocating for one [or] the other,” Olson said.

Olson can’t say who is correct, but he can rule out a few.

“What we can do is calculate the precise time, 5:51 p.m., when the photograph was taken. That does appear to rule out some of the widely accepted candidates,” he said.

The full article has been published in the August 2015 issue of Sky and Telescope magazine (sorry, you’ll have to buy a copy in you want to read the article). I also recommend clicking through the photographs in the press release; the captions of the photographs give many details of how the time of 5:51 pm was pinpointed.

15-year-old catches math error at the Boston Museum of Science

See the full article here: http://www.boston.com/news/local/massachusetts/2015/07/06/year-old-catches-math-error-the-museum-science/awhREamdn1KRg7nz2gGyPO/story.html?p1=Must_Reads

Arrangements of Stars on the American Flag

Reasonable star patterns on the American flag correspond to special factorizations; the density of such factorizations is less than the density of values in a multiplication table; Paul Erdös showed this density asymptotically approaches zero by considering the average number of prime factors of an integer. – See more at: http://www.maa.org/programs/maa-awards/writing-awards/lester-r-ford-awards/arrangements-of-stars-on-the-american-flag#sthash.e9PHpilF.dpuf