Medicine’s Uncomfortable Relationship With Math: Calculating Positive Predictive Value

Taken from: http://archinte.jamanetwork.com/article.aspx?articleid=1861033&utm_source=silverchair+information+systems&utm_medium=email&utm_campaign=archivesofinternalmedicine%3aonlinefirst04%2f21%2f2014

In 1978, Casscells et al¹ published a small but important study showing that the majority of physicians, house officers, and students overestimated the positive predictive value (PPV) of a laboratory test result using prevalence and false positive rate. Today, interpretation of diagnostic tests is even more critical with the increasing use of medical technology in health care. Accordingly, we replicated the study by Casscells et al¹ by asking a convenience sample of physicians, house officers, and students the same question: “If a test to detect a disease whose prevalence is 1/1000 has a false positive rate of 5%, what is the chance that a person found to have a positive result actually has the disease, assuming you know nothing about the person’s symptoms or signs?”

Approximately three-quarters of respondents answered the question incorrectly (95% CI, 65% to 87%). In our study, 14 of 61 respondents (23%) gave a correct response, not significantly different from the 11 of 60 correct responses (18%) in the Casscells study (difference, 5%; 95% CI, −11% to 21%). In both studies the most common answer was “95%,” given by 27 of 61 respondents (44%) in our study and 27 of 60 (45%) in the study by Casscells et al¹ (Figure).

Neil Armstrong on Being a Nerd

Statistics Done Wrong

I happily provide the following link to Statistics Done Wrong, a free e-book illustrating pitfalls when using statistical inference. From its description:

If you’re a practicing scientist, you probably use statistics to analyze your data. From basic t tests and standard error calculations to Cox proportional hazards models and geospatial kriging systems, we rely on statistics to give answers to scientific problems.

This is unfortunate, because most of us don’t know how to do statistics.

Statistics Done Wrong is a guide to the most popular statistical errors and slip-ups committed by scientists every day, in the lab and in peer-reviewed journals. Many of the errors are prevalent in vast swathes of the published literature, casting doubt on the findings of thousands of papers. Statistics Done Wrong assumes no prior knowledge of statistics, so you can read it before your first statistics course or after thirty years of scientific practice.

http://www.refsmmat.com/statistics/index.html

Statistical errors and their tendency to mislead

As a follow-up to yesterday’s post, here’s a recent article in the scientific journal Nature about the slippery nature of P-values, including a history about how reliance on P-values has evolved in the past 100 years or so: http://www.nature.com/news/scientific-method-statistical-errors-1.14700

While I’m personally familiar with many of the pitfalls mentioned this article, I have to admit that a couple of the issues raised are brand new to me. So I’ll refrain from editorializing until I’ve had some time to reflect more deeply on this article.

How Common Core Standards Kill Creative Teaching

While I have little patience for much of the fear-mongering tactics used by some critics of the Common Core, I do appreciate thoughtful criticism. A recent editorial in U.S. News & World Report (found at http://www.usnews.com/opinion/articles/2014/03/17/how-common-core-standards-kill-creative-teaching) definitely falls under the latter, succinctly summarizing my point of view:

Understandably, proponents of the Common Core say they want greater depth of instruction and lessons that engage students. They say that the standards are only a guide. But reformers betray their cause by over-emphasizing tests and grading teachers with formulas and test scores demanded by both No Child Left Behind and Race to the Top.

To try to live up to the new demands and ensure better test scores, states, districts and schools have purchased resources, materials and scripted curricular modules solely developed for test success. Being lost is the practical wisdom and planned spontaneity necessary to work with 20 to 35 individuals in a classroom. Academic creativity has been drained from degraded and overworked experienced teachers. Uniformity has sucked the life out of teaching and learning.

To me, the operative verb in the above citation is betray, because I certainly feel betrayed. While I personally had no input into the Common Core standards for mathematics, I’ve attended presentations as they were developed over past 8 or 10 years. And the presentations that I heard have little resemblance to the way that mathematics is being assessed in schools right now.

One more thought: I live in a non-Common Core state (Texas), but the same pressure to “follow the book” exists here. So the “follow the book” mentality is not unique to the Common Core.

Teaching for understanding and teaching procedures

Many critics of the current state of mathematics education take issue with asking students to explain their reasoning. They’d rather students just apply an algorithm and get the answer.

The following is quoted from QED: The Strange Theory of Light and Matter, where Richard Feynman describes how he’s going to explain for a lay audience the techniques behind quantum mechanics that earned him a Nobel Prize. (By the way, I highly recommend this book.)

How am I going to explain to you the things I don’t explain to my students until they are third-year graduate students? Let me explain it by analogy.

The Maya Indians were interested in the rising and setting of Venus as a morning “star” and as an evening “star” – they were very interested in when it would appear. After some years of observation, they noted that five cycles of Venus were very nearly equal to eight of their “nominal years” of 365 days (they were aware that the true year of seasons was different and they made calculations of that also). To make calculations, the Maya had invented a system of bars and dots to represent numbers (including zero), and had rules by which to calculate and predict not only the risings and settings of Venus, but other celestial phenomena, such as lunar eclipses.

In those days, only a few Maya priests could do such elaborate calculations. Now, suppose we were to ask one of them how to do just one step in the process of predicting when Venus will next rise as a morning star – subtracting two numbers. And let’s assume that, unlike today, we had not gone to school and did not know how to subtract. How would the priest explain to us what subtraction is?

He could either teach us the numbers represented by the bars and dots and the rules for “subtracting” them, or he could tell us what he was really doing: “Suppose we want to subtract 236 from 584. First, count out 584 beans and put them in a pot. Then take out 236 beans and put them to one side. Finally, count the beans left in the pot. That number is the result of subtracting 236 from 584.”

You might say, “My Quetzalcoatl! What tedium, counting beans, putting them in, taking them out – what a job!”

To which the priest would reply, “That’s why we have the rules for the bars and dots. The rules are tricky, but they are a much more efficient way of getting the answer than by counting beans. The important thing is, it makes no difference as far as the answer is concerned: we can predict the appearance of Venus by counting beans (which is slow, but easy to understand) or by using the tricky rules (which is much faster, but you must spend years in school to learn them).”

To understand how subtraction works – as long as you don’t have to actually carry it out – is really not so difficult.

That’s my position: I’m going to explain to you what the physicists are doing when they are predicting how Nature will behave, but I’m not going to teach you any tricks so you can do it efficiently. You will discover that in order to make any reasonable predictions with this new scheme of quantum electrodynamics, you would have to make an awful lot of little arrows on a piece of paper. It takes seven years – four undergraduate and three graduate to train our physics students to do that in a tricky, efficient way. That’s where we are going to skip seven years of education in physics: By explaining quantum electrodynamics to you in terms of what we are really doing, I hope you will be able to understand it better than do some of the students!

In the same way, I want students in 2nd and 3rd grades to understand what they are really doing when they subtract, and not just mindlessly follow a procedure to get an answer that they do not really understand.

Where I tend to agree with most critics of the Common Core is that students are asked to write miniature essays to explain their reasoning, and that’s probably a bad idea. Even though I want students to understand why subtraction works, 2nd and 3rd graders are still learning how to write complete sentences and can get easily frustrated with explaining their reasoning in paragraph form. I think there are better ways (like drawing pictures) of assessing whether young children really understand subtraction that is more developmentally appropriate.

The Doppler shift and Flight 370

The following report from CNN (http://www.cnn.com/2014/03/24/world/asia/malaysia-airlines-satellite-tracking/index.html?hpt=hp_t1; the video from CNN can be found at http://www.cnn.com/video/data/2.0/video/world/2014/03/24/lead-foreman-satellite-data-mh370.cnn.html) discusses in layman’s terms how applied mathematicians were able to track the final moments of Flight 370. Here are the relevant paragraphs:

The mathematics-based process used by Inmarsat and the UK’s Air Accidents Investigation Branch (AAIB) to reveal the definitive path was described by McLaughlin as “groundbreaking.”

“We’ve done something new,” he said.

Here’s how the process works in a nutshell: Inmarsat officials and engineers were able to determine whether the plane was flying away or toward the satellite’s location by expansion or compression of the satellite’s signal.

What does expansion or compression mean? You may have heard about something called the Doppler effect.

“If you sit at a train station and you listen to the train whistle — the pitch of the whistle changes as it moves past. That’s exactly what we have,” explained CNN Meteorologist Chad Myers,who has studied Doppler technology. “It’s the Doppler effect that they’re using on this ping or handshake back from the airplane. They know by nanoseconds whether that signal was compressed a little — or expanded — by whether the plane was moving closer or away from 64.5 degrees — which is the latitude of the orbiting satellite.”

Each ping was analyzed for its direction of travel, Myers said. The new calculations, McLaughlin said, underwent a peer review process with space agency experts and contributions by Boeing.

It’s possible to use this analysis to determine more specifically the area where the plane went down, Myers said. “Using trigonometry, engineers are capable of finding angles of flight.”

My understanding is that even though the pings from a satellite to the plane and back were occurring, even though the plane’s location was not being transmitted. From the Doppler shift of those pings, the plane’s trajectory could be reconstructed.

Someday, for teaching purposes, I hope that a formal write-up of this procedure is published. The details will probably be over the heads of most students, but this is a eye-catching, though indescribably tragic, example of how mathematics can be creatively used to solve a mystery.

Can You Solve This?

A friend forwarded this very interesting video to me. It’s not so much an exercise in mathematics but an exercise in problem-solving and logic and especially confirmation bias. I won’t ruin the video but I’ll give the punch line at the end:

If you think that something is true, you should try as hard as you can to disprove it. Only then can you really get at the truth and not fool yourself.