The End of “Statistical Significance”

I’ve linked to a number of articles about the misuse of p-values. Recently, I read a nice article in the October/November 2019 issue of MAA Focus summarizing a conversation between the Executive Directors of the Mathematical Association of America and the American Statistical Association about the ASA’s call to eliminate the use of p-values. Per copyright, I can’t copy the entire article here, but let me quote the lead paragraph:

In March 2016, the American Statistical Association took the extraordinary step of issuing a Statement on p-Values and Statistical Significance. This spring, the association went even further, publishing a massive special issue of its journal The American Statistician entitled Statistical Inference in the 21st Century: A World Beyond p<0.05. The lead editorial in that special issue called for the end of the use of the concept of statistical significance.

It’s going to be a while before entrenched statistics textbooks catch up with this new standard of professional practice.

Here’s an NPR article on the issue: https://www.npr.org/sections/health-shots/2019/03/20/705191851/statisticians-call-to-arms-reject-significance-and-embrace-uncertainty

Other articles cited in the MAA Focus article:

Adding by a Form of 0 (Part 4)

In my previous post, I wrote out a proof (that an even number is an odd number plus 1) that included the following counterintuitive steps:

2k = (2k - 1) + 1 = ([2k - 1 - 1] + 1) + 1

A common reaction that I get from students, who are taking their first steps in learning how to write mathematical proofs, is that they don’t think they could produce steps like these on their own without a lot of coaching and prompting. They understand that the steps are correct, and they eventually understand why the steps were necessary for this particular proof (for example, the conversion from 2k-1 to [2k - 1 -1]+1 was necessary to show that 2k-1 is odd).

Not all students initially struggle with this concept, but some do. I’ve found that the following illustration is psychologically reassuring to students struggling with this concept. I tell them that while they may not be comfortable with adding and subtracting the same number (net effect of adding by 0), they should be comfortable with multiplying and dividing by the same number because they do this every time that they add or subtract fractions with different denominators. For example:

\displaystyle \frac{2}{3} + \frac{4}{5} = \displaystyle \frac{2}{3} \times 1 + \frac{4}{5} \times 1

= \displaystyle \frac{2}{3} \frac{5}{5} + \frac{4}{5} \times \frac{3}{3}

= \displaystyle \frac{10}{15} + \frac{12}{15}

= \displaystyle \frac{22}{15}

In the same way, we’re permitted to change 2k-1 to 2k-1 + 0 to 2k -1 - 1 + 1.

Hopefully, connecting this proof technique to this familiar operation from 5th or 6th grade mathematics — here in Texas, it appears in the 5th grade Texas Essential Knowledge and Skills under (3)(H) and (3)(K) — makes adding by a form of 0 in a proof somewhat less foreign to my students.

Adding by a Form of 0 (Part 3)

As part of my discrete mathematics class, I introduce my freshmen/sophomore students to various proof techniques, including proofs about sets. Here is one of the examples that I use that involves adding and subtracting a number twice in the same proof.

Theorem. Let A be the set of even integers, and define

B = \{ n: n = m+1 for some odd integer m\}

Then A = B.

Proof (with annotations). Before starting the proof, I should say that I expect my students to use the formal definitions of even and odd:

  • An integer n is even if n = 2k for some integer k.
  • An integer n is odd if n = 2k+1 for some integer k.

To prove that A = B, we must show that A \subseteq B and B \subseteq A. The first of these tends to trickiest for students.

Part 1. Let n \in A. By definition of even, that means that there is an integer k so that n = 2k.

To show that n \in B, we must show that n = m + 1 for some odd integer m. To this end, notice that n = (n-1) + 1. Thus, we must show that n - 1 is an odd integer, or that n -1 can be written in the form 2k+1. To do this, we add and subtract 1 a second time:

n = 2k

= (2k - 1) + 1

= ([2k - 1 - 1] + 1) + 1

= ([2k-2] + 1) + 1

= (2[k-1] + 1) + 1.

By the closure axioms, k-1 is an integer. Therefore, 2[k-1] + 1 is an odd number by definition of odd, and hence $n \in B$.

The above part of the proof can be a bit much to swallow for students first learning about proofs. For completeness, let me also include Part 2 (which, in my experience, most students can produce without difficulty).

Part 2. Let n \in B, so that n = m + 1 for some odd integer m. By definition of odd, there is an integer k so that $m = 2k+1$. Therefore, n = (2k+1) + 1 = 2k+2 = 2(k+1). By the closure axioms, k +1 is an integer. Therefore, n is even by definition of even, and so we conclude that n \in A.

\square

For what it’s worth, this is the review problems for which I recorded myself talking through the solution for the benefit of my students.

In my opinion, the biggest conceptual barriers in this proof are these steps from Part 1:

2k = (2k - 1) + 1 = ([2k - 1 - 1] + 1) + 1.

These steps are undeniably awkward. Back in high school algebra, students would get points taken off for making the expression more complicated instead of simplifying the answer. But this is the kind of jump that I need to train my students to do so that they can master this technique and be successful in their future math classes.

Adding by a Form of 0 (Part 2)

Often intuitive appeals for the proof of the Product Rule rely on pictures like the following:

The above picture comes from https://mrchasemath.com/2017/04/02/the-product-rule/, which notes the intuitive appeal of the argument but also its lack of rigor.

My preferred technique is to use the above rectangle picture but make it more rigorous. Assuming that the functions f and g are increasing, the difference f(x+h) g(x+h) - f(x) g(x) is exactly equal to the sum of the green and blue areas in the figure below.

In other words,

f(x+h) g(x+h) - f(x) g(x) = f(x+h) [g(x+h) - g(x)] + [f(x+h) - f(x)] g(x),

or

f(x+h) g(x+h) - f(x+h) g(x) + f(x+h) g(x) - f(x) g(x).

This gives a geometrical way of explaining this otherwise counterintuitive step for students not used to adding by a form of 0. I make a point of noting that we took one term, f(x+h), from the first product f(x+h) g(x+h), while the second term, g(x), came from the second product f(x) g(x). From this, the usual proof of the Product Rule follows:

[(fg)(x)]' = \displaystyle \lim_{h \to 0} \frac{f(x+h) g(x+h) - f(x) g(x)}{h}

\displaystyle = \lim_{h \to 0} \frac{f(x+h) [g(x+h) - g(x)]}{h} + \lim_{h\ to 0} \frac{[f(x+h) - f(x)] g(x)}{h}

\displaystyle = \lim_{h \to 0} \frac{f(x+h) [g(x+h) - g(x)]}{h} + \lim_{h\ to 0} \frac{[f(x+h) - f(x)] g(x)}{h}

\displaystyle = \lim_{h \to 0} f(x+h) \frac{g(x+h) - g(x)}{h} + \lim_{h\ to 0} \frac{f(x+h) - f(x) }{h} g(x)

= f(x)g'(x) + f'(x) g(x)

For what it’s worth, a Google Images search for proofs of the Product Rule yielded plenty of pictures like the one at the top of this post but did not yield any pictures remotely similar to the green and blue rectangles above. This suggests to me that the above approach of motivating this critical step of this derivation might not be commonly known.

Once students have been introduced to the idea of adding by a form of 0, my experience is that the proof of the Quotient Rule is much more palatable. I’m unaware of a geometric proof that I would be willing to try with students (a description of the best attempt I’ve seen can be found here), and so adding by a form of 0 becomes unavoidable. The proof begins

\left[\left( \displaystyle \frac{f}{g} \right)(x) \right]' = \displaystyle \lim_{h \to 0} \frac{ \displaystyle \frac{f(x+h)}{ g(x+h)} - \frac{f(x)}{ g(x)}}{h}

= \displaystyle \lim_{h \to 0} \frac{ \displaystyle \frac{f(x+h) g(x) - f(x) g(x+h)}{ g(x) g(x+h)}}{h}

= \displaystyle \lim_{h \to 0} \frac{f(x+h) g(x) - f(x) g(x+h)}{ h g(x) g(x+h)}.

At this point, I ask my students what we should add and subtract this time to complete the derivation. Given the previous experience with the Product Rule, students are usually quick to chose one factor from the first term and another factor from the second term, usually picking f(x) g(x). In fact, they usually find this step easier than the analogous step in the Product Rule because this expression is more palatable than the slightly more complicated f(x+h) g(x). From here, the rest of the proof follows:

[(fg)(x)]' = \displaystyle \lim_{h \to 0} \frac{ \displaystyle \frac{f(x+h) g(x) - f(x) g(x+h)}{h }}{g(x) g(x+h)}

= \displaystyle \lim_{h \to 0} \frac{ \displaystyle \frac{f(x+h) g(x) - f(x)g(x) + f(x)g(x) - f(x) g(x+h)}{h }}{g(x) g(x+h)}

= \displaystyle \lim_{h \to 0} \frac{ \displaystyle \frac{f(x+h) g(x) - f(x)g(x)}{h} + \frac{f(x)g(x) - f(x) g(x+h)}{h }}{g(x) g(x+h)}

= \displaystyle \lim_{h \to 0} \frac{ \displaystyle \frac{f(x+h) g(x) - f(x)g(x)}{h} - \frac{f(x) g(x+h) - f(x)g(x)}{h }}{g(x) g(x+h)}

= \displaystyle \lim_{h \to 0} \frac{ \displaystyle \frac{[f(x+h) - f(x)] g(x)}{h} - \frac{f(x) [g(x+h) - g(x)]}{h }}{g(x) g(x+h)}

= \displaystyle \lim_{h \to 0} \frac{ \displaystyle \frac{f(x+h) - f(x) }{h} g(x) - f(x) \frac{ g(x+h) - g(x)}{h }}{g(x) g(x+h)}

= \displaystyle \frac{ f'(x) g(x) - f(x) g'(x)}{g(x)^2}

P.S.

  • The website https://mrchasemath.com/2017/04/02/the-product-rule/ also suggests an interesting pedagogical idea: before giving the formal proof of the Product Rule, use a particular function and the limit definition of a derivative so that students can intuitively guess the form of the rule. For example, if g(x) = x^2:

Adding by a Form of 0 (Part 1)

Adding by a form of 0, or adding and subtracting the same quantity, is a common technique in mathematical proofs. For example, this technique is used in the second step of the standard proof of the Product Rule in calculus:

[(fg)(x)]' = \displaystyle \lim_{h \to 0} \frac{f(x+h) g(x+h) - f(x) g(x)}{h}

\displaystyle = \lim_{h \to 0} \frac{f(x+h) g(x+h) - f(x+h) g(x) + f(x+h) g(x) - f(x) g(x)}{h}

\displaystyle = \lim_{h \to 0} \left[ \frac{f(x+h) g(x+h) - f(x+h) g(x)}{h} + \frac{f(x+h) g(x) - f(x) g(x)}{h} \right]

\displaystyle = \lim_{h \to 0} \frac{f(x+h) g(x+h) - f(x+h) g(x)}{h} + \lim_{h\ to 0} \frac{f(x+h) g(x) - f(x) g(x)}{h}

\displaystyle = \lim_{h \to 0} \frac{f(x+h) [g(x+h) - g(x)]}{h} + \lim_{h\ to 0} \frac{[f(x+h) - f(x)] g(x)}{h}

\displaystyle = \lim_{h \to 0} f(x+h) \frac{g(x+h) - g(x)}{h} + \lim_{h\ to 0} \frac{f(x+h) - f(x) }{h} g(x)

= f(x)g'(x) + f'(x) g(x)

Or the proof of the Quotient Rule:

\left[\left( \displaystyle \frac{f}{g} \right)(x) \right]' = \displaystyle \lim_{h \to 0} \frac{ \displaystyle \frac{f(x+h)}{ g(x+h)} - \frac{f(x)}{ g(x)}}{h}

= \displaystyle \lim_{h \to 0} \frac{ \displaystyle \frac{f(x+h) g(x) - f(x) g(x+h)}{ g(x) g(x+h)}}{h}

= \displaystyle \lim_{h \to 0} \frac{f(x+h) g(x) - f(x) g(x+h)}{ h g(x) g(x+h)}

= \displaystyle \lim_{h \to 0} \frac{ \displaystyle \frac{f(x+h) g(x) - f(x) g(x+h)}{h }}{g(x) g(x+h)}

= \displaystyle \lim_{h \to 0} \frac{ \displaystyle \frac{f(x+h) g(x) - f(x)g(x) + f(x)g(x) - f(x) g(x+h)}{h }}{g(x) g(x+h)}

= \displaystyle \lim_{h \to 0} \frac{ \displaystyle \frac{f(x+h) g(x) - f(x)g(x)}{h} + \frac{f(x)g(x) - f(x) g(x+h)}{h }}{g(x) g(x+h)}

= \displaystyle \lim_{h \to 0} \frac{ \displaystyle \frac{f(x+h) g(x) - f(x)g(x)}{h} - \frac{f(x) g(x+h) - f(x)g(x)}{h }}{g(x) g(x+h)}

= \displaystyle \lim_{h \to 0} \frac{ \displaystyle \frac{[f(x+h) - f(x)] g(x)}{h} - \frac{f(x) [g(x+h) - g(x)]}{h }}{g(x) g(x+h)}

= \displaystyle \lim_{h \to 0} \frac{ \displaystyle \frac{f(x+h) - f(x) }{h} g(x) - f(x) \frac{ g(x+h) - g(x)}{h }}{g(x) g(x+h)}

= \displaystyle \frac{ f'(x) g(x) - f(x) g'(x)}{g(x)^2}

This is a technique that we expect math majors to add to their repertoire of techniques as they progress through the curriculum. I forget the exact proof, but I remember that, when I was a student in honors calculus, we had some theorem that required an argument of the form

|x - y| = |x - A + A - B + B - C + C - D + D - E + E - F + F - y|

\le |x - A| + |A - B| + |B - C| + |C - D| + |D - E| + |E - F| + |F - y|

\le \displaystyle \frac{\epsilon}{7} + \frac{\epsilon}{7} +\frac{\epsilon}{7} +\frac{\epsilon}{7} +\frac{\epsilon}{7} +\frac{\epsilon}{7} +\frac{\epsilon}{7}

= \epsilon

But while this is a technique that expect students to master, there’s no doubt that this looks utterly foreign to a student first encountering this technique. After all, in high school algebra, students would simplify something like x - A + A - B + B - C + C - D + D - E + E - F + F - y into x-y. If they were to convert x-y into something more complicated like x - A + A - B + B - C + C - D + D - E + E - F + F - y, they would most definitely get points taken off.

In this brief series, I’d like to give some thoughts on getting students comfortable with this technique.

YouTube’s Automatic Closed-Captioning of Mathematical Speech (Part 2)

Last semester, as I spend untold hours editing the closed captioning automatically generated by YouTube on the math videos on my YouTube channel, I got a crash course on the capabilities and limitations of this system. This crash course was perhaps not legally necessary but extra work that I took on because a student with a hearing impairment was enrolled in my class, and I wanted to ensure that the review videos that I provide to my students were accessible to him also.

I think the resources offered by my university are fairly typical to ensure that instructors are able to reach all students and not just those who don’t have audio/visual impairments. After discussions with the cognizant people at my university, I’ve made a few conclusions:

  • Mostly by accident, my videos are ADA compliant since I made the decision to both write out the solutions and also talking through the solutions.
  • While the automatic closed-captioning provided by YouTube may be minimally compliant with ADA, I’m not sure that a student with a hearing impairment could always follow the transcriptions due to a number of errors.
  • Aside from punctuation, capitalization, and the occasional homonym (e.g., right vs. write), YouTube does a pretty good job at transcribing ordinary speech.
  • Naturally, YouTube’s automated closed-captioning is not to blame when I don’t enunciate clearly, have a rabbit trail of thought but then have to backtrack, use poor grammar, make a outright mistake, etc.
  • However, YouTube seems to have a lot of difficulty providing automatic closed-captioning of mathematical speech.

Fixing these transcription errors took an awful lot of time. I don’t want to know how many hours I devoted to fixing the 120 or so videos (each video is about 3-10 minutes long) recorded so that my hearing-impaired student could have full access to my class. About halfway into this project of fixing the closed-captioning errors, I started writing down some of the closed-captioning errors. I wish I had thought to do this near the start, but oh well.

Phonetically, I can understand why most of these errors were made. But these mistakes really shouldn’t have happened. Here are my favorite howlers that I recorded, showing both what I said and what YouTube thought I said.

  • “931,147,496” became “930 1,000,000 147,000 496”
  • “A \cap C,” pronounced “A intersect C,” became “A inner sexy”
  • “arithmetic” became “rhythm sick”
  • “capital X” became “Catholics”
  • “cardinality” became “carnality”
  • “divisible by 5” became “visited his wife live” (I have no idea how that happened)
  • “e^x” became “eat ooh the x”
  • “for succinctness” became “force the sickness”
  • “n \choose n,” pronounced “n choose n,” became “and shoes and”
  • “set containing” became “second taining”
  • “\sqrt{2}” became “squirt tuna”
  • “two ways in” became “too wasted”
  • “what f(3),” pronounced “what f of 3,” became “whateva 3”
  • x \in B, pronounced “x is in B,” became “sexism be”
  • x \in B \cap C, pronounced “x is in B and C,” became “x is Indiana see”
  • x \in C, pronounced “x is in C,” became “excellency”

Here’s the complete list of howlers that I recorded for posterity. If I’ve learned nothing else, it’s that I need to be more proactive about ensuring the mathematical accuracy of closed-captioning for my YouTube videos.

4 for
857 a 50 7
1232 1230 two
4761 4760 1
19,999 19,000 999
46,376 40 6376
123,552 120 3,552
5,565,120 five million 565,000 120
931,147,496 930 1,000,000 147,000 496
(2,\emptyset) 2d sent
(20,8) 28
[1,2] one too
12 \choose 4 12 juice 4
16 \choose 8 16 choosing
3 + 1 = 4 surplus one mix for
4 \choose 0 4 2 0
4 \choose k four twos k
49 \choose 5 49 she’s 5
50 \choose 6 52 six
8 \choose 2 a choose to
A \cap C A inner sexy
A \cap D a intersecting
A \cup B a you be
A \cup C a UNC
A \cup C a you will see
a proof approved
A^c a compliment
a_i asa by
all multiples of almost visit
an element of A known the debate
an element of A normal today
and divisible and as above
and positive 50 + + 50
and tens intense
and would let this be 3 andrew lippa p3
arithmetic earth to
arithmetic rhythm sick
As ace
B but not C be but not si
B \cap C b in a sexy
B if beef
bijection bi CH action
bijection bite jection
bijection by dejection
bijection by ejection
bijection by jection
bijection by Junction
both sets both says
capital X Catholics
cardinality carnality
Cartesian car to shull
codomain code Amin
coordinate cordon
coordinate court
coordinates corners
coordinates have cort in sap
cosine cosign
disjoint destroyed
divisible by 5 visited his wife live
e^x eat ooh the x
element of A illness of A
element of A mellow today
element x that Windex
elements of us
empty MQ
\emptyset descent
\emptyset intercept
equal able
exponent x1
factored acted
factorial fact welders
fill in film
flipping four coins philippine for coins
for succinctness force the sickness
hence in Hanson
i eye
i aye
If I divide by 15 If I / 15
in A nae
in there a bear
infinite if an
infinite imp an
infinite infant
into five in 2 5
is ice
j \choose r j choose arms
kth cave
kth kate
likewise lakh wise
n \choose n and shoes and
nth row nth throw
one-to-one 121
onto on 2
r \choose r our shoes are
r to art at
r to already
\mathbb{R}^2 are too
\mathbb{R}^2 our too
r‘s hours
same row samro
second coordinate sec cornered
set containing second inning
set containing second taining
set containing seconds hanging
set containing secretary
set containing 1 second anyone
since A has say has
sixth one six-month
square swear
\sqrt{2} score 2
\sqrt{2} squirt of tuna
team A teammate
term in it terminate
than zero gloves are off
that’s chosen that’s Showzen
then x the next
therefore there for
this entry in the century plus
to the k decay
two are to are
two ways in too wasted
union you need
up here pier
what f(3) whateva 3
will be 4 will before
with n=4 finials 4
would subtract was attract
writing riding
x is extras
x is in exiting
x is in A x as a native
x is in A x is nay
x is in B sexism be
x is in B and C x is Indiana see
x is in C excellency
x is in C X’s and see
x_2 next to
x_2 text too
x-coordinate export
y why
y wine
y is greater than or wider
ys wise

YouTube’s Automatic Closed-Captioning of Mathematical Speech (Part 1)

The biggest change that I’ve made to my teaching in the past ten years has been posting review videos for my students as they prepare for exams. The playlists that I post for my students can be found at my YouTube channel. The production quality of the videos is definitely low-budget: I just placed a ruler along the top of two textbooks of equal height, balanced a webcam on the ruler to point downward, and then recorded myself as I wrote out and talked through the solutions of the review problems. I’m not going for high production value in my videos, unlike excellent sites like Physics Girl or Numberphile, since my target audience is deliberately narrow (the students in my classes and, more recently, in some of my colleagues’ classes) and not worldwide.

For what it’s worth, I have recorded roughly 650 videos, each usually between 3 and 10 minutes long, which have collectively amassed over 200,000 views since I started recording them in 2011. Not bad for your friendly neighborhood mathematician.

Posting these videos have spurred some immediate changes to my pedagogical practice. First, I no longer give review lectures in class immediately before my exams. Instead, I ask students to take a shot at completing the review problems on their own, asking them to watch the videos only to check their work or else to get the answer if they get stuck. Students are still welcome to come to me for help during office hours or by appointment, but they’re expected to watch the videos first. In my end-of-semester evaluations, my students seem to really appreciate having these videos. They tell me that they like having after-hours help while studying for their exams and that, unlike a regular review lecture, they can rewind the video and start over again if they need to hear a concept repeated.

Another positive development is that eliminating the review lectures have given me three or four hours of extra contact time each semester with my students. Rather than add new material or cram in extra examples, I’ve mostly used these extra hours to slow down the pace of my lectures and to include group activities and other forms of student engagement during class time. I’m particularly happy that I have three dedicated days in my Discrete Mathematics class when my students can practice their new (for them) techniques of writing mathematical proofs. If they get stuck, I’m around to answer questions about the mechanics of proof-writing. If they don’t need help, they can get immediate affirmation from me about whether or not their proofs are correctly written. Discrete Mathematics is our math majors’ first introduction to writing mathematical proofs, and that my students have their initial struggles with this technique in class as opposed to when they do their homework on their own time.

So I intend to maintain this practice for the rest of my career.

However, there’s been one complication that I should have foreseen in 2011 but didn’t: the Americans with Disabilities Act. This had been mostly a potential problem for me that I hid away in the deep recesses of my mind until last semester, when a student with a hearing impairment was enrolled in my class.

In my next post, I’ll discuss some humorous examples of erroneous closed-captioning of mathematical speech which were automatically generated by YouTube.

Predicate Logic and Popular Culture (Part 206): Jack Johnson

Let H be the set of all things, let T be the set of all times, let G(x) be the proposition “x is good,” and let R(x,t) be the proposition “x remains at time t.” Translate the logical statement

\forall x \in H(G(x) \Longrightarrow \forall t \in T(R(x,t))).

This matches a line from “Mudfootball” by Jack Johnson.

Context: Part of the discrete mathematics course includes an introduction to predicate and propositional logic for our math majors. As you can probably guess from their names, students tend to think these concepts are dry and uninteresting even though they’re very important for their development as math majors.

In an effort to making these topics more appealing, I spent a few days mining the depths of popular culture in a (likely futile) attempt to make these ideas more interesting to my students. In this series, I’d like to share what I found. Naturally, the sources that I found have varying levels of complexity, which is appropriate for students who are first learning prepositional and predicate logic.

When I actually presented these in class, I either presented the logical statement and had my class guess the statement in actual English, or I gave my students the famous quote and them translate it into predicate logic. However, for the purposes of this series, I’ll just present the statement in predicate logic first.