When conducting an hypothesis test or computing a confidence interval for the difference of two means, where at least one mean does not arise from a small sample, the Student t distribution must be employed. In particular, the number of degrees of freedom for the Student t distribution must be computed. Many textbooks suggest using Welch’s formula:
rounded down to the nearest integer. In this formula, is the standard error associated with the first average , where (if known) is the population standard deviation for and is the number of samples that are averaged to find . In practice, is not known, and so the bootstrap estimate is employed.
The terms and are similarly defined for the average .
In Welch’s formula, the term in the numerator is equal to . This is the square of the standard error associated with the difference , since
This leads to the “Pythagorean” relationship
which (in my experience) is a reasonable aid to help students remember the formula for .
Naturally, a big problem that students encounter when using Welch’s formula is that the formula is really, really complicated, and it’s easy to make a mistake when entering information into their calculators. (Indeed, it might be that the pre-programmed calculator function simply gives the wrong answer.) Also, since the formula is complicated, students don’t have a lot of psychological reassurance that, when they come out the other end, their answer is actually correct. So, when teaching this topic, I tell my students the following rule of thumb so that they can at least check if their final answer is plausible:
To my surprise, I have never seen this formula in a statistics textbook, even though it’s quite simple to state and not too difficult to prove using techniques from first-semester calculus.
Let’s rewrite Welch’s formula as
For the sake of simplicity, let and , so that
Now let . All of these terms are nonnegative (and, in practice, they’re all positive), so that . Also, the numerator is no larger than the denominator, so that . Finally, we notice that
Using these observations, Welch’s formula reduces to the function
and the central problem is to find the maximum and minimum values of on the interval . Since is differentiable on , the absolute extrema can be found by checking the endpoints and the critical point(s).
First, the endpoints. If , then . On the other hand, if , then .
Next, the critical point(s). These are found by solving the equation :
Plugging back into the original equation, we find the local extremum
Based on the three local extrema that we’ve found, it’s clear that the absolute minimum of on is the smaller of and , while the absolute maximum is equal to .
In conclusion, I suggest offering the following guidelines to students to encourage their intuition about the plausibility of their answers:
- If is much smaller than (i.e., ), then will be close to .
- If is much larger than (i.e., ), then will be close to .
- Otherwise, could be as large as , but no larger.