Thoughts on Numerical Integration (Part 3): Derivation of left, right, and midpoint rules

Numerical integration is a standard topic in first-semester calculus. From time to time, I have received questions from students on various aspects of this topic, including:

Why is numerical integration necessary in the first place?
Where do these formulas come from (especially Simpson’s Rule)?
How can I do all of these formulas quickly?
Is there a reason why the Midpoint Rule is better than the Trapezoid Rule?
Is there a reason why both the Midpoint Rule and the Trapezoid Rule converge quadratically?
Is there a reason why Simpson’s Rule converges like the fourth power of the number of subintervals?

In this series, I hope to answer these questions. While these are standard questions in a introductory college course in numerical analysis, and full and rigorous proofs can be found on Wikipedia and Mathworld, I will approach these questions from the point of view of a bright student who is currently enrolled in calculus and hasn’t yet taken real analysis or numerical analysis.

For the sake of completeness, I discuss here the origins of the left-endpoint, right-endpoint, and midpoint rules of numerical integration. (These topics are often presented in calculus texts.) Consider the problem of finding $\displaystyle \int_a^b f(x) \, dx$, the area under a curve $f(x)$ between $x=a$ and $x=b$ .

To start the process of numerical integration, the interval $[a,b]$ is divided into subintervals. Usually, for convenience, the intervals are chosen to be the same length, a convention that I’ll follow in this series. That said, if the function is known to vary wildly on some parts of the domain but not so wildly on other parts, then computational efficiency can be gained by varying the sizes of the subintervals, choosing smaller subintervals for the places where the function varies wildly.

In any event, we’ll choose equal-sized subintervals for the duration of this series.

One numerical approximation can be made by choosing left endpoints. In the picture below, the interval $[a,b]$ was divided into four equal subintervals. Let $h = (b-a)/4$ , so that $x_0 = a$ , $x_1 = x_0 +h$ , $x_2 = x_0 + 2h$ , $x_3 = x_0 + 3h$ , and $x_4 = x_0 + 4h = b$ . We then can draw rectangles using the left endpoints of each subinterval. The sum of the areas of these rectangles below is

$hf(x_0) + hf(x_1) + hf(x_2) +hf(x_3)$ ,

and so this serves as an approximation to the area under the curve. In general, if there are $n$ subintervals and $x_k = x_0 + kh$ , then the integral may be approximated as

$\int_a^b f(x) \, dx \approx h \left[f(x_0) + f(x_1) + \dots + f(x_{n-1}) \right]$

That said, left endpoints were not necessary for making an approximation. We could have instead chosen the right endpoints of each subinterval. The sum of the areas of the rectangles below is

$hf(x_1) + hf(x_2) + hf(x_3) +hf(x_4)$ ,

and so this serves as an approximation to the area under the curve. In general, if there are $n$ subintervals, then the integral may be approximated as

$\int_a^b f(x) \, dx \approx h \left[f(x_1) + f(x_2) + \dots + f(x_n) \right]$

As a final approximation, any point in each subinterval could’ve been used for making an approximation. In the picture below, we use the midpoints of the subintervals, where $c_k = (x_k + x_{k-1})/2$ . The sum of the areas of the rectangles below is

$hf(c_1) + hf(c_2) + hf(c_3) +hf(c_4)$ ,

and so this serves as an approximation to the area under the curve. In general, if there are $n$ subintervals, then the integral may be approximated as

$\int_a^b f(x) \, dx \approx h \left[f(c_1) + f(c_2) + \dots + f(c_n) \right]$

Thoughts on Numerical Integration (Part 2): The bell curve

Numerical integration is a standard topic in first-semester calculus. From time to time, I have received questions from students on various aspects of this topic, including:

Why is numerical integration necessary in the first place?
Where do these formulas come from (especially Simpson’s Rule)?
How can I do all of these formulas quickly?
Is there a reason why the Midpoint Rule is better than the Trapezoid Rule?
Is there a reason why both the Midpoint Rule and the Trapezoid Rule converge quadratically?
Is there a reason why Simpson’s Rule converges like the fourth power of the number of subintervals?

In this post, I’d like to take a closer look at the indefinite integral $\displaystyle \int e^{-x^2} dx$ , which is closely related to the area under the bell curve $\displaystyle \frac{1}{\sqrt{2\pi}} e^{-x^2/2} dx$ . This integral cannot be computed using elementary functions. However, using integration by parts, there are some related integrals that can be computed:

$\displaystyle \int x e^{-x^2} dx = -\displaystyle \frac{1}{2} e^{-x^2}$

$\displaystyle \int x^3 e^{-x^2} dx = -\displaystyle \frac{x^2+1}{2} e^{-x^2}$

$\displaystyle \int x^5 e^{-x^2} dx = -\displaystyle \frac{x^4+2x^2+2}{2} e^{-x^2}$

$\displaystyle \int x^7 e^{-x^2} dx = -\displaystyle \frac{x^6+3x^4+6x^2+6}{2} e^{-x^2}$

Based on these examples, it stands to reason that, if $\displaystyle \int e^{-x^2} dx$ can be written in terms of elementary functions, it should have the form

$\displaystyle \int e^{-x^2} dx = f(x) e^{-x^2}$ ,

where $f(x)$ is some polynomial to be determined. We will now show that this is impossible.

Suppose $f(x) = \displaystyle \sum_{k=0}^n a_k x^k$ , a polynomial of degree $n$ to be determined. Then we have

$\displaystyle \frac{d}{dx} \left[ f(x) e^{-x^2} \right] = e^{-x^2}$

$f'(x) e^{-x^2} - 2 x f(x) e^{-x^2} =e^{-x^2}$

$f'(x) - 2x f(x) = 1$ .

In other words, all terms on the left-hand side except the constant term must cancel. However, this is impossible: $2x f(x)$ is a polynomial of degree $n+1$ while $f'(x)$ is a polynomial of degree $n-1$ . Therefore, the left hand side must have degree $n+1$ and therefore cannot be a constant.

A similar argument shows that $f(x)$ cannot have the form $f(x) = \displaystyle \sum_{k=0}^n a_k x^{b_k}$ , where the exponents $b_k$ may or may not be integers.

This may be enough to convince a calculus student that there is no elementary antiderivative of $\displaystyle e^{-x^2} dx$ . Indeed, although the proof goes well beyond first-year calculus, there is a theorem that says that if $\displaystyle \int x^a e^{bx^2}$ can be expressed in terms of elementary functions, then the antiderivative must have the form $f(x) e^{b x^2}$ . So the guess above actually can be rigorously justified. References:

Elena Anne Marchisotto and Gholam-Ali Zakeri, “An Invitation to Integration in Finite Terms,” The College Mathematics Journal , Sep., 1994, Vol. 25, No. 4 (Sep., 1994), pp. 295-308
J. F. Ritt, Integration in Finite Terms: Liouville’s Theory of Elementary Methods, Columbia University Press, New York, 1948

Thoughts on Numerical Integration (Part 1): Why numerical integration?

Numerical integration is a standard topic in first-semester calculus. From time to time, I have received questions from students on various aspects of this topic, including:

Why is numerical integration necessary in the first place?
Where do these formulas come from (especially Simpson’s Rule)?
How can I do all of these formulas quickly?
Is there a reason why the Midpoint Rule is better than the Trapezoid Rule?
Is there a reason why both the Midpoint Rule and the Trapezoid Rule converge quadratically?
Is there a reason why Simpson’s Rule converges like the fourth power of the number of subintervals?

First, let’s talk about why numerical integration is necessary in the first place. Indeed, I can still remember a high school calculus teacher asking me this question nearly 20 years ago, and this question really got me thinking about what we’re collectively teaching in the secondary curriculum. Indeed, in a Calculus I course, it seems like every integral can be computed if only the proper trick is used. We teach students to search for these different tricks:

Let $u = x^2+5$ to find $\displaystyle \int \frac{6x \, dx}{\sqrt{x^2+9}}$ .
Let $x = 3\tan \theta$ to find $\displaystyle \int \frac{6 \, dx}{\sqrt{x^2+9}}$
Use integration by parts to find $\displaystyle \int x^3 e^x \, dx$

In fact, we teach so many tricks that we may give the impression that every integral can be computed if only the proper trick is employed. Indeed, my university hosts an annual “Integration Bee” that challenges students to find the right technique(s) to evaluate some pretty tough integrals.

Unfortunately, not every integral can be solved in terms of a finite number of elementary functions (polynomials, rational functions, exponential functions, logarithms, trigonometric and inverse trigonometric functions). One function that is commonly known to many students which does not have an elementary antiderivative is $\displaystyle \frac{1}{\sqrt{2\pi}} e^{-x^2/2}$ , otherwise known as the bell curve. For most numbers $a$ and $b$ , the area

$\displaystyle \int_a^b \frac{1}{\sqrt{2\pi}} e^{-x^2/2}$

cannot be found exactly, and so we ask students to either use a table in the back of the textbook or else use a function on their scientific calculators to find the answer.

Just for the fun of it, I went through my Ph.D. thesis and wrote down some of the integrals that I had to integrate numerically while in school. As an applied mathematician, I was initially stunned by the teacher’s innocent question because so much of my work would be utterly impossible if it wasn’t for numerical integration. Here are some of the easier ones:

$\displaystyle \int_0^t \frac{1-e^{-x}}{x} dx$
$\displaystyle \int_{2R}^\infty t^2 g(t) \left( \frac{a_1 t^4 + a_2 t^2 + a_3}{(t^2-R^2)^7} +\frac{b_1 t^2 + b_2}{(t^2-R^2)^5} + \frac{c}{(t^2-R^2)^3} \right) dt$
$\displaystyle \int_{d_2}^\infty \int_0^{d_1} \frac{y^2-x^2}{(x^2+y^2)^2} \left(e^{-a(x+d_1)-b d_2} - c\right) dx \, dy$
$\displaystyle \int_{x/2}^\infty \sqrt{r^2 - k^2/4} \phi(r) \, dr$
$\displaystyle \int_{x/2}^\infty \left( \frac{z \sqrt{4r^2-z^2}}{4} + r^2 \arcsin \left( \frac{z}{2r} \right) \right) \phi(r) \, dr$
$\displaystyle \int_0^{2R} e^{-sz} \exp \left[ -c \left( z \sqrt{4R^2-z^2} + 4R^2 \arcsin \frac{z}{2R} \right) \right] dz$
$\displaystyle \int_0^d \exp \left[ -sz - \lambda \left(z - \frac{z^2}{4d} \right) \right] dz$
$\displaystyle \int_d^{d \sqrt{2}} \exp \left[ -sz - \lambda \left( \frac{d (\pi+1)}{2} - d \arcsin \frac{d}{z} + \frac{z^2}{4d} - \sqrt{z^2-d^2} \right) \right] dz$
$\displaystyle \int_0^\infty \exp \left[-sz - \eta \left(1 - e^{-cz/2} - \frac{cz}{4} e^{-cz/2} \right) \right] dz$
$\displaystyle \int_{-\infty}^x \frac{e^t}{t} dt$

All this to say, there are plenty of integrals that arise from a real-world context that have a numerical answer but cannot be computed using the techniques commonly taught in the first-year calculus sequence.

My Favorite One-Liners: Part 117

I absolutely love this joke. The integral looks diabolical but can be computed mentally.

For what it’s worth, while it was able to produce an answer to as many decimal places as needed, even Wolfram Alpha was unable to exactly compute this integral. Feel free to click the link if you’d like the (highly suggestive) answer.

Source: https://www.facebook.com/CTYJohnsHopkins/photos/a.323810509981/10151131549924982/?type=3&theater

Increasingly difficult integrals

Source: https://www.facebook.com/MathematicalMemesLogarithmicallyScaled/photos/a.1605246506167805.1073741827.1605219649503824/2209404589085324/?type=3&theater

Borwein integrals

When teaching proofs, I always stress to my students that it’s not enough to do a few examples and then extrapolate, because it’s possible that the pattern might break down with a sufficiently large example. Here’s an example of this theme that I recently learned:

No automatic alt text available.

Source: https://www.facebook.com/MathematicalMemesLogarithmicallyScaled/photos/a.1605246506167805.1073741827.1605219649503824/2080975208594930/?type=3&theater

For further reading:

My Favorite One-Liners: Part 101

In this series, I’m compiling some of the quips and one-liners that I’ll use with my students to hopefully make my lessons more memorable for them.

I’ll use today’s one-liner when a choice has to be made between two different techniques of approximately equal difficulty. For example:

Calculate $\displaystyle \iint_R e^{-x-2y}$ , where $R$ is the region $\{(x,y): 0 \le x \le y < \infty \}$

There are two reasonable options for calculating this double integral.

Option #1: Integrate with respect to $x$ first:

$\int_0^\infty \int_0^y e^{-x-2y} dx dy$

Option #2: Integrate with respect to $y$ first:

$\int_0^\infty \int_x^\infty e^{-x-2y} dy dx$

Both techniques require about the same amount of effort before getting the final answer. So which technique should we choose? Well, as the instructor, I realize that it really doesn’t matter, so I’ll throw it open for a student vote by asking my class:

Anyone ever read the Choose Your Own Adventure books when you were kids?

After the class decides which technique to use, then we’ll set off on the adventure of computing the double integral.

This quip also works well when finding the volume of a solid of revolution. We teach our students two different techniques for finding such volumes: disks/washers and cylindrical shells. If it’s a toss-up as to which technique is best, I’ll let the class vote as to which technique to use before computing the volume.

My Favorite One-Liners: Part 88

In this series, I’m compiling some of the quips and one-liners that I’ll use with my students to hopefully make my lessons more memorable for them.

In the first few weeks of my calculus class, after introducing the definition of a derivative,

$\displaystyle \frac{dy}{dx} = y' = f'(x) = \lim_{h \to 0} \displaystyle \frac{f(x+h) - f(x)}{h}$ ,

I’ll use the following steps to guide my students to find the derivatives of polynomials.

If $f(x) = c$ , a constant, then $\displaystyle \frac{d}{dx} (c) = 0$ .
If $f(x)$ and $g(x)$ are both differentiable, then $(f+g)'(x) = f'(x) + g'(x)$ .
If $f(x)$ is differentiable and $c$ is a constant, then $(cf)'(x) = c f'(x)$ .
If $f(x) = x^n$ , where $n$ is a nonnegative integer, then $f'(x) = n x^{n-1}$ .
If $f(x) = a_n x^n + a_{n-1} x^{n-1} + \dots + a_1 x + a_0$ is a polynomial, then $f'(x) = n a_n x^{n-1} + (n-1) a_{n-1} x^{n-2} + a_1$ .

After doing a few examples to help these concepts sink in, I’ll show the following two examples with about 3-4 minutes left in class.

Example 1. Let $A(r) = \pi r^2$ . Notice I’ve changed the variable from $x$ to $r$ , but that’s OK. Does this remind you of anything? (Students answer: the area of a circle.)

What’s the derivative? Remember, $\pi$ is just a constant. So $A'(r) = \pi \cdot 2r = 2\pi r$ .

Does this remind you of anything? (Students answer: Whoa… the circumference of a circle.)

Generally, students start waking up even though it’s near the end of class. I continue:

Example 2. Now let’s try $V(r) = \displaystyle \frac{4}{3} \pi r^3$ . Does this remind you of anything? (Students answer: the volume of a sphere.)

What’s the derivative? Again, $\displaystyle \frac{4}{3} \pi$ is just a constant. So $V'(r) = \displaystyle \frac{4}{3} \pi \cdot 3r^2 = 4\pi r^2$ .

Does this remind you of anything? (Students answer: Whoa… the surface area of a sphere.)

By now, I’ve really got my students’ attention with this unexpected connection between these formulas from high school geometry. If I’ve timed things right, I’ll say the following with about 30-60 seconds left in class:

Hmmm. That’s interesting. The derivative of the area of a circle is the circumference of the circle, and the derivative of the area of a sphere is the surface area of the sphere. I wonder why this works. Any ideas? (Students: stunned silence.)

This is what’s known as a cliff-hanger, and I’ll give you the answer at the start of class tomorrow. (Students groan, as they really want to know the answer immediately.) Class is dismissed.

If you’d like to see the answer, see my previous post on this topic.

My Favorite One-Liners: Part 87

In this series, I’m compiling some of the quips and one-liners that I’ll use with my students to hopefully make my lessons more memorable for them.

When teaching Calculus I, I use the following mantra throughout the semester. I heard this from my calculus instructor back in 1984, and I repeat it for my own students:

There are two themes of calculus: approximating curved things by straight things, and passing to limits.

For example, to find a derivative, we approximate a curved function by a straight tangent line and then pass to a limit. Later in the semester, to find a definite integral, we approximate the area under a curve by the sum of a bunch of straight rectangles and then pass to a limit.

For further reading, I’ll refer to this series of posts on what I typically do on the first day of my calculus class.

My Favorite One-Liners: Part 74

In this series, I’m compiling some of the quips and one-liners that I’ll use with my students to hopefully make my lessons more memorable for them.

After presenting the Fundamental Theorem of Calculus to my calculus students, I make a point of doing the following example in class:

$\displaystyle \int_0^4 \frac{1}{4} x^2 \, dx$

Hopefully my students are able to produce the correct answer:

$\displaystyle \int_0^4 \frac{1}{4} x^2 \, dx = \displaystyle \left[ \frac{x^3}{12} \right]^4_0$

$= \displaystyle \frac{(4)^3}{12} - \frac{(0)^3}{12}$

$= \displaystyle \frac{64}{12}$

$= \displaystyle \frac{16}{3}$

Then I tell my students that they’ve probably known the solution of this one since they were kids… and I show them the classic video “Unpack Your Adjectives” from Schoolhouse Rock. They’ll watch this video with no small amount of confusion (“How is this possibly connected to calculus?”)… until I reach the 1:15 mark of the video below, when I’ll pause and discuss this children’s cartoon. This never fails to get an enthusiastic response from my students.

If you have no idea what I’m talking about, be sure to watch the first 75 seconds of the video below. I think you’ll be amused.