The goal of this question is to find an error in a computation by a computer algebra system where the 'correct' answer (complete with correct reasoning to justify the answer) can be found in the literature. Note that the system must claim to be able to perform that computation, not implementing a piece of (really old) mathematics is sad, but is a different topic.
From my knowledge of the field, there are plenty of examples of 19th century mathematics where today's computer algebra system get the wrong answer. But how far back can we go?
Let me illustrate what I mean. James Bernoulli in letters to Leibniz (circa 1697-1704) wrote that [in today's notation, where I will assume that $y$ is a function of $x$ throughout] he could not find a closed-form to $y' = y^2 + x^2$. In a letter of Nov. 15th, 1702, he wrote to Leibniz that he was however able to reduce this to a 2nd order LODE, namely $y''/y = -x^2$. Maple can find (correct) closed-forms for both of these differential equations, in terms of Bessel functions.
An example that is 'sad' but less interesting is
$$r^{n+1}\int_0^{\pi}\cos(r\rho \cos (\omega))\sin(\omega)^{2n+1}d\omega$$
with $n$ assumed to be a positive integer, $r>0$ and $\rho$ real; this can be evaluated as a Bessel functions but, for example, Maple can't. Poisson published this result in a long memoir of 1823.
One could complain that (following Schloemilch, 1857) that he well knew that
$$J_n(z) = \sum_{0}^{\infty} \frac{(-1)^m(z/2)^{n+2*m}}{m!(n+m)!}$$
Maple seems to think that this sum is instead $J_n(z)\frac{\Gamma(n+1)}{n!}$, which no mathematician would ever write down in this manner.
Another example which gets closer to a real bug is that Lommel in 1871 showed that the Wronskian of $J_{\nu}$ and $J_{-\nu}$ was $-2\frac{sin(\nu\pi)}{\nu z}$. Maple can compute the Wronskian, but it cannot simplify the result to $0$. This can be transformed into a bug by using the resulting expression in a context where we force the CAS to divide by it.
For a real bug, consider
$$\int_{0}^{\infty} t^{-\lambda} J_{\mu}(at) J_{\nu}(bt)$$
as investigated by Weber in 1873. Maple returns an unconditional answer, which a priori looks fine. If, however, the same question is asked but with $a=b$, no answer is returned! What is going on? Well, in reality that answer is only valid for one of $0\lt a\lt b$ or $0\lt b \lt a$. But it turns out (as Watson explains lucidly on pages 398-404 of his master treatise on Bessel functions, this integral is discontinuous for $a=b$. Actually, the answer given is also problematic for $\lambda=\mu=0, \nu=1$. And for the curious, the answer given is
$$\frac{2^{-\lambda}{a}^{\lambda-1-\nu}{b}^{\nu}
\Gamma \left( 1/2\nu+1/2\mu-1/2\lambda+1/2 \right)} {
\Gamma\left( 1/2\mu+1/2\lambda+1/2-1/2\nu\right) \Gamma \left( \nu+1 \right)}
{F(1/2-1/2\mu-1/2\lambda+1/2\nu,1/2\nu+1/2\mu-1/2\lambda+1/2;\nu+1;{\frac {{b}^{2}}{{a}^{2}}})}
$$
EDIT: I first asked this question when the MO community was much smaller. Now that it has grown a lot, I think it needs a second go-around. A lot of mathematicians use CASes routinely in their work, so wouldn't they be interested to know the 'age' gap between human mathematics and (trustable) CAS mathematics?
Best Answer
If I recall correctly from ~30 years ago, on the Apple ][ the calculation 7^2 would return 49.0001. More than 100 years ago (or even 100 years before that), mathematicians already knew that the square of an integer is an integer.