The mathptmx package is not the best way to load the times font as the fonts are both incomplete and a patchwork of symbols from different sources.
Among the free fonts, you best choice today is probably newtx (alternatively, on older distributions, you have txfonts, but they have some spacing problems):
Use only \renewcommand\rmdefault{ptm} instead of your package.
You can also use a font package like newtxtext (as suggested by Manual), or tgtermes, or even times (which would also change the sans serif and the tt). All of them affect only the text font.
Best Answer
Add the following code to the preamble, basically what it does is changing the default symbol font for letters to the Roman font.
\DeclareSymbolFont{letters}{OT1}{cmr}{m}{n}