I assume this was done by design, seeing as the introduction of hyperlinks may clutter the user's view of the actual text. Moreover, not all hyperlink typesetting is printable - as you've mentioned, the PDF hyperlink is merely "a rectangular area of the page that is mouse-aware". However, if you want to do this, there are two options available
Manual
You deactivate the colorlinks
option so that hyperref
sets the link border:
\hypersetup{%
colorlinks=false,% hyperlinks will be black
linkbordercolor=red,% hyperlink borders will be red
pdfborderstyle={/S/U/W 1}% border style will be underline of width 1pt
}
and typeset the text manually using \color{<color>}
. For example:
...
\begin{document}
\section{To See}\label{tosee}
\hyperref[tosee]{\color{green}just to see}
\end{document}

Note that this is virtually the same as what hyperref
does internally, since the text colour is modified and will typeset this way even if the hyperlink is removed via printing to PDF (or flattening).
The advantage behind this approach (motivating to include it here) is that you can specify different colours for each hyperlink, if you so wish.
Automatic
You activate the colorlinks
option so that hyperref
sets the link colour in the text
\hypersetup{%
colorlinks=true,% hyperlinks will be coloured
linkcolor=green,% hyperlink text will be green
linkbordercolor=red,% hyperlink border will be red
}
and then add the following after the above \hypersetup{...}
:
\makeatletter
\Hy@AtBeginDocument{%
\def\@pdfborder{0 0 1}% Overrides border definition set with colorlinks=true
\def\@pdfborderstyle{/S/U/W 1}% Overrides border style set with colorlinks=true
% Hyperlink border style will be underline of width 1pt
}
\makeatother

Here is the pdfborderstyle
specification from Adobe:

Using linktoc=page
or linktocpage
, in the hyperref setup, allows you get sections' page numbers with red colored borders while the section name remains plain text.
This option doesn't change the rest of the document.
\documentclass[a4paper, 12pt]{amsart}
\usepackage{hyperref, xcolor}
\usepackage{cite}
\hypersetup{
backref,
colorlinks=false,
linktocpage,
pdfborder = {0 0 1 [1 0] }
}
\begin{document}
\title{test}
\maketitle
\tableofcontents
\section{Introduction}
In \cite[Thm 1]{work}...see section \S\ref{next} next
\section{Next}\label{next}
\begin{thebibliography}{99}
\bibitem[work]{work} \textit{July, 2013}
\end{thebibliography}
\end{document}

Best Answer
tufte-common.def
setswhen
hyperref
is loaded.So in order to get the borders back, we can just restore the default
See also What do the first two numbers in the pdfborder option of hyperref do?
MWE