\textls (from microtype) not working consistently for things in math mode

math-modemicrotypespacing

In the following code:

\documentclass{article}
\usepackage[letterspace= -150]{microtype}
\usepackage{amsmath}


\DeclareFontFamily{U}{BOONDOX-calo}{\skewchar\font=45 }
\DeclareFontShape{U}{BOONDOX-calo}{m}{n}{
  <-> s*[1.05] BOONDOX-r-calo}{}
\DeclareFontShape{U}{BOONDOX-calo}{b}{n}{
  <-> s*[1.05] BOONDOX-b-calo}{}
\DeclareMathAlphabet{\mathcalboondox}{U}{BOONDOX-calo}{m}{n}
\SetMathAlphabet{\mathcalboondox}{bold}{U}{BOONDOX-calo}{b}{n}
\DeclareMathAlphabet{\mathbcalboondox}{U}{BOONDOX-calo}{b}{n}
\usepackage[mathscr]{eucal}
\renewcommand{\cal}{\mathscr}
\newcommand{\scr}{\mathcalboondox}

\begin{document}

 Hilbert functor \textls{$\scr{Hilb}$}$_X: \textbf{\textsf{Scheme}}^\text{op} \to \textbf{\textsf{Set}}$ s.t. for $U \in \textbf{\textsf{Scheme}}$, we have $\textls{$\scr{Hilb}$}_X(U) = \text{Hom}(U, \text{Hilb}_X)$.

 $$\textls{$\scr{Hilb}$}$$
 $$\textls{$asdf$ asdf $asdf$}$$
\textls{$asdf$} \textls{\text{$asdf$}}
\end{document}

only the first “Hilb” in script font has the desired letter spacing. The other two, in math mode, do not. I assume there is some issue in the math mode only, but I don’t know what. I have looked at Problem with microtype's `\textls` and math mode and Microtype gives wrong letterspacing in math mode, but their issue seems to be about something bad happening after trying to exit the ‘lsstyle’ environment. Even https://ctan.math.illinois.edu/macros/latex/contrib/microtype/microtype.pdf has the words “math mode” only 6 times, and I don’t think my issue is addressed. If it matters, let’s just say I’m working in Overleaf, so whatever compiler Overleaf uses is the relevant one.

EDIT: I added some more experiments (“asdf” lines above) and found that \textls does not seem to work on things in math mode. A commenter already pointed this out, but leaves the “correct” spacing of the first “Hilb” unexplained. Even if \textls was not “meant” to handle math mode, it clearly had worked once, so even if it is “against the rules”, I am wondering if I can get it to work again.

enter image description here

EDIT2: if instead I do \usepackage[tracking=allmath, letterspace= -150]{microtype} then ALL the text including the math mode stuff is set so that the letter space is smaller. So clearly the microtype package has the power to change the letter spacing in math mode… it just, doesn’t want to?

enter image description here

EDIT3: per Robert's suggestion, I load microtype with the verbose option (verbose=true next to letterspace=...) and attach the log file: https://notepad.ltd/jyxrxf4t.

Best Answer

I would avoid a math alphabet for this.

\documentclass{article}
\usepackage{lmodern}
\usepackage{amsmath}
\usepackage{fixcmex}
\usepackage{microtype}

\DeclareFontFamily{U}{BOONDOX-calo}{\skewchar\font=45 }
\DeclareFontShape{U}{BOONDOX-calo}{m}{n}{
  <-> s*[1.05] BOONDOX-r-calo}{}
\DeclareFontShape{U}{BOONDOX-calo}{b}{n}{
  <-> s*[1.05] BOONDOX-b-calo}{}
\DeclareRobustCommand{\scrvar}[1]{%
  \text{\textls[-100]{\usefont{U}{BOONDOX-calo}{m}{n}#1}}%
}


\DeclareMathAlphabet{\mathsfbf}{\encodingdefault}{\sfdefault}{b}{n}

\newcommand{\Hilbs}{\scrvar{Hilb}}
\newcommand{\Hilbr}{\mathrm{Hilb}}
\newcommand{\Scheme}{\mathsfbf{Scheme}}
\newcommand{\Set}{\mathsfbf{Set}}
\newcommand{\op}{^{\mathrm{op}}}
\DeclareMathOperator{\Hom}{Hom}

\begin{document}

Hilbert functor $\Hilbs_X\colon \Scheme\op \to \Set$ s.t.\@ for $U \in \Scheme$, 
we have $\Hilbs_X(U) = \Hom(U, \Hilbr_X)$.

\end{document}

enter image description here

Related Question