[Tex/LaTex] \intextsep gives doubled space

floatsspacing

If a float (black box) follows my {example} environment (gray left line) the skip between them is too big. I guess it’s twice \baselineskip but it should be only one \baselineskip plus some glue. It should be the same space as between the float and following text.

error

I read in source2e that \intextsep doesn’t behave like \addvspace which causes the doubled space in my example. Can on fix this behavior or do I have to insert \vspace{-\baselineskip} manually everywhere?

I also tried to insert the space around {example} via \addvspace instead of the mdframed method (set skip as option) but that doesn’t work too.

Code

\documentclass[english]{scrartcl}

\usepackage{mdframed}
\newenvironment{example}[1][]{%
    \begin{mdframed}[%
        skipabove={\baselineskip plus 5pt minus 1.5pt},
        skipbelow={\baselineskip plus 5pt minus 1.5pt},
        middlelinewidth=3pt,
        middlelinecolor={black!60},
        splitbottomskip=1pt,
        splittopskip=8pt,
        innertopmargin=2pt,
        innerbottommargin=2pt,
        innerleftmargin=6.89749pt,
        innerrightmargin=0pt,
        topline=false,
        bottomline=false,
        rightline=false,
    ]%
}{%
    \end{mdframed}
}

\usepackage{babel,blindtext}
\begin{document}
\Blindtext
\begin{example}
\blindtext
\end{example}
\begin{table}[h]
\rule{\textwidth}{5cm}
\end{table}
\Blindtext
\end{document}

Update

After my first try to set \intextsep=0pt and add \addvspace{\intextsep} to the user level float environments didn’t work for all cases (the space was also inserted if the float floats away) I had a second look at source2e and found the \@addtocurcol macro which seems to be the one inserting the floats and the space around them. There are two lines in it inserting \intextsep via \vskip

\vskip \intextsep
\box\@currbox
\penalty\interlinepenalty
\vskip\intextsep

So I tried to change both of them width \pathcmd from etoolbox and it works for the case where a float follows an example but not if it preceeds.

\patchcmd{\@addtocurcol}%
    {\vskip \intextsep}%
    {\addvspace{\intextsep}}{}{}
\patchcmd{\@addtocurcol}%
    {\vskip\intextsep}{}{}{}

I also tried to place the second \addvspace somewhere else but without success.

I guess the the two \addvspaces of {table} and {example} can’t see each other to determine the correct spacing if the float precedes the example.

M(half)WE

\documentclass[english]{scrartcl}

\usepackage[framemethod=TikZ]{mdframed}
\newenvironment{example}[1][]{%
    \par\addvspace{\baselineskip}%
    \begin{mdframed}[%
%       skipabove={\baselineskip plus 5pt minus 1.5pt},
%       skipbelow={\baselineskip plus 5pt minus 1.5pt},
        middlelinewidth=3pt,
        middlelinecolor={black!60},
        splitbottomskip=1pt,
        splittopskip=8pt,
        innertopmargin=2pt,
        innerbottommargin=2pt,
        innerleftmargin=6.89749pt,
        innerrightmargin=0pt,
        topline=false,
        bottomline=false,
        rightline=false,
    ]%
}{%
    \end{mdframed}\par\addvspace{\baselineskip}%
}

\setlength{\intextsep}{\baselineskip}
\usepackage{etoolbox}
\makeatletter
\patchcmd{\@addtocurcol}%
    {\vskip \intextsep}%
    {\addvspace{\intextsep}}%
    {\typeout{*** SUCCESS ***}}{\typeout{*** FAIL ***}}
\patchcmd{\@addtocurcol}%
    {\vskip\intextsep}{}%
    {\typeout{*** SUCCESS ***}}%
    {\typeout{*** FAIL ***}}
\patchcmd{\@addtocurcol}%
    {\@inserttrue}%
    {\@inserttrue\addvspace{\intextsep}}%
    {\typeout{*** SUCCESS ***}}{\typeout{*** FAIL ***}}
\makeatother

\usepackage{babel,blindtext}
\begin{document}
\Blindtext\blindtext
\begin{table}[h]
    \rule{\textwidth}{5cm}
\end{table}
\begin{example}
    \blindtext
\end{example}
\begin{table}[h]
    \rule{\textwidth}{5cm}
\end{table}
\Blindtext
\begin{example}
    \blindtext
\end{example}
\blindtext
\end{document}

Result

result

Update 2

This is the code including Frank’s solution but it still doesn’t work correctly. There’s too much space between {example} and the first table, i.e. above the example. And it seems also not to work if I comment out the example: The two floats have more space between them as between a float and text.

\documentclass[english]{scrartcl}

\usepackage[framemethod=TikZ]{mdframed}
\newenvironment{example}[1][]{%
    \par\addvspace{\baselineskip}%
    \begin{mdframed}[%
%       skipabove={\baselineskip plus 5pt minus 1.5pt},
%       skipbelow={\baselineskip plus 5pt minus 1.5pt},
        middlelinewidth=3pt,
        middlelinecolor={black!60},
        splitbottomskip=1pt,
        splittopskip=8pt,
        innertopmargin=2pt,
        innerbottommargin=2pt,
        innerleftmargin=6.89749pt,
        innerrightmargin=0pt,
        topline=false,
        bottomline=false,
        rightline=false,
    ]%
}{%
    \end{mdframed}\par\addvspace{\baselineskip}%
}

\setlength{\intextsep}{\baselineskip}
\usepackage{etoolbox}
\makeatletter
\patchcmd{\@addtocurcol}%
    {\vskip \intextsep}%
    {\addvspace{\intextsep}}%
    {\typeout{*** SUCCESS ***}}{\typeout{*** FAIL ***}}

\patchcmd{\@addtocurcol}%
    {\vskip\intextsep}{\fix@second@intextsep}%
    {\typeout{*** SUCCESS ***}}%
    {\typeout{*** FAIL ***}}

\def\fix@second@intextsep {%
% was the float seen in vertical mode?
   \ifnum\outputpenalty <-\@Mii 
      \aftergroup\vskip\aftergroup\intextsep
  \else
      \vskip\intextsep
  \fi
}
\makeatother

\usepackage{babel,blindtext}
\begin{document}
\Blindtext\blindtext
\begin{table}[h]
    \rule{\textwidth}{5cm}
\end{table}
\begin{example}
    \blindtext
\end{example}
\begin{table}[h]
    \rule{\textwidth}{5cm}
\end{table}
\Blindtext
\begin{example}
    \blindtext
\end{example}
\blindtext
\end{document}

Best Answer

That's a slightly tricky one. Looks like you have found yet another deficiency in that part of 2e (if we don't want to call it a bug).

First we have to understand what \addvspace does: it looks if there is some positive (!) vertical glue just in front and if so does some magic to add the maximum of this glue and the one it is supposed to add. If it has to modify the existing glue it does this by first adding a negative glue which cancels the existing one and then adds its one glue. This roundabout way is necessary because one can't \unskip on the main vertical list, so real removal is not possible. It does nothing special for anything that follows --- this has to be handled by the next addvspaceor \addpenalty.

With that in mind the first change to \@addcurcol is correct: replacing \vskip \intextsep by \addvspace\intextsep will then nicely combine with any glue before the float.

However, it is pointless to try and change the second skip after the box. Why? Because before it there is just a box not any glue so \addvspacehere is equiv to \vskipexcept that it does some unnecessary lookups and tests to find out what is in from it it.

So why is then this skip not seen by a following \addvspacefrom the next environment? The answer to this lies in the way TeX implements the output routine call. Due to efficiency reasons (my guess) Don decided that keep a penalty item that triggers the output routine as the first item on the "recent contributions" and only change it to the value of 10000. In other words that penalty will never generate a break again. This is a somewhat nasty part of the algorithm and the cause for a lot of extra programming in various situations. Here it means that we will end up with the following sequence:

\glue \intextsep
\box (float)
\glue \intextsep
\penalty 10000   (from the initial \penalty -1000x triggered by the float)

Thus, any following `\addvspace will only see the penalty but not the glue.

so what could be done? The answer is to patch \@addcurcol differently:

\makeatletter
\patchcmd{\@addtocurcol}%
    {\vskip \intextsep}%
    {\addvspace{\intextsep}}%
    {\typeout{*** SUCCESS ***}}{\typeout{*** FAIL ***}}
\patchcmd{\@addtocurcol}%
    {\vskip\intextsep}{\aftergroup\vskip\aftergroup\intextsep}%
    {\typeout{*** SUCCESS ***}}%
    {\typeout{*** FAIL ***}}
\makeatother

This will result in the second \vskip being executed only after the output routine has ended and will therefore move the glue last so that it can be seen by a following \addvspace.

Update I + II

The above solution is unfortunately too simple minded. It works if the "here" float happens in vertical mode. But if the "here" float is placed in the middle of the paragraph then the "recent contributions" will not contain just the penalty but of course also any line from the paragraph which is after the line with the float. Thus our \aftergroups would not put the space after the float but would catapult it after the whole pararaph. So we need to make the code dependent on the mode the float was encountered in. Fortunately LaTeX signals this by using different penalties when triggering the output routine.

There is one further complication: in the 2e approach the is the line

  \ifnum\outputpenalty <-\@Mii \vskip -\parskip\fi

which backs up by \parskip if the float was encountered in vertical mode. The reason for this is that the next text line after the float will add a \parskip in that case. However, as Tobi mentioned in his comment, if two "here" floats come directly after each other then the space between them seems to be wrong. And indeed, in that case there is no \parskip added so canceling it means we are actually shortening the space by whatever amount it holds. Now there is no way to know, what comes after the float, so without any serious bookkeeping the only simple solution that I can see is to remove the above line (i.e., allow for the \parskip after the float in addition to \intextsep) and instead also add it before the float). Then the spacing around a "here" float in vertical mode will be always \parskip+\intextskip on either side and the same would happen between 2 such floats.

Finally, the space added above the float will just be inserted between the bottom of the last line of text (not measured from its baseline if the line has characters with descenders like "g"). However, below the float, there is not just the space added but also some extra space from the \baselineskip calculation (and funnily enough this also depends on the the \prevdepth of the line before the floats so it id doubly wrong). I therefore propose to suppress the \baselinskip correction altogether at this point using \nointerlineskip.

So the hopefully the (finally) correct solution looks like this:

\makeatletter
\patchcmd{\@addtocurcol}%
    {\ifnum\outputpenalty <-\@Mii \vskip -\parskip\fi}%
    {}%
    {\typeout{*** SUCCESS ***}}{\typeout{*** FAIL ***}}

\patchcmd{\@addtocurcol}%
    {\vskip \intextsep}%
    {\addvspace\intextsep
      \ifnum\outputpenalty <-\@Mii \vskip\parskip\fi}%
    {\typeout{*** SUCCESS ***}}{\typeout{*** FAIL ***}}

\patchcmd{\@addtocurcol}%
    {\vskip\intextsep}{\fix@second@intextsep}%
    {\typeout{*** SUCCESS ***}}%
    {\typeout{*** FAIL ***}}

\def\fix@second@intextsep {%
% was the float seen in vertical mode?
   \ifnum\outputpenalty <-\@Mii 
      \aftergroup\vskip\aftergroup\intextsep
      \aftergroup\nointerlineskip
  \else
      \vskip\intextsep
  \fi
}
\makeatother

Now this extended solution adds \parskip around the "here" floats in vertical mode so that there is now a noticable difference compared to "here" floats in the middle of a paragraph if this parameter has a positive value. Perhaps that makse this approach less attractive in such cases. However, detecting that two "here" floats really come directly after each other would be very difficult (and certainly not manageable by a few \patchcmd lines), so I guess this is as best as it can get.

Update III

Tobi tested the above solution and found one more case that it didn't cater for: if there are two "h" floats in succession and within a paragraph (i.e., within horizontal mode) then 2 instead of just one \intextsep spaces are inserted between them.

The reason for this behavior is that in hmode I add the \vskip\intextsep inside the output routine und thus before the penalty that triggered the OR and not after it using \aftergroup. There is a good reason for this (see discussion above) but of course if two such floats come directly after each other within a paragraph then this means that the \intextsepadded by the second one can't combine with the one from the previous float as it doesn't see it because of the penalty. So it looks like we are stuck: either the skip is kept hidden or it might get catapulted after the whole paragraph and thus would should up in the wrong place.

Fortunately there is still another trick we can apply: if we are not on the main vertical list (but building an internal vertical list and this is the case inside an output routine) we can make use of the primitive \unpenalty which looks at the last element on the current list and removes it if it is a penalty item. This way there is a possibility to get rid of this \penalty 10000 which is in our way and then \addvspace can act on a skip in front of this penalty.

Now one point we need to take into consideration is that there are "usually" 2 penalties we need to get rid off: the one from the output routine which is 10000 and the one that was inserted earlier in \@addtocurcol as \addpenalty\interlinepenalty. Now the question is: do we need this penalty at all if we immediately try to remove it again? The answer is "probably yes" at least without much more surgery. Point is, that because of the use of \addpenalty, there may be a skip after it (migrated from above). So we do need to try to \unpenalty twice and hope to catch this way the \penalty 10000 left over from the output routine. If the second penalty found this way is not 10000 we put the first one back (and hope for the best). This can happen in a number of scenarios so we need to account for it.

Another improvement we can try is to support two different skip values for floats within paragraphs and floats between paragraphs (after all the ones between paragraphs get the \parskipadded in addition, see previous update). So this gives us the following new set of patches:

\newlength\intextvsep  % like \intextsep but only for floats in vmode
\setlngth\intextvsep{\intextsep}

\patchcmd{\@addtocurcol}%
    {\vskip \intextsep}%
    {\edef\save@first@penalty{\the\lastpenalty}\unpenalty
     \ifnum \lastpenalty = \@M  % hopefully the OR penalty
        \unpenalty
     \else
        \penalty \save@first@penalty \relax % put it back
     \fi
      \ifnum\outputpenalty <-\@Mii
                         \addvspace\intextvsep
                         \vskip\parskip
      \else  
                         \addvspace\intextsep   
      \fi}%
    {\typeout{*** SUCCESS ***}}{\typeout{*** FAIL ***}}

\patchcmd{\@addtocurcol}%
    {\vskip\intextsep \ifnum\outputpenalty <-\@Mii \vskip -\parskip\fi}%
    {\fix@second@intextsep}%
    {\typeout{*** SUCCESS ***}}{\typeout{*** FAIL ***}}

If we use two separate parameters for the skip then we also need a slightly changed version of this command:

\def\fix@second@intextsep {%
% was the float seen in vertical mode?
   \ifnum\outputpenalty <-\@Mii 
      \aftergroup\vskip\aftergroup\intextvsep
      \aftergroup\nointerlineskip
  \else
      \vskip\intextsep
  \fi
}

If we use this is already looks reasonably good. However there is still a problem with it. If the user has an "h" float inside a paragraphs and tries to prevent the page break at the same line then things go wrong, e.g.:

Bla bla \nopagebreak[4] bla bla
\begin{table}[!h] -A- \end{table}
bla bla

This will produce a sequence of

<text line>
\penalty 10000   % from \nopagebreak not from a previous OR call !!!!!
\penalty 0       % from the interlinepenalty added by \@addtocurcol

So our two \unpenalty calls will see two penalties but will make the wrong deduction and remove the nobreak. Bad ...

One can fix even that but it means further changes: the penalty added by \nopagebreak is generated by some command called \getpen that looks at the optional argument and inserts different penalty values depending on the optional argument. A simple idea be to change the penalty there from \@M (10000) to, say, \@Mi since then our test above will fail and we are fine. But unfortunately \@getpen is also used for \pagebreak and this means we would there insert a penalty of -10001 and that signals a float not a pagebreak with the result that we get the LaTeX error "Float(s) lost". So we have to separate these cases and add different penalties when preventing and forcing page breaks:

\def\pagebreak{\@testopt{\@@pgbk}4}
\def\@@pgbk [#1]{%
  \ifvmode
    \penalty \@getpen@{#1}%
  \else
    \@bsphack
    \vadjust{\penalty \@getpen@{#1}}%
    \@esphack
  \fi}
\def\@getpen@#1{-\ifcase #1 \z@ \or \@lowpenalty\or
         \@medpenalty \or \@highpenalty
         \else \@M \fi}

% and here the definition used by \nopagebreak internally with a tiny change:
\def\@getpen#1{\ifcase #1 \z@ \or \@lowpenalty\or
         \@medpenalty \or \@highpenalty
         \else \@Mi \fi}

\@getpenis also used for \linebreak and friends but here 10000 or 10001 do not make a difference so we should be fine.

What this doesn't solve is the question of combining "h" floats with objects that look a little like "h" floats (such as listings or mdframed) but that do not use the float mechanism at all, i.e., do not trigger the output routine to place the object. If such objects are placed next to each other within a paragraph, then all the gymnastics with \unpenalty etc are not possible (since this is not allowed on the main vertical list) and thus we will have a situation where an \addvspace is unable to see the previous \intextsep from the "h" float and thus the spacing will get wrong without manual correction. The only way (that I can see) to fix that is to turn such objects into float classes as well and then use instances of "h" floats of those classes.

What else could be improved? As mentioned in "parskip inserts extra space after floats (and listings)" the vertical positioning for "h" floats within a paragraph is not perfect and a possible way to improve this is to use a \strut in that case. We can do this automatically and since we are patching this area here is also the code for this:

\patchcmd{\end@float}%
    {\vadjust}{\strut\vadjust}%
    {\typeout{*** SUCCESS ***}}{\typeout{*** FAIL ***}}

Summary

I think the whole area shows clear deficiencies in the 2e code (if not to say bugs) but I can state right now that for 2e there will be no change in the kernel. This would break/change too many documents which are formatted according to the current spacing rules.

I left the updates one after each other because I think it makes some of the problems around the OR algorithms easier to understand.

As Tobi mentioned below, the patches above need to be added in some shape or form also for the commands from the float package in case that package is used too.