To motivate that (admittedly) strange question:
When TeX builds a vertical list it keeps track of the depth of the last box added in a special variable called \prevdepth
which is used to calculate baselineskip glue for paragraph lines. When a new vertical list is being built (and the OR starts a new vertical list) TeX reinitializes \prevdepth
and when processing ends it resets it to the old (outer) value.
With normal output routines (that ship out pages) that's okay as the \prevdepth
(after the output routine ends) will match the last box on the main vertical list if there is a remainder that was not put into box255. If the main vertical list is empty the value is wrong but that doesn't make a difference because TeX will use \topskip
on top of the first line so it is essentially ignored.
But the situation is different if the output routine puts material back to be reprocessed. In that case the last box on the main vertical list is a) either the box that was not used in the OR and that has a depth matching \prevdepth
or it is the last box from the stuff that the OR contributed (which may have a different depth).
So to make baselineskip calculations work correctly we would need to know which situation we are in: has everything been packaged into box255 or is there an unused remainder? If that question could be answered that would also solve the issue but this is to my knowledge not possible.
An alternative solution would be to pretend that the material generated by the OR has the same depth as the outer \prevdepth
but for this I would somehow need to know its value.
Knowing its value is simple if we force the OR with an explicit penalty because then I can just save it and used the saved value. But what can be done if the OR is called by TeX on its own?
That's really the question above … any idea (as sneaky as it needs to be) to somehow get at this value in this particular case?
Or alternatively, any method to clearly determine whether or not the main vertical list is emptied.
Please note that I'm looking for a solution that works with all TeX engines, so doesn't make use of, say, lua programming.
To motivate a bit further, here is a short plain TeX file that shows the general problem (though here it could be fixed as we explicitly force the output routine):
\tracingonline=1
\showboxbreadth\maxdimen\showboxdepth\maxdimen
test with g to get a depth
\showthe\prevdepth % we see the prevdepth from the last line
% now assume we have some OR that traps the data and does something with it
% afterwards it is pushing back new or changed data t build pages, In the example I
% simply dropped the collected data and replaced it with soemthing else which has a different
% \prevdepth. A real life example would be rebalancing existing material.
\output{\setbox0=\vbox{\unvbox255}%
next line will be too close\par
\showlists
\showthe\prevdepth
}
\eject
% now we see that the prevdepth should be (and is 0pt) last line jiust contains
% characters without depth
\showlists
\showthe\prevdepth
% but now we got the old \prevdepth back even though it is no longer valid, as
% the OR simply pops the nest even though it is no longer valid.
% As a result we will get the wrong alignment on the next paragraph
Second line \par
\showlists
% here we can see that we are off by the 1.9...pt prevdepth as we should see
% 12pt baseline to baseline but we don't
\output{\plainoutput}
\bigskip
But we really should see:
Next line will not be too close \par Second line
\bye
If we run this we get:
and if we look into the log we can also see how the wrong \prevdepth
is messing up vertical spacing:
> 1.94444pt.
l.6 \showthe\prevdepth
% we see the prevdepth from the last line
?
### internal vertical mode entered at line 18 (\output routine)
\hbox(6.94444+0.0)x469.75499, glue set 337.7548fil
.\hbox(0.0+0.0)x20.0
.\tenrm n
.\tenrm e
.\tenrm x
.\tenrm t
.\glue 3.33333 plus 1.66666 minus 1.11111
.\tenrm l
.\tenrm i
.\tenrm n
.\tenrm e
.\glue 3.33333 plus 1.66666 minus 1.11111
.\tenrm w
.\tenrm i
.\tenrm l
.\tenrm l
.\glue 3.33333 plus 1.66666 minus 1.11111
.\tenrm b
.\kern0.27779
.\tenrm e
.\glue 3.33333 plus 1.66666 minus 1.11111
.\tenrm t
.\tenrm o
.\kern0.27779
.\tenrm o
.\glue 3.33333 plus 1.66666 minus 1.11111
.\tenrm c
.\tenrm l
.\tenrm o
.\tenrm s
.\tenrm e
.\penalty 10000
.\glue(\parfillskip) 0.0 plus 1.0fil
.\glue(\rightskip) 0.0
prevdepth 0.0, prevgraf 1 line
### vertical mode entered at line 0
### recent contributions:
\penalty 10000
prevdepth 1.94444, prevgraf 1 line
! OK.
<output> ...line will be too close\par \showlists
\showthe \prevdepth }
\break ->\penalty -\@M
l.18 \eject
?
> 0.0pt.
<output> ...se\par \showlists \showthe \prevdepth
}
\break ->\penalty -\@M
l.18 \eject
?
### vertical mode entered at line 0
### current page:
\glue(\topskip) 3.05556
\hbox(6.94444+0.0)x469.75499, glue set 337.7548fil
.\hbox(0.0+0.0)x20.0
.\tenrm n
.\tenrm e
.\tenrm x
.\tenrm t
.\glue 3.33333 plus 1.66666 minus 1.11111
.\tenrm l
.\tenrm i
.\tenrm n
.\tenrm e
.\glue 3.33333 plus 1.66666 minus 1.11111
.\tenrm w
.\tenrm i
.\tenrm l
.\tenrm l
.\glue 3.33333 plus 1.66666 minus 1.11111
.\tenrm b
.\kern0.27779
.\tenrm e
.\glue 3.33333 plus 1.66666 minus 1.11111
.\tenrm t
.\tenrm o
.\kern0.27779
.\tenrm o
.\glue 3.33333 plus 1.66666 minus 1.11111
.\tenrm c
.\tenrm l
.\tenrm o
.\tenrm s
.\tenrm e
.\penalty 10000
.\glue(\parfillskip) 0.0 plus 1.0fil
.\glue(\rightskip) 0.0
\penalty 10000
total height 10.0
goal height 643.20255
prevdepth 1.94444, prevgraf 1 line
! OK.
l.23 \showlists
?
> 1.94444pt.
l.24 \showthe\prevdepth
?
### vertical mode entered at line 0
### current page:
\glue(\topskip) 3.05556
\hbox(6.94444+0.0)x469.75499, glue set 337.7548fil
.\hbox(0.0+0.0)x20.0
.\tenrm n
.\tenrm e
.\tenrm x
.\tenrm t
.\glue 3.33333 plus 1.66666 minus 1.11111
.\tenrm l
.\tenrm i
.\tenrm n
.\tenrm e
.\glue 3.33333 plus 1.66666 minus 1.11111
.\tenrm w
.\tenrm i
.\tenrm l
.\tenrm l
.\glue 3.33333 plus 1.66666 minus 1.11111
.\tenrm b
.\kern0.27779
.\tenrm e
.\glue 3.33333 plus 1.66666 minus 1.11111
.\tenrm t
.\tenrm o
.\kern0.27779
.\tenrm o
.\glue 3.33333 plus 1.66666 minus 1.11111
.\tenrm c
.\tenrm l
.\tenrm o
.\tenrm s
.\tenrm e
.\penalty 10000
.\glue(\parfillskip) 0.0 plus 1.0fil
.\glue(\rightskip) 0.0
\penalty 10000
\glue(\parskip) 0.0 plus 1.0
\glue(\baselineskip) 3.11111
\hbox(6.94444+0.0)x469.75499, glue set 400.31046fil
.\hbox(0.0+0.0)x20.0
.\tenrm S
.\tenrm e
.\tenrm c
.\tenrm o
.\tenrm n
.\tenrm d
.\glue 3.33333 plus 1.66666 minus 1.11111
.\tenrm l
.\tenrm i
.\tenrm n
.\tenrm e
.\penalty 10000
.\glue(\parfillskip) 0.0 plus 1.0fil
.\glue(\rightskip) 0.0
total height 20.05556 plus 1.0
goal height 643.20255
prevdepth 0.0, prevgraf 1 line
! OK.
l.32 \showlists
Best Answer
@wipet in his answer has shown how to resolve the problem as long as we can assume that that the
\prevdepth
of all material in the document is sufficiently small (that it lies below\maxdepth
in fact). In that case we can use the depth of box 255 as a measure for the\prevdepth
calculations that will have taken place if there is any remainder in recent contributions, and use this to adjust the new depth to match that. An if there isn't any remainder then this doesn't really matter either. This trick is in fact already mentioned by Don Knuth in the TeXbook where he discusses an output routine that add index headings in random places in the text (and normal text has this nice property of having a depth smaller than\maxdepth
... normally but unfortunately not always.However, this stops working if this can't guaranteed and the general case is what I was and am after. He is also correct in stating that finding out what the value of
\prevdepth
on the main vertical list is isn't really going to help us as such, so my question above was partly incorrect: we also need to know if there is in fact something on the recent contributions and if so deal with it.So what I tried to get my head around over the last days if that is really impossible to find out within base TeX (or if it needs an extension in luatex or is already available there and resolved in ConTeXt ... would be interesting to learn about it beside that claim @Martin you made). And as it stands now it is possible in TeX after all. The solution is a bit complex and perhaps it can be simplified further but it is not so complex that it can't be used (and hopefully not too complex that I overlooked some cases).
The main idea is not to try to determine the situation directly but instead use 2 output routines in succession that just ensure that the recent contributions are empty. Along the way we gather enough information to afterwards dissect the material gathered and do whatever we like.
My initial idea of using
\aftergroup
to regain control and have a look doesn't work because the token inserted that way will not be executed when the output routine ends. Instead TeX immediately calls again on a procedure "buildpage" that takes anything on recent contributions and moves it to the main vertical list and only once that has happened my inserted token is looked at (in other words too late).So the more complicated approach is that the first OR puts box255 back but only it has changed
\vsize
to the largest possible dimen. Additionally it uses\aftergroup
so that we gain control later again. As we have changed\vsize
we will get everything including the material recent contributions onto the main vertical list and only then our control token kicks in. Finally we change the output routine to a second one and then return.The token inserted by
\aftergroup
will then issue a forcing penalty (actually it does a little more, see below) so that everything is grabbed and the second OR is called.Inside that OR we are now in a better situation:
\penalty 10000
)\vsplit
etc to get, for example the amount split of that we would have gotten in the first OR\aftergroup
to gain control after the OR has ended.\prevdepth
on the main vertical list to represent whatever is neededAnd that should (I believe) do the trick completely.
Here is the (more or less documented code including a bit of test data to play out various scenarios). It is a bit longish but largely because I tried to properly document the most important aspects and some of the more subtle points. Enjoy: