\insert is a TeX primitive not a plain TeX command.
If you go
\insert\boxregister{
.... vertical mode material ...
}
then two things happen, the vertical mode material is saved away and an insert node is placed in the current list.
If the current list is a horizontal list in a paragraph the node "migrates" to the surrounding vertical list.
Any insert nodes that end up being inside a box are essentially lost.
Insert nodes that end up on the main vertical list affect the page breaker in several ways.
depending on \count\boxregister and \skip\footins the output routine leaves space for the inserts when chopping off the page.
Inside the output routine \box\boxregister contains the contents of the insertion boxes u to a maximum of \dimen\boxregister. Any additional inserts are held over on to the next page (and the last insert on the page may be split if it doesn't fit (and is splittable)
so in your example the output routine will be handed \box\paragraphednote which will be a vbox with a sequence of hboxes. the output routine is responsible for adding (or not) those boxes to the page before it is shipped out, it could unbox the contents or process them in any way or it could decide not to make any inserts on the page being shipped out and re-insert them into the material returned to the main vertical list.
You probably need to add an actual runnable example showing any problem. The code you link to is rather long to take in by eye but as far as I can see the intention is that teh boxing is only temporary and the \removehboxes macro is invoked to unbox and re-package the notes.
You say that \hbox prevents line breaking but that does not appear to be the intention of the code.
The reason is in the definition:
\outer\def\bye{\par\vfill\supereject\end}
So \bye doesn't just issue \end (that would by itself issue \par, but not \vfill): it also performs \supereject, which is
\def\supereject{\par\penalty-\@MM}
Why a -20000 penalty? The reason is in the output routine:
\def\plainoutput{\shipout\vbox{\makeheadline\pagebody\makefootline}%
\advancepageno
\ifnum\outputpenalty>-\@MM \else\dosupereject\fi}
If the penalty that triggers the output routine is greater than -20000, nothing special is done after advancing the page number. Otherwise \dosupereject comes into action:
\def\dosupereject{\ifnum\insertpenalties>\z@ % something is being held over
\line{}\kern-\topskip\nobreak\vfill\supereject\fi}
The macro ensures all pending insertions are shipped out, by repeated call of \supereject until \insertpenalties is not positive any more.
One can call \supereject at any time (which is the reason for \par at the beginning), for instance when a chapter is beginning.
Some more ideas from comments to the question and to the answer
TeX can be run interactively and it goes interactive if the main input file ends without an \end command (or makes an “Emergency stop” if the running mode is not interactive). Due to this possible interactivity, an implied \end command would be undesirable.
Only a few operating systems add an implied <EOF> marker at the end of a file, which could be ^^D (EOT) or ^^Z. The ^^Z character used to be added in MS-DOS files, but not all text editors were compliant.
Depending on the operating system and the shell, hitting Control-D at the interactive prompt can stop the execution, but no \end command is executed.
Since TeX keeps a stack of the \input files, it knows when it has reached the end of the main file, so a possible feature could be “insert the token \end when you reach the end of the main file”. However, there should also be a “disable automatic \end” feature to allow interaction at the end of the file. There is a similar feature in TeX: when switching from horizontal to vertical mode due to finding a <vertical command>, TeX adds a \par token (which could not be the primitive \par any more). However, the situation for an “automatic \end” is very different: breaking a paragraph into files is a common TeX activity, while the end of the main file just happen once.
Having the terminating \end allows for adding structure to it. One could redefine \end (after keeping a copy of the primitive) to do other bookkeeping business like \bye does. So
\let\TeXend\end
\outer\def\bye{\par\vfill\supereject\TeXend}
could be thought of, but it was not Knuth's choice (and it wouldn't be mine as well).
Finally, having \end (or \bye or \end{document}) at the end of the main file allows for putting comments and additional material after the end marker, which wouldn't be possible otherwise. Since most programming languages require programs to be explicitly terminated, I don't see why TeX should be an exception.
Best Answer
The spacing for
$$is the same in plain TeX as in LaTeX and is only wrong if you use incorrect input with a blank line before the display. in LaTeX it's also wrong to leave a blank line before\[but it makes some attempt to catch that case.A plain TeX example is
Producing
with the default settings with a small
\parskipand\abovedisplayskipof similar size to\baselineskipthe spacing betweeen the two aaaaa and the following display may look similar in this image (they are actually the same) but the second one is completely incorrect.The log for the first shows
That is a line of text (aaa..) a no break penalty, then because the line was long and overlapped the formula,
\abovedisplayskipis inserted then a baselineskip to the formula x=1.The second equation spacing is constructed completely differently:
That is,
the line of text aaa
\parskipspacing (0pt here) and\baselineskipspacing to a spurious invisible paragrapha single line paragraph consisting of an indentation box and parfillskip and no text
a nobreak penalty
\abovedisplayshortskipas the empty paragraph does not overlap the formula\baselineskipto the formulaThe formula text
Apart from the fact that the visible space in the second case consists of
\abovedisplaysmallskipand\baselineskipNotable here is there is no nobreak penalty after the second line of aaa, a page break can happen there in which case the empty paragraph line will not be discarded at the page break as it is not vertical space so the equation will not start at the top of the page but be a line too low.If a setting with a non zero
\parskipis used this will affect the second form. The spaces in the image are only the same as (ignoring stretch and shrink)\abovedisplayskipis equal in plain TeX defauts to\abovedisplaysmallskip+\baselineskip`.LaTeX's
\[tries to catch the second case as it doesso it inserts a box over half of the linewidth to avoid abovedisplayshortskip being used, and avoids glue (and so a page break possibility) being added before this box.