Penalties are the main value that TeX tries to minimise when line or page breaking, They may be inserted explicitly (\penalty125 means that the penalty for breaking at that point is 125). Some penalties are built in to the TeX system and inserted automatically, LaTeX for example sets these default values for built in penalties
\linepenalty=10
the penalty added to the badness of each line within a paragraph (no associated penalty node) Increasing the value makes tex try to have fewer lines in the paragraph.
\interlinepenalty=0
value of the penalty (node) added after each line of a paragraph.
\hyphenpenalty=50
the penalty for line breaking at an automatically inserted hyphen
\exhyphenpenalty=50
the penalty for line breaking at an explicit hyphen
\binoppenalty=700
the penalty for breaking a line at a binary operator
\relpenalty=500
the penalty for breaking a line at a relation
\clubpenalty=150
extra penalty for breaking after first line of a paragraph
\widowpenalty=150
extra penalty for breaking before last line of a paragraph
\displaywidowpenalty=50
extra penalty for breaking before last line before a display math
\brokenpenalty=100
extra penalty for page breaking after a hyphenated line
\predisplaypenalty=10000
penalty for breaking before a display
\postdisplaypenalty=0
penalty for breaking after a display
\floatingpenalty = 20000
penalty for splitting an insertion (can only be split footnote in standard LaTeX)
e-tex adds some additional penalties (that are not set by default in LaTeX) but are supported by all three of the major engines
\interlinepenalties
An extension to \interlinepenalty that allows different penalties to be added after each line. It takes a space separated list of numbers as value. If n is positive interlinepenalties n i1... in specifies the penalty to be used after each of the lines in a paragraph, with the last value being repeated as often as needed if the paragraph has more then n lines.
\clubpenalties
An extension of \clubpenalty with the same syntax as \interlinepenalties.
\widowpenalties
An extension of \widowpenalty with the same syntax as \interlinepenalties (lines arecounted backwards from the last line in this case).
\displaywidowpenalties
An extension of \widowpenalty with the same syntax as \widowpenalties with lines counted backwards from each display math within a paragraph.
XeTeX supports one additional primitive penalty:
\XeTeXlinebreakpenalty
The full documentation of which is Inter-character linebreak penalty I believe it is the penalty inserted between characters for Asian typesetting which typically doesn't use inter-word spaces.
Other penalties are inserted by the format for example LaTeX defines a count
\newcount\interfootnotelinepenalty \interfootnotelinepenalty=100
but locally arranges that \linepenalty is set to this value while setting footnotes.
Similarly article class sets
\@lowpenalty 51
\@medpenalty 151
\@highpenalty 301
which are the values used by \pagebreak[1] ...
\samepage works by setting many of these penalties to "infinite" values:
\def\samepage{\interlinepenalty\@M
\postdisplaypenalty\@M
\interdisplaylinepenalty\@M
\@beginparpenalty\@M
\@endparpenalty\@M
\@itempenalty\@M
\@secpenalty\@M
\interfootnotelinepenalty\@M}
Note penalties are not just numeric values they are actual nodes that are inserted into the horizontal or vertical list. they can be inspected with \lastpenalty or removed (in some contexts) with \unpenalty so for example \count0=\lastpenalty will put the value of the penalty (even if it was automatically inserted) into the count register if the penalty s the last thing in the current horizontal or vertical list.
For example this bit of LaTeX which tries to insert italic corrections without disturbing the penalties in the horizontal list
\def \fix@penalty {%
\ifnum \lastpenalty=\z@
\@@italiccorr
\else
\count@ \lastpenalty
\unpenalty
\@@italiccorr
\penalty \count@
\fi
}
This is part of all the commands such as \textit that try to insert automatic italic correction.
Demerits
The statement above that TeX tries to minimise penalties is in fact a simplification. For line breaking, the actual quantity that is minimised is demerits which is (more or less) the sum of the squares of the penalties associated with the linebreaks in a paragraph and the badness of each line (which is a measure of how much white space has been stretched beyond its specified limits) plus three additional demerit parameters as described below. The exact formula is in the TeXBook, but isn't usually relevant.
The three demerit parameters and the default values given to them by LaTeX are:
\doublehyphendemerits=10000
Additional demerit added to the paragraph for each pair of consecutive lines ending in a discretionary (typically a hyphen).
\finalhyphendemerits=5000
Additional demerit added if the penultimate line ends with a discretionary.
\adjdemerits=10000
Additional demerit added for pairs of "incompatible" lines (ie a tight line in which white space is squeezed next to a loose line in which white space is stretched.)
Note, the values of demerit parameters are typically large (they combine with the square of penalty and badness units) also demerits are just a value used in the line breaking calculation. Unlike penalties they do not correspond to nodes that may be manipulated in TeX lists.
The command you defined does not take an optional argument, it takes a delimited argument. If you do:
\def\b[#1]#2{.#2.\bf #1}
\b[one]two
it will work fine, however if you remove the [one] TeX will throw an error:
\def\b[#1]#2{.#2.\bf #1}
\b two
! Use of \b doesn't match its definition.
l.5 \b t
wo
?
because when you define a command with \def\b[#1]#2{.#2.\bf #1}, TeX expects that when you use \b, the input matches exactly the parameter text (i.e., [#1]#2), which means that the next token must be [, and when it is not that error is raised. See here for a brief description of that.
When using just \def none of the arguments are optional! However, let's say you define:
\newcommand\b[2][--empty--]{.#2.\bf #1}
then the command will have 2 arguments, the first of which is optional, and if not given the default value is --empty--. When you use \b, the defined command does not actually take any argument, but it checks if the next character is a [. If it is, the command proceeds to use an "inner" \b (let's call it \b@opt), which is defined as you did, with \def\b@opt[#1]#2{.#2.\bf #1}. However if you use \b without the following [, then a \b@noopt is used, which is defined as \def\b@noopt{\b@opt[--empty--]}. So after all you end up using \b@opt, but the underlying definition provides the optional argument if you don't give it one.
You can manually define that with:
\makeatletter
\def\b{%
\@ifnextchar[%
{\b@opt}{\b@noopt}%
}
\def\b@noopt{\b@opt[--empty--]}
\def\b@opt[#1]#2{.#2.\bf #1}
\makeatother
Now, what makes the optional argument thing “fragile”?
A command is fragile when it can't work properly in an expansion-only context, which is usually when being written to a temporary file, like in section headings as you showed, captions, and the like, but also inside an \edef or, more recently, in \expanded.
It's said that commands with optional arguments are fragile because the mechanism that checks if an optional argument is there (precisely, the \@ifnextchar macro above) usually is fragile. It is possible, under some restrictions, to check for an optional argument expandably, like in xparse's \NewExpandableDocumentCommand, but usually that's not the case.
Taking the command defined above as example, if you do \edef\test{\b[one]{two}} (or \write or \expanded) TeX starts expanding from left to right, so the first thing it sees is \b, which is expanded to
\@ifnextchar[{\b@opt}{\b@noopt}
Next the \@ifnextchar test is expanded to:
\let\reserved@d=[%
\def\reserved@a{\b@opt}%
\def\reserved@b{\b@noopt}%
\futurelet\@let@token\@ifnch
Here the problem appears. \let, \def, and \futurelet aren't expandable, so TeX leaves them as they are, and proceeds expanding the rest. All other macros there will be expanded by TeX, but by doing so the \let and \def will not define \reserved@d and such, but their expansion, and this will make the code not work as intended.
Of course this is just an example, but the basic principle of fragility is that a command that contains non-expandable tokens is being used in an expansion-only context.
How to make a command robust?
Until a couple decades ago the only way to make a command robust was to prevent its expansion with \noexpand\command, which makes TeX temporarily treat \command as unexpandable and skip it in an expansion-only context. The downside of this is that as soon as the expansion was carried out the \noexpand would disappear and the command would be fragile again.
To circumvent this LaTeX defines \protect and the accompanying macros \protected@edef and \protected@write, which define \protect as \def\protect{\noexpand\protect\noexpand}. Then, in an expansion-only context \protect\command will expand to \noexpand\protect\noexpand\command. TeX will throw both \noexpands away, temporarily making \protect\command both unexpandable. If you happened to use the command again, it would continue being robust if you used the \protected@... macros instead of the normal ones.
Commands with optional arguments defined with LaTeX2ε's \newcommand and the like have a different look (but the same machinery underneath). If you define \newcommand\b[2][--empty--]{.#2.\bf #1}, then \b will actually be \protected@testopt \b \\b {--empty--} (that \\b is the command \\b, with two backslashes, not \\ then b). \protected@testopt will use the \protect machinery to test whether it can be safely expanded. If it cannot it will leave \protect\b, otherwise it will proceed to use \\b, which contains the actual definition of the command.
All this became easier when ε-TeX introduced the \protected primitive, which allows you to make a macro engine-protected. This means that instead of tricking TeX to \noexpand your macro, you will define the macro as robust with:
\protected\def\b{%
\@ifnextchar[%
{\b@opt}{\b@noopt}%
}
and then TeX itself will know that \b is not supposed to be expanded inside an \edef or \write or \expanded, without additional machinery.
LaTeX2ε doesn't use \protected to define robust macros because of backwards compatibility. LaTeX2ε predates ε-TeX, so the protection mechanism was established much earlier. LaTeX3, for example, dropped the 2ε protection mechanism and uses only \protected to define robust macros.
As a side note, I'd change that definition of yours to:
\newcommand\mybold[2][--empty--]{.#2.\textbf{#1}}
and use as:
\mybold[one]{two}
I changed the command to \mybold, as one-letter command names are not generally a good idea. I also changed \bf (which is deprecated for decades now) to \textbf and put the second argument in braces, so that the second argument is two, not just t.
Best Answer
Using
creates a macro
\foowhich has definitionInside a LaTeX 'protected expansion' command (
\protected@edef,\protected@xdef,\protected@write), the definition of\@protected@testoptis altered such that no further expansion takes place, so thatgives
\DeclareRobustCommandworks slightly differently, asgives a definition
where there is a space in the name of that internal macro: it's called '
\foo'. Doing a 'protected expansion' here givesi.e. the 'name with space' is retained. As this method adds
\protect'earlier' than the mechanism used by\newcommand, this is more robust at the cost of an additional cname.Neither of these mechanisms will prevent expansion inside a plain
\edef,\xdefor\write. For that, you need the e-TeX protected mechanism, which is wrapped up in a\newcommand-like way by\newrobustcmdfrom theetoolboxpackage. There, doinggives a definition
This will never expand in an expansion context, as the engine is doing the protection.