% This file is part of HINT
% Copyright 2017-2022 Martin Ruckert, Hochschule Muenchen, Lothstrasse 64, 80336 Muenchen
%
% Permission is hereby granted, free of charge, to any person obtaining a copy
% of this software and associated documentation files (the "Software"), to deal
% in the Software without restriction, including without limitation the rights
% to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
% copies of the Software, and to permit persons to whom the Software is
% furnished to do so, subject to the following conditions:
%
% The above copyright notice and this permission notice shall be
% included in all copies or substantial portions of the Software.
%
% THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
% IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
% FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
% COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
% WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT
% OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
% THE SOFTWARE.
%
% Except as contained in this notice, the name of the copyright holders shall
% not be used in advertising or otherwise to promote the sale, use or other
% dealings in this Software without prior written authorization from the
% copyright holders.

\input cwebmac
\input btxmac.tex
\input hintmac.tex

\hyphenation{HINT-ver-sion}
\hyphenation{HINT-mi-nor-ver-sion}
\hyphenation{HINT-image}
\hyphenation{HINT-global-color}
\hyphenation{HINT-color}
\hyphenation{HINT-end-color}
\hyphenation{HINT-start-link}
\hyphenation{HINT-end-link}
\hyphenation{HINT-global-link-color}
\hyphenation{HINT-link-color}
\hyphenation{HINT-dest}
\hyphenation{HINT-out-line}
\hyphenation{HINT-set-page}
\hyphenation{HINT-stream}
\hyphenation{HINT-set-stream}
\hyphenation{HINT-before}
\hyphenation{HINT-after}

\makeindex
\maketoc

\titletrue

\null

\font\largetitlefont=cmssbx10 scaled\magstep4
\font\Largetitlefont=cmssbx10 at 40pt
\font\hugetitlefont=cmssbx10 at 48pt
\font\smalltitlefontit=cmbxti10 scaled\magstep3
\font\smalltitlefont=cmssbx10 scaled\magstep3

\hbox{}
\vskip 0pt plus 1fill
{
  \baselineskip=1cm\parindent=0pt
  \largetitlefont\raggedright  Hi\TeX\par
  \vskip 10pt plus 0.1fill
  \leftline{\smalltitlefont User Manual} 
  \vskip-3pt
  \vskip 10pt plus 0.5fill
  \hskip 0pt plus 2fill \it  F\"ur Beatriz\hskip 0pt plus 0.5fill\hbox{}
  \vskip 10pt plus 3fill
  \leftline{\smalltitlefont Version 1.1 (Draft)}
  \bigskip
  \raggedright\baselineskip=12pt
  \bf MARTIN RUCKERT \ \it Munich University of Applied Sciences\par
  \bigskip
}
\eject

\titletrue
\begingroup
\figrm
\parindent=0pt

{\raggedright\advance\rightskip 3.5pc 
The author has taken care in the preparation of this document,
but makes no expressed or implied warranty of any kind and assumes no
responsibility for errors or omissions. No liability is assumed for
incidental or consequential damages in connection with or arising out
of the use of the information or programs contained herein.
\bigskip
{\def\:{\discretionary{}{}{}}
Internet page  {\tt http:\://hint.\:userweb.\:mwn.\:de/\:hint/hitex.html}
may contain current information, downloadable software,
and news.}

\vfill
Copyright $\copyright$ 2022 by Martin Ruckert
\smallskip
All rights reserved.
\smallskip
This publication is protected by copyright, and permission must be
obtained prior to any prohibited reproduction, storage in
a~retrieval system, or transmission in any form or by any means, electronic,
mechanical, photocopying, recording, or likewise.
To obtain permission to use material from this work, please submit a written
request to Martin Ruckert,
Hochschule M\"unchen,
Fakult\"at f\"ur Informatik und Mathematik,
Lothstrasse 64,
80335 M\"unchen,
Germany.
\medskip
{\tt ruckert\:@cs.hm.edu}
\medskip
Last commit: Tue Jan 21 19:09:13 2025
\par
}
\eject
\endgroup

\frontmatter
\pageno=3%

\tableofcontent

\mainmatter

%\def\rs{\penalty100\hskip 2pt plus 3pt minus 2pt\penalty100\relax}
%\def\rs{\kern 2pt}
\def\rule#1:#2.{\par{\hangindent32pt\hangafter1\parindent0pt\rightskip 0pt plus 60pt#1{\bf:}\quad%
  \hskip 0pt plus 60pt\penalty-300\hskip 0pt plus -60pt#2{\bf.}\par}}
\def\prim#1.{\par{\hangindent32pt\hangafter1\parindent0pt\rightskip 0pt plus 60pt#1\par}}  
\def\sym#1{$\left<\right.${\it #1\/}$\left.\right>$}
\def\OR{${}\vert{}$}
\def\opt#1{$\left[\right.$#1$\left.\right]$}
\def\ctl#1{{\tt\BS #1}}


\section{Introduction}
When I started the \HINT\ project in 2017,
I tried to keep the project as small as possible to increase the
chances that I would be able to complete it. So one design decision
was to keep things simple---or to quote an aphorism attributed to
Albert Einstein: ``Make things as simple as possible, but not simpler''.
The other imperative was:
keep things out of the viewer if possible because I do not know
how much processing power or battery power is available.

As a consequence, I focused on Donald Knuth' original \TeX,
disregarding all later developments like \eTeX\ or \LaTeX, and I
decided that the \TeX\ interpreter would not need to run in the
viewer.
Of course \TeX's line breaking routine will run in the viewer
and modifications of \TeX's page breaking routine. 
But the decision to keep the \TeX\ interpreter
out of the \HINT\ viewer implies that \HINT\ files do
not contain token lists and that there are
neither output routines nor marks.
To replace them, the \HINT\ file format includes
page templates. I have described the technical
means to specify page templates below and try to explain
the rationale behind it, but \HINT's page templates 
are at the time of this writing a largely untested area.

By now, the state of the \HINT\ project is far beyond of what
I had expected then, and the processing power of even low-cost
mobile devices is far better than expected especially when programming the
graphics card directly using OpenEGL.

The following sections will describe all the primitive control sequences
that are special for Hi\TeX. I tried to be as close to similar primitives
that have proven to be useful in other engines, notably pdf\TeX, to make
it easy for package writers to support the Hi\TeX\ engine.

While currently Hi\TeX\ is the only \TeX\ engine that supports output in the \HINT\ file
format, this might not be so forever. To avoid unnecessary complications for 
package writers, it is strongly suggested that all such \TeX\ engines implement
the same primitives according to the same specification. The following is the first
draft of this specification. All the primitives use {\tt HINT} as a prefix to
avoid name conflicts. The prefix {\tt HINT}, as opposed to e.g. {\tt HiTeX},
was chosen to stress the idea that these primitives are specific for the
output format---not for the \TeX\ engine. 

It is common practice in other \TeX\ engines to support the \ctl{special} 
primitive to insert raw code snippets in the output. Using this primitive,
it is possible to insert PostScript code into a PS file, or PDF code in a 
PDF output file. It is currently not planed to support this mechanism for
\HINT\ output files for two reasons: 
First, the development of Hi\TeX\ is closely related to the development of
the \HINT\ file format and therefore features that are part of the \HINT\
file format will enjoy support in Hi\TeX\ by corresponding primitives.
Everything that is not available through primitives in Hi\TeX\ should
be considered ``internal'' and might change in the future.
Second, Hi\TeX\ is not considered a replacement for but 
a supplement to other engines. If your aim is the production of a printed
book, your will target one of the engines that produce PDF output.
But if, on occasion, you want to read what you wrote on a computer screen,
you might just use Hi\TeX\ to process your source file. At this point you
do not want to write \ctl{special} commands for the new target; you want
Hi\TeX\ as a plug-in replacement for your main target engine, even if it
is not completely faithful to your final printed book.  

\section{Hi\TeX\ primitives}

Because this is the first specification that will reach a wider user base,
it is reasonable to expect changes to occur in the future. Therefore it is
recommended that these primitives should not be used directly in a
\TeX\ document; instead they should be encapsulated in suitable
macros so that the consequences of any change to the primitives will 
be local to these macros.

\subsection{Syntax Description}
In the following, we describe the syntax of primitive control sequences which were
added to \TeX.

\itemize
\item We use a {\tt typewriter font}\index{typewriter font}
for text that occurs \index{verbatim}verbatim in the \TeX\ document.
\item We use \sym{italics} enclosed in pointed brackets to denote symbols\index{symbol+\sym{symbol}}.
\item We use rules\index{rule} to define the meaning of symbols.
A rule starts with the symbol
to be explained, followed by a colon ``{\bf :}'', and then the text that this symbol
stands for. A rule ends with a period ``{\bf .}''.
\item Optional\index{optional+\opt{optional}} parts of the rule's text
are enclosed in \opt{square brackets}.
\item Alternatives\index{alternative} are separated by a vertical bar ``\OR''\index{\OR}.
\item Some symbols refer to text that is defined as part of standard \TeX. These are explained here by an example:

\medskip
\rule\sym{integer}: \index{integer+\sym{integer}}
  an integer as in  \ctl{penalty}\sym{integer}.
\rule\sym{number}: \index{number+\sym{number}}
  a general number as in  \ctl{kern}\sym{number}\.{pt}.
\rule\sym{normal dimension}:\index{normal dimension+\sym{normal dimension}}
  a dimension as in \ctl{hrule} \.{width} \sym{normal dimension}.
\rule\sym{dimension}:\index{dimension+\sym{dimension}}
  a dimension as in \ctl{vskip} \.{0pt} \.{plus} \sym{dimension}.
\rule\sym{name}:\index{dimension+\sym{dimension}}
  a name as in \ctl{input} \sym{name}.
\rule\sym{vertical list}:\index{vertical list+\sym{vertical list}}
  a token list  with matching braces as in
  \ctl{vbox}\.{\{}\sym{vertical list}\.{\}}.
\rule\sym{horizontal list}:\index{horizontal list+\sym{horizontal list}}
  a token list  with matching braces as in
  \ctl{hbox}\.{\{}\sym{horizontal list}\.{\}}.
\rule\sym{general text}:\index{general text+\sym{general text}}
  a token list with matching braces as in
  \ctl{write}\.{\{}\sym{general text}\.{\}}.
\medskip
\enditemize

\subsection{Version and Revision}
The control sequences \ctl{HINTversion}\index{HINTversion+\ctl{HINTversion}}
and \ctl{HINTminorversion}\index{HINTminorversion+\ctl{HINTminorversion}} are
used to determine the major and minor version numbers of the \HINT\ output format
that is generated by Hi\TeX. It can be used as part of the output as 
in \verbatim|\the\HINTversion|.
The most important use, however, is testing whether the current \TeX\ engine
will in fact produce \HINT\ output.
As an example the file {\tt ifhint.tex}\index{ifhint.tex+{\tt ifhint.tex}}
contains the following code:


\verbatim/
\newif\ifhint
\expandafter
  \ifx\csname HINTversion\endcsname\relax
  \hintfalse\else\hinttrue\fi/


\subsection{Images}
The primitive \ctl{HINTimage}\index{HINTimage+\ctl{HINTimage}}
includes an image\index{image} in a document.
The syntax is as follows:

\medskip
\prim\ctl{HINTimage}  \opt{\.{=}} \sym{name}
\opt{\sym{width}} \opt{\sym{height}}. 
\medskip

The optional equal sign can be added to make the code look nicer.
The \sym{name} specifies the image file.
The width specification determines the width of the image. If omitted,
Hi\TeX\ tries to determine the image's width from the image file.
The same holds for the height specification.

\medskip
\rule \sym{width}\index{width+\sym{width}}:\.{width} \sym{normal dimension}.
\rule \sym{height}\index{height+\sym{height}}:\.{height} \sym{normal dimension}.
\medskip

Note that a \sym{normal dimension} that is computed from \ctl{hsize}
or \ctl{vsize} retains this dependency when processed by Hi\TeX.
This allows an image to adapt to the size of the viewing area.
Scaling in the \HINT\ viewer will, however, never change the
aspect ratio of an image. So it may become smaller or larger,
but it will never be distorted.
For this reason, Hi\TeX\ will inspect the image file to determine the
aspect ratio\index{aspect ratio} of the stored image.
The width and height values as given in the \TeX\ file serve
as the maximum values for the actual width and height. When rendering,
the image will become as large as possible within the given bounds.
If \TeX\ does not specify neither width nor height, the image file
must specify the absolute width and height of the image.
It is considered an error if valid settings for the image's width and height
can not be obtained.

\subsection{Colors}
Since the \HINT\ file format is designed for on-screen viewing,
the only color model supported is the RGBA model, where a color is
specified by four values: the red, the green, the blue, and the alpha
value. The first three determine the light intensity of the red,
green, and blue component of a pixel; the alpha value determines
the relative share of a color when displaying
one color on top of another color.
Because in practice most display devices use one byte for
each of the four values that define a color, the \HINT file
format stores the four color
components using integer values in the range 0 to 255.
Independent of the input format, Hi\TeX\ will convert all
colors to this format when storing them in the output file.

\subsubsection{Foreground Color}
The most common color specification is the specification of a
foreground color. (We will consider background colors below.)
A foreground color can be specified using
the following syntax:

\medskip
\rule\sym{foreground}\index{foreground+\sym{foreground}}:
   \.{FG\{} \sym{integer} \sym{integer} \sym{integer} \opt{\sym{integer}} \.{\}}.
\medskip

Note that for convenience, the alpha value
is optional; if no alpha value is given, the value 255 will be used
and the color is completely opaque.

Here are some examples:
\.{FG\{255 0 0\}}, \.{FG\{255 0 0 255\}},
both specify the same plain opaque red;
\.{FG\{0 0 255\}} is plain blue;
\.{FG\{255 255 0 127\}} is  a transparent yellow.
Because each value fits in a single byte, the values are often given in
hexadecimal notation. In \TeX, hexadecimal values are written with
a \.{"}~prefix. The same colors as before are then written
\.{FG\{"FF 0 0\}}, \.{FG\{"FF 0 0 "FF\}},
\.{FG\{0 0 "FF\}} and \.{FG\{"FF "FF 0 "7F\}}. 
Values greater than 255 or less than 0 are not allowed.

A common alternative to the color representation just described
is the device independent notation where each
value is a real number in the interval from 0 to 1.
To keep both representations apart, the device independent
representation (with the smaller numbers) uses the lowercase
keyword \.{fg} instead of \.{FG}. Here is the syntax:

\medskip
\rule\sym{foreground}\index{foreground+\sym{foreground}}:
\.{fg\{} \sym{number} \sym{number} \sym{number} \opt{\sym{number}} \.{\}}.
\medskip

Using the new syntax, the colors above are written
\.{fg\{1 0 0\}}, \.{fg\{1 0 0 1\}},
\.{fg\{0 0 1\}} and \.{fg\{1 1 0 0.5\}}.
Values greater than 1 and less than 0 are not allowed.
Note that \.{fg\{1 1 1\}} is pure white while  \.{FG\{1 1 1\}} is
the darkest possible gray, which on most devices is indistinguishable
from pure black.

When specifying colors for computer screen, using red, green, and blue
components is natural. For printing on paper, the specification using
cyan, magenta, yellow, and black is the default.
Since collections of named colors using the latter format
are common, Hi\TeX\ allows the use of this format by prefixing the numbers with
the keyword \.{cmyk}. Specifying the keyword \.{rgb} is also possible and has the
same effect as giving no keyword. Using the new syntax the transparent yellow
can be written \.{fg\{cmyk 0 0 1 0 0.5\}},  \.{FG\{cmyk 0 0 "FF 0 "7F\}},
 \.{fg\{rgb 1 1 0 0.5\}}, or \.{FG\{rgb "FF "FF 0 "7F\}}.

The additional syntax rules are:

\medskip
\rule\sym{foreground}\index{foreground+\sym{foreground}}:
\.{fg\{ rgb \sym{number} \sym{number} \sym{number} \opt{\sym{number}} \}}.
\rule\sym{foreground}\index{foreground+\sym{foreground}}:
\.{fg\{ cmyk \sym{number} \sym{number} \sym{number} \sym{number} \opt{\sym{number}} \}}.
\rule\sym{foreground}\index{foreground+\sym{foreground}}:
   \.{FG\{ rgb \sym{integer} \sym{integer} \sym{integer} \opt{\sym{integer}} \}}.
\rule\sym{foreground}\index{foreground+\sym{foreground}}:
   \.{FG\{ cmyk \sym{integer} \sym{integer} \sym{integer} \sym{integer} \opt{\sym{integer}} \}}.
\medskip

\subsubsection{Defining and Using Colors}
As we will see, colors come in whole sets of colors.
To define such a set of colors, Hi\TeX\ provides
the primitive \ctl{HINTcolor}. It syntax is
\medskip
\prim\ctl{HINTcolor} \.{\{ \sym{color specification} \}}.
\medskip
Before we give the complete definition of a  \sym{color specification},
we start with some examples.
In its simplest form this primitive just specifies a single color.
For example \ctl{HINTcolor}\.{\{fg\{0 0 0\}\}} specifies
the foreground color black which is then used for rules and glyphs.
In addition to the foreground color, you can specify a background color.
For example, black text on white background is specified by
\ctl{HINTcolor}\.{\{fg\{0 0 0\}} \.{bg\{1 1 1\}\}} or
\ctl{HINTcolor}\.{\{fg\{0 0 0\}} \.{BG\{"FF "FF "FF\}\}}.

The viewer for \HINT\ files may provide a ``dark'' mode, and as a document
author, you can specify the colors also for dark mode.
If you like white letters on dark blue background you can write
\ctl{HINTcolor}\.{\{fg\{0 0 0\}} \.{bg\{1 1 1\}}
\.{dark} \.{fg\{1 1 1\}} \.{bg\{0 0 0.3\}\}}.

There are two more colors that an author might care about: When searching
for a text, all occurrences of the search phrase are highlighted by
using a different color. And while the user iterates over the occurrences
on the page, one occurrence has the ``focus'' and is rendered again in a
different color. You can specify the highlight color right after
the normal text color and the focus color right after the highlight color.
The same can be done for the colors in ``dark'' mode.

Here are the remaining rules that complete the \sym{color specification}:

\medskip
\rule\sym{color specification}\index{color specification+\sym{color specification}}: \sym{color set} \opt{\.{dark} \sym{color set}}.
\rule\sym{color set}\index{color set+\sym{color set}}:
    \sym{color} \opt{\sym{color} \opt{\sym{color}}}.
\rule\sym{color}\index{color+\sym{color}}:
    \sym{foreground} \opt{\sym{background}}.
\rule\sym{background}\index{foreground+\sym{foreground}}:
   \.{FG\{ \opt{\.{rgb}} \sym{integer} \sym{integer} \sym{integer} \opt{\sym{integer}} \}}.
\rule\sym{background}\index{foreground+\sym{foreground}}:
   \.{fg\{ \opt{\.{rgb}} \sym{number} \sym{number} \sym{number} \opt{\sym{number}} \}}.
\rule\sym{background}\index{foreground+\sym{foreground}}:
   \.{FG\{ cmyk \sym{integer} \sym{integer} \sym{integer} \sym{integer} \opt{\sym{integer}} \}}.
\rule\sym{background}\index{foreground+\sym{foreground}}:
  \.{fg\{ cmyk \sym{number} \sym{number} \sym{number} \sym{number} \opt{\sym{number}} \}}.
\medskip
If some of the optional parts in the \sym{color specification} are missing,
the corresponding colors from the set of default colors, as described below,
are used.

Note that the background colors for highlighted text and focus text
can be given, but current viewers ignore these background specifications.
Further note that the current specification of the \HINT\ file format
limits the total number of different color specifications in a document to 255.


The colors given in \ctl{HINTcolor} will have an immediate effect
on all following rules and glyphs and the background of the enclosing box.
The effect will persist until the next change of colors or until
the end of the box---whatever occurs first.

The line breaking algorithm of Hi\TeX\ 
tracks changes in color within a paragraph and reinsert an appropriate
color change at the start of every \ctl{hbox} that contains a new line.
In this way local color changes inside a paragraph can span multiple lines
but do not affect the inter line glue or material that is inserted with
\ctl{vadjust}. Similarly, spliting off the initial part of a vertical
box with \ctl{vsplit} will insert a color node in the remaining part
if necessary to keep the color consistent accross the split.

Special care is needed if background colors are used.
Unless the background color is completely transparent
with an alpha value equal to zero,
the background color will fill a vertical box from left to right
and a horizontal box from top to bottom. Since height, depth, and width of
boxes often depend on the text that is inside, which in turn
might depend on the outcome of line breaking, it is strongly recommended
to use background colors with caution, and use \ctl{strut}s to enforce
a fixed height and depth of horizontal boxes.


\subsubsection{Default Colors}
The \HINT\ file format specifies default values for all colors.
Hi\TeX\ provides the primitive \ctl{HINTdefaultcolor} to overwrite these
default colors. This primitive must not be used after defining any custom
colors using \ctl{HINTcolor}. Its syntax is
\medskip
\prim\ctl{HINTdefaultcolor} \.{\{} \sym{color specification} \.{\}}.
\medskip
The \HINT\ format specifies the following default colors:
Normal text is black \.{FG\{0 0 0\}},
highlight text is a slightly dark red  \.{FG\{"EE 0 0\}},
and focus text is slighty dark green \.{FG\{0 "EE 0\}}.
The background is transparent white \.{BG\{"FF "FF "FF 0\}}.
In dark mode the background is transparent black \.{BG\{0 0 0 0\}},
normal text is white \.{FG\{"FF "FF "FF\}},
and a slightly lighter red \.{FG\{"FF "11 "11\}},
and green \.{FG\{"11 "FF "11\}}, are used for highlighted and focus text.

\subsubsection{Nesting Colors}
A color change is limited to the enclosing box. Hence the
nesting of boxes leads to a nesting of color definitions.
So for example a transparent background color in the inner box
will not completely replace the background color of the enclosing
box but will only modify this color like seeing it through colored glas.

A color change ends not only at the end of the enclosing box,
it will also end at the next use of the \ctl{HINTcolor}
or \ctl{HINTendcolor} primitive:
The \ctl{HINTcolor} primitive will replace the current colors by
a new set of colors; the \ctl{HINTendcolor} primitive will resume
the color specification that was valid just before the matching use
of \ctl{HINTcolor}. Hi\TeX\ maintains a color stack tracking
local color changes within a box or paragraph, and uses it to
insert appropriate color changes so that the \ctl{HINTendcolor} primitive
will simply cancel the color change by the matching \ctl{HINTcolor} primitive.
If there is no matching \ctl{HINTcolor} primitive,
the \ctl{HINTendcolor} primitive is silently ignored.
Note that within a single box, there is at any point only a single
background color: The color stack will switch from one background
color to an other background color but will not overlay an ``inner''
background color over an ``outer'' background color.
This is only the case when multiple boxes are nested as described above.

Here is an example:
Suppose we want the \TeX\ logo to be rendered in light red,
and notes in dark green. You can write
\medskip
{\tt\parindent 0pt\rightskip=0pt plus 160pt
\verbatim/\def\redTeX{\HINTcolor{fg{1 0.3 0.3}}\TeX\HINTendcolor}
\def\beginnote{\HINTcolor{fg{0 0.5 0}}}% dark green
\def\endnote{\HINTendcolor}/
\medskip
This is an example showing the \ctl{redTeX}\BS\ logo in red color.
\ctl{beginnote}\ Note how the \ctl{redTeX}\BS\ logo is still red inside
this note.\ctl{endnote}\par}
\medskip

After the first occurrence of the red \TeX\ logo, the color will be switched
back to normal black, while after the second occurrence the color will
be switched back to dark green. The color switching will work as intended
even if the paragraph is spread over several lines by the line breaking routine.

\subsubsection{Colors for Pages}
When a page get rendered in the \HINT\ file viewer,
the renderer starts with the default colors and the page is initially
cleared using the default background color. If a different page
color is desired, color changes can be added to the page templates.

In a vertical box, the color stack of Hi\TeX\ has a similar effect as in
a horizontal box. Similar to the precautions in the line breaking routine,
Hi\TeX\ will insert color changes when splitting a vertical box with \ctl{vsplit}.
Complications arise from color changes in the top level vertical list
which is split into pages in the \HINT\ viewer at runtime.
Because the page builder in the viewer has no global information and
should not need global information, Hi\TeX\ will insert copies of the
local color information after every possible breakpoint in the top
level vertical list. This will ensure that page breaks will not
affect the colors of the displayed material.
Note, however, that \TeX\ considers glue (and kerns) as discardable
and will remove these items from the top of a new page. Because glues and kerns
are colored using the current background color, these items might be visible
on a page but disappear when they follow immediately after a page break.
So if you want the effect of a colored glue or kern that is not affected by
a page break, you should include it inside a box or use a colored rule instead.

\subsubsection{Colors for Links}
The most common change in color is caused by the use of links.
To support this changing of colors, the primitives
\ctl{HINTstartlink}\index{HINTstartlink+\ctl{HINTstartlink}}
and \ctl{HINTendlink}\index{HINTendlink+\ctl{HINTendlink}}
(see section~\secref{llo})
cause an automatic change of the color specification.
A document author can set the default colors used for links
with the  primitive \ctl{HINTdefaultlinkcolor} and change
the current link color with the primitive \ctl{HINTlinkcolor}.
The syntax is:
\medskip
\prim\ctl{HINTdefaultlinkcolor} \.{\{} \sym{color specification} \.{\}}.
\prim\ctl{HINTlinkcolor} \.{\{} \sym{color specification} \.{\}}.
\medskip

For convenience, the \HINT\ file format specifies default colors
for links as well: links use dark blue \.{FG\{0 0 "EE\}} and in dark mode
light blue \.{FG\{"11 "11 "FF\}}.
The primitive \ctl{HINTdefaultlinkcolor} is used
to partly or completely redefine these defaults.

Later uses of \ctl{HINTlinkcolor} will set new current link colors.
Colors that are missing in the new link color specification are taken
from the corresponding default colors for links.

Whenever the \ctl{HINTstartlink} primitive is used, its effect on the
colors is equivalent to the \ctl{HINTcolor} primitive using the current
link color. This implies that the color change caused by \ctl{HINTstartlink}
is local to the enclosing box.

Whenever the \ctl{HINTendlink} primitive is used, it will restore
the color stack of Hi\TeX\ to its state before the matching \ctl{HINTstartlink}.
It is the responsibility of the \TeX\ source code (or package) to keep the
sequence of  \ctl{HINTstartlink}, \ctl{HINTendlink},
\ctl{HINTcolor}, and \ctl{HINTendcolor} properly nested.
A sequence like  ``\ctl{HINTstartlink} \dots\ \ctl{HINTcolor}
 \dots\ \ctl{HINTendlink} \dots\ \ctl{HINTendcolor}'' is possible,
but it will cause \ctl{HINTendlink} to restore the colors to those
in effect before the \ctl{HINTstartlink}.
The following \ctl{HINTendcolor} will then either restore
a color of a matching \ctl{HINTcolor} preceeding
the link in the same box or it will restore the
color in the outer box, or it will be ignored.
In effect, the color changes inside a link stay local to the link.

\subsubsection{\LaTeX\ Support}
Starting with \TeX\ Live 2025, there is a limited support for the \.{xcolor}
package.

After \ctl{usepackage}\.{\{xcolor\}} you can use the predefined standard colors;
for example \ctl{color}\.{\{red\}}.
If you add one (or several) of the named color
options \.{x11names}, \.{svgnames}, or \.{dvipsnames} to the package,
you can also use commands like
\ctl{color}\.{\{Tomato4\}} (x11),
\ctl{color}\.{\{BlanchedAlmond\}} (svg), or
\ctl{color}\.{\{Plum\}} (dvips).


To define your own colors you can use for example

\ctl{definecolor}\.{\{mypink1\}\{rgb\}\{0.858, 0.188, 0.478\}},

\ctl{definecolor}\.{\{mypink2\}\{RGB\}\{219, "30, 122\}},

\ctl{definecolor}\.{\{mypink3\}\{cmyk\}\{0, 0.7808, 0.4429, 0.1412\}}, or

\ctl{definecolor}\.{\{mygray\}\{gray\}\{0.2\}}.

The mixing of colors is supported as well. For example a
mixture of 40\% green and 60\% yellow look is produced by
\ctl{color}\.{\{green!40!yellow\}}.

The colors for links and other references can be given as options
to the \.{hyperref} package. For example as in

\ctl{usepackage}\.{[linkcolor=green,urlcolor=red]\{hyperref\}}

\subsubsection{Differences between \LaTeX\ and Hi\LaTeX}

\noindent{\bf Colors and Groups}\par\smallskip\noindent
In \LaTeX, colors are local to the group. So by writing
``\.{text 1} \.{\{}\ctl{color}\.{\{blue\}} \.{text 2} \.{\}} \.{text 3}''
after \.{text~1} the color of \.{text~2} will change to  blue and after \.{\}}
marking the end of the group, the color of \.{text~3} will revert to the
color of \.{text~1}. Hi\LaTeX\ emulates this behaviour by inserting
\ctl{HINTendcolor} at the end of the group.

When it comes to paragraphs,
the scoping rules of colors in Hi\LaTeX\ are however slightly
different from the \LaTeX\ scoping rules.
In \TeX\ and \LaTeX, boxes and references all have there own group,
but this is not true for paragraphs.
So \TeX\ or \LaTeX\ will allow you to start a new group in one paragraph and
end the group in the next paragraph, while it is not possible to start a
group in one box and end the group in another box. As a consequence,
you can switch to blue text color in the middle of a paragraph and end the blue
color in the middle of the next paragraph.
In Hi\TeX, on the other hand, when it comes to colors,
paragraphs behave pretty much like boxes: The effect of a color change inside
a paragraph will not extend past the end of the paragraph. The closing of the
group in the next paragraph will then have no effect.

\bigskip\noindent{\bf Colors in vertical Lists}\par\smallskip\noindent
The \HINT\ file format allows color specifications in horizontal boxes
and---unlike the \PDF\ file format---in vertical boxes as well.
Together with the mode switching of \TeX, which goes into horizontal mode
when it sees the beginning of a paragraph and back into vertical mode at
the end of the paragraph, this can cause unexpected color effects.

There is for example a big difference between

\medskip
\verbatim/\color{blue}
The first paragraph ...

The second paragraph ...
/
\medskip
and
\medskip
\verbatim/\indent
\color{blue}
The first paragraph ...

The second paragraph ...
/
\medskip

In the first case, the color change is part of the vertical list
and the letter ``\.T'' starts the paragraph. 
As a consequence, the color change is still in effect when the second
paragraph starts.
In the second case, the \ctl{indent} command puts \TeX\ into horizontal mode
and the color change becomes part of the first paragraph.
As a consequence, the color change will end with the first paragraph, as explained
in the previous section.

Even more surprising is this:
\medskip
\verbatim/{\color{blue} Blue} The first paragraph ...

The second paragraph ...
/
\medskip


\TeX\ finds the begining of the group \.{\{} and the color change in vertical mode
and it puts the color change into the vertical list. Then it finds the letter
``\.B'' and starts the paragraph. When \TeX\ encounters the end of the group,
there is no local color change {\it inside} the paragraph and the text
continues to be blue. Even the second paragraph and all following paragraphs
will continue in blue until the end of the vertical list.

The confusion that such behaviour might create has its root in \TeX's
mode switching which is not synchronized with \TeX's grouping.
While grouping is typically visible in the source text, the mode
switching remains largely invisible. 


\bigskip\noindent{\bf Future Changes}\par\smallskip\noindent
While it may be questionable whether all the color changes shown above
makes sense, it is definitely undesirable if Hi\LaTeX\ and \LaTeX\
behave differently.
As a consequence,  Hi\LaTeX\ might very well change in this respect in a
later version, so that  Hi\TeX\ will no longer treat the begining
and ending of paragraphs as the beginning and ending of a group.
It is an open question how Hi\TeX\ should handle the end of a group
in the middle of a paragraph ending a color change that started in
the enclosing vertical list. Currently a \ctl{HINTendcolor} at that
position would be silently ignored because it can only undo local
changes {\it inside} the paragraph. Should Hi\TeX\ instead change the color
of the enclosing vertical box immediately? What does
it mean to do this change immediately? At the baseline?
Before the next interline glue? What are the implications for the
rendering engine? How complicated can it be to look ahead
for color changes that occur depply nested inside a vertical list?
Would it not be better to demand the use of \ctl{vadjust} for such an effect?
Should Hi\TeX\ postpone the color change in the enclosing vertical
box until the end of the paragraph?

\bigskip\noindent{\bf Default Colors}\par\smallskip\noindent
Because complete color specifications are pretty long. It is important
to provide usefull defaults. Currently missing elements of a color specification
are taken from a single default color specification. It might be convenient
to be able to provide a way to define color specifications using the
current color as a basis for missing elements.

\bigskip\noindent{\bf Color Numbers}\par\smallskip\noindent
The \HINT\ file format references a color set by a number in the range
0 to 254.
So Hi\TeX\ assigns each color specification a number, using the same
number for two identical color specifications.
One extension to the above specification of Hi\TeX's color primitives
could be to make these numbers accessible to document authors or
package programmers. For example \ctl{the}\ctl{HINTcolor} could
expand to the number $n$ of the current color set and
\ctl{HINTcolor}$n$ would be equivalent to a use of \ctl{HINTcolor}
with a full color specification that is equivalent to the color specification
belonging to $n$.
This would be quite efficient; it would not be necessary
to scan the color specification and search the existing color specifications
for the matching specification with number $n$.

The \LaTeX\ named colors are stored as macros, which has the advantage that
loading a whole package of color names does not use any of Hi\TeX's
color numbers. Only colors that actually get used (probably only a few)
will get a color number. This works well in practice.
So currently, there are no plans to implement this extension.

\subsection{Links, Labels, and Outlines}\label{llo}
A link\index{link} in a \HINT\ document refers to another location in the same document.
It can be used to navigate to that location.
A link is defined using the primitives 
\ctl{HINTstartlink}\index{HINTstartlink+\ctl{HINTstartlink}}
and \ctl{HINTendlink}\index{HINTendlink+\ctl{HINTendlink}}.
Neither of them can be used in vertical mode.
The text between the start and the end of the link
constitutes the visible part of the link. Depending on the user interface, clicking
or tapping or otherwise activating the link (e.g. pronouncing)
will navigate to the destination of the link.
The user interface might provide a visual clue to make the user aware of the
available links for example using a special cursor when hovering over a link.
But it also may choose to leave the visual clues completely to the author
of the document (e.g. using a special colors, images, or fonts).

The syntax is 
\ctl{HINTstartlink}  \sym{destination}
and
\ctl{HINTendlink}
with

\medskip
\rule \sym{destination}\index{destination+\sym{destination}}:\.{goto} \sym{label}.
\rule \sym{label}\index{label+\sym{label}}:
  \.{name} \.{\{}\sym{general text}\.{\}} \OR\ \.{num} \sym{integer}.
\medskip

As you can see, the link refers to its destination using a label
which is either a name or a number.
The destination can be defined by using the 
\ctl{HINTdest}\index{HINTdest+\ctl{HINTdest}} primitive.
Forward and backward links are allowed; the definition of a label can either
precede or follow the use of the label. If at the end of the document a label
is undefined, a warning is given, and the label will reference the beginning of the
document.

The syntax is
\medskip
\prim\ctl{HINTdest} \sym{label} \opt{\sym{placement}}.
\medskip
with
\medskip
\rule\sym{placement}\index{placement+\sym{placement}}:
\.{top}\index{top+{\tt top}} \OR\ \.{bot}\index{bot+{\tt bot}}.
\medskip

The optional placement argument specifies how to build the page 
containing the destination location. \.{top} demands
a page starting with the destination location. This is useful
if the destination is for example the start of a section or chapter heading.
Similarly \.{bot} asks for a page that ends with the destination location.
The most common case is to omit the placement argument. In this case, the
viewer will build a ``good'' page that includes the given destination.
In case of a section heading, for example, it will most probably start the
page with the section heading because section headings are usually preceded
by a negative penalty that will convince the page builder that this is a good
place to break the page. But if the section heading is immediately preceded
by a chapter heading, the negative penalty found there will probably
persuade the page builder to start with the chapter heading instead.

There is a special label that has the form
\.{name} \.{\{}\.{HINT.home}\.{\}}\index{HINT.home+{\tt HINT.home}}.
It is used to mark the ``home page''\index{home page} of the document. User interfaces
are encouraged to offer a button or keyboard shortcut to navigate to the
document location labeled this way. The page should be a convenient
starting point to explore the document. The typical place for this label
would be the documents table of content.

The labels that identify destinations in a document can also be used
to define document outlines. A document outline\index{outline} is a document summary
given as a hierarchical list of headings where each of them
refers to a specific location in the document.

The syntax is
\medskip
\prim\ctl{HINToutline}\index{HINToutline+\ctl{HINToutline}}
\sym{destination} \opt{\sym{depth}} \.{\{}\sym{horizontal list}\.{\}}.
\medskip
\rule \sym{depth}\index{depth+\sym{depth}}: \.{depth} \sym{integer}.
\medskip

The user interface can format the \sym{horizontal list} much like 
a \ctl{hbox} would do and display it to the user. When the user selects
this text, the document will be repositioned to show the destination location
in the same way as with a link. In order to support also simpler
user interfaces, the current \HINT\ backend also extracts the characters
(and spaces) from the horizontal list (in top-left to bottom-right order)
and makes this character string available to the user interface.


The order in which outline items are defined is important because
this is the order in which they will be presented to the reader of the
document. The optional depth argument allows to structure 
the list of outline items as a hierarchy. Outline items with a higher depth value are considered to be sub-items of items earlier in the list with lower
depth values. If no depth value is given, the depth value is set to zero.
It is not necessary to define depth values strictly consecutive.


\subsection{Page Templates and Streams}\index{page template}\index{stream}

To produce the final page, \TeX\ uses a special piece of program
called the output routine\index{output routine}.  Because a \HINT\
file is pure data, it can not contain output routines.  Instead it
uses page templates to assemble pages from the main text, footnotes,
floating illustrations, and other material.  I start here by
describing how \HINT's page templates work and the special syntax used
to specify them in a \TeX\ file that is about to be processed with
Hi\TeX.  For those interested in how the design decision was made and
how page templates relate to \TeX's page building mechanism, a
separate section follows at the end.

The syntax of a page template specification is:
\medskip
\prim\ctl{HINTsetpage}\index{HINTsetpage+\ctl{HINTsetpage}}
\sym{integer} \opt{\.{=}} \sym{name} 
\opt{\sym{priority}} \opt{\sym{width}} \opt{\sym{height}}
\.{\{}\sym{vertical list} \sym{stream definition list}\.{\}}.
\medskip

The \sym{integer} specifies the page templates number in the range 1
to 255.  The number 0 is reserved for the build in page template of
the \HINT\ file format, which is used if no other page template has
been defined. The page template 0 can not be redefined.
The \sym{name} associates a name with the page template.  The name can
be displayed by the \HINT\ viewer to help the user selecting a
suitable page template.

After the name follows an optional priority; it is used to select the
``best page template'' if multiple templates are available. The
default value is~1. The build-in template has priority~0.

\medskip
\rule\sym{priority}\index{priority+\sym{priority}}: {\tt priority} \sym{integer}.
\medskip


After that follows an optional width and height of the full page
including the margins.  After subtracting \ctl{hsize} from the width
and \ctl{vsize} from the height, the remainder is used for the margins
around the displayed text.  For example giving the width as
1.2\ctl{hsize} will leave 0.1\ctl{hsize} for the margins on both sides.
In this case the margins will grow together with the available width
of the display.  If the width is computed by adding 20pt
to \ctl{hsize}, the margin will be 10pt on both sides.  In this case
the margin will not grow with the size of the display, but it will
grow with the magnification factor.  Of course both methods can be
used together.  The default is \ctl{hsize} for the width and \ctl{vsize}
for the height so there are no margins.

The following \sym{vertical list} defines the page itself. It should assign suitable values
to \ctl{topskip} and \ctl{maxdepth} because the values valid at the end of the vertical list
are stored in the page template and are used in the page building process. 
The vertical list usually also specifies the insertion of content streams
using a \sym{stream insert point}.

\medskip
\rule\sym{stream insert point}\index{stream insert point+\sym{stream insert point}}:
  \ctl{HINTstream} \sym{integer}.
\medskip

Here  the \sym{integer} must be in the range 0 to 254. The value 255 is invalid;
the value 0 indicates the main body of text (what \TeX's page builder would normally put into
box 255 before calling the output routine).
Otherwise, the \sym{integer} is \TeX's insertion number, that is the number of \TeX's box 
containing the insertions. As usual, this box is filled using \TeX's \ctl{insert} primitive. 
So after plain \TeX\ has defined \ctl{footins},
the footnotes for the current page can be inserted after the main body of text in the \sym{vertical list}
by saying \ctl{HINTstream}\.0 followed by \ctl{HINTstream}\ctl{footins}.
Of course you might want to have a footnote rule and a small skip to separate the 
footnotes ---if there are any---from the main text. This can be achieved by a suitable
\sym{stream definition} in the \sym{stream definition list}.

\medskip
\rule\sym{stream definition list}\index{stream definition list+\sym{stream definition list}}:
  \OR\ \sym{stream definition list} \sym{stream definition}.
\rule\sym{stream definition}\index{stream definition+\sym{stream definition}}:
  \ctl{HINTsetstream} \sym{integer}  \opt{\.{=}}
\opt{{\tt preferred} \sym{integer}}
\opt{\.{next} \sym{integer}}
\opt{\.{ratio} \sym{integer}} \.{\{}\sym{vertical list}\.{\}}.
\medskip


The first \sym{integer} is the streams insertion number $i$,
and it must match the \sym{integer} 
previously used in the \sym{stream insert point}.
Then follows the optional specification of a preferred stream with insertion number $p$,
a next stream with insertion number $n$, and a split ratio $r$.
If $r>0$, the contributions to stream $i$ are split between
stream $p$ and $n$  in the ratio $r/1000$ for $p$ and $1-r/1000$ for $n$
before contributing streams $p$ and $r$ to the page.
Else if $p\ge0$ any insertion to stream $i$ is moved to stream $p$ as long as possible,
and if $n\ge0$ we move an insert to stream $n$ if there is ``no room left'' in $p$ nor in $i$.
How much ``room'' is available for the insertions is specified inside the vertical list
that follows.
Here \ctl{dimen}$i$ should be set to the maximum total height of the insertions in class $i$ per page. 
\ctl{count}$i$ should be set to the magnification factor $f$,
such that inserting a box of height $h$ will contribute $h*f/1000$ to the main page;
and \ctl{skip}$i$ should be set to the extra space needed if an insertion in class $i$ is present.

This extra space is usually taken up by material that is inserted before and after the insertions,
such as for example the footnote rule. This material can be defined by a 
\sym{before list} and an \sym{after list}.

\medskip\index{HINTbefore+\ctl{HINTbefore}}\index{HINTafter+\ctl{HINTafter}}
\rule \sym{before list}\index{before list+\sym{before list}}:
  \ctl{HINTbefore} \opt{\.{=}} \.{\{}\sym{vertical list}\.{\}}.
\rule \sym{after list}\index{after list+\sym{after list}}:
  \ctl{HINTafter} \opt{\.{=}} \.{\{}\sym{vertical list}\.{\}}.
\medskip

If you are interested in the design decision that motivate the definitions that have
be given in this section, you should read section~\secref{build}.

\section{Other Primitives}

Since I consider the support for \LaTeX\ to be crucial for
the success of the \HINT\ project, quite a few primitives
have been added to Hi\TeX\ that go beyond \TeX's original
specification. 

\subsection{\eTeX}
First, the primitives of \eTeX\ have been
added with the exception of those primitives that deal with
line breaking, with right to left reading, and with marks. 
Here is a list of \eTeX\ primitives that are missing in Hi\TeX:
\itemize
\item\ctl{TeXXeTstate} (current reading direction)
\item\ctl{beginL}, \ctl{endL}  (switching reading direction)
\item\ctl{beginR}, \ctl{endR} (switching reading direction)
\item\ctl{predisplaydirection}  (reading direction)
\item\ctl{lastlinefit}  (line breaking)
\item\ctl{marks}  (multiple marks)
\item\ctl{botmarks}, \ctl{splitbotmarks}   (multiple marks)
\item\ctl{firstmarks}, \ctl{splitfirstmarks}  (multiple marks)
\item\ctl{topmarks}  (multiple marks)
\enditemize 

\subsection{\LaTeX\ and \Prote}
Second, the primitives required to support
\LaTeX\ were added using Thierry Larondes implementation of \Prote.

\itemize
\item\ctl{Proteversion}, \ctl{Proterevision} (version information)
\item\ctl{resettimer}, \ctl{elapsedtime} (timing information)
\item\ctl{creationdate}, \ctl{filemoddate}, \ctl{filesize},
     \ctl{filedump}, \ctl{mdfivesum} (file information)
\item\ctl{shellescape} (Currently only a dummy implementation.)
\item\ctl{setrandomseed}, \ctl{randomseed},
     \ctl{normaldeviate}, \ctl{uniformdeviate} (random numbers)
\item\ctl{expanddepth}, \ctl{expanded} (programming)
\item\ctl{ifincsname}, \ctl{ifprimitive} \ctl{primitive} (programming)
\item\ctl{savepos}, \ctl{lastxpos}, \ctl{lastypos}, \ctl{pageheight},
     \ctl{pagewidth} (Only dummy implementations since this information
      is not available to Hi\TeX\ at runtime.)
\item\ctl{strcmp} (comparing strings)
\enditemize 


\subsection{{\tt kpathsearch} and \ctl{input}}
In Don Knuth's implementation of \TeX, the \ctl{input} primitive
will add the extension {\tt .tex} to any filename that does not have an
extension. This implies that a file without extension cannot be opened
as an input file. The usual engines do not add such an extension but
pass the filename as given to \verbatim/kpse_find_file/ function. 
Hi\TeX\ does the same. The {\tt kpathsearch} library will find files
in a variety of directories and yes, it will also find files without
extension. Using this library, or equivalent functionality, is just about
mandatory for any engine that wants to process \LaTeX\ input.


\section{Replacing \TeX's Page Builder}\label{build}

\TeX\ uses an output\index{output routine} routine to finalize the page. 
The output outline takes the material which the page builder had accumulated in {\tt box255}
and attaches headers, footers, and floating material
like figures, tables, and footnotes. The latter material is specified by insert nodes
while headers and footers are often constructed using mark nodes.
Running an output routine requires the full power of the \TeX\ engine and is not 
part of the \HINT\ viewer. Therefore, \HINT\ replaces output routines by page templates\index{template}.
\TeX\ can use different output routines for different parts of a book---for example
the index might use a different output routine than the main body of text.

\TeX\ uses insertions to describe floating content that is not necessarily displayed 
where it is specified. Three examples may illustrate this:
\itemize
\item Footnotes\footnote*{Like this one.}  are specified in the middle of the text but are displayed at the
bottom of the page.  Several
footnotes\index{footnote} on the same page are collected and displayed together. The
page layout may specify a short rule to separate footnotes from the
main text, and if there are many short footnotes, it may use two columns
to display them.  In extreme cases, the page layout may demand a long
footnote to be split and continued on the next page.

\item Illustrations\index{illustration} may be displayed exactly where specified if there is enough
room on the page, but may move to the top of the page, the bottom of the page,
the top of next page, or a separate page at the end of the chapter.

\item Margin notes\index{margin note} are displayed in the margin on the same page starting at the top
of the margin.
\enditemize

\HINT\ uses page templates and content streams to achieve similar effects.
But before I describe the page building\index{page building} mechanisms of \HINT,
let me summarize \TeX's page builder.

\subsection{\TeX's page building mechanism}
\TeX's page builder ignores leading glue\index{glue}, kern\index{kern},
and penalty\index{penalty} nodes until the first
box\index{box node} or rule\index{rule node} node is encountered;
whatsit\index{whatsit node} nodes do not really contribute 
anything\footnote*{This changes when images are implemented as whatsit nodes.} to
a page; mark\index{mark node} nodes are recorded for later use.  Once
the first box, rule, or insert\index{insert node} arrives, \TeX\ makes
copies of all parameters that influence the page building process and
uses these copies. These parameters are the \.{page\_goal} and the
\.{page\_max\_depth}. Further, the variables {\tt page\_total}, {\tt page\_shrink},
{\tt page\_stretch}, {\tt page\_depth}, and {\tt in\-sert\_pe\-nal\-ties} are
initialized to zero.  The top skip\index{top skip} adjustment is made
when the first box or rule arrives---possibly after an insert.
Now the page builder accumulates material: normal material goes
into {\tt box255}\index{box 255} and will change {\tt page\_total}, {\tt page\_shrink}, 
{\tt page\_stretch}, and {\tt page\_depth}. The latter is adjusted so that 
is does not exceed {\tt page\_max\_depth}.

The handling of inserts\index{insert node} is more complex.
\TeX\ creates an insert class using {\tt newinsert}. This reserves a number $i$
and four registers: {\tt box\hair$i$} for the inserted material,
{\tt count\hair$i$} for the magnification factor $f$, {\tt dimen\hair$i$}
for the maximum size per page $d$, and {\tt skip\hair$i$} for the
extra space needed on a page if there are any insertions of class $i$.

For example plain \TeX\ allocates $n=254$ for footnotes\index{footnote} and sets
{\tt count254} to~$1000$, {\tt dimen254} to 8in, and {\tt skip254} to 
{\tt \BS big\-skip\-amount}.

An insertion node will specify the insertion class $i$, some vertical material,
its natural height plus depth $x$, a {\tt split\-\_top\-\_skip}, a {\tt split\-\_max\_depth},
and a {\tt floa\-ting\-\_pe\-nal\-ty}. 


Now assume that an insert node with subtype 254 arrives at the page builder.
If this is the first such insert, \TeX\ will decrease the {\tt page\_goal}
by the width of {\tt skip254} and adds its stretchability and shrinkability
to the total stretchability and shrinkability of the page. Later,
the output routine will add some space and the footnote rule to fill just that
much space and add just that much shrinkability and stretchability to the page.
Then \TeX\ will normally add the vertical material in the insert node to
{\tt box254} and decrease the {\tt page\_goal} by $x\times f/1000$.

Special processing is required if \TeX\ detects that there is not enough space on
the current page to accommodate the complete insertion.
If already a previous insert did not fit on the page, simply the {\tt floating\_penalty}
as given in the insert node is added to the total {\tt insert\_penalties}.
Otherwise \TeX\ will test that the total natural height plus depth of {\tt box254} 
including $x$ does not exceed the maximum size $d$ and that the 
${\tt page\_total} + {\tt page\_depth} + x\times f/1000 - {\tt page\_shrink} \le {\tt page\_goal}$.
If one of these tests fails, the current insertion
is split in such a way as to make the size of the remaining insertions just pass the tests
just stated.

Whenever a glue node, or penalty node, or a kern node that is followed by glue arrives
at the page builder, it rates the current position as a possible end of the page based on
the shrinkability of the page and the difference between {\tt page\_total} and {\tt page\_goal}.
As the page fills, the page breaks tend to become better and better until the
page starts to get overfull and the page breaks get worse and worse until
they reach the point where they become {\tt awful\_bad}. At that point,
the page builder returns to the best page break found so far and fires up the 
output routine.


\subsection{\HINT\ Page Templates}
Let's look at the problems that show up when implementing a replacement for \TeX's
page building mechanism.

\enumerate
\item 
An insertion node can not always specify its height $x$ because insertions may contain paragraphs that need
to be broken in lines and the height of a paragraph depends in some non obvious way on
its width. 

\item 
Before the viewer can compute the height $x$, it needs to know the width of the insertion. Just imagine
displaying footnotes in two columns or setting notes in the margin. Knowing the width, it
can pack the vertical material and derive its height and depth.

\item
\TeX's plain format provides an insert macro that checks whether there is still space
on the current page, and if so, it creates a contribution to the main text body, otherwise it
creates a {\tt topinsert}. Such a decision needs to be postponed to the \HINT\ viewer.

\item
\HINT\ has no output routines that would specify something like the space and the rule preceding the footnote.

\item 
\TeX's output routines have the ability to inspect the content of the boxes,
split them, and distribute the content over the page.
For example, the output routine for an index set in two column format might
expect a box containing index entries up to a height of $2\times\.{vsize}$.
It will split this box in the middle and display the top part in the left
column and the bottom part in the right column. With this approach, the
last page will show two partly filled columns of about equal size.

\item
\HINT\ has no mark nodes that could be used to create page headers or footers.
Marks, like output routines, contain token lists and need the full \TeX\ interpreter
for processing them. Hence, \HINT\ does not support mark nodes.
\endenumerate


Instead of output routines, \HINT\ uses page templates.
Page templates are basically vertical boxes with \sym{stream insert points} marking the 
positions where the content of the box registers, filled by the page builder,
should appear. 
To output the page, the viewer traverses the page template,
replaces the placeholders by the appropriate box content, and 
sets the glue. 

It is only natural to treat the page's main body,
inserts, and marks using the same mechanism. We call this
mechanism a content stream\index{stream}. 
Content streams are identified by a stream number in the range 0 to 254;
the number 255 is used to indicate an invalid stream number.
The stream number 0 is reserved for the main content stream; it is always defined.

\medskip
{\small \advance \leftskip by 30pt \advance \rightskip by 30pt 
It is planed to implement a replacement for \TeX's mark nodes using
different types of streams:
\itemize
\item normal streams correspond to \TeX's inserts and accumulate content on the page,
\item first\index{first stream} streams correspond to \TeX's first marks and will contain only the first insertion of the page,
\item last\index{last stream} streams correspond to \TeX's bottom marks and will contain only the last insertion of the page, and
\item top\index{top stream} streams correspond to \TeX's top marks. Top streams are not yet implemented.
\enditemize
\medskip
}

Nodes from the content section are considered contributions to stream~0 except
for insert nodes which will specify the stream number explicitly. 
If the stream is not defined or is not used in the current page template, its content is simply ignored.

The page builder needs a mechanism to redirect contributions from one content
stream to another content stream based on the availability of space.
Hence a \HINT\ content stream can optionally specify a preferred stream number,
where content should go if there is still space available, a next stream number,
where content should go if the present stream has no more space available, and
a split ratio if the content is to be split between these two streams before
filling in the template.

Various stream parameters govern the treatment of contributions to the stream
and the page building process.

\itemize
\item The magnification factor $f$: Inserting a box of height $h$ to this stream will contribute $h\times f/1000$
to the height of the page under construction. For example, a stream
that uses a two column format will have an $f$ value of 500; a stream
that specifies notes that will be displayed in the page margin will
have an $f$ value of zero.

\item The height $h$: The extended dimension $h$ gives the maximum height this 
stream is allowed to occupy on the current page.
To continue the previous example, a stream that will be split into two columns
will have $h=2\cdot\.{vsize}$ , and a stream that specifies
notes that will be displayed in the page margin will have
$h=1\cdot\.{vsize}$.  You can restrict the amount of space occupied by
footnotes to the bottom quarter by setting the corresponding $h$ value
to $h=0.25\cdot\.{vsize}$.

\item The depth $d$: The dimension $d$ gives the maximum depth this 
stream is allowed to have after formatting.

\item The width $w$: The extended dimension $w$ gives the width of this stream 
when formatting its content. For example margin notes
should have the width of the margin less some surrounding space.

\item The ``before'' list $b$: If there are any contributions to this
stream on the current page, the material in list $b$
is inserted {\it before\/} the material from the stream itself. For
example, the short line that separates the footnotes from the main
page will go, together with some surrounding space, into the list~$b$.

\item The top skip glue $g$: This glue is inserted between the material
from list $b$ and the first box of the stream, reduced
by the height of the first box. Hence it specifies the distance between
the material in $b$ and the first baseline of the stream content.

\item The ``after'' list $a$: The list $a$ is treated like list $b$ but
its material is placed {\it after\/} the  material from the stream itself.

\item The ``preferred'' stream number $p$:  If $p\ne 255$, it is the number of 
the {\it preferred\/} stream. If stream $p$ has still
enough room to accommodate the current contribution, move the
contribution to stream $p$, otherwise keep it.  For example, you can
move an illustration to the main content stream, provided there is
still enough space for it on the current page, by setting $p=0$.

\item The ``next'' stream number $n$: If $n\ne 255$, it is the number of the 
{\it next\/} stream. If a contribution can not be
accommodated in stream $p$ nor in the current stream, treat it as an
insertion to stream $n$.  For example, you can move contributions to
the next column after the first column is full, or move illustrations
to a separate page at the end of the chapter.

\item The split ratio\index{split ratio} $r$: If $r$ is positive, both $p$ and $n$ must 
be valid stream numbers and contents is not immediately moved to stream $p$ or $n$ as described before.
Instead the content is kept in the stream itself until the current page is complete.
Then, before inserting the streams into the page template, the content of
this stream is formatted as a vertical box, the vertical box is
split into a top fraction and a bottom fraction in the ratio $r/1000$
for the top and $(1000-r)/1000$ for the bottom, and finally the top
fraction is moved to stream $p$ and the bottom fraction to stream
$n$. You can use this feature for example to implement footnotes
arranged in two columns of about equal size. By collecting all the
footnotes in one stream and then splitting the footnotes with $r=500$
before placing them on the page into a right and left column.  Even
three or more columns can be implemented by cascades of streams using
this mechanism.
\enditemize

\HINT\ allows multiple page templates but Hi\TeX\ currently does not implement
restricting them to individual page ranges and the viewer selects
the page template with the highest priority. To support different output media, the page
templates are named and a suitable user interface may offer the user a selection
of possible page layouts. In this way, the page layout remains in the hands of the
book designer, and the user has still the opportunity to pick a layout that best fits
the display device.

The build-in page template with number 0 is always defined and has priority 0.
It will display just the main content stream. It puts a small margin 
of $\.{hsize}/8 -4.5\hbox{pt}$ all around it.
Given a letter size page, 8.5 inch wide, this formula yields a margin of 1 inch,
matching \TeX's plain format. The margin will be positive as long as
the page is wider than $1/2$ inch. For narrower pages, there will be no
margin at all. In general, the \HINT\ viewer will never set {\tt hsize} larger
than the width of the page and {\tt vsize} larger than its height.

%8.5 in should give 1 inch margin 2/17
%612pt should give 72pt margin
%72pt = 612/8-4.5pt
%This would give a positive margin starting at 36pt or 1/2 inch


%\appendix

%\plainsection{References}

%{\baselineskip=11pt
%\def\bfblrm{\small\rm}%
%\def\bblem{\small\it}%
%\bibliography{../hint}
%\bibliographystyle{plain}
%}

\plainsection{Index}
{
\def\_{{\tt \UL}} % underline in a string
\catcode`\_=\active \let_=\_ % underline is a letter
\input hitexman.ind
}

\write\cont{} % ensure that the contents file isn't empty
%  \write\cont{\catcode `\noexpand\@=12\relax}   % \makeatother
\closeout\cont% the contents information has been fully gathered

\inx
\fin
\end