Section 211: The semantic nest

TeX is typically in the midst of building many lists at once. For example, when a math formula is being processed, is in math mode and working on an mlist; this formula has temporarily interrupted from being in horizontal mode and building the hlist of a paragraph; and this paragraph has temporarily interrupted from being in vertical mode and building the vlist for the next page of a document. Similarly, when a \vbox occurs inside of an \hbox, is temporarily interrupted from working in restricted horizontal mode, and it enters internal vertical mode. The “semantic nest” is a stack that keeps track of what lists and modes are currently suspended.

At each level of processing we are in one of six modes:

  • VMODE stands for vertical mode (the page builder);
  • HMODE stands for horizontal mode (the paragraph builder);
  • MMODE stands for displayed formula mode;
  • −VMODE stands for internal vertical mode (e.g., in a \vbox);
  • −HMODE stands for restricted horizontal mode (e.g., in an \hbox);
  • −MMODE stands for math formula mode (not displayed).

The mode is temporarily set to zero while processing \write texts.

Numeric values are assigned to VMODE, HMODE, and MMODE so that ’s “big semantic switch” can select the appropriate thing to do by computing the value abs(mode) + cur_cmd, where mode is the current mode and cur_cmd is the current command code.

constants.h
#define VMODE 1                         // vertical mode
#define HMODE (VMODE + MAX_COMMAND + 1) // horizontal mode
#define MMODE (HMODE + MAX_COMMAND + 1) // math mode
other_printing.c
// << Start file |other_printing.c|, 1382 >>

// prints the mode represented by |m|
void print_mode(int m) {
    if (m > 0) {
        switch(m / (MAX_COMMAND + 1)) {
        case 0:
            print("vertical");
            break;
        
        case 1:
            print("horizontal");
            break;
        
        case 2:
            print("display math");
        }
    }
    else if (m == 0) {
        print("no");
    }
    else {
        switch ((-m) / (MAX_COMMAND + 1)) {
        case 0:
            print("internal vertical");
            break;
        
        case 1:
            print("restricted horizontal");
            break;
        
        case 2:
            print("math");
        }
    }
    print(" mode");
}

Section 212

The state of affairs at any semantic level can be represented by five values:

  • mode is the number representing the semantic mode, as just explained.
  • head is a pointer to a list head for the list being built; link(head) therefore points to the first element of the list, or to null if the list is empty.
  • tail is a pointer to the final node of the list being built; thus, tail = head if and only if the list is empty.
  • prev_graf is the number of lines of the current paragraph that have already been put into the present vertical list.
  • aux is an auxiliary memory_word that gives further information that is needed to characterize the situation.

In vertical mode, aux is also known as prev_depth; it is the scaled value representing the depth of the previous box, for use in baseline calculations, or it is −1000pt if the next box on the vertical list is to be exempt from baseline calculations. In horizontal mode, aux is also known as space_factor and clang; it holds the current space factor used in spacing calculations, and the current language used for hyphenation. (The value of clang is undefined in restricted horizontal mode.) In math mode, aux is also known as incompleat_noad; if not null, it points to a record that represents the numerator of a generalized fraction for which the denominator is currently being formed in the current list.

There is also a sixth quantity, mode_line, which correlates the semantic nest with the user’s input; mode_line contains the source line number at which the current level of nesting was entered. The negative of this line number is the mode_line at the level of the user’s output routine.

In horizontal mode, the prev_graf field is used for initial language data.

The semantic nest is an array called nest that holds the mode, head, tail, prev_graf, aux, and mode_line values for all semantic levels below the currently active one. Information about the currently active level is kept in the global quantities mode, head, tail, prev_graf, aux, and mode_line, which live in a Pascal record that is ready to be pushed onto nest if necessary.

constants.h
#define IGNORE_DEPTH -65536000 // |prev_depth| value is ignored

⟨ Types in the outer block 18 ⟩+≡

typedef struct {
    int mode_field;
    pointer head_field;
    pointer tail_field;
    int pg_field;
    int ml_field;
    memory_word aux_field;
} list_state_record;

Section 213

datastructures.h
#define mode            cur_list.mode_field // current mode
#define head            cur_list.head_field // header node of current list
#define tail            cur_list.tail_field // final node on current list
#define prev_graf       cur_list.pg_field   // number of paragraph lines accumulated
#define aux             cur_list.aux_field  // auxiliary data about the current list
#define prev_depth      aux.sc              // the name of |aux| in vertical mode
#define space_factor    hh_lh(aux)          // part of |aux| in horizontal mode
#define clang           hh_rh(aux)          // the other part of |aux| in horizontal mode
#define incompleat_noad aux.integer         // the name of |aux| in math mode
#define mode_line       cur_list.ml_field   // source file line number at beginning of list

⟨ Global variables 13 ⟩+≡

list_state_record nest[NEST_SIZE + 1];
int nest_ptr;               // first unused location of |nest|
int max_nest_stack;         // maximum of |nest_ptr| when pushing
list_state_record cur_list; // top "top" semantic nest
int shown_mode;             // most recent mode shown by \tracingcommands

Section 214

Here is a common way to make the current list grow:

datastructures.h
#define tail_append(X) link(tail) = (X); tail = link(tail)

Section 215

We will see later that the vertical list at the bottom semantic level is split into two parts; the “current page” runs from PAGE_HEAD to page_tail, and the “contribution list” runs from CONTRIB_HEAD to tail of semantic level zero. The idea is that contributions are first formed in vertical mode, then “contributed” to the current page (during which time the page-breaking decisions are made). For now, we don’t need to know any more details about the page-building process.

⟨ Set initial values of key variables 21 ⟩+≡

nest_ptr = 0;
max_nest_stack = 0;
mode = VMODE;
head = CONTRIB_HEAD;
tail = CONTRIB_HEAD;
prev_depth = IGNORE_DEPTH;
mode_line = 0;
prev_graf = 0;
shown_mode = 0;
// << Start a new current page, 991 >>

Section 216

When ’s work on one level is interrupted, the state is saved by calling push_nest. This routine changes head and tail so that a new (empty) list is begun; it does not change mode or aux.

modes.c
// << Start file |modes.c|, 1382 >>

// enter a new semantic level, save the old
void push_nest() {
    if (nest_ptr > max_nest_stack) {
        max_nest_stack = nest_ptr;
        if (nest_ptr == NEST_SIZE) {
            overflow("semantic nest size", NEST_SIZE);
        }
    }
    nest[nest_ptr] = cur_list; // stack the record
    incr(nest_ptr);
    head = get_avail();
    tail = head;
    prev_graf = 0;
    mode_line = line;
}

Section 217

Conversely, when is finished on the current level, the former state is restored by calling pop_nest. This routine will never be called at the lowest semantic level, nor will it be called unless head is a node that should be returned to free memory.

modes.c
// leave a semantic level, re-enter the old
void pop_nest() {
    free_avail(head);
    decr(nest_ptr);
    cur_list = nest[nest_ptr];
}

Section 218

Here is a procedure that displays what is working on, at all levels.

other_printing.c
void show_activities() {
    int p;         // index into |nest|
    int m;         // mode
    memory_word a; // auxiliary
    pointer q, r;  // for showing the current page
    int t;         // ditto

    nest[nest_ptr] = cur_list; // put the top level into the array
    print_nl("");
    print_ln();
    for(p = nest_ptr; p >= 0; p--) {
        m = nest[p].mode_field;
        a = nest[p].aux_field;
        print_nl("### ");
        print_mode(m);
        print(" entered at line ");
        print_int(abs(nest[p].ml_field));
        if (m == HMODE && nest[p].pg_field != 0x830000) {
            print(" (language");
            print_int(nest[p].pg_field % 0x10000);
            print(":hyphenmin");
            print_int(nest[p].pg_field / 0x400000);
            print_char(',');
            print_int((nest[p].pg_field / 0x10000) % 64);
            print_char(')');
        }
        if (nest[p].ml_field < 0) {
            print(" (\\output routine)");
        }
        if (p == 0) {
            // << Show the status of the current page, 986 >>
            if (link(CONTRIB_HEAD) != null) {
                print_nl("### recent contributions:");
            }
        }
        show_box(link(nest[p].head_field));
        // << Show the auxiliary field, |a|, 219 >>
    }
}

Section 219

⟨ Show the auxiliary field, a 219 ⟩≡

switch(abs(m) / (MAX_COMMAND + 1)) {
case 0:
    print_nl("prevdepth ");
    if (a.sc <= IGNORE_DEPTH) {
        print("ignored");
    }
    else {
        print_scaled(a.sc);
    }
    if (nest[p].pg_field != 0) {
        print(", prevgraf ");
        print_int(nest[p].pg_field);
        print(" line");
        if (nest[p].pg_field != 1) {
            print_char('s');
        }
    }
    break;

case 1:
    print_nl("spacefactor ");
    print_int(hh_lh(a));
    if (m > 0 && hh_rh(a) > 0) {
        print(", current language ");
        print_int(hh_rh(a));
    }
    break;

case 2:
    if (a.integer != null) {
        print("this will begin denominator of:");
        show_box(a.integer);
    }
} // there are no other cases

This code is used in section 218.