nyweb

v3.03

nyweb is just another incarnation of Knuth's WEB, this time language-independent, notation based on XML, and heavily inspired by nuweb.

Essentially, it is a preprocessor, separating comment from content (the program itself), outputting both into a different file (or, more precisely, multiple files as specified). Other major features are parametrizable macros definition/replacement and conditional "translation". But, contrary to the previous implementations of WEB, nyweb does not pretend that the comments are usable as true documentation, therefore no prettyprinting is supported.

Rather, it is seen that the comments are valuable only as the programmer's tool, in the process of creation. Documentation can be created together with the program, though, simply as just another type of content, written in some of documentation formats (e.g. TeX (and derived formats), or html, or formats derived from xml), output to a separate file(s).

Instead of "autodocumentation"/prettyprinting, a different but major idea of WEB is brought into front in nyweb - notably, that it supports the natural way how programs are created. First, there is a rough description in human language - equivalent of which is in the freelance and unpublished comments - and the sketch of a skeleton of the program can be written at top level, replacing the key elements simply by their desctiption, to be written (here usually as macros) and included later. Then, these lower-level elements are written, possibly in the same manner: outlining their function and replacing the details again by inclusion marks, the body of which is to be written later. After the first version of the program is written in this top-to-down manner, it is not uncommon that new functionality and features are added, often influencing more parts of the original program, and not rarely requiring a different structure than the original program was written with. In (ny)web, elements of such new functionality is written as a separate "chapter" of a set of macros, included; thus grouping together functionality, but enabling to position these elements into separate parts of the program as needed. This partially replaces and augments the natural structures provided by many programming languages, but does this without need to sacrifice efficiency (as unnecessary "artificial" functions calls would).

nyweb aims to build also on an other aspect of modern programming. We have seen in the last years a growth in "intelligent" programmer's editors. They now can highlight syntax, fold functions or other structures, create lists of functions and other structures, autocomplete names of already written/used functions and variables etc. These editors doubtlessly increased efficiency of a programmer, especially working on a big project. However, these editors rely on recognition of the programming language syntax (basically they perform the first steps of compilation, "online"), therefore are language dependent, can be confused by new features of a language, have troubles in recognising "mixed" languages/projects - in short, they are rigid and fragile.

Therefore, nyweb is seen only as an intermediate step towards - or basis of - a new programmer's editor, providing a high degree of flexibility in defining structure by the programmer, while maintaining efficiency of the resulting compiled code. It would also provide markup needed for fast and efficient movement within sources and instant lookup of key program elements, fully under the control of programmer, not exhausting him with unnecessary details of similar markups (e.g. functions lists) created automatically. Also, the "interwoven documentation" can be easily accomplished in such editor, in a very similar manner to what hypertext editors/viewers provide. This is why xml was used as the underlying markup language, enabling to add an arbitrary number of tags and marks, even in a forward-compatible manner (simply ignoring tags unknown for a given version). However, it is clear, that such editor would be a major undertaking, far beyond the capabilities of the one-man-show nyweb currently is.

Usage of nyweb

nyweb is a command-line tool, being run as
 nyweb inputfile.w [outputfile.txt]  
Comments (i.e. all text outside emit tags - see below) are output into the output file. If not given as the second parameter, the output goes to standard output (console). Often, it is wise to use nul as output file, to flush the - for most uses unnecessary - comments. The content (real program) output files are given explicitly in the source .w file - see emit tag.

Errors and warnings as they occur during processing are printed to the stderr device. Upon occurence of the first error further input processing stops and the program exists with exit code 1. On success, the exit code is 0.

Basic technicalities of nyweb

nyweb currently reproduces much of nuweb's functionality, except prettyprinting/TeX support. However, before describing the tags themselves, a few words about the xml-related stuff:

XML CDATA facility

Since v2.02, accordingly to XML rules, text enclosed within <![CDATA[ and ]]> is not parsed for markups, i.e. for the < and & characters. Thus, in such block of text, these characters could be used freely.

This greatly facilitates writing HTML documentation or XML files in nyweb.

Basic tags of nyweb

emit tag

This tag encloses the "real" program (or documentation or any other content), to be saved into a file. It has one attribute, file, to determine the filename to be output into:
 
<emit file="myprogram.c">
  for (i = 0; i &lt; 99; i++) {
    printf("Blahblah %d\n", i);
  }
</emit>
  
(Note the usage of &lt; instead of < in the example above 1 ). There may be any number of emit tags in a single input file, even emitting to the same file: the outputs are concatenated in the order they appear in the input file. However, emit tags cannot be nested.

One more feature is taken over from nuweb: the emitted files are during processing saved to a temporary file; and after the whole input file is processed, these temporary files are compared against older version of the same file. Only if a change occurs (or the file did not exist previously), the new file is overwritten over the old file; otherwise the temporary file is silently discarded. This feature enables to use tools such as make, which rely on the timestamp of a source file to determine whether it has changed or not.

dependencies attribute of emit tag - using this attribute, a file containing dependencies for the emitted file on its .w sources. This is intended to be an input to programs like make. The value of this attribute determines the dependency file name (traditionally, it is placed into .dep subdirectory and has the same name as the emitted file plus a .d suffix). If there are multiple emit tags targeting the same emitted file, the first which has a non-empty dependency tag determines the dependency file name.

macro tag

This tag encloses any content, which won't appear in any output file directly, but can be included either in comment or emit - and it can be included any number of times - see use tag below.

Macro has one required attribute, name, which uniquely identifies the macro:
 
<macro name="print blahblah">
  printf("Blahblah\n");
</macro>
  
Macros can be defined only outside any other functional tag (i.e. macro, emit or use). However, use tags can be nested into macros. Macros can be defined in any order, they don't need to be defined before the use tag where they are invoked.

Multiple macros with the same name can be defined, they are then concatenated in the order they appear in the source file. This ordering can be overriden explicitly by adding an order attribute to the macro, with an integer as the value. These macros are then concatenated in ascending order (macros of the same "order" again in their order of appearance), with those macros which don't have theg order attribute defined added to the end of chain of macros. For example:
 <macro name="fruits">  Apple </macro>
<macro name="fruits">  Banana </macro>
<macro name="fruits">  Orange </macro>  
when invoked as <use name="fruits"/> would appear as " Apple Banana Orange ", but if we define the same as
 <macro name="fruits" order="20">  Apple </macro>
<macro name="fruits">  Banana </macro>
<macro name="fruits" order="10">  Orange </macro>  
when invoked as <use name="fruits"/> would appear as " Orange Apple Banana ".

use tag

This tag is replaced by the content of macro with the same name tag. It can occur in comment area as well as inside emit tags.

Since v1.06, <use macro="xxx"/> is an alternative form of <use name="xxx"/> (this is en par with the alternative form of parameter invocation, <use param="xxx"/>).

htmlize attribute of use tag - this attribute can be used with a top-level use tag (i.e. such which is not embedded within a macro). It can have any non-empty value (e.g. htmlize="1"), the value itself is ignored. This attribute causes to output or emit &lt; instead of < and &amp; instead of &, which might be useful if these escaped characters are to be outputted into a HTML (or other xML) file.

attributes for stripping whitespace - adding non-empty slw/stw/sltw attribute (with any non-empty value, which is ignored) to use tag, or to param tag when it is in macro (i.e. when it causes inclusion of the parameter or a table item), will strip the leading/trailing/both whitespaces of the included macro or table item. (Note: As complete processing of the trailing whitespace stripping is very complex, currently only a simplified version is implemented, which strips whitespace preceding any &lt; character, even if it is the start of a nested tag).
Similarly, non-empty scrlf attribute removes all the newlines (since v2.10).

attributes for limiting output length - number of characters output for a given use/param (as counted after whitespace stripping) may be limited to the value of maxlen attribute (since v2.11). If this is exceeded, a warning is printed, which can be suppressed by adding nonempty "nowarn" attribute.

nowarn attribute - adding non-empty nowarn attribute (with any non-empty value, which is ignored) to use tag, or to param tag when it is in macro (i.e. when it causes inclusion of the parameter or a table item), will suppress the "macro of that name not found", "parameter not found" and "no such row in the table" warnings

Parametrized use of macros: params and tables

param tag

Parametrized macros are accomplished using the param tag. This tag has dual usage:

  1. inside the use element, it contains definitions of the parameter body
  2. within a macro definition, it is replaced by the respective definition when the macro is invoked from a use
Except for the alternative forms mentioned above, both the "definition-form" and "use-form" param has a required name attribute, as an identifier.

Both usage and processing of the param elements are very similar to the usage and processing of macros, except

It is not necessary to have all parameters used in a macro defined when the macro is invoked from a use. For the undefined params, a warning is issued, but processing continues.

Example:
 
<macro name="filled pie"> <param name="filling"/> pie</macro>
<emit file="menu.txt">
  <use name="filled pie"> <param name="filling">Cherry</param> </use>,
  <use name="filled pie"> <param name="filling">Apple</param> </use>,
  <use name="filled pie"> <param name="filling">Chocolate</param> </use>.
</emit>   
will create the following menu: Cherry pie, Apple pie, Chocolate pie.

Tables (lists): table and item tags, and table attribute in use tag

Tables (available since v2.0; in the "planned" features they were mentioned under an older name as lists) are rows of items, each item containing a source text. They also can be seen as sets of params, which can be used in expansion of an appropriately written macro, one row at a time, through use of that macro together with table.

Defining tables

Tables are defined one row at a time, through a table tag. Each row contains definition of items through item tag; both having a name attribute. Rows are mutually independent, and each can have a different set of items defined. The order of definition of items within a row is irrelevant.

Rows within the table can be ordered through order attribute of the table tag; this determines the order how they are used.

As tables are primary sources of text, their definition is always at the top level, same as with ordinary macro. The text items behave as macros, i.e. they can have embedded use.

Using tables

Tables are used through use of an appropriate macro. The use will have an additional table attribute, which determines the used table and indicates that this use shall be repeated for all rows of table. Upon each iteration of use, items of the given table row behave exactly as params with the same name as the given item. This simply recycles the param mechanism. The used macro then simply has these params embedded exactly as with "ordinary" params.

There still can be params defined within use, even if it has the table tag; and these params may have the same name as items in the table. When <param/> is encountered in the expanded macro, the current table row is scanned first for item of a given name, and if not found, the set of params defined in the use is scanned for a param of the same name. This provides a convenient mechanism for default value, if not defined within a table row. (This behaviour can be modifyind by adding a non-empty deep attribute to the use-form param.)

As a planned feature, "derived tables" could be created and subsequently used using "filters" and "sorters", but that's upon further development.

A simple example

This is a similar example than at the explanation of param:
 
<table name="pies"> <item name="filling">Cherry</item>    <item name="topping">chocolate</item> </table>
<table name="pies"> <item name="filling">Apple</item>     <item name="topping">sugar icing</item> </table>
<table name="pies"> <item name="filling">Chocolate</item> <item name="topping">whipped cream</item> </table>
<macro name="menu line"> <param name="filling"/> pie topped by <param name="topping"/>.
</macro>
<emit file="menu.txt">
  <use name="menu line" table="pies"/>
</emit>   
will create the following menu into menu.txt:
 
Cherry pie topped by chocolate.
Apple pie topped by sugar icing.
Chocolate pie topped by whipped cream.  

A sligthly more complex example

Framework (boring, but it needs to be done only once):
 
<emit file="source.h">
// global variables declarations
  <use name="external variables declaration template" table="global variables">
    <param name="type"><use name="default type"/></param>
  </use>

// function prototypes
  [...]
</emit>

<emit file="source.c">
// includes, defines, etc.
  [...]

// global variables
  <use name="global variables definition template" table="global variables">
    <param name="type"><use name="default type"/></param>
    <param name="init"> 0 </param>
  </use>


// functions etc.
  [...]
</emit>



<macro name="external variables declaration template">  extern <param name="type"/> <param name="var"/>;
</macro>

<macro name="global variables definition template">  <param name="type"/> <param name="var"/> = <param name="init"/>;
</macro>


<macro name="default type"> uint32_t </macro>
  

Then, whenever needed, we can "create" global variables at various places throughout the .w source simply by...
 
<table name="global variables"> <item name="type"> int </item>   <item name="var">  a </item> <item name="init"> 5 </item> </table>
[...]
We want this variable to be defined as first, for some allocation reasons; and we want it to be of the default type:
<table name="global variables" order="1">    <item name="var">  b </item> <item name="init"> -9 </item> </table>
[...]
We want this variable to be initialized with the default initializer:
<table name="global variables"> <item name="type"> char </item>   <item name="var">  c </item>  </table>
  

... and we get... (source.h):
 

// global variables declarations
    extern  uint32_t    b ;
  extern  int    a ;
  extern  char    c ;


// function prototypes
  [...]
  
(source.c):
 

// includes, defines, etc.
  [...]

// global variables
     uint32_t    b  =  -9 ;
   int    a  =  5 ;
   char    c  =  0 ;



// functions etc.
  [...]
  

Filters on tables

While there can be sophisticated filters and sorters "invented" and applied on tables, at the moment (as of v2.01), only very simple filters are implemented.

A trivial case: row attribute

Table rows can be individually "picked" from the table, by labelling them using a row attribute when defined. Then, if such table is used, using again the row attribute in the use tag

In case there are more rows in the table with the same row attribute, only the first one will be taken into account. If there is no such row in the table, a warning will be printed and the use will be ignored.

A somewhat more sophisticated method: has_item attribute

Functionally, this is a superset of the trivial row-attribute-based method. Placing has_item attribute into the use tag causes that only those rows, which explicitly contain an item of matching name, will be used. There may be multiple such rows in a table, all of them will be used consecutively, as if a subset of the table would be applied to the macro.

If there is no such row in the table, a warning will be printed and use will be ignored.

Since v2.11, there is also a has_item_not attribute available, with the expected function: selecting rows which don't contain the named item.

Pandora's box

macro and table indirection in param and item

Since v3.00, definition-form param and table-row item may take either table or macro attribute.

These may replace the macro and/or table specified in <use name="xxx" table="yyy"/>. Prior to "execution" of use, value of name attribute of all current param/item is compared to xxx/yyy, and if a match is found, the value of respective macro/table attribute is substituted for the name of used macro/table. A simple, but quite useless example*:

 
  <emit file="test.txt">
    <use name="test macro"> <param name="test table" table="table 1"></param> </use>
    <use name="test macro"> <param name="test table" table="table 2"></param> </use>
  </emit>

  <macro name="test macro"> Things are: <use name="list things" table="test table"/> </macro>
  <macro name="list things"> <param name="thing"/>, </macro>

  <table name="table 1"><item name="thing">cake</item></table>
  <table name="table 1"><item name="thing">bread</item></table>
  <table name="table 1"><item name="thing">steak</item></table>

  <table name="table 2"><item name="thing">fork</item></table>
  <table name="table 2"><item name="thing">spoon</item></table>
  

yields:

 
     Things are:  cake,  bread,  steak,
     Things are:  fork,  spoon,
  

A more convoluted example:

 
  <emit file="menu.txt"><use name="execute processes" table="processes"/>
  </emit>
  <macro name="execute processes"><param name="header"/>: <use name="process" table="pies"/>.
  </macro>

  <table name="processes"> <item name="header">Fillings</item> <item name="process" macro="list fillings"></item> </table>
  <table name="processes"> <item name="header">Toppings</item> <item name="process" macro="list toppings"></item> </table>
  <table name="processes"> <item name="header">Menu</item> <item name="process" macro="make menu"></item> </table>

  <macro name="list fillings"><if iter=">0">, </if><param name="filling"/> filling</macro>
  <macro name="list toppings"><if iter=">0">, </if><param name="topping"/></macro>
  <macro name="make menu"><if iter=">0">, </if><param name="filling"/> pie topped by <param name="topping"/></macro>

  <table name="pies"> <item name="filling">Cherry</item>    <item name="topping">chocolate</item> </table>
  <table name="pies"> <item name="filling">Apple</item>     <item name="topping">sugar icing</item> </table>
  <table name="pies"> <item name="filling">Chocolate</item> <item name="topping">whipped cream</item> </table>
  
Here, macro indirection is demonstrated. In emit which creates "menu.txt" file, macro "execute processes" is invoked with "processes" table. As this table has three rows, this invocation generates three lines, starting with "header" item, followed by the output of (redirected) "process" invocation with table "pies". In each "execute processes" invocation, "process" is replaced by respective "item macro" from the currently used "processes" table's row, thus invoking three different macros, "list fillings", "list toppings" and "make menu", in this order; supplying to all of the the three-row sized "pies" table. The resulting "menu.txt" file is thus:
 
  Fillings: Cherry filling, Apple filling, Chocolate filling.
  Toppings: chocolate, sugar icing, whipped cream.
  Menu: Cherry pie topped by chocolate, Apple pie topped by sugar icing, Chocolate pie topped by whipped cream.
  
The table-redirection form of items allow a tree-like structure to be created, for example:
 
  <emit file="menu.txt"><use name="execute level 1"/></emit>
  <macro name="execute level 1">
  General G.W. consults <use name="list names" table="colonels"><param name="rank">colonel</param></use>.
  <use name="execute level 2" table="colonels"/></macro>
  <macro name="execute level 2">  Colonel <param name="name"/> instructs <use name="list names" table="captains"><param name="rank">captain</param></use>.
  <use name="execute level 3" table="captains"/></macro>
  <macro name="execute level 3">    Captain <param name="name"/> orders <use name="list names" table="soldiers"/>.
  </macro>

  <macro name="list names"><if iter=">0">, </if><param name="rank" nowarn="1"/> <param name="name"/></macro>

  <table name="colonels"> <item name="name">H.K.</item> <item name="captains" table="captains of H.K."/> </table>
  <table name="colonels"> <item name="name">A.H.</item> <item name="captains" table="captains of A.H."/> </table>

  <table name="captains of H.K."> <item name="name">E.W.</item> <item name="soldiers" table="soldiers of E.W."/> </table>
  <table name="captains of H.K."> <item name="name">R.F.</item> <item name="soldiers" table="soldiers of R.F."/> </table>
  <table name="captains of H.K."> <item name="name">S.D.</item> <item name="soldiers" table="soldiers of S.D."/> </table>
  <table name="captains of A.H."> <item name="name">G.I.</item> <item name="soldiers" table="soldiers of G.I."/> </table>
  <table name="captains of A.H."> <item name="name">L.C.</item> <item name="soldiers" table="soldiers of L.C."/> </table>

  <table name="soldiers of E.W."> <item name="name">J.1.D.</item> </table>
  <table name="soldiers of E.W."> <item name="name">J.2.D.</item> </table>
  <table name="soldiers of E.W."> <item name="name">J.3.D.</item> </table>
  <table name="soldiers of E.W."> <item name="name">J.4.D.</item> </table>
  <table name="soldiers of R.F."> <item name="name">J.5.D.</item> </table>
  <table name="soldiers of R.F."> <item name="name">J.5.D.</item> </table>
  <table name="soldiers of R.F."> <item name="name">J.6.D.</item> </table>
  <table name="soldiers of R.F."> <item name="name">J.7.D.</item> </table>
  <table name="soldiers of R.F."> <item name="name">J.8.D.</item> </table>
  <table name="soldiers of R.F."> <item name="name">J.9.D.</item> </table>
  <table name="soldiers of S.D."> <item name="name">J.A.D.</item> </table>
  <table name="soldiers of G.I."> <item name="name">J.B.D.</item> </table>
  <table name="soldiers of G.I."> <item name="name">J.C.D.</item> </table>
  <table name="soldiers of G.I."> <item name="name">J.D.D.</item> </table>
  <table name="soldiers of L.C."> <item name="name">J.E.D.</item> </table>
  <table name="soldiers of L.C."> <item name="name">J.F.D.</item> </table>
  <table name="soldiers of L.C."> <item name="name">J.G.D.</item> </table>
  <table name="soldiers of L.C."> <item name="name">J.H.D.</item> </table>
  
results in
 
General G.W. consults colonel H.K., colonel A.H..
  Colonel H.K. instructs captain E.W., captain R.F., captain S.D..
    Captain E.W. orders  J.1.D.,  J.2.D.,  J.3.D.,  J.4.D..
    Captain R.F. orders  J.5.D.,  J.5.D.,  J.6.D.,  J.7.D.,  J.8.D.,  J.9.D..
    Captain S.D. orders  J.A.D..
  Colonel A.H. instructs captain G.I., captain L.C..
    Captain G.I. orders  J.B.D.,  J.C.D.,  J.D.D..
    Captain L.C. orders  J.E.D.,  J.F.D.,  J.G.D.,  J.H.D..
  

Similarly to the use-form param, redirection for macro/table are first searched in the current table row items, then in params, so a redirection-form param can serve as a fallback default. Contrary to the use-form param, here, a "true" top-level macro/table may be available, too, as an additional fallback default.

param/item tag with macro/table attribute still may enclose a content, which is output if use-form param of the same name is encountered. This is not a very useful feature, rather, a (harmless) side-effect of simplified processing.

Since v3.02, param/item tag with table attribute may have an optional has_item or has_item_not attribute. This will override the analogous attribute of the table-to-be-overridden, allowing to filter tables by the invoking use.

* Note, that as param with macro/table attribute is always the define-form param (i.e. contained within use), it must not be of the empty form (i.e. <param name="name" table="table"/> is NOT allowed!).

pass attribute of use: parameters and table-row nesting

Since v3.00, adding pass attribute to use allows the current-table-row items and current params to be accessed from the invoked macro.

Three forms of pass are available: pass="params", pass="items" and pass="all"; their purpose is self-explanatory.

deep attribute of use-form param

When use-form of param is encountered during outputting content of a macro, first, table items are searched for a matching name, then params defined in the invoking use, and if pass is employed, this search goes on recursively as described above. Normally, the first match is used, allowing for the "deeper levels" to provide a default in case when item/param in "nearest level" is missing.

Since v3.03, using a non-empty deep attribute in the use-form (empty) param (or the equivalent use param construction) modifies this behaviour, so, that the "deepest" occurence of item/param is used. This allows param to override item, and invoker to override the invoked.

<param attrib="debug">

Both table redirections and param/item nesting may result in complex param/item patterns. To help development, <param attrib="debug"> outputs a list of params/items available at the current level.

Params are listed with prefix "P-" and current-table-row items with prefix "I-". If param/item is of a redirect form, it is suffixed by ":M-macro_name" or ":T-table_name", for macro/table redirection respectively. Nesting using "pass" is indicated by "=[xx]=>", where xx is P for params pass, I for items pass, and p/i for params/items pass suppressed by missing pass at a higher-level. For example:

   I-t1r2r:T-t2,I-t1r2i1,P-p3b,P-p3a:M-m4=[P]=>P-p2b,P-p2a=[Pi]=>P-p1b,P-p1a
  
At the current level, there is a table row containing <item name="t1r2r" table="t2"> and <item name="t1r2i1> and two params, p3b and p3a, the latter redirecting to macro m4. There is a pass to higher level, passing only params, through which two params, p2b and p2a, are available. Then there is another pass to even higher level; that pass is "all", but due to the lower-level pass being params-only, the item-part of this pass is suppressed. Two params are available through this pass, p1b and p1a.

allow_recursion attribute of use

Use of macros and tables is subject to check for recursive use, as such may lock up the program (given time, it will eventually lead to memory exhaustion and crash). However, using tables, there may be legal re-uses of tables (e.g. if in an item the same table is run for a row count, or if one column references table with a different column) and even macros (using them with different tables). Such usage is very difficult to be checked for correctness, so since v3.01, allow_recursion attribute is added to use to allows the user explicit exemptions from the recursion check.

Three forms of allow_recursion are available: allow_recursion="macro", allow_recursion="table" and allow_recursion="both".

Other tags

Including unparsed "character" macro: cmacro tag

The cmacro tag starts a macro, which is literally included, ignoring the "reserved" characters (& and <). This ends with /cmacro tag placed on a new line, preceded by only whitespace characters.

This allows to write freely source texts in common programming languages within such macros, with the only limitation, that a < character cannot appear as the first character on a new line (not after newline and whitespace).

Except that tags (including use) nested into cmacro are ignored, cmacro blocks are processed in the same way as macro, i.e. they can chain, be ordered, even macro and cmacro can be mixed together (if they have the same name attribute).

Note: the </cmacro> string has to be entered EXACTLY in this way - it is case sensitive, and no whitespaces inside < > are allowed!

Note: this feature violates rules of XML, so if strict XML-conformance is required for some reason, don't use cmacro at all.

See also CDATA and cinclude.

Input files nesting: include tag

As could be expected, include tag simply literally includes the file specified by the file attribute, during processing.
 <include file="common subroutines.w"/>  

Including unparsed "character" file: cinclude tag

The cinclude tag not only literally includes the file specified by the file attribute, but does this without parsing its content. This is useful to directly add content of this file to an output file.

The "c" in "cinclude" alludes to CDATA, meaning it is a "character"-oriented (rather than XML-oriented) operation.

Conditional processing: define, if/else and comment tags

This is similar to conditional compilation in many programming languages:
 <define name="tropical"/>

<if defined="tropical">
  <macro name="fruit">Banana</macro>
  <macro name="fruit">Tangerine</macro>
  <macro name="fruit">Orange</macro>
<else/>
  <macro name="fruit">Cherry</macro>
  <macro name="fruit">Apple</macro>
</if>  
As macros are invoked in a different order than they are placed in the input file, placing define and if tags inside macros might lead to a difficult interpretation of things, and is strongly discouraged (although not disabled, for the benefit of those who desperately want to shoot themselves into their foot). A warning is issued if a define is encountered for a symbol, which has already been tested by an if.

Alternative forms of if

Since v2.12, there is also an <if has_item="name"/> form, which in macros used with table allows to selectively emit text depending on whether the named item is present in current table row or not. See also the
has_item attribute used with use . Since v2.13, <if is_param="name"/> in macros allows to selectively emit text depending on whether the named param is present in the "calling" use.
Since v3.00, after pass attribute has been added to use, both <if has_item="name"/> and <if is_param="name"/> search only in the set of params/table row for the current level of use. But, there is a new form, <if param="name"/> , which searches for a param or table item through nested levels of use/param, following the same rules as the use-form of param (or use with param attribute).
Since v3.00, there is an <if iter="value"> form of if, to be used within macros invoked by use with table. "value" may be "0" or ">0"; the outcome is then true/false respectively for the first "invocation" of that macro within a table, and false/true for subsequent rows. This feature may be useful for inserting separators (commas, semicolons) between items produced by use table. (See also param attrib="iter".)

comment

comment is just another tag, which content is simply ignored:
 
<comment>
  This text is completely ignored.
  <macro name="fruit">This macro is completely ignored, too.</macro>
</comment>  

comment-s and if/else-s can be freely nested to each other into any levels.

Miscellaneous: nyweb tag

The nyweb tag is used for various ad-hoc functions, determined by the attribute.

Unused macros list

Macros unused yet can be printed using nyweb tag with list="unused macros" attribute. It is wise to place this tag at the end of the source .w file:
 <nyweb list="unused macros"/>  

List of (header) tags

All header tags (i.e. those enclosed within <hN> and </hN>) from the source .w file can be listed in parseable form together with file name and line number, into an output file with nyweb tag with list="headers" attribute. This allows to input this file into some of the editors/IDEs as an "error" output, to facilitate "jumping" at a given position in the .w sources. The location of this tag in the source .w file is arbitrary.
 <nyweb list="headers" [file="filename"] />  

Timing

Duration of processing of part of source file can be printed out, using a couple of nyweb tags with time="begin" and time="end" attribute. In the latter tag, a file attribute determines the output file name where the timing is written to. Two times are written, one for each processing pass. It is wise to place this couple of tags at the beginning and end of the source .w file:
 <nyweb time="begin"/>
<nyweb time="end" file="timing.txt"/>  
Note that the total of two times even in this case will be slightly less than the total duration of nyweb run - the time needed to check the temporary output file(s) against its older version (see emit tag) is not included.

Usage tips

nyweb and C

The necessity to use escape sequences for & and < gets pretty soon annoying, when writing in C. These two characters are used quite often as operators - < as less-than, less-or-equal and left-shift; & as bitwise and logical AND and the reference operator. As an additional annoyance, if the escaped versions are used in the .w source, this cannot be simply copy/pasted into a .c source say in a different project.

While there is no good solution to this problem (until a full-featured nyweb-friendly editor gets available, which is nowhere since now), it can be partially side-stepped by replacing the operators in question by predefined macros. These macros can then be stored in a "compatibility" header, which can be easily made be available in its C version too. Such header may then contain macro definitions such as (in .w notation):
 
    #define ANL &amp;&amp;
    #define AND &amp;
    #define OR  |
    #define ORL ||
    #define SHL &lt;&lt;
    #define SHR >>
    #define XOR ^

    #define LT  &lt;
    #define GT  >
    #define LE  &lt;=
    #define GE  >=
    #define EQ  ==
    #define NE  !=

    #define PTR &amp;

  
(while definitions for OR, SHR etc. are not necessary, they may be then used for "symmetry" reasons).

Planned features

Changelog


- v 1.01 - added a diagnostic dump of current escaped character upon errors in parsing escaped character

- v 1.02 - added input file "caching"

- v 1.03 - added <nyweb list="headers"> tag

- v 1.04 - slightly improved error checking for macro and use tags and more informative output printouts for the most common types of error
         - bug fixed (r1.w) - undefined tag in emit crashed nyweb


- v 1.05 - memory "leak" fixed: input file "cache" destroyed at end of program

- v 1.05 - params behaviour fixed, now param expansion (use) inside param is possible

- v 1.06 - added alternative tags:
         - "use macro" as an alias for "use name"
         - empty "use param" as an alias for empty "param" (i.e. param "invocation")

- v 1.07 (20101230)
         - added define_after_used check and corresponding warning; removed warnings for defines/if-s inside macros

- v 1.08 (20110331)
         - added htmlize attribute to use tag
         - added dependencies attribute to emit tag

- v 2.00 (20120510)
         - implemented "table" and related - see chapter "Tables and their use"
         - fixed line number printed with warnings/errors (was one off)

- v 2.01 (20121018)
         - implemented simple filters for table - "row" and "has_item" attributes
  
- v 2.02 (20121018)
         - implemented CDATA to the base XML parser
  
- v 2.03 (20130709)
         - implemented cinclude tag
         - fixed minor bug which prevented include tag directly in emit

- v 2.04 (20130812)
         - added cmacro

- v 2.05 (20140316)
         - enhancement: row and has_item on tables with no such row now cause the whole use to be ignored
         - added whitespace stripping attributes to use/param tags
  
- v 2.06 (20140613)
         - added <param attrib="order">

- v 2.07 (20140714)
         - added nowarn attribute to param and use

- v 2.08 (20140715)
         - added <param attrib="iter">

- v 2.09 (20141022)
         - improved table handling in use

- v 2.10 (20150202)
         - added scrlf attribute to use/param tags
         - all "strip" attributes now apply to all nested levels

- v 2.11 (20150306)
         - added "maxlen" attribute to use/param tags
         - improved cmacro end parsing
         - implemented "has_item_not" attribute for tables

- v 2.12 (20150323)
         - added "has_item" form of "if" tag

- v 2.13 (20160721)
         - added "is_param" form of "if" tag

- v 2.14 (20160725)
         - added <param attrib="seq">

- v 3.00 (20160805) 
         - added "param" form of "if" tag
         - added "iter" form of "if" tag
         - implemented table and macro indirection through "table"/"macro" attribute in "item"/"param" tag
         - implemented parameter passing in nested "use"/"param" through "pass" attribute of "use"
         - added <param attrib="debug">
         - various bug fixes, mainly for nested use/param chains

- v 3.01 (20160812) 
         - allow_recursion attribute added to use
         - bugfix (param attrib fail when no table in use)

- v 3.02 (20161121)
         - implemented has_item/has_item_not in table-form of "item/param"

- v 3.03 (20161201)
         - added "deep" attribute of the use-form param