Core PDML Specification

Overview

In this chapter you'll be presented with a birds-eye view of Core PDML, in order to become acquainted with its terminology and rules, and grasp a general understanding. Throughout the following chapters, each element will be explained in full detail. You may skip this chapter if you're already familiar with the basic Core PDML concepts.

A PDML document is structured as a tree that can contain three types of nodes:

• Tagged Branch Node

  This node has a tag (aka label or name) and one or more child nodes.

  A tag is a sequence of one or more Unicode code points (e.g. config, chapter).

  Each child node can be anyone of the three node types.

  Syntax: "[" tag separator node + "]"

  The separator can be a space, tab, or line break.

  Example

  You can use tagged branch nodes, for instance, to encode a record (aka structure, struct, or compound data). Here's a simple example of a node tagged dimensions containing two child-nodes, the first tagged width and the second one height:

  [dimensions [width 200][height 100]]

  In other words, the above code represents a record (tagged dimensions) with two fields: width and height. Each field is a key–value pair. In our example, the key width contains the value 200).

  We can enhance readability by indenting nested nodes:

  [dimensions
      [width 200]
      [height 100]
  ]

  In XML, the semantically equivalent tree looks like this:

  <dimensions>
      <width>200</width>
      <height>100</height>
  </dimensions>

  And here's the same data structure expressed in JSON:

  {
      "dimensions": {
          "width": 200,
          "height": 100
      }
  }

• Tagged Leaf Node

  This node has a tag, but no child nodes.

  Syntax: "[" tag "]"

  Example

  Tagged leaf nodes are used to encode record fields that have no value (aka "absence of value"; null, nil, nothing, void in programming languages).

  Suppose that a user didn't enter a value for field remark in a data entry form. Then the code representing this field (i.e. its key–value pair) would look like this:

  PDML: [remark]

  XML: <remark></remark> or <remark />

  JSON: { "remark": null }

  Besides being used for record fields with no values, tagged leaf nodes are also useful to represent native elements, especially in markup documents.

  For example, a horizontal rule (hr) in a markup document would be inserted using the following syntax:

  PDML: [hr]

  HTML: <hr /> or <hr>

• Untagged Text Leaf Node

  This node is composed of text (Unicode code points), and it doesn't have a tag.

  Example

  Untagged text leaf node: Lorem ipsum ...

  An untagged text leaf is always contained in a tagged branch node. Here's a branch node tagged p (paragraph) containing the untagged text leaf Lorem ipsum ...:

  PDML: [p Lorem ipsum ...]

  HTML: <p>Lorem ipsum ...</p>

The root node of a PDML document must be a tagged branch node or a tagged leaf node.

Here's a PDML document containing a mixture of data and markup code:

[document
    [data
        [message
            [id 123]
            [content All is well! 👍]
            [public? yes]
            [remark]
        ]
    ]
    [markup_code
        [p We can write text in [b bold], [i italic], or [b [i bold and italic]].]
    ]
]

The following image shows the tree structure of node data, stripped of indentation:

A PDML document is structured in a way that resembles the tree of directories and files on your PC.

To help you grasp the essence of PDML terminology, the following analogies may prove useful:

• A directory containing other directories and/or files is like a tagged branch node in PDML: It has a name (called tag in PDML) and child-nodes.

• An empty directory is like a tagged leaf node in PDML: It only has a name (a tag).

• A non-empty text file is like an untagged text leaf node in PDML: It contains a sequence of one or more characters. However, while a file has a name, an untagged text leaf has no tag.

Basic PDML Terminology

Some commonly used IT-terms are defined as follows in the context of PDML:

• Character

  A single Unicode code point.

  Examples: a, Z, é, 日, 0, 9, !, 😀

• Text

  A sequence of one or more characters.

  Examples: Hello, Guten Tag!, やあ, a = b + 1

• Whitespace character

  Any one of the following characters:

  Unicode Code Point	Unicode Name	C-style Syntax
  U+0009	Character Tabulation (HT, TAB)	'\t'
  U+000A	End of Line (EOL, LF, NL)	'\n'
  U+000C	Form Feed (FF)	'\f'
  U+000D	Carriage Return (CR)	'\r'
  U+0020	Space (SP)	' '

• Whitespace

  A sequence of one or more whitespace characters.

• Line break

  One of the following:

  • LF: a single End of Line character (U+000A), also called Unix/Linux line break

  • CRLF: a sequence of a single Carriage Return character (U+000D) followed by a single End of Line character (+U000A), also called Windows/DOS line break

Tree Structure

As mentioned in Overview, a PDML document is structured as a tree that can contain three types of nodes:

• Tagged Branch Node: has a tag and one or more child nodes.

  A child node can be anyone of the three node types.

• Tagged Leaf Node: has a tag, but no child nodes.

• Untagged Text Leaf Node: is composed of text.

The root node must be a tagged branch node or a tagged leaf node.

quality
Qualität               // German for "quality"
คุณภาพ                 // Thai for "quality"
text_color
list.index-a
123
2d_point
2025-01-07
_
😍                     // Unicode emoticon

Net\sWeight

Net Weight
[kg]

Net\sWeight\n\[kg\]

"[" tag separator node + "]"

[greeting Hello world]

[color green]
      ^

[color  green]
      ^^

[color green]

[color
green]

[color
    green
]

[b [i huge]]
  ^  ^

[b[i huge]]
  ^          INVALID!

[foo [key1 value1] [key2 value2]]
                  ^

[a\sb c]

"[" tag "]"

[remark]

[remark ]
       ^    INVALID! 

[remark  ]
       ^^

[message All is well! 👍]

[ข้อความ ทุกอย่างดี 👍]

[1 2 3]

[warning Characters \\, \[, \], and \^ must be escaped.]

Invalid Characters

The following characters (Unicode code points) are invalid anywhere in a PDML document (including tags and text leaves), because they can cause readability, interpretation, and security issues.

• C0 Controls in the range U+0000 to U+001F (included), except:

  • U+0009 (Character Tabulation)

  • U+000A (End of Line)

  • U+000C (Form Feed)

  • U+000D (Carriage Return)

• C1 Controls in the range U+0080 to U+009F (included)

Unicode surrogate code points in the range U+D800 to U+DFFF are also invalid anywhere in a PDML document, because they are reserved for encoding Unicode code points beyond U+FFFF in UTF-16 encodings.

Character Escape Sequences

Some characters in tags and text leaves must be escaped.

A character escape sequence starts with a backslash (\), followed by a second character that denotes the escaped character (as in C-like programming languages).

For example, the character escape sequence \t is used to escape a Character Tabulation (TAB).

The following table shows the character escape sequences:

All characters in the above table must be escaped in tags.

However, only characters in rows marked as "yes" in the last column must be escaped in text leaves. Characters in rows marked as "no" may be escaped.

For instance:

• The text snippet a[1] must be written as a\[1\] in tags and in text leaves.

• The text snippet a=b must be written as a\=b in tags; in text leaves it can be written as a=b or a\=b.

Escape sequences not listed in the above table are invalid and parsers must generate an error if they occur in a PDML document. For example, if \m occurs in a PDML document, the parser must generate an error — it must not convert \m to m or ignore the invalid escape sequence.

A node tagged Note (important) with text content Characters \, [, ], and ^ must be escaped. is written as follows:

[Note\s\(important\) Characters \\, \[, \], and \^ must be escaped.]

Whitespace Rules

A PDML document may contain whitespace before or after the root node. Any other character is invalid (including escaped whitespace characters like \t and \n). Whitespace before or after the root node must be ignored.

Here's a valid PDML document that has one empty line before the root node, and two empty lines after it:

[root
    [child text]
]

Whitespace within PDML documents (i.e. the text contained between the opening and closing delimiters of the root node) must be preserved by parsers.

Consider the following PDML snippet:

[a  foo   [b]
    2 [c] [d]
]

Here, node a contains seven child nodes:

• text leaf {space}foo{space}{space}{space}

• tagged leaf node b

• text leaf {line break}{space}{space}{space}{space}2{space}

• tagged leaf node c

• text leaf {space}

• tagged leaf node d

• text leaf {line break}

Note that all line breaks are stored in the AST as they are encountered, without any transformations such as normalization to LF. Hence, an AST might contain a mixture of Unix line breaks (LF) and Windows line breaks (CRLF).

Encoding

PDML documents must be encoded in UTF-8.

Versioning

This specification adopts Semantic Versioning.

Useful Links

• Core PDML Grammar, expressed in Extended Backus–Naur form (EBNF) notation

• Official website: pdml-lang.dev

• Core PDML Examples

Glossary

[size big]
^        ^

[color dark green]
      ^

[doc
    [data ...]
    [markup ...]
]

[pi 3.1415]
 ^^

[dimensions
    [width 200]
    [height 100]
]

Length Limits

The specification does not define a length limit (i.e. maximum number of characters/Unicode points) for tags, text leaves, or the whole document. Practical limitations depend on software implementations (e.g. parsers) and should be documented.

In case of very large PDML documents, you may encounter performance issues or out-of-memory errors, depending on the capabilities of the system.

If you create a parser designed to read very large documents, consider creating a streaming parser that processes the document in smaller chunks. For example, instead of loading the whole document into memory, the parser only holds the currently parsed token in memory (e.g. a single tag or text leaf). The PDML reference implementation includes a streaming parser that uses this technique and generates events to be consumed by a handler.

Tags

[Net\sWeight\s\[Estimate\] 200]

["Net Weight [Estimate]" 200]

[names
    [name Tim]
    [name Tom]
    [name Tam]
]

[names
    [_ Tim]
    [_ Tom]
    [_ Tam]
]

[map
    [one 1]
    [two 2]
    [one 1]
]

How to Handle Whitespace

IT history has repeatedly proven that whitespace (though seemingly harmless) can cause unexpected results and malfunctions when handled incorrectly. Problems and unanticipated edge cases can arise due to the following factors:

• There is no universal, precise, one-size-fits-all definition for the term whitespace in the IT world — this term can mean different things in different domains and contexts.

• To enhance readability and maintainability, we often insert insignificant whitespace (e.g. indents) into text documents containing data or markup.

• Line breaks are defined differently in Unix/Linux (LF) and Windows (CRLF).

  Moreover, editors and other tools might silently transform line breaks (e.g. CRLF to LF), and these transformations depend on the tools used, the OS, as well as configuration files installed on your PC.

• Editors and other tools often render tabs as spaces, making it difficult for humans to distinguish between the two.

  Software applications might also silently transform tabs to spaces, or vice versa.

To help you avoid unexpected issues, this section offers some tips for managing whitespace in PDML documents.

[2d_point [x 1.234][y 567.8]]

[2d_point
    [x 1.234]
    [y 567.8]
]

[2d_point
    [x   1.234]
    [y 567.8  ]
]

[article [title Harmonious States][p This article [i demonstrates] that ...][p You'll also discover ...]]

[article [title Harmonious States]
    [p This article [i demonstrates] that ...]
    [p You'll also discover ...]
]

[article [title Harmonious States]
        
    This article [i demonstrates] that ...
        
    You'll also discover ...
]

[birthdate
    2000-01-01
]

line 1
line 2

line 1\r\nline 2

line 1\nline 2

Encoding Binary Data

To encode binary data in PDML documents (e.g. images, audios, and videos), you can use common and portable binary-to-text encoding methods like Base64.

The following example shows a tiny (1x1) PNG image encoded in Base64:

[image
    [title Example Image]
    [format PNG]
    [data
        iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwC
        AAAAC0lEQVR42mP8/wQAAwAB/AL+j3sAAAAASUVORK5CYII=
    ]
]

Copyright

Copyright © 2019-2025, Christian Neumanns.

Everyone is permitted to:

• Copy and distribute verbatim copies of this document (changing it is not allowed).

• Create verbatim translations of this document into other human languages and distribute them.

• Create verbatim conversions of this document into other text formats and distribute them.

Acknowledgments

Many thanks to Tristano Ajmone for his useful feedback to improve this document.

Markup

This document is written in PML, a lightweight markup language based on PDML.

Revision History

• 2025-xx-xx: version 2.0.0

• 2021-12-03: version 1.0.1

• 2021-06-13: version 1.0.0