Introducing Features for Superlanguages

Modern programming languages increasingly contain features that allow the host language system to be extended by the programmer. The requirements for extensibility stem from a desire to allow problem and solution domain features to be represented directly or declaratively in the program and for the execution engine of the host language to directly execute new features, thereby having knowledge of the language extensions. Features such as meta-programming in Smalltalk, the CLOS MOP, reflection in LISP and Java, Java Annotations, C++ Templates etc., in addition to features of more specialized languages, all offer various ways to achieve extensibility. However these languages are complex, and extensibility features are often incomplete. We will use the term superlanguage to describe a programming system that can be extended. This article is the first of a series that describes a simple core language that captures the essential features of being a superlanguage and shows examples of how it can be used to build languages from modular units.

A superlanguage offers the following features:

• Concrete syntax processing. Such a feature must allow new language constructs to be defined.
• First-class language modules. Language definitions should be data values, subject to normal scoping rules, parametric etc.
• Meta-Programming. Arbitrary computations can be performed when processing syntax in order to process declarative language constructs.
• Access to the abstract syntax ADT of host programming language that can be used to construct and transform arbitrary programs.
• Control over the scope of languages so that, once defined, a new language feature can be placed in scope for a given extent of program text.
• The ability to construct recursive definitions including those that involve grammars and language constructs.

This is the first in a series of posts that analyze essential features of superlanguages. More detail about superlanguages and a simple example language is found here.