Some of the code in this article may need updating?
A Unifying Language for Database and User Interface Development(c) Software Lab. Alexander Burger
IntroductionIn this pager, a language framework is presented which closes the semantic gap between database, application logic, and user interface. We introduce the concepts of Prefix Classes and Relation Maintenance Daemon Objects to suggest a unified development style.
We use the PicoLisp interpreter to build a vertical, unified solution to these problems. It allows to describe a direct mapping between application structures and database objects, so that the underlying machine can handle the most tedious parts of DB/GUI programming automatically.
The solution is "vertical", because it extends from the virtual machine level up into the application and GUI levels. And it "unifies", in a consistent way, the Lisp interpreter with
- a flexible object architecture
- an object-oriented database
- relation maintenance daemon objects
- a Prolog-equivalent query language.
- and a user interface strategy
PicoLisp was chosen as the base language, because its interpreter is simple and completely written in C (or asm for the 64-bit version). This makes it easy to incorporate the necessary extensions to the virtual machine. Besides this, two other features (of Lisp in general):
- dynamic data types and structures
- and formal equivalence of code and data
Object ArchitecturePicoLisp employs a very simple object architecture. It uses symbols for the implementation of both classes and objects. In PicoLisp each symbol has a value cell, a property list, and a name.
For a symbol representing an object, the value cell holds a list of the object's classes, the property list holds the object's attributes (instance variables), and the name is usually empty (anonymous symbol).
For a symbol representing a class, the value cell holds an association list with the class' methods, concatenated with a list of the class' superclasses, the property list may hold some class attribues (class variables), and the symbol's name is the name of the class.
When a message (also a symbol) is sent to an object, that object's list of classes - and recursively those classes' superclasses - is searched from left to right (in a depth-first manner) for that message, and the corresponding method body is executed. (In effect, this is a multiple-inheritance late-binding strategy.)
In that way, a class +MyCls can be defined to inherit from three classes +Cls1, +Cls2 and +Cls3 (by convention, class names start with a `+'):
(class +MyCls +Cls1 +Cls2 +Cls3)Then, an object can be created with
(new '(+MyCls))or another object with equivalent behavior:
(new '(+Cls1 +Cls2 +Cls3))In both cases the resulting objects will inherit method definitions from +Cls1, +Cls2 and +Cls3. Because of the depth-first and left-to-right search order, however, methods in the class hierarchy of +Cls1 will override methods with the same name anywhere in the class hierarchies of +Cls2 and +Cls3.
This is a "horizontal" inheritance, as opposed to - and in addition to - the normal "vertical" inheritance. +Cls1 and +Cls2 can surgically alter the behavior of +Cls3, in a very fine-grained manner. Thus - as +Cls3 will typically define the general behavior - +Cls1 and +Cls2 are called "Prefix Classes" of +Cls3.
This object architecture is used throughout the whole system, including the DB and GUI. And prefix classes are an essential part of it: The expressive power of Lisp's equivalence of code and data is augmented in combination with prefix classes.
DatabaseThe PicoLisp database is built upon that object architecture.
On the lowest level, a database is a collection of persistent objects. PicoLisp supports persistent symbols, called external symbols, as a first-class data type. These symbols are known to - and handled in special ways by - the interpreter.
External symbols are stored in a file, in linked blocks of fixed size, where each symbols's starting block address is computable from the symbols's name.
A read or write access to an external symbol's value or properties causes that symbol to be automatically fetched from the database file. At the end of any transaction, modified symbols are written back (commit) or reverted (rollback). The garbage collector knows about the state of external symbols, and purges currently unused symbols from memory (Note: These symbols are only temporarily removed from memory, not from the database. The latter is done separately by a DB-level garbage collector.).
On the higher levels, these external symbols are organized into class hierarchies, reflecting the application's organizational structures.
As opposed to simple two-dimensional tables, they form arbitrary data structures like lists, stacks, trees and graphs.
The connections (relations) between objects in the database are not established by index lookup, but by explicit inclusion. That is, an object referring to another object can explicitly hold (i.e. contain a pointer to) that object, and can get access to it very rapidly. There is no explicit select operation, everything is simply available when it is needed.
Relation DaemonsGenerally, instances of persistent database objects are called "Entities". In our system, there is an additional separate class hierarchy of "Relations": Instances of these classes we call "relation maintenance daemon objects".
Relation daemons are a kind of metadata, controlling the interactions between entities, and maintaining database integrity. They are the concrete realization of an abstract Entity/Relationship.
Like other classes, relation classes can be extended and refined, and in combination with proper prefix classes a fine-grained description of the application's structure can be produced.
Besides some primitive relation classes, like +Number, +String or +Date, there are
- relations between entities, like +Link (uni-directional link), +Joint (bi-directional link) or +Hook (object-local index root)
- relations that bundle other relations into a single unit (+Bag)
- a +List prefix class
- prefix classes that maintain index trees, like +Key (unique index), +Ref (non-unique index) or +Idx (full text index)
- prefix classes which in turn modify index class behavior, like +Sn (soundex algorithm (Donald E. Knuth: "The Art of Computer Programming", Vol.3, Addison-Vesley, 1973, p. 392) for tolerant searches)
- a +Need prefix class, for existence checks
- a +Dep prefix class controlling dependencies between other relations
(rel attr (+Cls ..) Arg ..)rel is called with an attribute name attr, a list of relation classes (+Cls ..) and - depending on these classes - other optional arguments. Basically, this function simply does a
(new '(+Cls ..) Arg ..)and assigns the resulting daemon object to the attr-property of the current entity class. Relation Prefix Usage
For example, a simple entity "Person" can be defined, having just a "name" attribute:
(class +Person +Entity) # class '+Person' (rel name (+String)) # relation 'name'If the name relation needs a unique index, it is written as:
(rel name (+Key +String))And - for an extended example of prefix classes - if name should be a mandatory list of names, each with an index using the soundex algorithm:
(rel name (+Need +List +Sn +Idx +String))This demonstrates the power of combined prefix classes, which allow to define complex object behavior "on the fly", without the need to leave the current programming focus. This is even more important in user interface programming (see section GUI Integration).
Entity LinkageA uni-directional link to another entity might be, for example, the person's address:
(rel adr (+Link) (+Address))This assumes that some entity class +Address is defined:
(class +Address +Entity)In reality, the relation will probably be bi-directional, with several persons living at some address:
(class +Person +Entity) (rel adr (+Joint) prs (+Address)) (class +Address +Entity) (rel prs (+List +Joint) adr (+Person))This says: The adr-attribute of +Person (a +Joint) is an address (connected to the prs-attribute of +Address), while the prs-attribute of +Address (a +List +Joint) is a list of persons (connected to the adr-attribute of each person).
So, when a person's adr is assigned to some address, the +Joint relation daemon will take care of updating the list of persons in that address, and vice versa. Query Language
For extensive searches in the database, as they are needed for reports or user-specified queries, a Prolog engine was incorporated into PicoLisp. Prolog is similar to SQL, due to its declarative nature, but much more powerful because of its rule-deriving and backtracking capabilities.
Prolog is easy to implement in Lisp (see J. A. Campbell: Implementations of Prolog, Ellis Horwood Limited, 1984). We extended the basic inference machine to iterate also through facts in the database, and the basic search/backtracking algorithm to a self-optimizing parallel search through multiple index trees.
The details of the query language are beyond the scope of this paper. It is mentioned here because our production system would be incomplete without it.
GUI IntegrationThe connection between database objects and GUI components is established with only two prefix classes: +E/R and +Obj.
Normally, GUI components are created at runtime with the gui function, e.g.:
(gui '(+TextField) "Text" 8) (gui '(+NumField) "Number" 8)for a text field and a numeric field, each 8 characters wide.
+E/R (Entity/Relation)The +E/R prefix connects the GUI field with a given relation of an entity:
(gui '(+E/R +NumField) '(n . Obj) "Number" 8)The specification (n . Obj) is passed as an additional argument. It indicates that this numeric field is "connected" to the n-property of the database object Obj.
Nothing else has to be done by the programmer. The field will automatically display the value of n from the database, and modifications entered by the user into this field will be written automatically to the database value of n, in the object Obj.
+Obj (Object)The +Obj prefix extends the GUI field types, from primitives like numbers or strings, to database objects.
That is, a GUI field can "contain" a database object, just like a text field contains a string, and a numeric field contains a number. An object can be "set" into (assigned to) the field, and retrieving the value of the field will result in that object.
The field cannot, of course, directly display the complete database object. But it can show a typical attribute of that object, e.g. the object's name in a text field, or the object's ID number in a numeric field:
(gui '(+Obj +NumField) '(id +Cls) "ID" 8)The +Obj prefix extends the capabilities of the basic field type (here a +NumField), in such a way that
- a proper attribute value is displayed when an object is assigned to the field
- the field is set to the corresponding object when a legal attribute value is entered
- the field performs keyboard auto-expansion to legal attribute values directly out of the database
- the field can display a choice list for matching attribute values
- the fields opens an editor for that object when it is double-clicked (Hyper-Link)
(gui '(+E/R +Obj +NumField) '(x . Obj) '(id +Cls) "Number" 8)they effectively establish the GUI for an entity linkage (+Link, +Joint etc.), as described in section Entity Linkage.
An ExamplePutting it all together, the total effect of the described concepts can best be explained with the help of a detailed and complete example.
Assuming a simple family database, we represent a network of family relationships. For each person, we have attribues for name, sex, date of birth, and we want to have links to father, mother, husband/wife and children.
For a better understanding, we first present the traditional, relational representation (Note that we have to introduce a unique ID number for each record. Also, in such a tabular representation, it is more convenient to store the ID of father and mother (instead of the children).):
ID nm sex dat mate pa ma 1 John M 22jan1954 2 2 Mary F 01feb1958 1 3 Thomas M 26jan1988 1 2 4 Claudia F 15jul1989 1 2 5 Michael M 03jan1992 1 2When viewing the these family members as a graph, we get the following object structure:
Example object structureThe bi-directional relations, connecting the parents to the children and to each other, lend themselves to the +Joint entity linkage, resulting in the following definition for a person:
(class +Person +Entity) (rel nm (+Key +String)) # Name (rel pa (+Joint) kids (+Man)) # Father (rel ma (+Joint) kids (+Woman)) # Mother (rel mate (+Joint) mate (+Person)) # Partner (rel dat (+Date)) # bornFrom this base class, we can derive two classes +Man and +Woman:
(class +Man +Person) (rel kids (+List +Joint) pa (+Person)) (class +Woman +Person) (rel kids (+List +Joint) ma (+Person))To produce a corresponding GUI...
It would be cool to link to a live example here, that people could play with
allowing to view and edit the family members in the database, the following code is sufficient:
(row (gui '(+E/R +TextField) '(nm : home obj) "Name" 20) (gui '(+E/R +DateField) '(dat : home obj) "born" 10) (gui '(+ClassField) '(: home obj) "Sex" '(("Male" +Man) ("Female" +Woman)) ) ) (----) (row (gui '(+E/R +Obj +TextField) '(pa : home obj) '(nm +Man) "Father" 20 ) (gui '(+E/R +Obj +TextField) '(ma : home obj) '(nm +Woman) "Mother" 20 ) ) (gui '(+E/R +Obj +TextField) '(mate : home obj) '(nm +Person) "Partner" 20 ) (---- T) (gui '(+E/R +Chart) '(kids : home obj) 4 '("Children" "born" "Father" "Mother") (quote (gui '(+Obj +TextField) '(nm +Person) "" 15) (gui '(+Skip +Lock +DateField) "" 10) (gui '(+ObjView +TextField) '(: nm) "" 15) (gui '(+ObjView +TextField) '(: nm) "" 15) ) )The above block of code will produce exactly the layout and functionality of the example GUI display. Without explaining all details here, suffice it to say that the row function arranges the components horizontally (while otherwise the default is vertically), (----) groups components into separate panels, and a +Chart creates an array of its argument components.
The point is that this is the complete program, not just some important details. It specifies the whole database and GUI application.
ConclusionThe previous sections and the example show that application programming does not need to involve any concerns about database access (select, insert, update etc.) and database integrity maintenance.
The advantages are derived from the use of prefix classes and relation daemons. They allow to specify the complete program behavior and appearance in a single place of definition, and in a very concise form. Typically, the names of prefix classes are simply chained together, and intermix freely with Lisp's formal indifference of code and data.
This removes the need of maintaining separate resource files, class and data declarations, and program code.
Start CodingHead over to PicoLisp Application Development to learn how to actually use the framework. And then start writing your own apps, or check out A Minimal DB/GUI Application for a detailed explanation of a tiny but useful PicoLisp app.
Finally, be sure to consult The Common Index as needed. There is a lot of documentation concerning the GUI framework, its functions and prefix classes.