Vip - Vi-Style Editor in PicoLisp

A complete editor in less than 2000 lines of code

There is an old saying: Once in your life you should build a house, plant a tree, and write an editor. I decided to start with the last one.

I implemented a subset of Vim in PicoLisp. It feels and behaves almost identically to what I'm used to, and what I have configured in my .vimrc. It knows a lot more than Vim about PicoLisp and the currently running program. I never really understood the quirks of vimscript, or why the bottom command line of Vim does not allow editing in Vi-style. And it is easier for me to write the whole beast in PicoLisp anyway.

Not that the world needs yet another editor! Rather, I see Vip as a proof of concept, and as a collection of examples for PicoLisp coding. And of course it is fun, and finally evolved into a full PicoLisp-IDE.

The "P" in "Vip" may mean "PicoLisp", but also "personalized". It does not even attempt to cover the full power of Vim, and is not intended as a replacement. But it is now my default editor for PicoLisp code and e-mails, and of course for writing this article. So I should first report what it does not support: (and probably other things which I did not care about)

Simplicity was a major design goal, perhaps at the cost of efficiency. This may result in poor performance when editing large files with Vip.

On the other hand, there are some dedicated PicoLisp features not easily available in Vim (at least I could not figure out how to do them in vimscript): In any case, Vip supports most expected features of a usable Vim subset, including:

The Source Code

Vip is included in the PicoLisp distribution. It consists of the library in "@lib/vip.l", and two executable scripts "@vip" and "@bin/vip". These scripts are analog to the "pil" interpreter startup scripts: "@bin/vip" is for calls in a global installation, and "@vip" for calls with relative or absolute path names in local installation(s).

Also, there is an an extension in "@lib/vip/draw.l" for drawing s-expressions as ASCII graphics.

The first versions used the "ncurses" library, which is still the standard Unix way of character screen handling. But as curses turned out to be a mess, it now uses only direct ANSI escape sequences (VT-100).

If a file ~/.pil/viprc exists, it is 'load'ed as a config file. It may contain arbitrary configurations and extensions. There is an example file in "@doc/viprc.sample" in the PicoLisp distribution.

Vip defines two classes for buffers and windows, 
   (class +Buffer)
   ...
   (class +Window)
   ...
A +Buffer holds the edited text in the text property as a list of lists of characters. Each character is a transient symbol, storing markup information in its value.

The markup is generated by the markup function. This function scans the whole text (a relatively expensive operation) using a 'finite state' machine. As a result, each character is known to be either normal text, the member of a string, or of a comment of a certain nesting depth. This information is used later for highlighting strings (with underlines) and comments (cyan color), and to handle indentation and matching parentheses.

A +Window displays the contents of a buffer. There may be multiple windows on a buffer, but no window without a buffer. Each window is linked to its neighbors with the prev and next properties.

The bottom window, which is created implicitly with a default size of one line, is called the "command window" (stored in the *CmdWin global). It can get permanent focus with normal window-change commands (e.g. qj or ^Wj), or temporary focus with commands like :, /, ?, & or SPACE in a text window.

All windows except the command window have a status line at the bottom. It displays (from left to right) If the path name is too long, the status toggles between path-only and the other informations upon each move or edit operation.

The goto function is the main workhorse to move around (jump, scroll) in the buffer and to display its contents.

getch waits for - and returns - key-presses (NIL if Escape was hit).

Modifications of the text go through change, undo and redo, to keep track of the undo and redo stacks for each buffer. The changes are done with a mixture of destructive and non-destructive list operations, to keep the amount of garbage small, yet still have data fragments which are shareable across the undo/redo steps. Usually this is achieved by destructively modifying a single cell in the top level list, saving its CAR and CDR for undo/redo.

Movement functions like goLeft, goFind or word are the primitives to be combined in move with change, delete, yank and external filters, optionally with a count and other informations. It is important that they are usually never executed directly, but used to build - sometimes with the help of 'curried' functions - executable expressions which are indeed executed, but also kept in the *Repeat global to be repeatable with the . dot command.

command interprets and executes text in the command window, and the long loop at the end of the source is the main program. Start with inspecting the case statements to find out available commands, and how Vip handles them.

Running Vip

To run Vip on your machine, three conditions must be satisfied: The UTF-8 locale is a general PicoLisp requirement. Due to the use of namespaces, native calls and SIGCONT handling, Vip runs only on pil21.

If you have a global installation of PicoLisp on your machine, you can just move the file "@bin/vip" into some directory in your execution $PATH, and you are done.

Alternatively, you can install PicoLisp locally, and put symbolic links into /usr/bin and /usr/lib as described in INSTALL or adjust the hash-bang in the first line to point to your local installation.

The easiest is defining an alias in your shell, e.g. 
   alias vip=~/pil21/vip


Invocation from a Shell

You can invoke Vip either as a shell command or with the function vi in a REPL.

To start Vip on the shell, call it with one (or more) file names: 
   $ vip file.l
It will display the first file in the large main window above the single-line command window. If called without arguments, it opens an empty temporary file "~/.pil/tmp/<pid>/empty".

Optionally you can pass a search pattern with + 
   $ vip +word file.l
causing to put word into the search buffer. Hitting n then jumps directly to the next exactly matching word.

If the + is instead followed by a number 
   $ vip +123 file.l
then Vip immediately jumps to that line number, or with 
   $ vip + file.l
to the end of the file.

If called in a pipe 
   $ echo ok |vip
it opens a temporary file "~/.pil/tmp/<pid>/stdin".

You can pass any number of files on the shell command line. With 
   :n
or F6 you can switch to the next file(s), and with 
   :N
or F5 you can step backwards through the files.

Invocation from a REPL

Vip is available if PicoLisp was started in debug mode. 
   $ pil +
To edit a file, pass it as a single argument: 
   : (vi "file.l")
   -> "file.l"  # Return value is NIL if ":q"
You can edit the definition of a Lisp symbol 
   : (vi 'pad)
which may also be a built-in function 
   : (vi 'car)
or a method definition 
   : (vi 'put!> '+Entity)
You can then move the cursor onto other symbols in the source, and press K (or ^] like in Vim) to jump to the definition of those symbols. Q (or ^T) pops you back.

If the cursor is at the beginning of an s-expression (open parenthesis or the first character of an atom), ^E evaluates that expression and displays the result in the command window.

gf opens the file whose name is under the cursor.

A backslash \ switches to the most recent buffer in this window, and if you prefix with a count, the corresponding buffer is selected.

Direct Editing of Lisp Symbols

If you pass - instead of individual symbol(s) - a list of symbols to vi 
   : (vi '(sym1 sym2))
these symbols are not used to open a file, but instead can be edited directly.

This means, their names, values and property lists are pretty-printed to a temporary file, that file is opened in a buffer, may be edited normally, and when changes are saved with :w or :x etc., the values and/or properties in the symbols may be modified.

This allows quick changes without touching the original sources, like inserting debug breaks or diagnostic messages.

And it is very convenient to browse the database (external symbols).

The K command now adds the symbol under the cursor on top (instead of jumping to a source file).

There is a non-evaluating function v as a shortcut to the above call with a list of symbols: 
   : (v sym1 sym2)
This makes it convenient to copy/paste e.g. external symbols (which are otherwise tedious to type in) from debug screens to the command line.

Viewing Directory Contents

Whenever a directory is accessed instead of a normal file (e.g. on the shell command line, with the :e <file>} command, or by pressing gf on a path name, a listing of that directory - sorted by modification date - is displayed in an editable buffer.

This happens in two modes: Flat or Recursive. If the specified path name ends with a slash /, the directory is displayed in Flat mode, listing only the top level files and subdirectories. You can now press gf on files to open them normally, or on directories (which are formatted with a slash at the end) in Flat mode again.

If a directory name without a trailing slash is accessed, it is opened in Recursive mode, showing all files in this directory and all subdirectories, again sorted by modification date. This is very useful to quickly see in a project tree which files were recently modified.

You can toggle Flat and Recursive display with the :E command.

Vip as a Coroutine

Technically, Vip is alway running as a 'coroutine' in the current process.

This makes it possible during development and debugging of a project to press qz, to suspend it temporarily and drop back into the normal REPL. This is similar to Unix job control, where ^Z suspends the current foreground process and drops into a shell.

Vip can be resumed later, by calling 
   : (v)
It will continue at exactly the same point, with all buffers and windows preserved.

Window Management

So qs (or ^Ws) splits the current window into to halves, and you can scroll to other parts of the file, or open other files with :n or e <file> etc.

qx exchanges the contents of two windows, qk changes the focus to the window above, and qj to the window below (possibly the command window).

qq closes the current window, or Vip if this was the last (non-command) window.

+ increases the size of the current window, and - decreases it. = makes all windows of equal size.

Tag Stack

K (or ^] like in Vim) opens the source of the symbol under the cursor in the current window. Note that this works only when Vip was started in debug mode. Either in a debug REPL, or on the shell as: 
   $ vip file1.l file2.l +


With Q (or ^T like in Vim) the tag stack can be popped, going back to the previous location.

Note that this "tag" functionality does not use a tag file. Instead, the debug properties of the internal symbols are used directly.

Differences to Vim

There are some deliberate - though not mission-critical - differences in behavior between Vim and Vip. Some of them are because I prefer them this way, and some as a consequence of the implementation.

One is the w word motion command. The normal Vi/Vim philosophy is that "change", "delete" etc. operate on the range specified by the motion. w, however, behaves differently in Vi/Vim for moves (going to the beginning of the next word) and changes (going only to the end of the current word).

To keep Vip simple, and also for reasons of consistency, a command like cw or dw will change the text up to including the first character of the next word. This is usually not what you want. Instead, use ce (change to end of word) or de.

A consistent addition is TAB in command mode, which goes to the last space before the next word. so cTAB or dTAB will more behave like cw and dw in Vim.

Similar differences may also exist for other motion commands. As a rule, assume that Vip simply does what follows from the pure motion logic.

When undoing a change operation, it often requires 2 presses of u, because the removal of the old and the insertion of the new text are stored internally as separate steps. This is a bit odd at first, but turns out useful in certain situations.

Not really a difference is the behavior of the left and right arrow keys. To have the same behavior in Vim, I have 
   map <LEFT> zh
   map <RIGHT> zl
in ".vimrc".

Another difference is the behavior of the command window.

The Command Window

Vip does not implement the legacy ex commands which are available in Vi/Vim.

Instead, we can consider the command window as a specialized REPL. It takes commands in special formats, and evaluates them.

In some cases, it behaves very similar to Vi. For example, pressing : in a text window will cause the focus to go to the command window, display a colon, and wait for a command.

One important difference is: Writing ": " (colon plus space) followed by a Lisp expression causes this expression to be evaluated in the command window. 
   : (* 3 4)
   -> 12
If immediately following the colon (no space), currently these commands are supported: For some commands (e.g. ls) it may be useful to increase the size of the command window.

Similarly, pressing / or ? in a text window will cause the focus to go to the command window, and start a search. Note, however, that regular expressions are not supported. You can either search for a constant text (case sensitive), or use @ in the search pattern as a wildcard.

In any case, the command window can be edited just like any other window, to go back in the "history", and individual lines can be re-executed by pressing ENTER.

Other Commands

The , comma command is redefined in Vip to fix the indentation of a paragraph (i.e. until the next empty line) according to the PicoLisp rules (3 spaces per level). It uses the internal markup information to handle strings, (nested) comments and (super)parentheses.

F1 toggles the markup highlight display (strings, comments) on and off.

F2 shows the differences (changes) of the current file and a file with "-" appended to the name (see the bak and kab commands above).

F3 is similar, but calls a custom "dif" program. I use it with dedicated scripts which show the differences to files on other machines and/or in the incremental backups. The count prefix is used then to select the backup version.

F4 formats the current paragraph, calling the Unix "fmt" command.

F5 steps backwards through the buffers.

F6 steps forward through the buffers.

F7 through F12 may be defined in ~/.pil/viprc.

https://picolisp.com/wiki/?vip

07apr24    abu