Websockets with PicoLisp

See the pl-web page for practical terse info on how to setup, run the demo etc if you feel like tl;dr, you might be interested in the WSS section at the end though.

The typical demo of websockets is the realtime chat which PicoLisp has its own version of in the form of web.l's websocket demo.

However my initial requirement - which in the end is the reason for integration of websockets into pl-web - was a realtime notification system. Various events are generated on/for users via cron jobs, admin actions and user actions, the goal was for some of these events to popup immediately for logged in users.

As the main system in this case was based on LAMP the natural thing to do was to check if Apache implemented the protocol which the used version did not do at the time of writing. The next step was to look for a raw PHP solution of which there are several, the most developed one is Ratchet but Ratchet doesn't have a client which per the requirements is a necessity.

There were several clients written in PHP though, none of which were able to communicate with Ratchet, or were out of reach due to them requiring a too high PHP version.

It was at this point I decided to take a closer look at the websocket code of web.l which José graciously permitted me to copy into pl-web.

Using one of the downloaded PHP clients to send a chat message to the webtest.l demo I verified that I was able to communicate with José's setup. Notifications from PHP to the websockets server and onwards to the browser were therefore a possibility if the websockets server was made with PicoLisp.

As opposed to the webtest.l demo I would need to be able to keep track of the clients in order to be able to send notifications only to the person they pertain to. I would also need to enable administrators to be able to broadcast notifications to all users. In addition to this it would be good if users could be tagged so that the same websocket server could be used for many different services, for instance if a support chat is added in the future.

Therefore pl-web requires clients to register with a tag, a unique ID is optional, this ID should be impossible to guess, if supplied, otherwise unauthorized people could send messages to people they should not have access to. All client process IDs are then simultaneously kept track of with the help of two associative lists, one with the unique ID in the CAR and the PID in the CDR, the other with the tag in the CAR and the list of PIDs in the CDR. These two association lists will be periodically cleaned up to eliminate terminated PIDs in the worker websocket process, of which all client processes are siblings. Furthermore, once a registration has taken place all PicoLisp websocket processes periodically send ping calls to their clients to keep the connection alive.

Since non-human clients typically just connect, send their message/notification and disconnect it was necessary to implement the concept of actions in order to prevent them from being unnecessarily added to the above mentioned lists. Browser clients who maintain a persistent connection in order to listen for notifications send a "set" action in order to set the tag they will be listening to, ie being added to the tag list.

Finally, to enable some clients to send messages to all clients tagged with "notifications" while at the same time disable the same for other clients, some authentication logic was required. Certain tag and action combinations can be protected by password, or key if you will. Only clients supplying the password in their registration call are able to broadcast to that tag using that action.

WSS

Sockets created in the context of a page loaded through HTTPS need to be secure too. Luckily this is easily achived through the httpGate. The httpGate needs a single PEM file as argument which in my case was achieved through conatenating the various files like this: cat main_ca_cert.crt intermediate_ca.crt main.key > cert.pem.

Launching is then done like this: "bin/httpGate 9091 9090 cert.pem" if the websocket server is listening on 9090 (unencrypted) and you want httpGate to listen for encrypted traffic on 9091 and forward it to 9090.

Here is how all of the above, including WSS, can be setup from scratch with the 64bit version on Ubuntu 12.04:

apt-get install gcc make libssl-dev mercurial
wget http://software-lab.de/picoLisp-3.1.7.tgz
wget http://software-lab.de/x86-64.linux.tgz
tar -xzvf picoLisp-3.1.7.tgz
tar -xzvf x86-64.linux.tgz
mv picoLisp /opt/picolisp
cp src64/* /opt/picolisp/src64/
cd /opt/picolisp/src64/
make
ln -s /opt/picolisp /usr/lib/picolisp
ln -s /usr/lib/picolisp/bin/picolisp /usr/bin
ln -s /usr/lib/picolisp/bin/pil /usr/bin
cd /opt/picolisp
hg clone https://bitbucket.org/hsarvell/ext
hg clone https://bitbucket.org/hsarvell/pl-web
cd /opt/picolisp/src/
make gate

https://picolisp.com/wiki/?websockets

18may15    hsarvell
Revision History