From Jupyter to the Moon
A new Jupyter kernel for Lua based on Xeus
A key principle in Jupyter’s design is language agnosticism, and one of the main extension points of the Jupyter ecosystem is the kernel, the part of the architecture responsible for executing the user’s code.
In the end, a Jupyter kernel is merely an executable implementing a well-defined inter-process communication protocol.
A reference implementation of the kernel is ipykernel, which provides the default Python kernel, and utilities to build other language kernels. Another reference implementation of the protocol is Xeus.
Xeus
Xeus is C++ library for writing Jupyter kernels. It is not a kernel but a library that simplifies the process of writing kernels. Xeus implements the Jupyter messaging protocol, so that kernel authors only need to implement a handful of methods related to the actual interpreter, like executing pieces of code and auto-completing. There is a long list of scripting languages which can be embedded in C/C++. With the help of Xeus, it is very easy to write kernels for such languages. In fact, Xeus has already been used for making several language kernels:
- xeus-cling is a Xeus-based C++ kernel.
- xeus-python is a Xeus-based Python kernel.
- xeus-sql / xeus-sqlite are Xeus-based kernel for SQL / SQLite databases.
- xeus-robot is a Jupyter kernel for the Robot Framework .
- LFortran is a modern interactive Fortran compiler with a Xeus-based Jupyter kernel.
xwidgets
The xwidget library is a C++ implementation of the Jupyter Interactive Widget protocol. Xwidgets were initially developed for xeus-cling, the Xeus C++ kernel, but any Xeus-based kernel can reuse these components.
A xeus-based Lua kernel
Today, I am pleased to announce the release of xeus-lua, a new Jupyter kernel for Lua based on Xeus.
Lua
Lua, which means “Moon” in Portuguese, is a lightweight embeddable scripting language written in C. While Python is one of the leading scripting languages for scientific application, Lua is the leading scripting language used in games / game engines. The list of video games using Lua is long and includes triple-a games like Crisys and Farcry. Not only games rely on Lua, but also applications like Adobe Photoshop Lightroom.
Sol2
While Lua itself can be embedded in a C++ application quite easy, it is even easier when using Sol2. Sol2 is a C++ library that simplifies the usage of Lua from C++. Furthermore Sol2 can be used to expose C++ classes to Lua, in a similar fashion as pybind11 can be used to expose C++ classes to Python.
Xeus-lua
I am pleased to announce xeus-lua, a new Jupyter kernel for the Lua language based on Xeus. Using Sol2 and Xeus, it was very easy to implement a feature rich kernel using modern C++. With the help of Sol2, it was trivial to expose the xwidgets classes to Lua such that we can use the widgets within xeus-lua notebooks.
But why a Lua kernel?
While Lua might not be the hippest language in 2021, there are multiple reasons while I choose to implement such a Lua kernel, and why such a kernel can be useful:
- Writing a Xeus-based kernel is the best way to learn and familiarize oneself with the xeus ecosystem. And since Lua is so easy to build and embed, Lua and Xeus are perfect match. Implementing this kernel gave me great insights into Xeus and Jupyter itself.
- Learning Lua with xeus-lua is fun! Before writing xeus-lua, I had zero Lua experience. While implementing xeus-lua I learned Lua from within Jupyter by playing around with xwidgets from within Lua — this was a very joyful experience.
- The xeus-lua implementation might serve as reference for future kernel authors. In particular the integration of xwidgets in xeus-lua might be useful for future implementations.
- Since Lua is so easy to build, xeus-lua can be built for WebAssembly and will be integrated in JupyterLite — stay tuned!
Features
Obviously xeus-lua supports the execution of code.
Streams are redirected to the notebooks output
Errors messages are shown in the notebooks output
Input streams are redirected to the notebooks input system
Statements can be auto-completed with a press of tab
Data of different sources as Latex, JSON and html can be displayed.
Interactive widgets can be added to the notebook!
We can feed the widgets with data!
Try It Out
You can try it right now with binder, without the need of installing anything on your computer, just by clicking on this link.
Other Lua kernels
There are several Lua kernels available:
- lupyter: A Python based implementation of a Lua Jupyter kernel. The heavy lifting is done in a C-extension.
- ILua: A feature rich implementation. Ilua is implemented in Python and does the communication with Lua via IPC.
- IPyLua: IPyLua is a pure Lua implementation.
While these implementations are feature rich and work well, none of the above supports interactive widgets. Since xeus-lua is based on Xeus, we can reuse components from the Xeus ecosystem like xwidgets with minimal effort.
Acknowledgments
We are grateful to Juliette Taka for the illustration.
About the Author
Thorsten Beier is a Scientific Software Engineer at QuantStack. Before joining QuantStack, he graduated in computer science at the University of Heidelberg and worked at the EMBL. As an open source developer, Thorsten worked on a variety of projects, from nifty, and vigra in C++ to inferno, kipoi and ilastik in Python.
