
Jupyter Games
Ipycanvas + Box2D
Motivation
Making their own tiny video games can be a great way for kids to learn programming in a playful matter. While Jupyter is widely used as a scientific and educational tool, Jupyter is seldom used as a platform for game development. In this blog post, we show how Jupyter can be used to develop tiny games based on Box2D. While Jupyter is language agnostic and kernels exist for many programming languages, we will focus on the Python programming language in this blog post.
Mini-Games
When talking about video-games one might think of games such as:
Games as the ones above are usually developed with an engine like:
Not only do these games require complex engines to be developed, but also a whole team developing them and a lot of time and money.
Obviously, these are not the kind of games we can and want to develop in Jupyter. We want tiny games (~ 1000 lines of code) which can be implemented by kids in a matter of a few hours rather than many days, i.e. games like Pong, Pinball, and AngryBirds. In fact, we will focus on a certain class of games: 2D physics-based games, since these can be implemented very easily with the use of a 2D physics simulation engine like Box2D or Chipmunk2D.
Box2D
Box2D is a 2D rigid body simulation library for games. Box2D is best explained by the demo below.

Box2D makes it very easy to implement physics-based games like Angry Birds or World of Goo.
Box2D Debug Draw
DebugDraw
Box2D has an abstract class DebugDraw with a handful of methods to draw simple primitives. The API is given by the following:


Implementing this abstract class allows you to draw the physics for your UI back-end. Box2D is shipped with a glfw based DebugDraw implementation. For Python one can implement a pygame or kivy based DebugDraw implementation. For a finished product, one wants to replace the debug-draw-based renderings with custom shiny rendering routines. But using only debug-draw-based renderings is more than enough to quickly test game ideas and play around with Box2D.
LiquidFun
LiquidFun is a 2D rigid-body and fluid simulation library for games written in C++ based upon Box2D. LiquidFun is best explained by the demo below:

More than just Games
Even though Box2D is advertised as a 2D physics engine for games, Box2D can also be used for educational purposes:
The YouTube channel iforce2d has done some impressive things with Box2D like a Box2D-based combustion engine and a wind tunnel.


Box2D Python
Box2D has bindings for many languages and can be compiled to Wasm. Several Box2D demos exist:
- http://www.iforce2d.net/embox2d/testbed.html (only HTTP, no HTTPS 😢 )
- https://birchlabs.co.uk/liquidfun-wasm/
- https://google.github.io/liquidfun/
We are particularly interested in using Box2D / LiquidFun from Python.
Here we have the following possibilities:
pybox2d
With pybox2d, Box2D has matured and robust Python bindings with extensive documentation. Unfortunately, pybox2d has a few shortcomings:
- It only works with an old version of Box2D.
- There is no support for LiquidFun.
- pybox2d is mostly unmaintained.
- The Python bindings are generated with SIWG (not really a shortcoming, but I personally strongly prefer pybind11).
pyb2d
A disclaimer first, I am the author of the pyb2d Box2D Python bindings. The motivation for creating pyb2d was having Box2D / LiquidFun bindings created with pybind11. While pyb2d works with the brand-new 2.4.1 release of Box2D, has support for LiquidFun and pybind11 has been used for generating the Python bindings, there are also some downsides:
- pyb2d has fewer examples compared to pybox2d
- pyb2d is not as mature and robust as pybox2d
- pyb2d is not yet as well documented as pybox2d
A particularly useful feature of pyb2d is its BatchDebugDraw implementation: While we could implement the above mentioned debug draw API directly in Python, this would have a few drawbacks:
- There is a certain overhead when calling C++ from Python and vice versa. When our game contains a lot of shapes, for instance circles, we would have to call drawCircle very often and have a lot of calls from C++ to Python and vice versa.
- Some back-ends provide batch-drawing capabilities such that we can draw multiple primitives at once. For instance, drawing 100 circles at different locations with a single function call where the function arguments are NumPy nd-arrays with the centers/radii of the circles. Using such a batch API can lead to tremendous speedups.

To address these issues pyb2d provides a BatchDebugDraw implementation where we first collect all the individual calls like drawCircle, drawSegment, drawPolygon etc. We store the arguments of these calls in NumPy arrays. After collecting all the shapes we call the Python API with functions like draw_circles, draw_segments, draw_polygons, etc. where all the shapes are passed to Python in a single function call. On the Python side, one can pass these batched draw instructions to a batch-drawing API of the UI backed. (Spoiler: We will use the batch drawing API of ipycanvas for the Jupyter pyb2d integration.)
Jupyter Box2D Integration: Requirements
To have a platform to develop tiny games from within Jupyter we need at least the following:
- A Canvas: We need a surface/canvas on which we can draw the content of the games.
- Input Devices: Games would be boring without any user input. We need access to input devices like the mouse, the keyboard or even game-pads.
Ipycanvas, Ipywidgets, Ipyevents
Jupyter has a huge ecosystem of extensions. For the Jupyter Box2D integration, we just need to pick the appropriate Jupyter extensions: Ipycanvas gives us access to the HTML Canvas from within Python kernels in Jupyter. This serves as the drawable surface for our games. Ipywidets and Ipyevents give us access to input devices like game-pads, the keyboard, and the mouse.
Ipycanvas

Ipycanvas is a lightweight library developed by Martin Renou exposing the browser’s Canvas API to Jupyter. It allows drawing simple primitives directly from Python like text, lines, polygons, arcs, images, etc. With a few tricks, ipycanvas is fast enough to draw smooth animations. This even works when the server and the client are not on the same machines (when we run ipycanvas on MyBinder for example).

An Ipycanvas-based DebugDraw:
The first step of integrating pyb2d in Jupyter notebooks is implementing an ipycanvas based DebugDraw. We recently released a new version of ipycanvas which provides an extended batch API to draw things very fast. We utilize this batch API when implementing the above mention batch-debug-draw API.
Handling events in Jupyter:
We can use Ipyevents for the event handling in Jupyter. Obviously, we want to handle the events while our game is running. To achieve this within a Jupyter-notebook we need to run our game-loop in a dedicated thread and listen for events in the main thread.
Adding Buttons:
We want to have a few buttons to start, pause and reset the Box2D based games. We can add these with ipywidgets.
Putting it all together
With all the above we can put together a pyb2d based Python Box2D integration in Jupyter. As a proof of concept we implemented a few mini-games:
Billiard:
A very simple billiard game. Try it on binder!

Angry Shapes:
An Angry Birds-like game implemented with pyb2d. Try it on binder!

World of Goo homage:
A World of Goo homage implemented with pyb2d. Try it on binder!

Rocket:
Fly a rocket controlled with the keyboard, but avoid the black hole! Try it on binder!

Compatibility
All the pyb2d examples shown above can also be run in a pygame window or a kivy window since pyb2d also provides back-ends for these. But even when one prefers to experiment with pygame, the Jupyter back-end can still be interesting since one can put the testbed-examples on MyBinder and have them accessible in a convenient fashion.
Caveats
The Jupyter integration of pyb2d gives usable frame rates, even when run through MyBinder. But nevertheless, a pygame based back-end usually leads to a better frame rate and smoother gameplay.
Outlook / Sneak preview
We currently work on:
- Adding more examples and better documentation to pyb2d.
- Improve the performance of the Jupyter integration.
- Add pyb2d to JupyterLite.
- Implement remote multiplayer-based gaming via the collaborative mode of JupyterLab.

Acknowledgments
I would like to thank Martin Renou for his help with ipycanvas.
About the author
Thorsten Beier is a Scientific Software Engineer at QuantStack. Before joining QuantStack, he graduated in computer science at the University of Heidelberg and worked at the EMBL. As an open-source developer, Thorsten worked on a variety of projects, from nifty and vigra in C++ to inferno, kipoi, and ilastik in Python.