JupyterLab, the Cloud Robotics Command Station
The ROS open-source project (Robot Operating System) has become a de-facto industry standard in the robotics community. Very much like with Project Jupyter, a large collection of software has been built upon well-specified communication protocols. ROS has become a rich ecosystem of standardized tools for building and distributing ROS-based software, exploratory robotics development, and much more.
With Industry 4.0, the number of robots and smart devices has been increasing exponentially. We are talking about fully automated factories with hundreds of robots and thousands of connected sensors, generating large amounts of raw data.
There is an opportunity in bridging the ROS and the open-source data science ecosystems and tools.
For this reason, we set ourselves to bring together these ecosystems,
- by producing a collection of JupyterLab extensions for ROS, turning JupyterLab into a cloud robotics command station,
- by integrating ROS packages with conda / mamba package management system, to enable their installation alongside other open-source data-science packages.
JupyterLab, the cloud robotics command station
While the traditional developer tools from the ROS stack such as RViz are desktop-based, we think that a robotics development environment running in a web browser would provide a lot of flexibility. Also, the ideal robotics developer platform should be plugin-based, and easily extensible to account for the diversity of use cases to enable, from simple interactive scripting to visualization or large datasets. Finally, we should build this environment upon broadly adopted foundations, with open governance, and not let a single entity hold the keys to the ROS ecosystem.
Jupyter is a natural candidate to serve as the foundation for a web-based robotics development environment.
JupyterLab is a web-based interactive development environment that has many of the much-needed features
- In JupyterLab, everything is a plugin, including the core components to the application. Anyone can produce a “remix” of core and third-party JupyterLab extensions to their needs.
- Very much in the spirit of ROS, one of the keys to Jupyter’s success is that the project was built upon well-documented and specified protocols and file formats that anyone could implement.
- Jupyter is a multi-stakeholder project, not backed by a single corporation, but by a community of developers at a variety of companies, universities, as well as individual contributors.
Today, we are announcing the first beta of JupyterLab-ROS, a collection of JupyterLab plugins to integrate ROS with the JupyterLab platform. We combine the best of the data-science and robotics worlds to help developers build custom solutions for the industry. ROS can run in a high-performance server while JupyterLab is accessed in a web-browser (on Windows, OS X or Linux).
JupyterLab-ROS automatically starts a rosmaster and rosbridge_server making all the painful setup of WebSockets connections completely transparent to users. A status bar widget allows you to start and stop the rosmaster server with a simple click and displays its state in real-time. Internally this widget runs a launch file which can be changed from the JupyterLab settings UI to launch additional nodes.
Another important component of the ROS stack is RViz, a desktop-based 3-D visualization tool. It’s web-based counterpart in JupyterLab is Zethus, started by Rapyuta Robotics. This plugin supports most of the display types supported by RViz. It also provides an info panel that displays the raw messages in realtime, and a web-based version of rqt_graph for visualizing the ROS node graph.
A robotics station would be incomplete without a tool for debugging. For that purpose, ROS provides rosconsole, a package which allows developers to send messages to rosout and make them available on every node. The console gives access to these debugging messages in JupyterLab and provides additional features such as filtering by level and by node.
Robotics Notebooks
A key component to the Jupyter stack is the Notebook, an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. It’s frequently used to do data cleaning and transformation, numerical simulation, statistical modelling, data visualization, machine learning etc.
The notebook could be the perfect tool for robotics researchers and educators to make (and share) experiments in control algorithms, working in a dynamic and interactive development environment that allows quick prototyping and exploratory analysis while having access to the sophisticated mathematical libraries already well integrated with Jupyter. It can be tedious to have a different platform for interacting with robots and processing the data obtained from these, JupyterLab-ROS combines both toolsets and enables an integrated workflow.
At the same time, many projects are becoming more and more interdisciplinary. Currently, it is common to see projects mixing topics like machine learning and robotics. JupyterLab-ROS opens a new world of possibilities for machine learning researchers that are interested in applying their models to robots: now they have the possibility of connecting a robot to a development environment that they are very familiar with.
JupyROS is another powerful library that leverages ipywidgets and bqplot to integrate ROS in Jupyter Notebooks, being able to show real time plots from ROS messages and create intuitive forms to control robots in minutes.
ROS Package management
Modern scientific computing applications typically depend on a large number of libraries. Conda-forge is a community initiative to package scientific libraries for the conda package manager, and has become a de-facto standard source of packages for the scientific computing community, with tens of thousands of available packages, and thousands of contributors.
During the last year, we have worked hard to package more and more of ROS as conda packages. Conda-packages for ROS Melodic, including RViz are now available on all three operating systems! You can find the packages on the RoboStack channel (a more detailed blog post about this is going to follow soon).
Conda can be used to create virtual environments for ROS projects, which makes it possible to install Melodic, Kinect and Dashing side-by-side on the same Linux system, and have exact control over package versions.
We are also working on Mamba, a conda-compatible package manager implemented in C++. Together with Boa this effort will help to continuously build and release ROS packages on conda.
Try it online!
You can try the JupyterLab-ROS extension online without installing anything by just clicking on the following binder link:
JupyterLab-ROS is under active development. We are working on new features such as a rosbag widget to play and record bag files and improving existing ones like the roslaunch widget. If you are interested in contributing to JupyterLab-ROS you can find the code at this repository in the RoboStack organization. Contributions are always welcome!
Installation
You can install JupyterLab-ROS using Mamba, the first step to create a new environment with JupyterLab and the ROS packages necessary, then you can install the extension.
mamba create -n test -c conda-forge -c robostack python=3.6 nodejs=12 jupyterlab ros-melodic-ros-core ros-melodic-rosauth ros-melodic-rospy ros-melodic-rosbridge-suite ros-melodic-rosbag ros-melodic-tf2-web-republisherconda activate testpip install jupyter-ros-server
If you have any questions don’t hesitate to contact us on the RoboStack or QuantStack chats.
This work has many collaborators around the world, and we are especially grateful to Deep Chaitanya & Ashish Chaudhary from Rapyuta Robotics, Sean Yen from Microsoft for his amazing work on vinca that automagically generates the conda-recipes, Tobias Fischer for many patches and pushing us to bring OS X support to the conda packages as well. Many thanks to Jeremy Tuloup for his help with the JupyterLab extension system.
About the authors
Wolf Vollprecht works as a scientific and robotics software developer for QuantStack in Paris and Berlin.
Carlos Herrero is a Computer Engineer passionate about AI and his applications on robotics. Currently working at QuantStack helping to develop Open Source projects.
