I’ve been working for several months on the next version of rtpmidid, and finally, it’s time to unleash it.
In brief, I rewrote the entire server itself and fixed some bugs in the librtpmidi library.
The core of the changes in the server is to centralize all MIDI routing into a single object: the MIDI router.
The MIDI router knows about all MIDI peers, whether they are rtpmidi clients, servers, ALSA connections, listeners, etc. Thanks to this architecture, it will be easy to add new functions in the future. More on that later.
What’s rtpmidid?
rtpmidid is a GNU/Linux daemon that allows connecting MIDI devices to and from the local network. This enables all MIDI devices to be accessed from afar, with no limit on the number of devices, very low latency, and the ability to be shared between several hosts. It frees each MIDI device from the USB/MIDI DIN.
For example, I use it on a small Single Board Computer Raspberry Pi-style (an Orange Pi Zero), which has all my MIDI synthesizers connected to a USB HUB. Then I can easily play one synth from another (some don’t have keyboards but sound amazing!). One of the devices is an MPC One connected via a USB MIDI adapter, allowing me to jam DAWless in 30 seconds. But sometimes I want to go full DAW, so I just fire up my Bitwig with rtpmidid on Linux or Tobias Erichsen’s rtpmidi for Windows and make my synths play without any latency. For some time, I used Linux gadget support to provide USB MIDI to the MPC One…
The Old Architecture
What was wrong with the current architecture? It was a mess of signals and connections. It did work fine but was difficult to expand and understand after some time. Especially the latter. As Kernighan and Plauger said, “Everyone knows that debugging is twice as hard as writing a program in the first place. So if you’re as clever as you can be when you write it, how will you ever debug it?” So I decided it was time to simplify it. All interactions are managed by the router, and there is a lot of telemetry to see what’s going on.
It may be less efficient, use more memory, but finally, I can expand it and not pull my hair out.
New Features
- midirouter. All communication between peers (ALSA or rtpmidi) is via the router, allowing us to centralize many things like communication, lifecycle, ownership. It is much easier to reason about this architecture. With this, the code is now properly segregated into different files that do only one task, not all with a bit of a mess of signals. The signals are still there, but now they are properly assigned to the right objects.
- INI configuration file. Instead of passing many command-line parameters, it is now possible to use a .ini file with details of default behavior, initial connections, and so on. It really helps to have a sane default configuration that does not feel like a kludge. For installations that require some default configuration, I think it is a great addition.
- Avoid malloc/free in the hot path. Even logging was using malloc. Now it’s not. There is no separation of hot and cold paths, so a malloc can appear at any moment (new connection! a lot of mallocs! disconnection! a lot of frees!). But at least in the most common path of playing MIDI, there are none.
- Improved statistics and command management. The command line interface via Unix sockets is now easier to expand. With the midirouter in the middle of all the peers, it is possible to really know what’s going on inside the connections, including the number of packets, latency (including the average for the last 2 minutes and standard error!). It is also possible to tweak the internal connections and many small details like that. Old commands like connect still work as normal. The “help” command shows all available commands on the server with a description included.
- top like command line management – To easy my life in the development and bug fixing, just staring at the status from the command management was not enough. So I created a basic TUI that shows some real time data (updated every second), and allows some interactivity. Think of this as a proof of concept for a latter more use friendly GUI management.
- ninja compilation. Just to speed up compilation a bit and use precompiled headers. Both are optional but enabled by default.
librtpmidid
The library itself got changed in the way of better management of signals, which always return a handler that can only be moved and will disconnect when the handler is destroyed (lifecycle management basically). The same for poller connections, and improved statistics (average latency and std error).
Future Proofing
With the new architecture, it is possible to add new functionalities much easier. And I have a lot of ideas.
We can add more transports such as simpler TCP MIDI, jack midi, pipewire MIDI, raw midi, UART, or even add some MIDI2 support. There are also many proprietary MIDI network transports like the AKAI Network. We can add any other transport and make them easily communicate with each other. But for this version, it’s the same behavior as before: ALSA to rtpmidi.
Other ideas include adding an HTTP server for management, adding some connection management similar to AseqRC but more powerful. For example, it can show some statistics on current connections; maybe we can add MIDI filters or even add support for Raspberry Pi and family as MIDI gadgets.
Also, a big change I would like is to separate hot and cold paths into different threads so we can make the hot path really low latency, avoiding huge no-nos such as malloc and free, and doing kernel calls for IO. But being rtpmidi a network protocol, I have to think about how to do it better. Maybe with IO uring?
Anyway, all these dreams sound like not only rtpmidi anymore, so maybe I should change the name… I thought of midirouterd… but nothing has been decided. Also, I see a huge gap in sharing audio over the network with open protocols and easy management. Some synths nowadays also export the internal audio over USB.
Final words
As always, every comment, bug report, pull request, or just using it is appreciated. Thanks!