Q&A: Introducing the Eclipse Amlen IoT Message Broker

Eclipse Amlen may be a new project at the Eclipse Foundation, but the software is already well-established in the industry as the IBM Watson IoT Platform Message Gateway. With IBM’s decision to contribute the message broker software to the Eclipse Foundation, it’s the ideal time for everyone in the Eclipse Foundation community to learn more about this mature, production-ready code base and its role in IoT communications.

Here’s how Eclipse Amlen project lead and IBM software architect, Jon Levell, explains the message broker software, its uses, and its benefits.

Q. What does the Eclipse Amlen software do?

A. Amlen is message gateway software that makes it easy to reliably and securely connect mobile and IoT devices and data. As shown in the illustration below, the software sits between the cloud or data center and devices such as cars, home appliances, mobile phones, web browsers, and embedded devices.
 
The message gateway is often implemented as a highly available primary and standby pair so one gateway can take over from the other during software upgrades, or if there’s a hardware failure, to ensure communications are never interrupted. Typically, the MQTT protocol is used to transport the messages, but Jakarta Messaging (formerly JMS) can be used as well.

Q. Can you tell us more about how the message broker is used today?

A. Today, the message broker is used to connect all sorts of different devices to backend applications.

It’s widely used in the automotive industry. For example, a number of car companies use it for communications between the car and the base, so for diagnostics, weather reports, and those types of communications. However, it’s also used in manufacturing facilities to monitor the equipment that’s making vehicles and ensure the factory is operating as efficiently as possible.
 
A number of banks are also using the message broker in their mobile applications and on their websites.
 
And, of course we see it used in more traditional IoT devices as well. So, you can use a mobile to open your garage door when you don’t have line of sight. Or you can monitor your internet-connected washing machine so you know when it’s time to empty it. Also, if something goes wrong with an appliance, the manufacturer can use the technology to retrieve diagnostic information for troubleshooting.

Q. What does the Amlen software include?

A. Amlen is a standard piece of Linux software that’s packaged in RPM format. I’m hoping that community contributors will also make packages for other Linux distributions.
 
The software supports containers, so we provide an example Docker file, and we provide a web UI, shown below, that can be used to manage servers, see what’s going on, set policies, and add automation.

The software also includes a message gateway bridge that allows you to create hierarchies of message gateway servers, and to connect a network of Amlen servers and other MQTT brokers. The bi-directional bridge ensures that messages arriving on one topic on one message broker are always forwarded to other topics on other brokers. The message gateway bridge also supports one-way message forwarding to Apache Kafka.

Q. What level of performance, reliability, and scalability can developers expect?

A. Due to the history of the software at IBM, Amlen includes a lot of very efficient, performance-sensitive code written in C. In the architecture illustration below, the components shown in blue are all written in C.

In terms of performance, a single instance of the message gateway can support:

• Hundreds of thousands of connections. The rough rule is 4,000 connections per 1 Gb of RAM.
• Throughput of hundreds of thousands of messages per second.

Also, as I mentioned earlier, the software supports use of primary and standby message gateway servers to increase reliability. And it supports clusters of message gateways for scalability.
 
With a cluster of message gateways, or message gateway pairs, you can share a topic tree over a number of servers. If a device connected to one server subscribes to a topic, and a device connected to a different server in the cluster publishes on that topic, the infrastructure ensures the subscribing device receives the message. This allows you to create systems with extremely large numbers of devices.
 
Anyone who is interested in the performance details can download the IBM performance report detailing the machine specifications and test scenarios used.

Q. Why did IBM contribute the software to the Eclipse Foundation?

A. We wanted to build a broader community around the software and enable more people to contribute to it.

IBM developers will continue contributing code to Amlen because it’s an important component of our plans going forward. For example, we plan to develop offerings that include the Amlen code base, but will differentiate themselves in the way they analyze data and manage IoT assets.
 
We named the project Amlen, which is the Welsh word for envelope, because it's a good metaphor for a message broker; protecting the message en route to its destination.

Q. What are the next steps for Amlen?

A. Once we have stable builds running, we very quickly want to create documentation and tutorials to help people start using the software. We want to make it as easy as possible for more people to get involved, so we’ll be creating content that explains how to build the software and how to make changes. We’ll also provide suggested new features for people who aren’t familiar with the code.

After that, we’ll extend and improve the software. For example, Amlen can be used in containers, so we need a Kubernetes operator that allows you to control the number of instances so it’s easier to use with Kubernetes. We’d also like to see more plug-in APIs so Amlen can be extended in other ways.

Q. How can developers learn more and get involved in the Amlen project?

A. We’re at the early stages of our Eclipse Foundation development, but we’re very keen to have developers join the effort and help in whatever way they can. Here are a few ways to get started:

• Join the Slack channel: #amlen on eclipse-iot-wg.slack.com
• Subscribe to the dev mailing list
• Visit the Amlen website

Another good way to learn more about Amlen is to watch the webinar I presented as part of Virtual IoT and Edge Day 2021.