How Eclipse Sparkplug Is Standardizing MQTT Communications

The MQTT protocol, which was originally developed for an oil pipeline-monitoring system, has since become ubiquitous across industries. But it lacks the underlying language that enables greater industrial IoT (IIoT) interoperability. The Eclipse Sparkplug Working Group is developing a specification to provide that common language and enhance developers’ ability to use MQTT in more modern IoT applications.

To understand why that specification is necessary, it’s necessary to understand MQTT’s origins.

MQTT Started as a Flexible but Focused Solution

MQTT began its life as a publish-subscribe messaging protocol for Supervisory Control and Data Acquisition (SCADA) systems, where it needed to be lightweight, and capable of operating in low-power and low-bandwidth environments. Because it lived in a self-contained ecosystem, it didn’t need a common language, which made it very flexible. MQTT also decouples the application producing the data from the application controlling applications, which enables other players to consume that data. 

These attributes made it well-suited to IIoT applications, and MQTT began to be used in applications it had never been intended for. It became more and more obvious that MQTT needed a common language to define how the data is encoded — similar to the way HTTP needs HTML. 

This led to the founding of the working group, which currently includes:

• Chevron
• Canary Labs
• Cirrus Link Solutions
• HiveMQ
• Inductive Automation
• Opto 22
• Intel
• Flow Software
• Signalfire telemetry
• The University of Sheffield
• Warner Bio Systems

The group’s overall goal is to build a common language for industrial control applications over MQTT, including an MQTT topic namespace, standard payload definition, and a generically applicable session-state management approach. This will enable key functionality for edge devices.

Sparkplug Makes the Edge the Single Source of Truth

As a common language, Eclipse Sparkplug shifts the source of truth towards the network edge in two main ways.

First, Sparkplug extends MQTT’s report-by-exception approach, often described as  event-based reporting, which has been shown to reduce network bandwidth consumption by 80% to 95%. MQTT sessions using Sparkplug begin with birth certificates and end with death certificates: As long as TCP and MQTT sessions remain open between those bookends, data from an edge system can be published whenever it changes. 

Second, Sparkplug automatically propagates data definitions from the edge, upstream through to the MQTT server, eliminating data duplication and reducing the need to validate records. By defining the content of messages, Sparkplug provides three key functionalities: 

• Describing the state and capabilities of devices, edge nodes, and the host application
• Data transfer from edge-of-network nodes to the central server
• The ability to send commands back to those nodes

And by moving the single source of truth (SSOT) to the edge, Sparkplug enables faster and more accurate decision-making. 

While other specifications, such as OPC Unified Architecture (OPC UA), are already used in industrial applications, Sparkplug is meant to complement them. However, Sparkplug does offer some key advantages over OPC UA, such as: 

• An architecture that use publish-subscribe message-oriented middleware to connect devices directly to Operational Technology
• Sufficiently lightweight construction to enable sensor and device telemetry communications
• Implementation capability on any type of edge device

Improved Interoperability With Existing Systems a Major Goal

The Sparkplug specification aims to enable  interoperability without disrupting existing systems. Because hardware is often deployed in the field for decades at a time. Sparkplug is being developed so that even the latest version of the specification can communicate with a device that’s 20 or 30 years old.

In 2019, Cirrus Link Solutions contributed the then current version of the Sparkplug specification, v2.2, to the Eclipse Foundation. The Sparkplug working group is now preparing the first release of the specification under the vendor-neutral Eclipse Foundation Specification Process. The goal of this initial release is not to add new features to the protocol. 

Rather, the goal of the working group is to further clarify and make the language more precise in the specification. The target release date for the next version is summer 2022. 

The TCK and Compatible Implementations Are Underway

To release a new version of Sparkplug, two other artifacts must become available at the same time.

The first is the technology compatibility kit (TCK) that allows developers to test whether they’ve accurately implemented the specification. Given MQTT’s nature as a network of interacting systems, developing the TCK will be more complicated than it would for an API-based project.

The other requirement is for the three core profiles in Sparkplug — edge node, MQTT server, and host application — to have compatible open source implementations. Eclipse Mosquitto already provides a compatible open source MQTT server, while the Eclipse Tahu project, an existing implementation of the standard, is currently working on implementations of the other two. 

Get Involved in Sparkplug

We encourage everyone in the community to use the tools currently available in Eclipse Tahu and Sparkplug and let us know how well they’re working. 

We’re also hoping for feedback from businesses that currently use Sparkplug, particularly those that own pipelines or manufacturing plants and those that build SCADA and other MQTT-based applications that leverage Sparkplug. Of course, feedback from prospective adopters of Sparkplug is also welcome.

To keep up with the project or participate:

• Sign up for the mailing list
• Provide feedback in GitHub
• Join the Sparkplug Working Group’s Slack Workplace
• Join the Sparkplug Working Group