Optimising Zenoh-Flow: A Comprehensive Guide to Zenoh and Zenoh-Flow Introduction to Zenoh-Flow Zenoh-Flow is a distributed data flow programming framework based on Zenoh. It focuses on performance, real-time capabilities, and control over these aspects thanks to a declarative approach. It is developed in Rust, a programming language gaining traction in the automotive industry due to its memory and concurrency safety guarantees. Genesis of Zenoh-Flow Zenoh-Flow started as an exploration of the potential of Zenoh as a performant and easy-to-use building block for distributed applications. The purpose was to replace the underlying data flow framework used in the Pylot project — an autonomous driving stack aimed at researchers developed by the University of Berkeley — and assess the gains in terms of ease of use, ease of deployment, and performance. This first iteration has already yielded improvements on all fronts! In particular, Zenoh-Flow removed a bottleneck in the processing, which translated to a continuous rendering of the autonomous driving simulator in some scenarios. Benefits Zenoh-Flow aims to offer several benefits compared to existing solutions. Some of these key benefits include: Ease of use and fine-grained control: Through its declarative approach, developers and integrators can quickly modify and tweak their data flow. Seamless integration with automotive middleware, ensuring effective communication and data transfer across different systems leveraging Zenoh. Real-time capabilities, including: associating a priority and a time budget to tasks, pinning them to a specific CPU core and parking them. Julien Loudet, the Project Lead for Zenoh-Flow, hosted a presentation during the Zenoh User Meeting 2023 and delved deeper into the performance of Zenoh-Flow and what’s coming next. Video to be embedded: https://www.youtube.com/watch?v=Txpujyo0GG0&list=PLZDEtJusUvAZ1ISiU0qAQ011m-wuDwNtC&index=17 Real-time Capabilities Real-time capabilities are essential for the automotive industry and, in general, for any safety-critical application. Hence, one of Zenoh-Flow's main objectives is to offer precise yet easily adjustable control over each computation. This control will eventually allow to address these requirements. The possibility to easily (though safely) tweak the execution will allow for quick development cycles and, ultimately, faster delivery. In addition to what we describe below, these offerings position Zenoh-Flow as a valuable framework for future automotive developments. Assigning Priorities When working with Zenoh-Flow, it's important to assign priorities to tasks. By assigning priorities, you can ensure that higher-priority tasks are executed first, depending on the specific requirements of your environment. This allows for efficient task management, especially in real-time computing scenarios. In addition to this static assignment, Zenoh-Flow will be able to dynamically adjust priorities, catering to the unique needs of automotive applications in changing environments and ensuring seamless task execution. CPU Core Pinning Though in a prototyping stage, Zenoh-Flow offers the ability to pin a task to a specific CPU core, providing control over task allocation and ensuring optimal performance. By pinning a task to a specific CPU core, you can effectively manage resource allocation and guarantee safety in automotive applications. This feature is another major step towards precise control over computational resources, enhancing the real-time capabilities of Zenoh-Flow and making it a valuable framework for automotive developments. Integration with Automotive Middleware Zenoh-Flow's integration with automotive middleware is a critical aspect that ensures effective communication and data transfer within the automotive stack. By leveraging Zenoh, Zenoh-Flow is already compatible with automotive middleware, such as Cyclone DDS or the upcoming µProtocol, which guarantees efficient data transfer, making it a valuable framework for future automotive developments. This integration is another example of how Zenoh-Flow caters to the specific requirements of automotive applications. Rust Language and Zenoh-Flow Software-defined vehicles need a distributed, declarative data flow framework that offers full control over the flow of data. Zenoh-Flow is developed in Rust, a programming language gaining traction in the automotive industry due to its memory safety and concurrency features. Rust has been recognised and certified by industry leaders. The first RUST compiler was certified for ISO 26262 and testimonials from Volvo engineers make it an ideal choice for automotive applications. Renault does it as well and Frédéric Ameye (a Cybersecurity Architect previously working at Renault) gave a talk about it on RUST Nation UK in 2024. Furthermore, the limitations of other programming languages, such as C and C++, have led to the increasing preference for Rust in modern development, especially for autonomous driving applications. Conclusion and Future of Zenoh-Flow As an open-source initiative, Zenoh-Flow is continuously evolving to meet the dynamic demands of the automotive industry. With ongoing work on custom scheduling, real-time capabilities, and seamless integration with automotive middleware, Zenoh-Flow is poised to become a valuable resource for future automotive applications. The framework's development in Rust, a programming language gaining traction in the automotive industry, further solidifies its potential for growth and innovation. If you want to try the code yourself, Julien Loudet wrote a blog post on how to get started. Read it, hack it and let us know your thoughts. Expect to see significant advancements and contributions to Zenoh-Flow in the coming years, positioning it as a leading framework for real-time computing and data flow programming in the automotive sector! As an open-source initiative, Zenoh-Flow encourages contributions from the development community. More information is available on GitHub. If you have questions, send us an email at contact@zettascale.tech or talk with our engineers on Discord. FAQ 1. What is Zenoh-Flow? Zenoh-Flow is a distributed data flow programming framework based on Zenoh, designed to offer better performance and enhanced capabilities for real-time computing in the automotive industry. 2. What are the advantages of using Zenoh-Flow? Zenoh-Flow offers several advantages, including a declarative approach, better performance, seamless integration with automotive middleware, real-time capabilities, and minimal overhead in execution times. 3. How does Zenoh-Flow integrate with automotive middleware? Zenoh-Flow seamlessly integrates with automotive middleware, such as Cyclone DDS, to ensure effective communication and data transfer within the automotive stack, making it a valuable framework for future automotive developments. 4. What are the real-time capabilities of Zenoh-Flow? Zenoh-Flow offers real-time capabilities, allowing for the parking of tasks, assignment of priorities to tasks, and pinning of tasks to specific CPU cores, catering to the specific requirements of automotive applications. 5. How can users contribute to Zenoh-Flow? As an open-source initiative, Zenoh-Flow encourages contributions from the development community, welcoming any contributions on GitHub that aim to enhance its capabilities and performance.

Optimising Eclipse Zenoh-Flow: A Comprehensive Guide

| 6 min read

Zenoh-Flow is a distributed data flow programming framework based on Zenoh. Both of them are Eclipse Foundation open-source projects. Zenoh-Flow focuses on performance, real-time capabilities, and control over these aspects thanks to a declarative approach. 

It is developed in Rust, a programming language gaining traction in the automotive industry due to its memory and concurrency safety guarantees.

Genesis of Zenoh-Flow

It started as an exploration of the potential of Zenoh as a performant and easy-to-use building block for distributed applications. 

The aim was to replace Pylot’s data flow framework, assess gains in ease of use, deployment, and performance.

This first iteration has already yielded improvements on all fronts! 

In particular, Zenoh-Flow removed a bottleneck in the processing, which translated to a continuous rendering of the autonomous driving simulator in some scenarios.

Zenoh-Flow for Software Defined Vehicle Architectures

It aims to offer several benefits compared to existing solutions. Some of these key benefits include:

  • Declarative approach allows quick modification of data flow, ensuring ease of use and fine-grained control for developers and integrators.
  • It enables seamless integration with automotive middleware, ensuring effective communication and data transfer across systems.
  • Real-time capabilities include task priority association, time budget allocation, CPU core pinning, and task parking.

Julien Loudet, the Project Lead for Zenoh-Flow, hosted a presentation during the Zenoh User Meeting 2023 and delved deeper into the its performance and what’s coming next. 

Real-time Capabilities

Real-time capabilities are essential for the automotive industry and, in general, for any safety-critical application. Hence, one of the streamlining mechanism’s main objectives is to offer precise yet easily adjustable control over each computation

This control will eventually allow to address these requirements. The possibility to easily (though safely) tweak the execution will allow for quick development cycles and, ultimately, faster delivery.

In addition to what we describe below, these offerings position Zenoh-Flow as a valuable framework for future automotive developments.

Assigning Priorities

When working with a streaming framework, it’s crucial to prioritize tasks. Prioritizing ensures higher-priority tasks are executed first as needed. By assigning priorities, you can ensure that higher-priority tasks are executed first, depending on the specific requirements of your environment. 

This allows for efficient task management, especially in real-time computing scenarios

In addition to this static assignment, Zenoh-Flow will be able to dynamically adjust priorities, catering to the unique needs of automotive applications in changing environments and ensuring seamless task execution.

CPU Core Pinning

Though in a prototyping stage, it offers the ability to pin a task to a specific CPU core, providing control over task allocation and ensuring optimal performance. 

By pinning a task to a specific CPU core, you can effectively manage resource allocation and guarantee safety in automotive applications.

This feature is another major step towards precise control over computational resources, enhancing the real-time capabilities of Zenoh-Flow and making it a valuable framework for automotive developments.

Integration with Automotive Middleware

Zenoh-Flow’s integration with automotive middleware is a critical aspect that ensures effective communication and data transfer within the automotive stack. 

By leveraging Zenoh, Zenoh-Flow is already compatible with automotive middleware, such as Cyclone DDS or the upcoming µProtocol, which guarantees efficient data transfer, making it a valuable framework for future automotive developments.

This integration is another example of how Zenoh-Flow caters to the specific requirements of automotive applications.

Rust Language and Zenoh-Flow

Zenoh-Flow, in Rust, a programming language gaining automotive traction, offers control over data flow for software-defined vehicles.

Rust has been recognised and certified by industry leaders. The first RUST compiler was certified for ISO 26262 and testimonials from Volvo engineers make it an ideal choice for automotive applications. Renault does it as well and Frédéric Ameye (a Cybersecurity Architect previously working at Renault) gave a talk about it on RUST Nation UK in 2024

Limitations in languages like C and C++ drive rising preference for Rust in modern development, particularly in autonomous driving.

Conclusion and Future of Zenoh-Flow

As an open-source initiative, Zenoh-Flow is continuously evolving to meet the dynamic demands of the automotive industry.

Ongoing work on custom scheduling, real-time capabilities, and seamless integration positions Zenoh-Flow as a valuable resource for future automotive apps.

The framework’s development in Rust, a programming language gaining traction in the automotive industry, further solidifies its potential for growth and innovation. 

Do you want to know more about it?

If you want to try the code yourself, Julien Loudet wrote a blog post on how to get started. Read it, hack it and let us know your thoughts.

Expect to see significant advancements and contributions to Zenoh-Flow in the coming years, positioning it as a leading framework for real-time computing and data flow programming in the automotive sector!

As an open-source initiative, Zenoh-Flow encourages contributions from the development community. More information is available on GitHub. If you have questions, send us an email at contact@zettascale.tech or talk with our engineers on Discord

FAQ section

FAQ Section
What is Zenoh-Flow?
Zenoh-Flow is a distributed data flow programming framework based on Zenoh, designed to offer better performance and enhanced capabilities for real-time computing in the automotive industry.
What are the advantages of using Zenoh-Flow?
Zenoh-Flow offers several advantages, including a declarative approach, better performance, seamless integration with automotive middleware, real-time capabilities, and minimal overhead in execution times.
How does Zenoh-Flow integrate with automotive middleware?
Zenoh-Flow seamlessly integrates with automotive middleware, such as Cyclone DDS, to ensure effective communication and data transfer within the automotive stack, making it a valuable framework for future automotive developments.
What are the real-time capabilities of Zenoh-Flow?
Zenoh-Flow offers real-time capabilities, allowing for the parking of tasks, assignment of priorities to tasks, and pinning of tasks to specific CPU cores, catering to the specific requirements of automotive applications.
How can users contribute to Zenoh-Flow?
As an open-source initiative, Zenoh-Flow encourages contributions from the development community, welcoming any contributions on GitHub that aim to enhance its capabilities and performance.

Author

ZettaScale

ZettaScale’s mission is to bring to every connected human and machine the unconstrained freedom to communicate, compute and store — anywhere, at any scale, efficiently and securely. The company develops open source communication middleware that underpins next-generation robotics, transportation and mission critical applications. As devices in the physical world become connected, instrumented and interdependent, ZettaScale enables device creators to easily achieve robust, secure and scalable communication.

ZettaScale was spun out from ADLINK Technology Inc. in 2022, with a strategic investment from TTTech Auto. ZettaScale has offices in the U.K., France and The Netherlands.

FOLLOW
Scroll to Top