Eclipse Dirigible: Where We Are Now and How We Got Here

More than eight years ago, the Eclipse Dirigible project set out to achieve the shortest possible turnaround time for software development in the cloud. That was our mission statement from day one, and we’ve come a long way in the years since. 

The latest version of the software has several changes, large and small. But understanding where the project is today requires you to understand how it started. 

Dirigible Enables In-System Development

Back in 2014, the software industry in general had slow, tedious, and error-prone software development and deployment processes. 

It could take anywhere from five to 20 minutes to compile, deploy, and run code in the target environment. To save time, there were options to run code on local servers and local services instead of going to the cloud. 

However, once the source code, which had been carefully built, tested, and deployed locally, was deployed in the real environment, it usually crashed. Sometimes it was due to edge cases, or “insignificant” differences in the behaviour of the local and cloud services, or simply due to missing configurations, permissions, or service quota.

We decided to fix all these issues with one big shot — move it to the cloud, with real services, permissions, and configurations. There would be no differences between the local and cloud environments. We called this In-System Development. The hardware world has a similar term — In-System Programming — used in the context of programmable logic devices, microcontrollers, and embedded devices.

Finding a JavaScript Engine to Overcome Compilation Challenges

The other big hurdle hampering turnaround time was compilation. Because all Java code needs to be compiled before being executed, that was a real challenge. We tried to overcome this issue using techniques for dynamic Java compilation and loading dependencies while being deployed in the target environment. But back then, this approach was quite unstable and didn’t provide the expected results. We had to think outside the box, and we had a few ideas:

• Does Java need to be the target language for end users?
• Would a more dynamic language like JavaScript, Python, or Ruby work better?
• Could we run and orchestrate a program written in JavaScript, Python, or Ruby from the Java process?

Then we found Rhino —a JavaScript engine written in Java, and it served us well. For years, Rhino was the heart of Eclipse Dirigible, the core module for JavaScript execution, and everything in Eclipse Dirigible was built around it. We did try Nashorn and J2V8, and for some versions of Eclipse Dirigible it was possible to choose any of these three JavaScript engines as the default. 

In 2019, we finally found what we were looking for, a solution that resolves all our JavaScript engine challenges, and that’s faster and more powerful than any of our previous JavaScript engines: GraalVM and GraalJS. With that find, we could close the “compilation challenge” chapter of the Eclipse Dirigible story.

Focusing on APIs and Enhancements

Over the years, we built a set of enterprise JavaScript APIs that Eclipse Dirigible provides out of the box. The APIs provide essential functionality such as:

• File system access
• Database queries
• HTTP calls
• REST framework
• Kafka/MongoDB clients 

With the most recent versions of Eclipse Dirigible, we also provide support for ECMAScript Modules. The ECMAScript Modules come with enhanced code completion support in the Monaco Editor (Figure 1).

Figure 1: Enhanced Code Completion Support in Monaco Editor

Later versions will support TypeScript, so type errors can be displayed in the editor and to improve object and function documentation. Another major improvement coming up is integration between ChromeDevTools and the GraalJS engine, which will enhance debugging in Eclipse Dirigible.

Ensuring a Consistent Look and Feel Across Applications

The In-System Development experience isn’t complete without a web IDE, and we started developing our own from day one. It’s changed a lot over the years, going from being OSGi-and RAP-based to a fully webby interface with REST. There are at least three versions of the Eclipse Dirigible Web IDE that I can count. 

We decided to adopt the Fundamental Library, an open source project from SAP, to create a consistent look and feel among the different applications, and to incorporate the SAP user experience standards. We’re also introducing small design changes to further improve the user experience and the usability of the Eclipse Dirigible platform (Figure 2).

Figure 2: Design Changes in Eclipse Dirigible

Simplifying the Development Process With a Mono-Repo

Kind of like the movie Inception, we’re using Eclipse Dirigible to develop Dirigible itself. This proves we’re using the In-System Development capabilities to the maximum, but we had one big challenge to overcome. We wanted to simultaneously have:

• Small modules, to separate concerns
• Flexibility, so you could have a build-your-own-Dirigible with your own set of dependencies
• A quick development process

As a result, we now have more than 300 Maven modules. Most of them are for the web IDE and its perspectives and views (packaged as WebJars). To resolve this, we introduced DirigibleLabs, where all non-Java components of Eclipse Dirigible are stored. 

The idea behind this decision was quite simple — store each web IDE component in a dedicated repository and develop there. Then, at some point, transfer the content from DirigibleLabs to the main branch in the Eclipse Dirigible git repository. 

We started with just a few repositories, but after several years, we realized we had more than 480 repositories in DirigibleLabs. After receiving this issue, we decided it was finally time to refactor the structure and the entire setup.

With the recent updates, it’s now possible to clone https://github.com/eclipse/dirigible.git inside Eclipse Dirigible Git perspective. This archives the web IDE and the API repositories in DirigibleLabs, and the entire development process happens in one place: the new Eclipse Dirigible mono-repo.

The Dirigible Community Is Growing

The Eclipse Dirigible community has been busy this past year. We had a major increase in the number of issues, pull requests, and contributors (Figure 3). 

Figure 3: Eclipse Dirigible Activity 2021-2022

The increase in the activity is mostly related to project XSK — a compatible environment for applications based on SAP HANA Extended Application Services (XS). It’s a compatibility environment based on Eclipse Dirigible that leverages and extends the In-System Development paradigm and the web IDE. We’re expecting a few other white-label platforms that are currently being developed to drive even more interest and contributions.

With the increased adoption of Eclipse Dirigible, we also now have enterprise support provided by codbex, Quanterall, Arcware, and others.

Get Involved in Eclipse Dirigible

Of course, we want our community to keep growing. If you’d like to learn more about Eclipse Dirigible or get involved, consider trying it out and joining the discussion, and be sure to read out contributor guide.