Eclipse AQAvit Provides Comprehensive Quality Assurance for Java

Procurement teams and architects within enterprises must secure and verify the quality of the software coming into their supply chain. Some of the common questions we hear from enterprises are:

• Is this software ready for enterprise use?
• Is there a more cost-effective way to validate quality?
• Has someone already tested this, and can we trust those results?

The Eclipse AQAvit project was created to answer these questions. As more distributors are using AQAvit, more enhancements are being made to the suite and associated tools, ultimately leading to higher quality runtimes in the ecosystem.

Java Fundamental to Many Software Supply Chains

Enterprises have built significant dependencies on open source software, especially Java (OpenJDK). As a recent blog post put it: “Java is the ‘write once, run everywhere’ language. ... and arguably is the most pervasive choice for enterprise software for many reasons.”

There are many reasons for Java’s supremacy, including its:

• Stability
• Portability
• Ease-of-learning
• Affordability
• Community support
• Scalability
• Performance 

Java also provides economic benefits in intensive, production-critical business systems from its shared libraries and maintenance.

The open source supply chain for enterprises that use Java either for internal or external applications looks like Figure 1. Organizations concerned with quality and security are encouraged to shift left, which involves taking more ownership of these concerns and addressing them earlier in the supply chain.

Figure 1: Open Source Supply Chain for the Enterprise: Shift Left Concept
 

Rapid OpenJDK Release Cadence

Ever since OpenJDK accelerated its release cadence, enterprises have had to become more agile with application testing and integration. Enterprises need to pull in multiple Java versions, which makes it more difficult to provide secure-in-production versions and productivity enhancements and keep pace with third-party dependencies.

AQAvit supports all maintained Java versions from Java 8 onwards for all platforms upon which Java is ported and eases the challenge of balancing multiple versions by standardizing quality assurance activities across them all.

This standardization yields many benefits, not the least of which are comparable results across quality criteria across the different versions of Java that may be rolled out across an enterprise.

AQAvit Reduces Cost of Validating Dependencies

Enterprises spend a lot of effort validating suppliers and the software provided by those suppliers, quantified as cost of quality (CoQ). It is important that organizations have a method for determining the cost incurred to ensure products meet a certain quality bar and recognize the cost when products fail to meet quality standards.

As shown in Figure 2, the cost of quality is defined as:

CoQ = Prevention + Appraisal + Internal Failure + External Failure

Figure 2: Total Cost of Quality (from AQQ)

 

By collaborating on quality assurance and verification, enterprises share the prevention and appraisal costs with a community of quality-minded developers.

Community Validation of Dependencies Reduces Risk and Cost

Java is a standardized product built and distributed by many different vendors. Enterprises often validate their dependencies on Java in isolation, and the results of this validation are not shared.

AQAvit offers unique value by providing a well-resourced public location to compare notes, reproduce issues, and transparently demonstrate failure scenarios and/or fixes on any Java build (Figure 3). By verifying earlier in the supply chain, issues are found and can be resolved before integration.

Figure 3: Query on the State of a Test  
 

When enterprises come together around a quality initiative, it elevates the quality bar for the entire ecosystem, saving time and resources that enterprises can put towards other goals.

AQAvit Simplifies Support for Multiple Java Implementations, Platforms, and Versions

Many enterprises also end up supporting multiple Java implementations, due to acquisitions or the need for extended platform support, which can rapidly expand (Figure 4). AQAvit can be applied against all OpenJDK distributions from any vendor across any platform where Java has been ported, making it easier to manage multiple implementations.

Figure 4: Example Test Matrix 
 

Transparent Triage of Issues Accelerates Resolution

The AQAvit project also makes it easier to raise issues against the OpenJDK project when the cause is rooted upstream. By providing an open and transparent community location to run, reproduce, and demonstrate the issue, it becomes much easier to demonstrate the core issue. This aids the upstream development teams, which generally expedites a fix.

 For example, say you encounter an issue running Camel v2.15.0 against the Temurin JDK 17 distribution from Adoptium on the x86-64_linux platform (Figure 5). It is not immediately obvious where the problem lies. Is it in Camel v2.15.0? Is it in the Temurin distro? Or elsewhere?

 The AQAvit project can automatically triage these types of issues and allows users to easily swap factors in any of the rows of Figure 5 to hone in on the root cause. The test can then be repeated by varying the combinations of inputs, such as different versions of the application and of OpenJDK, as well as different distributions, different architectures, and different operating systems.

Figure 5: Example Triage Scenario  
 

Collaboration with research focused on developing machine-learning models that can automate this type of activity is ongoing.

Save Effort and Secure Software Supply Chains by Engaging With AQAvit

To summarize, engaging with AQAvit can have numerous benefits for your enterprise:

• Save cost in time, money, and resources
• Ensure vendors support your dependencies and requirements effectively
• Continuously improve your ability to deliver high-quality products and services
• Leverage collaboration to gain expert management of your Java dependencies
• Keep pace with a rapid release cadence

You are invited to engage on the topic of quality assurance at the Eclipse AQAvit project.

Many activities that contribute to the reliability of Java are borne by the project team. If you would like to help keep surprises at bay, consider getting involved:

• Provide feedback and use cases to the AQAvit project
• Contribute tests into the AQAvit test suite that represent your use cases
• Sponsor AQAvit to help the project maintain high-quality assurances for OpenJDK builds
• Join the Adoptium working group and actively guide the direction of AQAvit
• Use AQAvit verification as a qualification when appraising and rating suppliers 

If you’d like to learn more about AQAvit, you can read about it here. 

If you are interested in engaging in the project, want help assessing your current cost of quality, or have questions, you can join the Adoptium Slack workspace, connect in the #testing-aqavit channel, or engage directly with us in GitHub via our Contributing Guide.