CODECO: Supporting the Edge-Cloud Continuum Via an Open, Cognitive, Decentralized Framework

Thursday, July 27, 2023 - 06:00

The current internet evolution, with an increasing focus on the intertwining of smart, mobile, decentralized IoT services, brings new challenges to the deployment of applications. Applications and their components need to be lightweight, highly portable, and mobile, to run anywhere and everywhere. Secondly, there is a need to handle large amounts of data often with very different features both in terms of computation (processing) and exchange (networking), including those with low latency demands. Third, it is important to guarantee a smooth data flow, to allow for a smooth and secure integration end-to-end, across edge-cloud environments. Fourthly, real-time decision-making on where to compute and to store data must be supported.

CODECO, a Horizon Europe project, is attempting to answer these needs (Figure 1), framing itself as an intelligent and decentralized edge-cloud orchestration framework that addresses the existing infrastructure by considering data, compute, and network resources.

Figure 1: The key principles and objectives of CODECO

CODECO develops an ecosystem consisting of open-source toolkits, large-scale experimentation, training tools and events, use-cases across four vertical domains (Smart Cities, Energy, Manufacturing, Smart Buildings) and multiple events integrated into a unique Innovation and Research Community Engagement Programme. The open source software, which is pluggable and interoperable with Kubernetes (K8s) will be made available to the community through the Eclipse Foundation.

A Novel Approach to Orchestration: Data-Compute-Network

CODECO will implement software toolkits suitable for smarter management of highly-distributed environments based on heterogeneous networks and by integrating mobile, resource-constrained devices. The CODECO components, illustrated in Figure 2, shall extend the management and orchestration of edge-cloud services with cognitive cross-layer adaptability and with features that allow for intelligent decisions regarding computational offloading and network adaptation while taking into consideration application and networking requirements, data security and sensitivity, as well as other context-specific aspects related to the data flow. CODECO plans to support both single cluster and federated cluster operations.

Figure 2: The CODECO open-source framework and its components.

Key research contributions in this context are:

Automated configuration and cognitive edge-cloud management, taking into account cross-layer behavior (application to the network), data and meta-data management, and policy enforcement
Data privacy-based decentralized learning and context-awareness, addressed in a cross-layer manner, and focusing on federated learning methods that are better suited to the orchestration of data/workflow/computation operations and networking.
Dynamic scheduling and workload migration. Addresses challenges derived from the higher degree of automation due to container and workload mobility between nodes in a single cluster and across different clusters.
Joint orchestration of data, computational, and networking resources. On the one hand, the networking infrastructure needs to become more flexible and adapt to the real-time needs of running applications. On the other hand, the edge-cloud continuum needs to be addressed as a joint data-computational-networking system. By doing so, it is feasible to develop a privacy-preservation end-to-end system capable of adapting to challenges such as mobility, different namespaces and data spaces.
Global view of data in the IoT-edge-cloud continuum. Where are datasets stored, what are their characteristics, and what constraints limit how data can be processed and replicated? To achieve this, CODECO will collect and manage metadata and make it available to the orchestration engine and other system components.

CODECO Main Assets

The CODECO ecosystem is established via the development of six assets (depicted in Figure 3):

A1: Open, cognitive toolkits and smart Apps, integrating the elastic and advanced concepts to manage, in a smart and flexible way, containerized applications across Edge and Cloud (dynamic cluster and multi-cluster environment).
A2: A developer-oriented open-source software repository, to be available in an early stage of the project, thus allowing for early exploitation of initial, advanced results and a better adaptation throughout the project lifetime.
A3: Training tools and events, to support the development of services based on the CODECO framework.
A4: Six use-cases across four domains (smart cities, energy, manufacturing, mobility), to be deployed in operational environments.
A5: Research and Innovation Community Engagement Programme, based on the different use-cases and including the different CODECO stakeholders.
A6: CODECO framework integration into the large-scale EdgeNet, experimental infrastructure, to assist in the building of experimentation and novel concepts by the research community.

Figure 3: CODECO Main Assets

Use Cases

CODECO aims to research and to contribute to a smoother operation across edge-cloud, in particular envisioning current and future data and application centres. The CODECO key technological blocks shall be explored via the use of multiple use-cases which will serve as basis for experimentation and demonstrations. In this proposal, partners have agreed to support the development of six use-cases across four different European competitiveness markets: smart cities, energy, manufacturing and smart buildings. A summary of the six CODECO use cases is provided in Figure 4.

Figure 4: CODECO Use Cases

Current Status and Phases of Development

To allow for a faster delivery of early code to the community, CODECO has been set to be developed with different software releases. The first set of tools is expected to be released after 18 months (June 2024), providing the full operation of CODECO for the setup and management of one cluster, with a pre-release expected in September 2023. Then, the final release envisioning the CODECO operations within federated clusters shall take place in December 2025.

The CODECO Toolkit will be provided under a business-friendly open source license allowing for both academic and commercial exploitation. The development is embedded in the Eclipse Research Labs environment, a combination of infrastructure (GitLab) and training resources to implement open source best practices right from the start and increase the chances of delivering high-quality, professional open source results.

CODECO is currently in its sixth developmental month, having released its high-level architectural design and the CODECO use-cases design. All documents are freely available through the CODECO ZENODO Community.

Project Consortium

The CODECO consortium led by the fortiss research institute comprises of 16 partners across Europe and its associated states Israel and Switzerland. The consortium partners represent several types of organizations, ranging from SMEs with a focus on open source software and innovation management (Inova Mais, Eclipse Foundation, Almende); renowned universities (University of Göttingen, Universidad Politecnica de Madrid, University of Pireus Research Center); research institutes (fortiss, I2CAT, ATHENA) and large companies (ATOS, Telefonica, Siemens, Intracom-Telecom, Red Hat, Netsoft-Intrasoft, IBM). Three affiliated entities participate in the development of CODECO: Universidad Carlos III de Madrid, ATOS IT and City of Göttingen.

The CODECO project has received funding from the European Commission program Horizon Europe, under grant agreement number: 101092696.