Unifying EDITO-MODELLAB and EDITO-Infra for a seamless virtual environment
The EDITO-Model Lab Integration Workshop, held on September 12-13, 2024, at the Barcelona Supercomputing Center (BSC-CNS) in Barcelona, Spain, was a significant event aimed at reviewing the status of and making advances on the integration of the EDITO-Model Lab components within EDITO-Infra and the broader EDITO virtual environment. This hybrid workshop gathered 51 experts, developers, and stakeholders to review the progress, discuss challenges, and establish clear objectives for the next phases of integration.
The workshop builds on progress made over the past year through the co-design workshops, the Digital Ocean Forum high-level event, and the initial integration tests, considering the specific requirements of Focus Applications (FAs) and What-if Scenarios (WiS). This foundational work sets the stage for a more streamlined integration of both EDITO-Model Lab and EDITO-Infra developments, ensuring these components are well-aligned and compatible for future use.
An important objective of the workshop was to assess the integration progress of various components, within the virtual environment with a strong focus on the Virtual Ocean Model Lab (VOML). This assessment aimed to provide a clear picture of where the integration phase stands, outline what needs to be accomplished in the coming months and identify potential roadblocks. This was made possible through demo sessions (showing how to run workflows and applications from the VOML in cloud or HPC infrastructure) and hands-on sessions (testing the demos individually and adapting them as a result) carried on in MareNostrum5, preceded by introductory presentations of the VOML and the EDITO platform.
Key areas of focus:
- Integration of Demos
The workshop reviewed the integration of six essential demonstrators into the EDITO Platform. These demos are pivotal for testing the platform’s capacity to launch FAs on demand and test WiS. Successful integration is crucial for demonstrating the platform’s capabilities towards high Technical Readiness Levels. The integration also involves ensuring that these demos work seamlessly with the underlying EDITO-Infra, which provides the necessary infrastructure support. - AI-Based Emulators
The workshop discussed the deployment of AI-based emulators on the EDITO platform, ensuring they run efficiently on both HPC and cloud environments provided by EDITO-Infra. The emulators are a vital component of FAs/WiS workflows, and their integration will enable more advanced data analysis and predictive modelling capabilities on the platform. - Model Optimization and Deployment
Models optimized for high performance need to be seamlessly integrated into the EDITO Platform. Utilizing HPC platforms like MareNostrum5 via the Autosubmit workflow manager, participants examined how models can be deployed and orchestrated from the EDITO Platform. EDITO-Infra’s capabilities play a crucial role here, providing the necessary infrastructure for instantiating and interacting with models and providing additional computational resources to run them. - Graphical User Interfaces (GUIs)
To enhance usability, the integration of GUIs is necessary to facilitate the interaction with the models and the tools available within the platform. The workshop emphasized the need for user-friendly and harmonized interfaces that leverage both EDITO-Model Lab and EDITO-Infra software, data and infrastructure components, ensuring that users of all levels can interact with the system in an efficient and seamless manner. - Data Transfer to Data Lake
An essential objective discussed was the transfer of reference simulations to the EDITO Data Lake, making them accessible from the platform for further analysis. The integration of EDITO-Infra interfaces here is crucial, as it ensures the efficient storage, management, and retrieval of vast datasets that are integral to the platform’s operation. - Integration of Diagnostic Tools
The integration of tools to calculate diagnostics, metrics, and indicators directly from the EDITO Platform was another key focus. These tools will leverage both the computational resources of EDITO-Infra and the advanced functionalities of the EDITO-Model Lab to provide valuable insights and analytics to users
The workshop also laid the groundwork for upcoming integration review meetings. Discussions included the need to continue engaging intermediate and end-users and to manage resources efficiently to accommodate new users. In 2025, EDITO-Model Lab will organize a hackathon for this purpose, beside a few other activities already planned.
The next steps involve preparing for the integration review meetings, where more detailed plans will be laid out for addressing identified challenges, optimizing workflows, and enhancing EDITO functionalities, based on user feedback. This roadmap also envisions further beta testing and user engagement opportunities, and the development of targeted tutorials to ensure the platform meets the needs of its diverse user base.
Overall, the EDITO-Model Lab Integration Workshop marked an important milestone in integrating complex modelling tools and advanced infrastructure into a unified virtual environment. The collaboration between the EDITO-Model Lab and EDITO-Infra is essential for creating a comprehensive and powerful platform that can support cutting-edge scientific research and operational applications.
To stay updated about the EDITO initiative and ensure you do not miss how we add value directly for intermediate users and indirectly for end-users and society, follow us on LinkedIn and X, and subscribe to our mailing list.
EDITO-Infra: The backbone of the European Digital Twin of the Ocean
