Resources

EASA and Collins Aerospace, Formal Methods use for Learning Assurance (ForMuLA), Technical Report, April 2023

The aim of this report is to present the outcome of the collaboration between EASA and Collins Aerospace on an Innovation Partnership Contract (IPC) that investigated the use of formal methods as part of the learning assurance building block of the EASA AI Roadmap [4]. The project ran from Oct 2021 to Mar 2023. The IPC project titled "Formal Methods use for Learning Assurance” (ForMuLA) focused on emphasizing opportunities for the adoption of formal methods techniques in the design assurance process of machine learning enabled systems. This resulted in the following key achievements:

• Proposed use of formal methods as anticipated means of compliance for a set of key certification objectives from the EASA Concept Paper for Level 1&2 Machine Learning Applications. This supported the update of definitions in the concept paper and the clarification of objective LM-11 on learning algorithm and trained model stability, which has been split into objectives LM-11 and LM-12 in the transition to the new version of the concept paper.
• Detailed discussion of relevant formal methods (FM) technologies and supporting statistical methods, and their possible role in the development and validation and verification (V&V) of machine learning enabled systems. Emphasis has been made on innovative FM applications specific to the robustness assessment of machine learning models.
• Practical demonstration of the use of formal methods on an industrial use case of a deep learning-based estimator for remaining useful life of mechanical bearings in airborne equipment. The output of the estimator is used for on-ground maintenance applications. Demonstrations provided concrete evidence of how FM and supporting statistical techniques can be used as part of the verification activities to deal with data quality assessment, ML stability, robustness and intended behavior verification.

The considerations summarized in the report apply to machine learning in general, but particular emphasis has been placed on specific challenges related to neural networks. Discussion of formal methods applications are purposefully kept generic. This report does not recommend specific methods or tools, but is rather intended to motivate opportunities from a theoretical perspective. Where applicable, a reference is made to concrete methods and tools.

EASA publication page

PDF

Fabio Federici, Davide Martintoni and Valerio Senni

This paper considers the domain of Industrial Internet of Things (IIoT) infrastructures and the recurring need for collaboration across teams and stakeholders by means of remote access. The paper describes a secure solution beyond the traditional perimeter-based security approach, which consists of an architecture that supports multi-level authorization to achieve fine-grained access control, better scalability, and maintainability. An implementation of the proposed solution, using open-source technologies, is also discussed and covers the protection of both the network and edge domains of a complex IIoT infrastructure. Finally, the paper presents a risk-driven and model-based process that is designed to support the migration of existing infrastructures to the solution architecture. The approach is validated, taking as a reference two relevant scenarios for the aerospace industry.

View 'A Zero-Trust Architecture for Remote Access in Industrial IoT Infrastructures' on mdpi.com

Raul de la Cruz, Philip Harris, Samuel R. Thompson, Christos Evripidou, Tim Loveless, Juan M. Reina, Mikel Fernandez, Enrico Mezzetti, Francisco J. Cazorla, Embedded Real-Time Systems

Driven by the increasing compute performance required by modern autonomous systems, high-integrity applications are moving to multi-core processors as their main computing platform. Using multi-core processors in avionics is particularly challenging since the timing behavior of the software is not only affected by its inputs but also by software running simultaneously on other cores. To address this challenge the MASTECS project has developed a methodology for multicore timing analysis together with a supporting toolset. In this work, we show the results of evaluating this methodology and tools on a representative avionics use case.

View MASTECS Multicore Timing Analysis on an Avionics Vehicle Management Computer

Fateh Kaakai, Konstantin Dmitriev, Sridhar (“Shreeder”) Adibhatla, Elgiz Baskaya, Emanuele Bezzecchi, Ramesh Bharadwaj, Barclay Brown, Giacomo Gentile, Corinne Gingins, Stephane Grihon, Christophe Travers, SAE International Journal of Aerospace

This article presents a new machine learning (ML) development lifecycle which will constitute the core of the new aeronautical standard on ML called AS6983, jointly being developed by working group WG-114/G34 of European Organisation for Civil Aviation Equipment (EUROCAE) and SAE. The article also presents a survey of several existing standards and guidelines related to ML in aeronautics, automotive, and industrial domains by comparing their scope, purpose, and results. Standards and guidelines reviewed include the European Union Aviation Safety Agency (EASA) Concept Paper, the DEEL (DEpendable and Explainable Learning) white paper “Machine Learning in Certified Systems”, Aerospace Vehicle System Institute (AVSI) Authorization for Expenditure (AFE) 87 report on Machine Learning, Guidance on the Assurance of Machine Learning for use in Autonomous Systems (AMLAS), Laboratoire National de Metrologie et d’Essais (LNE) Certification Standard of Processes for AI, the Underwriters Laboratories (UL) 4600 Safety Standard for Autonomous Vehicles, and the paper on Assuring the Machine Learning Lifecycle. These standards and guidelines are examined from the perspective of the learning assurance objectives they propose, and the means of evaluation and compliance for achieving these learning objectives. The reference used for comparison is the list of learning assurance objectives defined within the framework of AS6983 development. From this comparative analysis, and based on a coverage criterion defined in this article, only three (3) standards and guidelines exceed 50% coverage of the Machine Learning Development Lifecycle (MLDL) learning assurance objectives baseline. The next steps of this work are to update the AS6983 learning assurance objectives, improve the associated means of compliance to approach a coverage score of 100%, and offer a certification-based process to other domains that could benefit from the AS6983 standard.

This publication is available for purchase from SAE International.

Giuseppe Cammarata, Gabriele Giunta, Lorenzo F. Sutton, Riccardo Orizio, Thu Le Pham, Stefano Sebastio, Piotr Sobonski, Jack Boyd, Filippo Leddi and Carina Pamminger, in Cyber-Physical Threat Intelligence for Critical Infrastructures Security by John Soldatos and Isabel Praça

In this chapter, challenges and approaches for effective Data Visualisation aimed at enhancing Situational Awareness in Sensitive Industrial Sites and Plants (SIPS) Critical Infrastructure are discussed. In the H2020 InfraStress project, a set of specific visualisation tools and dashboards have been developed for SIPS, including for real-time events monitoring and augmented reality. These tools have been integrated in a unified environment and with a set of other Cyber-Physical security solutions, aimed at collecting and presenting visually relevant data to users. The dashboards have been tested within the Piloting activities of the InfraStress project. In particular, in the pilot carried out at the De Puy Synthes site in Ireland (DPS), cyber-physical visualization was an important asset to enable operators to gain knowledge on the detected threats, as well as to receive advanced mitigation and reaction strategies, and therefore improve the site resilience. The first part of the article discusses the general dashboard architecture and core visualisation items (and related paradigms) as well as specifics about the DPS pilot deployment and its interactions with other InfraStress components. The second part elaborates on deployment experience that is critical in successful operation and critical site infrastructure supervision from the Cyber Physical Systems threats perspective. Finally, the article presents main user feedback and conclusions from the InfraStress pilot activities, with a particular focus on enhanced site resilience.

View Data Visualisation for Situational Awareness in Industrial Critical Infrastructure: An InfraStress Case Study