Security

The First of Six Key Challenges

4.1 Security

The following sections provide a deeper dive into the key areas suggested in section 2.The chain of assurance box at the end of each section provides questions that could be asked by stakeholders, auditors, or regulators seeking assurance in the key area. These questions have been compiled drawing on the useful work of the EU’s High Level Expert Group on Ethical AI.⁶4.1 SecurityFor security audits, verifiability of security claims continues to be the basis for assurance. It enables relying parties to make decisions about the appropriate usage of systems. Under a legislative regime or in situations where there is an issue of liability, such as potential for human injury and environmental harm, a high level of audit and documentation is required for all systems. Using a chain of assurance approach enables AI systems to meet those audit requirements.We distinguish two aspects of security in relation to the implementation of trustworthy AI systems: cybersecurity in general and AI-specific security.

4.1.1 CybersecurityThe aim of cybersecurity in a more traditional sense is to implement systems that protect the confidentiality and integrity of assets, while guaranteeing a certain level of availability.Traditional cybersecurity is a constantly evolving field and there are many approaches, but at Arm we believe a security-by-design system is based on the four key principles:

Analyze: Make a threat model to determine your security requirements.

Architect: Use an established security architecture.

Implement: Create a high-quality implementation.

Certify: Get an independent, unbiased security evaluation of the system to create assurance.

The foundations of a secure hardware platform implementation are:

A hardware Root of Trust that is resistant to certain physical attacks.

Hardware-backed isolation primitives, such as the Realms in the Arm Confidential Computing Architecture, which protect against software and hardware attacks by untrusted third parties.

Robust use of encryption for communication attacks on data at rest and in transit.

Secure lifecycle management to protect against supply chain attacks.

More information about secure hardware platforms can be found at the PSA Certified website. For more information about using confidential computing to increase the security of AI systems, see section 5.2.We cannot stress enough the importance of platform security, especially secure lifecycle management. This lies at the heart of many of the mitigations we mentioned. It also gives us the ability to revoke or update components in the systems, whether they are models or part of the compute subsystem.

Source: PSA Certified 2022 Security Report

4.1.2 AI-Specific SecurityThere are also specific threats that are introduced by the development and deployment of AI systems, primarily related to the data and models inherent in these AI systems. We can categorize them in terms of attacks on:

Data. During the input stage to achieve a future adverse outcome, or at the inference stage, to gain knowledge of the input data used.

Models and algorithms. To achieve an adverse outcome, or to gain knowledge of the model/algorithm itself which were not intended.

Adversarial data poisoning is an effective attack against machine learning and threatens model integrity by introducing poisoned data into the training dataset."

Cornell University, 2020.

4.1.2.1 Compromising Data

Poisoned Data Backdoor Attacks. This attack injects poisoned sampling data at the development stage by a model developer or at the deployment stage by a service provider, such that a specific unintended outcome (backdoor) can be triggered by a specific set of inputs.⁷
Data Protection and Privacy Risks. For information about privacy risks, see section 4.3. One specific example of a privacy risk is a violation of data privacy by adversaries using member inference attacks.
Member Inference Attacks. A member inference attack attempts to establish whether a subject belongs to a specific data set by probing APIs and running numerous inferences through machine-learning-as-a-service.⁸ This presents privacy risks for private data sets, or when the data set is facial or other biometric data.

4.1.2.2. Attacks for Models and Algorithms

Physical Adversarial Example attacks on Deep Learning Models. This is a well-known attack vector, where small perturbations in input can lead to a high rate of misclassification and mislead systems to potentially dangerous outcomes.^{9, 10}

Model Extraction Attacks. Like member inference attacks, model extraction attacks attempt to infer properties of a model through manipulation of input data and output analysis.¹¹

Chain of assurance questions for security

Seeking assurance

Have the technology developer, system integrator, and device provider considered security and vulnerability?
Have the technology suppliers considered relevant trustworthy AI vulnerabilities during the design and development phase?

Providing detailed information

Does the hardware design include a hardware Root of Trust?
Does the system use secure lifecycle management to protect throughout its design, deployment, and maintenance?
What technology does the system use to ensure the integrity of data sets?
How does the system protect the security of models and algorithms?