Information systems and the data they contain face a wide range of risks. Organizations rely on technology for their vital operations and activities, that’s why they need to be aware of the range and nature of those threats. Different events or incidents can compromise information systems resulting in adverse impact on the organization's operations. These effects can range from insignificant to catastrophic. Evaluating the probability (likelihood) of various types of events or incidents with their projected impact, in case they occur, is a common way to assess and measure IT risks. Additional methods of measuring IT risk involve assessing other factors such as threats, vulnerabilities, exposures, and asset values. With the rapid growth and adoption of artificial intelligence, security issues and risks that are specific to AI arise. The AI risks most frequently encountered are:

AI Risk	Real Case Scenario
Data and Privacy Breaches AI systems rely on large amounts of data to learn and improve their performance. However, this data could be compromised by hackers or malicious actors, who could gain unauthorized access, steal, or leak sensitive information.	An attacker could use AI to generate realistic fake identities and bypass biometric authentication systems. Data breaches could also result from human errors, such as misconfiguring cloud storage or sharing data with unauthorized parties.
Model Poisoning AI systems are vulnerable to adversarial attacks, which aim to manipulate or degrade their functionality.	An attacker could inject malicious data into the training set of an AI system, causing it to learn incorrect or harmful behaviors.
Adversarial Attacks An attacker could introduce subtle perturbations to the input data of an AI system, causing it to produce erroneous or misleading outputs.	These attacks could have serious consequences for AI systems used in critical domains, such as: ֎ Healthcare - Patient safety compromise, privacy breaches, erosion of trust, legal and ethical impact. ֎ Finance - Financial loss, market instability, regulations scrutiny, reputation damage. ֎ Defense - National security threats, loss of military advantage, escalation of conflict, costly revisions and rebuilding.
Plagiarism and Copyright Infringement AI systems can generate original content, such as text, images, audio, and video. This content could be exposed to plagiarism or infringement on the intellectual property rights of others.	Copying the style or content of a published author, artist, or musician without their consent or attribution, violating their creative rights and damaging their reputation. Creating content that is similar or identical to existing works, resulting in legal disputes or claims of plagiarism.
Ethical and Social Implications AI systems can make decisions that affect human lives and their well-being. The decisions made by the AI systems may not always align with human values and rules.	Exhibiting bias or discrimination against certain individuals or groups, based on factors such as race, gender, age, or religion. Causing harm or suffering to humans or other sentient beings, either intentionally or unintentionally. These issues raise ethical and social questions about the responsibility and accountability of AI systems and their developers.

 

Cognizance of AI Security Policies and Risk Assessment

AI security policies and risk assessment in AI are critical to ensure that AI systems are trustworthy, reliable, and resilient to adversarial attacks. AI systems are increasingly used in critical domains such as healthcare, finance, and defense, where the consequences of a security breach can be severe. Therefore, it is essential to identify and mitigate AI risks and vulnerabilities throughout their lifecycle, from design to deployment to operation. 

AI Best Practices for Security and Risk Management 

The following AI best practices allow organizations to improve their ability to secure their AI systems and protect them from malicious actors. 

1. Adopt a security mindset when developing and deploying AI systems.
2. Consider possible attack vectors and scenarios which could compromise AI system's integrity, confidentiality, availability, or accountability. 
3. Use a risk-based approach to prioritize the security measures and controls for different AI components and scenarios, based on the impact and likelihood of an attack. 
4. Apply security principles such as defense-in-depth, least privilege, and separation of duties to the AI system architecture and operations. 
5. Use secure coding practices and tools to prevent common vulnerabilities. 
6. Establish clear AI security policies and governance that ensure compliance with relevant standards and regulations, such as the NIST AI Risk Management Framework which provides a voluntary guidance for incorporating trustworthiness considerations into the design, development, use, and evaluation of AI products, services, and systems. 
7. Implement security AI systems monitoring and testing. 

Monitor: Identify and Categorize AI Risks 

The ATT&CK Matrix for Enterprise, created and maintained by MITRE, is a widely used framework for understanding and categorizing cyber threats and attack techniques. It provides a comprehensive and structured view of various attack tactics, techniques, and procedures (TTPs) that adversaries may employ during a cyberattack. While the ATT&CK Matrix is not specific to AI security and risk management, it can be valuable for AI security practitioners in several ways: 

AI Risk Real Case Scenario

Activity	Description	ATT&CK Matrix
Understanding Attack Vectors	AI systems can be vulnerable to various types of attacks, such as adversarial attacks, data poisoning, and model inversion.	Can help security professionals understand the potential attack vectors and tactics that adversaries might use against AI systems.
Mapping AI-Specific Threats	While not exhaustive, security experts can map AI-specific threats and risks to relevant ATT&CK Matrix categories.	Can aid in identifying potential weaknesses and developing appropriate mitigation strategies.
Incident Response	In the event of a security incident involving AI systems, security teams can use the ATT&CK Matrix to categorize and analyze the attack techniques used.	Can facilitate incident response efforts and help organizations learn from past incidents.

Testing AI Security 

Counterfit is an open-source automation tool developed by Microsoft specifically designed for security testing AI and machine learning systems. It complements the ATT&CK Matrix for Enterprise by providing practical means to test and secure AI systems against these threats. 

Simulate AI Attacks and Vulnerabilities

Counterfit helps security professionals simulate real-world attacks and vulnerabilities on AI models and systems. It provides a practical way to assess the security of AI systems and validate defenses against known attack techniques. 

ATT&CK Mapping

Counterfit can be used to demonstrate how various attack techniques from the ATT&CK Matrix can be applied to AI systems. This allows security teams to assess their AI systems' resilience against specific threats. 

Security Training

Counterfit can be used for educational purposes and security training to help organizations understand AI security risks and develop effective defense strategies. 

Using both the ATT&CK Matrix and Counterfit together can enhance an organization's AI security and risk management efforts by providing a clear roadmap of where the security threats and potential risks lie in the organization.

___________________

The information provided here is solely for informational purposes and does not imply any affiliation, representation, or endorsement by the respective organizations mentioned. We are not affiliated with or representing MITRE, ISO 27001, GDPR (General Data Protection Regulation), NIST (National Institute of Standards and Technology), or Microsoft. 

Each of these organizations, their trademarks, and their copyrighted materials are the sole property of their respective owners. Any references made to MITRE, ISO 27001, GDPR, NIST, or Microsoft are for informative purposes only, and this content is not a substitute for official guidance or legal advice provided by these organizations. 

Please note that the mentioned organizations may update their standards, regulations, or guidelines over time. It is essential to refer to their official publications, websites, or legal counsel for the most current and accurate information related to their respective areas of expertise and authority.

___________________

References:

AI and Ethics: Balancing progress and protection. Dataconomy. (2023, January 16). https://dataconomy.com/2023/01/16/artificial-intelligence-security-issues/

Cybersecurity and AI: The challenges and opportunities. World Economic Forum. (n.d.). https://www.weforum.org/agenda/2023/06/cybersecurity-and-ai-challenges-opportunities/

Will Pearce, R. S. S. K. (2023, May 16). Best practices for AI security risk management. Microsoft Security Blog. https://www.microsoft.com/en-us/security/blog/2021/12/09/best-practices-for-ai-security-risk-management/

Marr, B. (2023, June 5). The 15 biggest risks of Artificial Intelligence. Forbes. https://www.forbes.com/sites/bernardmarr/2023/06/02/the-15-biggest-risks-of-artificial-intelligence/.

OWASP AI Security and Privacy Guide. OWASP AI Security and Privacy Guide | OWASP Foundation. (n.d.). https://owasp.org/www-project-ai-security-and-privacy-guide/.

AI security threats: The real risk behind science fiction scenarios. Security Intelligence. (2023, August 30). https://securityintelligence.com/articles/ai-security-threats-risk/.