Adapting security for AI 15 AI-specific security measures to rethink their data practices and security controls to ensure safe use of AI: AI applications are fundamentally different Attack simulation (red team/pen testing) than traditional applications. Traditional has to operate differently, focusing on using applications are deterministic, which means human language to trick AI models in addition they generate the exact same output every to exploiting deterministic code vulnerabilities. time they get the same input. Today’s security controls and security assumptions are built Security and technology roles have to rely around that predictability. heavily on threat models to evaluate these AI based applications that use generative AI new system designs until a knowledgebase of models are different because they are dynamic security controls are established for standard in nature—the model will generate a different application patterns. output each time they are run with the exact Business and AI application roles that are same input. For example, asking an image sponsoring and developing AI projects need generation model to “draw a picture of a to work with security teams to understand the kitten in a security guard uniform” repeatedly inherent risks for AI and available mitigations. is unlikely to generate the exact same picture twice (though they will all be similar). Data owners need to work with security This dynamism offers new value for teams to ensure that sensitive data is classified businesses but also introduces new types and handled properly by AI (which may be of security risks. This dynamism also means excluding its use by AI). that current (deterministic) security controls The image below illustrates how AI applications designed will not be effective against AI are typically a combination of both predictable applications. This requires the organization deterministic logic and dynamic AI logic: Different technical Classic app components AI components use exploits and defenses for: use predictable logic dynamic logic Precise interruption/ Consistent (deterministic) A pattern of variable outcomes redirection of logic flow outcomes based on execution based on model design, training General biases and of classic programing data, real-time inputs, etc. hallucinations in outcomes For more information on the types of threats to AI, see Microsoft AI Red Team