This paper was accepted on the Workshop on Regulatable ML (ReML) at NeurIPS 2025.
Latest developments in AI governance and security analysis have known as for red-teaming strategies that may successfully floor potential dangers posed by AI fashions. Many of those calls have emphasised how the identities and backgrounds of red-teamers can form their red-teaming methods, and thus the sorts of dangers they’re prone to uncover. Whereas automated red-teaming approaches promise to enhance human red-teaming by enabling larger-scale exploration of mannequin conduct, present approaches don’t think about the function of id. As an preliminary step in direction of incorporating individuals’s background and identities in automated red-teaming, we develop and consider a novel technique, PersonaTeaming, that introduces personas within the adversarial immediate era course of to discover a wider spectrum of adversarial methods. Particularly, we first introduce a technique for mutating prompts based mostly on both “red-teaming skilled” personas or “common AI consumer” personas. We then develop a dynamic persona-generating algorithm that mechanically generates varied persona varieties adaptive to completely different seed prompts. As well as, we develop a set of latest metrics to explicitly measure the “mutation distance” to enhance present range measurements of adversarial prompts. Our experiments present promising enhancements (as much as 144.1%) within the assault success charges of adversarial prompts by way of persona mutation, whereas sustaining immediate range, in comparison with RainbowPlus, a state-of-the-art automated red-teaming technique. We focus on the strengths and limitations of various persona varieties and mutation strategies, shedding mild on future alternatives to discover complementarities between automated and human red-teaming approaches.
- † Carnegie Mellon College
- ‡ Unbiased Researcher
- ** Work performed whereas at Apple
