OpenAI's Red Teaming Network: Strengthening AI Model Resilience

In an ongoing commitment to enhance the robustness of its AI systems, OpenAI has officially launched the OpenAI Red Teaming Network. This network consists of contracted experts tasked with providing invaluable insights to inform the company's AI model risk assessment and mitigation strategies.

The practice of red teaming has gained increasing importance within the AI model development sphere, particularly with the proliferation of generative technologies. Red teaming serves as a critical checkpoint in identifying and addressing biases present in models such as OpenAI's DALL-E 2. For instance, it has come to light that DALL-E 2 can unintentionally magnify stereotypes related to race and gender. Additionally, red teaming plays a vital role in uncovering issues within text-generating models like ChatGPT and GPT-4, where they may bypass safety filters.

OpenAI acknowledges its prior collaborations with external experts in benchmarking and testing its models, including participants in its bug bounty and researcher access programs. However, the introduction of the Red Teaming Network formalizes and intensifies these efforts. OpenAI's objective is to deepen and broaden its collaboration with scientists, research institutions, and civil society organizations, as outlined in the company's recent blog post.

OpenAI states, "We view this initiative as a complement to established governance practices, such as third-party audits." The network members will be called upon based on their specific expertise to contribute to red teaming activities at various stages of model and product development.

Beyond commissioned red teaming campaigns, members of the Red Teaming Network will have opportunities to interact with one another regarding general red teaming methodologies and discoveries. It's worth noting that not every member will participate in evaluating each new OpenAI model or product. Time commitments, which may range from as little as 5 to 10 hours per year, will be determined on an individual basis in consultation with OpenAI.

OpenAI seeks a diverse array of domain experts for participation, including individuals with backgrounds in linguistics, biometrics, finance, and healthcare. Prior experience with AI systems or language models is not a prerequisite for eligibility. Nevertheless, OpenAI cautions that participation in the Red Teaming Network may involve non-disclosure and confidentiality agreements that could affect other research pursuits.

OpenAI emphasizes the importance of experts' willingness to engage and contribute their unique perspectives to assess the impact of AI systems. The company invites applications from experts worldwide, with a particular focus on geographic and domain diversity in the selection process.

The question arises: Is red teaming alone sufficient? Some argue that it may not be.

In a recent article, Wired contributor Aviv Ovadya, affiliated with Harvard's Berkman Klein Center and the Centre for the Governance of AI, advocates for "violet teaming." This concept involves identifying how a system like GPT-4 might pose risks to institutions or the public good and then actively supporting the development of countermeasures using the same system to protect these interests. This notion holds merit, but as Ovadya points out, there are few incentives to implement violet teaming, let alone slow down AI releases sufficiently to allow for its effective implementation.

For now, it appears that red teaming networks, such as OpenAI's initiative, represent the best available solution to address these challenges.