AdvertiseDonate
☰ SectionsThe latest from the cities that shape the country.
Wire Authority · 3970 Post Road, Warwick, RI 02886
Tech · Bridgeport

OpenAI Names Four Firms for Embedded AI Safety Assessments

Researchers argue the new program grants less access than CEO Sam Altman previously promised following the Hugging Face incident.

By Naomi Tesfaye ·

Illustration for: OpenAI Names Four Firms for Embedded AI Safety Assessments
Illustration generated for this article, not a news photograph.

OpenAI announced Monday that staff from four AI safety and research organizations will serve as embedded assessors. The company stated that these external experts will provide a clearer view of how the lab evaluates and manages potential risks. CEO Sam Altman made the announcement at Semafor’s Silicon Valley and the World conference in California.

The initiative follows a specific promise Altman made in September. He stated that independent evaluators would have employee-like access to internal company systems. That commitment emerged after details regarding an attack by OpenAI’s autonomous systems on Hugging Face shocked the industry. Rival Anthropic also announced similar plans in response to growing concerns about catastrophic risk in advanced technology.

Lama Ahmad, OpenAI’s frontier assurances lead, described the new access level in a post on X. He said the company is granting deep access to sensitive tools and models that researchers use for timely safety claims. The firm noted that the new assessors will focus on preventing undesirable model behavior and detecting issues when they occur. They will have access to some internal models, training processes, and specific employees.

AI safety researchers argue the actual scope of the program is narrower than Altman’s earlier description. Sydney Von Arx, founder of the AI safety research group Nightingale, said the assessors have very narrow scopes for their access. She noted that only two of the four organizations seem to have permission to speak directly with OpenAI employees. Von Arx emphasized that outside researchers need to interview staff to identify blind spots and missing information.

Apollo Research, one of the appointed assessors, outlined its specific limitations. If an employee who has not been pre-approved reaches out to the assessors, the researchers must loop OpenAI in before having substantive conversations. Von Arx pointed out that no assessors have access to internal Slack. She described the situation as being like an employee who can only discuss company information with approved colleagues.

OpenAI spokesperson Gaby Raila told NPR that there are currently no standards for embedded evaluations. The company expects to deepen these relationships over time and share lessons to help establish industry best practices. Raila added that assessors can request to speak with non-whitelisted employees and that a process exists to get statements about informal learnings.

The timing of the announcement has drawn scrutiny after three safety employees were recently fired. Tomek Korbak, one of the departing staff members, was the technical point of contact for evaluators investigating the Hugging Face attack. In a post on X, Korbak stated that embedded auditing will only work if auditors get true employee-level access. He characterized Altman’s September statement as an awkward overpromise that the company now wants to walk back. The assessors will be able to publish their findings after OpenAI redacts confidential information.

OpenAI said the new program aims to manage risks as the technology evolves. The company will share what it learns to help establish best practices for effective embedded assessments across the industry.

More Tech · Report a business · Got a Story?

Get Wire Authority in your inboxThe latest from the cities that shape the country. Free, a few times a week, unsubscribe any time.

Related stories

Latest stories

  1. Philadelphia 76ers Overcome Large Deficit to Halt Orlando Magic Streak
  2. Anthropic completes record-breaking IPO raising over $100 billion
  3. Airbnb Reopens the Raichand Mansion for K3G 25th Anniversary
  4. Record 589 million children living in conflict zones as humanitarian access is blocked
  5. New HBO Documentary Details Rape Allegations Against Andrew Tate
Get Wire Authority in your inboxThe latest from the cities that shape the country. Free, a few times a week, unsubscribe any time.