OpenAI recently announced the establishment of a Preparedness team dedicated to assessing, forecasting, and safeguarding against the risks associated with highly-capable AI systems. In addition to this, it also announced the launch of the Preparedness Challenge,

The challenge is aimed at identifying less obvious areas of concern related to catastrophic misuse prevention. The challenge offers up to $25,000 in API credits to the top 10 submissions, with the potential to discover candidates for the Preparedness team among the leading contenders.

Click here to apply.

OpenAI, with its mission of creating safe artificial general intelligence (AGI), has consistently emphasised the importance of addressing safety risks across the entire spectrum of AI technologies, from existing models to the potential future superintelligent systems. This endeavour is in line with the voluntary commitments made by OpenAI and other leading AI research labs in July, focusing on promoting safety, security, and trust within the AI domain.

OpenAI’s Preparedness team, under the leadership of Aleksander Madry, will play a pivotal role in this effort. The team’s scope extends from assessing the capabilities of upcoming models to those with AGI-level proficiency. Their mission encompasses a wide range of categories, including individualized persuasion, cybersecurity, and the management of threats related to chemical, biological, radiological, and nuclear (CBRN) domains. Additionally, the team will address issues concerning autonomous replication and adaptation (ARA).

The core of OpenAI’s approach lies in understanding and mitigating the risks associated with frontier AI models. These models, expected to surpass the capabilities of today’s most advanced AI systems, hold immense potential for the betterment of humanity. However, they also pose severe and complex risks, necessitating thorough preparedness and precautionary measures.

The company is also actively seeking talent from diverse technical backgrounds to join the Preparedness team and contribute to the enhancement of frontier AI models.

Meanwhile, speculations are rife that at the upcoming OpenAI DevDay conference, the company might just announce their first completely autonomous agent which might ultimately lead to AGI. OpenAI chief Sam Altman is known to tease people with hints that the company has achieved AGI internally. While he has clarified that he was kidding, later on, things might take a pretty interesting turn this time around.