OpenAI yönetim kuruluna katılma hakkında kişisel beyan

I am excited to be joining the OpenAI nonprofit board, serving on the Safety and Security Committee to support safety oversight. Based on the recent trajectory of capabilities and the continued difficulty of alignment, I now believe there is a meaningful risk that rapid acceleration in AI capabilities leads to catastrophic and irreversible loss of control in the very near term.1 I do not think that the AI industry in general, including OpenAI, is currently on track to reduce this risk to an acceptable level. I’m joining because I believe that if OpenAI rises to the occasion we could significantly reduce risk. The SSC has an important and challenging role in overseeing risk management at OpenAI, and I hope to help provide expertise and assistance in a critical moment. My joining is not an endorsement or criticism of OpenAI’s safety practices in particular; I hope that all frontier companies strengthen safety oversight and I am excited to work on this at OpenAI. I believe that the rest of the world should judge OpenAI, and all AI developers, by externally verifiable behavior and results. In the rest of this post, I’ll explain why I believe loss-of-control risk is now acute and how I think about the current situation in the AI industry. First, automated AI R&D could lead to a very rapid acceleration in AI capabilities very soon.

OpenAI has predicted that we might have capabilities sufficient to fully automate AI research within 18 months; my personal forecast is extremely uncertain and I think it could easily take anywhere from several months to several years. Full automation of AI R&D means that improvements in training and algorithms can directly increase the quality and quantity of automated AI researchers available to do additional research. Existing evidence is very uncertain but suggests that this positive feedback loop might be strong enough to overcome diminishing returns and compute bottlenecks, leading to a rapid intelligence explosion. If this happens, then within six months of full AI R&D automation we could see more algorithmic progress than has occurred since the development of the Transformer nearly a decade ago. I believe this would result in superintelligent AI systems. Second, we currently train our AI agents with RL to get as much reward as they can.