OpenAI’s plan to become a for-profit company could encourage the AI startup to take shortcuts on security, a whistleblower said.
William Saunders, a former research engineer at OpenAI, told The Guardian that he was concerned by reports that the ChatGPT developer was preparing to change its corporate structure and would no longer be controlled by its nonprofit board.
Saunders, who expressed his concerns in testimony before the united states senate This month, he said he was also concerned about reports that OpenAI CEO Sam Altman could have a stake in the restructured business.
“What concerns me most is what this means for the governance of security decisions at OpenAI,” he said. “If the nonprofit’s board of directors is no longer in control of these decisions and Sam Altman has a significant ownership stake, this creates more incentives to compete and cut corners.”
OpenAI was founded as a non-profit entity and your letter commits the startup to building artificial general intelligence (AGI), which it describes as “systems that are generally smarter than humans,” that benefits “all of humanity.” However, the potential power of an AGI system has alarmed experts and professionals, including Saunders, amid fears that the competitive race to build such technology could lead to security concerns being overridden.
Saunders said in written testimony before the Senate that he left the company. because “he lost faith” that OpenAI would make responsible decisions about AGI. Saunders was a member of OpenAI’s super-alignment team, a now-disbanded group tasked with ensuring that powerful AI systems adhere to human values and goals.
after newsletter promotion
Saunders has now said that a move to a for-profit entity could undermine the goals of the structure current, according to which OpenAI has a for-profit entity that limits returns for investors and employees. All profits generated above that limit are returned to the nonprofit organization for “the benefit of humanity.”
Saunders said a purely for-profit entity cannot distribute its profits to society if, for example, it developed technology that made a significant number of jobs obsolete.
“OpenAI was only supposed to allow limited profits for investors and employees, and give the rest to nonprofits,” he said. “So if OpenAI created AI technology that caused large-scale unemployment, they wouldn’t be keeping the profits themselves and giving back to society. “Changing to a for-profit organization suggests this is no longer a priority.”
OpenAI has been contacted for comment. Its bylaws state that the company is “committed to conducting the research necessary to make AGI safe” and recently made its safety and security committee an independent entitywhich also meant that Altman resigned as one of its members.
OpenAI is reportedly considering restructuring as a public benefit corporation, which has no limit on its profits but is committed to making a positive impact on society. Reuters also reported this week that the nonprofit entity would have a stake in the new business.
OpenAI declined to comment on the details of the restructuring, but said the nonprofit would continue to exist. In a statement issued Thursday, OpenAI president Bret Taylor said the board had discussed giving Altman a stake in the business, but no specific figures had been discussed. It has been reported that Altman may be given a 7% stake in OpenAI, which is seeking $6.5 billion in fundraising investment that could take its value to $150 billion.
“The board has discussed whether it would be beneficial to the company and our mission for Sam to be compensated with stock, but no specific numbers have been discussed nor have any decisions been made,” he said.
