Ethereum Vitalik Buterin: ‘Even Mars May Not Be Safe’ If AI Turns On Us

On Monday, Ethereum creator Vitalik Buterin reflected on “techno-optimism” inspired by Marc Andreessen's comments on AI in October's Techno-Optimist Manifesto. While Buterin agrees with Andresen's positive outlook, Buterin also points to the importance of how AI develops and the future direction of the technology.

Buterin acknowledges the dangers of artificial intelligence, including the extinction of humanity.

“This is quite a claim: there are many islands of civilization that will remain untouched to pick up the pieces, as catastrophic as climate change, or man-made pandemics, or nuclear war could cause.” he said.

But a super-intelligent AI, if it decides to turn on us, could leave no survivors and end humanity for good, Buterin said. “Even Mars may not be safe.”

Buterin in 2010 According to the AI ​​Impacts by 2022 survey, between 5% and 10% of participants said they believe that humans will be wiped out by AI or that humans will be unable to control AI. The security-focused open source movement is better suited to lead AI development than closed and proprietary corporations and venture capital funds, he said.

“If we want a future where humans are not just pets, but hold meaningful agency in the world – something like this is the most natural option,” he said.

What is needed, Buterin continues, is an active human desire to choose its direction and consequences. “The ‘profit maximization' formula doesn't immediately occur to them,” he said.

Buterin said he loves technology because it expands human capabilities, pointing to the history of inventions from hand tools to smartphones.

“I believe these things are great, and expanding humanity's reach to the planets and stars is great, because I believe humanity is deeply good,” Buterin said.

While Buterin believes that transformative technology offers a bright future for humanity, he rejects the idea that the world should remain as it is, with only a little greed and concern for public health.

“There are certain types of technology that make the world safer than other types of technology,” Buterin said. “There are certain types of technology that, if developed, can reduce the negative effects of other types of technology.”

Buterin warned of digital authoritarianism and the rise of surveillance technology against those who oppose or oppose a government controlled by a small group of technocrats. He says most people would rather see massively advanced AI a decade later than be monopolized by one group.

“My fundamental fear is that the same management technologies that allow OpenAI to serve more than a hundred million customers with 500 employees will also allow a 500-person political elite or a 5-person board to keep an iron fist over the entire country,” he said.

While Buterin says he sympathizes with the effective acceleration (also known as “e/acc”) movement, he has mixed feelings about its enthusiasm for military technology.

Referring to the idea that military technology is good, he said, “The enthusiasm for modern military technology as a good force seems to require a belief that the technological force will be one of the most reliably good in most conflicts.” It's good because it's being built and controlled by America and by America.

“Does being an E/A require being America's top professional, betting everything on the government's present and future morale and the nation's future success?” he said.

Buterin warned against giving “extreme and vague power” to small people. He says this mindset can be adapted to effective altruists, libertarians, populists, blockchain advocates, and sun and moon punks.

“A defense-friendly world is a better world for many reasons,” Buterin said. “First, of course, is the direct security benefit: fewer people die, less economic impact, less time wasted on conflict.”

“What is less appreciated, however, is that a world favored by protectionism is conducive to healthier, more open and more freedom-respecting forms of governance,” he concludes.

Buterin said that although they think that it is necessary to build and speed up, the society should constantly ask what we are speeding up. Buterin suggests that the 21st century may be the “decisive century” for humanity that could determine humanity's fate for millennia.

“These are challenging problems,” Buterin said. But I look forward to seeing and participating in the great collective effort of our eyes to find the answer.

Edited by Ryan Ozawa.