OpenAI said on Thursday, Oct. 8, that it has banned ChatGPT accounts tied to two covert influence operations, campaigns that try to sway public opinion or political outcomes while hiding who is really behind them. One came from Russia and one from Iran. Both hid behind what OpenAI calls false fronts: a research group in Latin America that OpenAI says was secretly run from Russia, and seven made-up journalists who pitched articles to news sites around the world. A false front is a real-looking group or writer that carries a campaign's message, so readers never see where it came from.
In the Russian case, OpenAI says the operators controlled a group called the Social Research Center through a fake persona named "Mia Clark." Because OpenAI does not allow access from Russia, they reached ChatGPT through VPNs, services that hide where a user is connecting from. OpenAI says the evidence suggests the center's staff in Latin America did not know they worked for a Russian group, and the center's website carried more than 60 articles, most of them original. The operators mainly used ChatGPT to write reports to an unknown boss, and sometimes to draft fakes, such as a letter meant to tie Ukraine's honorary consul to Panama to corruption.
The operators claimed that in May they used a fake email address posing as Lima's regional education office to tell Peruvian schools to hold events on Ukraine that mentioned Stepan Bandera, a nationalist seen by some Ukrainians as an independence figure but linked in Russia and Poland with fascism and wartime atrocities. They said some schools replied. OpenAI says the story was later reported in Latin America and Europe and appears to have drawn comment from a Polish member of the European Parliament, and that the fake fed into old tensions between Ukrainians and Poles. The operators also claimed a similar fake in June, about schools in Ecuador pledging allegiance to President Daniel Noboa and to Erik Prince, the former head of the military contractor Blackwater. OpenAI found news coverage matching that claim and a detailed rebuttal from Ecuador's education minister.
OpenAI rates the Russian campaign a 5 on the Breakout Scale, which runs from 1 to 6 by how far a campaign spreads, and says it is the first it has rated that high among the 30 campaigns it has exposed in two and a half years. OpenAI says this fits a pattern across all 30: campaigns that get their material into real news outlets reach far more people than those that rely on fake social media accounts. It rates the Iranian campaign's articles a 4 and its social media comments a 2. Its fake journalists placed almost 100 articles, most of them about the US-Iran conflict, across about a dozen small and medium online outlets, one with almost 2 million followers on Facebook, while its comments drew little response. OpenAI says it looked like a commercial firm working for hire but could not say which.
Much of the evidence comes from the operators' own reports. OpenAI says the Russian operators often took credit for events they had nothing to do with, and the Iranian ones counted their reach in a way that greatly overstated it, so their claims of success cannot be taken at face value. Not all of the content that reached news sites came from OpenAI's models, and the company says the campaigns closely resembled those run before AI, with AI making some of the work easier. OpenAI says it has shared both cases with authorities. It is not yet known whether any government will act, or whether the research group will keep running, though OpenAI notes that similar fronts in the past stopped once they were exposed.