OpenAI said on Monday, Oct. 19, that its newest AI models are much better at writing software in COBOL, Fortran, Visual Basic and other less common programming languages, and at rewriting old programs in newer ones. These languages have less code and documentation for AI models to learn from than newer ones such as Python and JavaScript, and developers and business customers told OpenAI that coding models often struggle with them. Yet they still run essential systems. IBM says COBOL is behind more than 40% of online banking systems, 80% of in-person credit card transactions and 95% of ATM transactions, and OpenAI notes that Fortran is used in weather forecasting and nuclear reactor safety analysis.
The hard part is often moving an old program into a modern language without changing what it does. Documentation may be incomplete, the original programmers may have left, and important business rules may exist only in the code. Small details matter: COBOL cuts off extra digits after the decimal point unless a program asks it to round, so a rewrite that rounds instead can run without errors and still give different financial results, OpenAI notes.
To measure this, OpenAI built a test that asks a model to move COBOL programs into C++, Java, Kotlin, Python or Rust, and counts a move as a success only if the new program passes every one of a set of hidden tests, giving exactly the same output as the original, character for character. At a low reasoning setting, where the model does less step-by-step work before it answers, GPT-6.1 Sol succeeded 86.6% of the time. GPT-5.6 Sol managed 5.4% at the same setting, and OpenAI says the newer model cost 39% less to run on that test. At a higher reasoning setting, GPT-6.1 Sol reached 97.9%, near the top of the scale.
A second test, MultiLangBench, asks a model to build its own version of SQLite, a widely used program that stores and looks up data, from scratch in each of 20 languages, checked against tests adapted from SQLite's own. At its highest setting, GPT-6 Astra passed 87.4% of the tests on average, up from 26.8% for GPT-5.6 Sol, and did better in every language. COBOL stayed the hardest, at 72.1%, up from 10.9%. GPT-6 Luna, a cheaper model, averaged 40.3% at its highest setting, at what OpenAI estimates is about 78% lower cost than GPT-5.6 Sol, and in COBOL it scored 17.1%, against 10.9% for GPT-5.6 Sol. OpenAI gave scores for different models on each test: it did not report GPT-6 Astra on the COBOL rewriting test or GPT-6.1 Sol on the database test.
Both tests were built and run by OpenAI. It has released the database test's setup and test suite so others can see what it checks and run it themselves, but has not said it will release the COBOL test. OpenAI also says that using these models on real systems still needs expert review and testing against each business's own rules and connections to other software. It has not said how rival companies' models score on either test, or whether any bank or government is using its models to move COBOL systems today.