AI Signal
Your rewrite can change the money: OpenAI tests COBOL migration
OpenAI’s October 19 evaluations test whether code migrations preserve behavior. COBOL rounding shows why a working rewrite can change results. Company tests need real-world review.
· 1:13
Transcript
The original narration, preserved as published. Relative dates refer to the episode’s coverage window.
Would you trust a rewrite that quietly changes your money? OpenAI published new coding evaluations on October nineteenth. One tests a deceptively hard job: moving old business software into another language without changing what it does.
Take COBOL, still used in transaction systems. Its decimal arithmetic can discard extra digits unless rounding is requested. A rewrite that rounds instead can run perfectly and produce different financial results. The bug is the changed rule.
OpenAI tests migrations into languages including Python and Rust. Hidden tests compare observable output byte for byte. Every test must pass for a migration to count. That checks behavior, beyond whether the new code builds.
The company reports eighty six point six percent accuracy for GPT six point one Sol at low reasoning effort, against five point four percent for GPT five point six Sol. These are OpenAI evaluations, not independent production results.
The separate MultiLangBench test suite is public on GitHub. It tests building a SQLite compatible database across programming languages. Watch whether these gains survive your own business rules, dependencies and expert review before replacing a live system.