Explaining how an AI-driven decision was reached remains a challenge for many businesses, even when the technology is meeting their financial expectations. A new FICO survey of 1,004 senior data, analytics and AI leaders found that 85.1 percent of respondents said AI had met or exceeded their initial ROI expectations. However, just 28.5 percent of decisions that directly affect customers, such as personalization or low-risk customer service, were made by AI, and only 5.2 percent said they were very confident explaining AI-driven decisions to regulators and customers.
Rachael Hadaway, vice president of AI product at FICO, said companies can get returns before using AI in decisions that carry direct financial or regulatory consequences. She said some decisions may require a company to explain to the customer involved or a regulator how the decision was reached. FICO, a global analytics and decision management software company, examined AI returns, governance and explainability in its new report, “State of Responsible AI 2026: The Confidence Gap in AI Decisions.” The ROI numbers come with an important qualifier: Respondents were comparing actual returns with their own initial expectations, not a common financial benchmark.
Scott Zoldi, chief analytics officer at FICO, said describing how an AI model typically handles similar cases is different from explaining why it reached a particular decision. He said companies should also be able to trace what data went into decisions such as credit eligibility or fraud-related transaction blocking and where it came from. His concern is that knowing how a model generally behaves does not necessarily tell a customer or regulator why one person received a particular outcome.
The survey found another gap between respondents’ confidence about regulatory readiness and the extent to which they said responsible AI standards were incorporated into operations. Among respondents, 82.8 percent described their organizations as in line with or ahead of anticipated regulatory requirements. Responsible AI adherence ranked lowest among five operational standards measured, behind model performance monitoring, security, regulatory and compliance adherence and continuous data quality monitoring.
Only 60.5 percent rated it a four or five on a five-point scale. Zoldi said the finding troubled him. He argued that confidence may partly depend on how much an organization is currently required to demonstrate.
Depending where I am, very little, unless there’s some sort of customer impact,” he said. The finding also comes with a geographic caveat. Sixty-seven percent of respondents were based in North America, a concentration the report says likely shapes how they assess anticipated regulation.
Extract — continue reading at the source.
