Not all AI tasks might be winners.
So CIOs should apply “fail quick” rules to their AI initiatives, deciding as shortly as attainable when a promising thought is simply not going to pan out.
That is simpler mentioned than finished. The MIT report “State of AI in Enterprise 2025” discovered that 95% of 153 senior leaders surveyed “are getting zero return.”
To grasp how CIOs resolve when to cease an AI undertaking, we requested two IT leaders: What’s the particular pink flag that tells you an AI pilot has change into a sunk value and must be killed? Each recognized clear, telltale indicators {that a} undertaking is off observe.
-
Soo-Jin Behrstock, chief info know-how officer at Nice Day Enhancements, a direct-to-consumer dwelling transforming firm, mentioned missed milestones are a warning signal to pivot — and that cautious upfront planning makes killing an AI undertaking virtually nonexistent for her.
-
Ed Clark, CIO of California State College, which serves practically 500,000 college students, mentioned stalled progress and weak adoption are clear indicators {that a} undertaking is foundering — and that it is essential to observe for these indicators so leaders can redeploy sources to extra promising efforts.
Under are Behrstock and Clark’s responses to our query, edited for readability and size.
Soo-Jin Behrstock, chief info know-how officer, Nice Day Enhancements
Behrstock: ‘Begin with: What does success appear like?’
“Once we tackle AI initiatives, I all the time begin with: What does success appear like, and the way are we going to measure it?
“For instance, if we’re utilizing AI for gross sales or advertising and marketing predictions, we begin with a small pattern of knowledge that we all know rather well. Based mostly on that, we’ve a superb sense of what the output ought to appear like. If the output is just not directionally proper, then that often tells us one thing is off — it might be the info, the method or the mannequin.
“From there, we set quick milestones, often each couple of weeks, to see if we’re getting nearer to the end result we outlined with measurable outcomes.
“If we’re not [getting closer to the outcome], then we pivot or defer. I don’t consider in pushing AI ahead only for the sake of claiming we’re doing AI. If success is just not clearly outlined or we can not measure progress towards it, that could be a pink flag.
“I do not learn about killing [an AI initiative] except you identify it isn’t aligned with the enterprise.”
‘Pivot to get to success’
“One factor that I do discover is usually some builders get into evaluation paralysis by way of how the AI ought to work. That extends the timeline and the funds. However when you’ve the incremental milestones that are not being met, it’s essential to ask what wants to vary to get to success?
“Let me give an instance: Proper now, we’re engaged on utilizing AI predictive modeling. We’re taking a small pattern of knowledge that we’re actually conversant in, and we’re measuring what the output is, so we are able to say, ‘Here is what good actually appears like. This works.’ Then we’re including extra knowledge into it, so we are able to measure whether or not our mannequin is working accurately or whether or not we have to pivot.
“In such circumstances [where we need to pivot], it might be that we do not have the fitting sources or abilities, so we could have to accomplice with consulting firms to assist us.”
The worth of being ‘very intentional’
“I have never needed to be able to say, ‘Let’s kill it.’ However I might see doing that if what we thought would make sense for the enterprise, we later decide does not. However I have never been in that scenario but as a result of all the pieces’s been very intentional. I am actually cautious to set expectations upfront and outline success. And so if we’re not hitting milestones, it is often [because of issues] round knowledge and course of, so we decide the place we have to regulate and we simply pivot.”
Ed Clark, CIO, California State College System
Clark: A listing of pink flags
“In my thoughts, that pink flag is when the pilot now not has a transparent path to create strategic worth to your group.
“One other pink flag is when the group will get caught in a loop, after they come again with the identical standing updates and also you’re seeing no progress, whenever you see the identical slides, the identical hurdles, whenever you hear, ‘We’re virtually there’ and nothing is going on, and there are not any deliverables. Then you understand this factor is caught.
“One other factor to search for is when adoption is weak, whenever you’ve rolled out one thing that everybody mentioned, ‘Oh, that is going to be so cool,’ however then nobody makes use of it.
“Additionally, if the manager sponsorship disappears, that is one other factor I search for.
“And one other sign that is actually vital — and this occurs on a regular basis — is when distributors are making a core functionality for his or her platform [that’s similar to the AI project you’re developing]. We’re not within the enterprise of competing with these distributors.
“After which the very last thing — and this occurs particularly in synthetic intelligence — is when the unique use case that you just’re all enthusiastic about is simply sort of out of date as a result of the know-how strikes so quick.
“Any of these might be pink flags.”
Discovering the explanations behind the pink flags
“It’s a must to ask why some tasks find yourself with pink flags.
It might be what’s being requested for is just too out of the vary of what your group is ready to accomplish. Then it’s important to work out whether or not [the AI project] is an thought ok to pursue — the place I wish to chase it down and perhaps usher in exterior sources to get it finished, or whether or not it is a pilot the place it is OK to your group to only observe and be taught, or whether or not the manager who was enthusiastic about it however will not meet with us about it actually does not care about it anymore, so that you should not be pursuing it.
“All the cash and energy you are spending might be going towards one thing else that would obtain the goals of the group.”
The pilot that didn’t take maintain
“I can inform you a selected instance: One of many issues that we’re always is affordability. And we thought we might make open textbooks — these free textbooks — extra accessible to college students by creating an AI overlay that [functions as] a tutor.
“So we tried to pilot this factor, however there was no adoption. It was irritating as a result of we noticed a method to make open textbooks extra helpful for our college students by including this assist system.
“It seems that college generally do not like open textbooks, as a result of they do not include the educating sources they need. And so although it was an exquisite thought that may assist serve our mission and advance our strategic objectives — and that executives initially thought can be nice — we needed to kill the concept.”
What the group discovered from killing the undertaking
“It did harm to make that decision, as a result of I feel the quantity our college students [collectively] spend in textbooks yearly is within the tons of of tens of millions of {dollars}. However we discovered quite a bit engaged on that undertaking. Like, if we’re actually going to do that, we want to ensure it is multilingual and that it could actually deal with mathematical symbols. We discovered issues which can be going to be helpful for our group that may be utilized elsewhere.”
