Don't replace the human. AI's role in real innovation
Are AI feedback loops replacing real customer insights in innovation?
Key Points
Innovation requires real customer feedback, not just AI simulations
AI should augment exploration, not replace customer validation
Strong innovation foundations are essential when using AI
The wisdom as I read and hear it is that innovation will accelerate, move even more quickly as generative AI drives speed - it accelerates everything. Speed to idea, to prototype, to "feedback," to decisions. Even to building the solution. For many, I imagine it both feels and sounds like the dream: insights at scale, ideation on tap, simulation and global validation and knowledge - without users. A lot of that I'm sure is true, but there's always a second side to the ledger.
Are we going to see something subtle but potentially dangerous? After years of hard work convincing organizations that users matter, feedback loops that once centered around customers are being quietly re-routed.
Are teams testing with users - or testing with the model? Are they using AI to validate, or just to echo? When AI becomes your primary feedback mechanism, you're not really accelerating innovation based on user knowledge and purchase patterns; you're building in a closed loop.
I don't think this is a theoretical risk. Unless the social-media hype can be completely discarded, you see/hear products and businesses " launching" with "AI" stuff, and I wonder what problems that are being solved, based on real need. There are certainly some real breakthroughs happening on the back of generative AI, but it's not just me that can hear it in planning meetings where "customer needs" are summarised via ChatGPT. There needs to be more specificity and nuance to make decisions.
Speed isn't strategy
Probably ten years ago with [Guy Geron](https://www.linkedin.com/in/guygeron) & some from [Thoughtworks](https://www.thoughtworks.com) we used the Google Design Sprint on some experimental work at IOOF. I was looking for the teams to challenge assumptions fast. They had people in the room from Day 1. It wasn’t perfect, but it demanded discipline: clarity of the problem, collaboration, and human input. It didn’t guarantee success—but it made better decisions possible. We built software that was used, matters, and in fact still exist today. I use it.
I wonder if AI is now collapsing much of that structure. It can:
• Surface themes from thousands of survey responses in minutes
• Generate a dozen design concepts before a workshop even starts
• Simulate user flows and stress-test ideas at unprecedented scale
But is all that 'power and speed' pushing innovation knowledge away from the problem rather than into it?
The moment a team replaces customer interviews with AI-generated personas, or accepts model outputs as "validation," they've left the realm of product discovery. They're optimising for coherence, not truth and reality.
And the outputs will almost always sound smart—because the model is trained to be smart.
The comfort of coherence
There's an old product mantra: fall in love with the problem, not the solution. But today, teams are falling in love with the synthesis. With the simulation. With the illusion of understanding. And especially the speed of producing some software without developers and the hassle that comes with it.
AI Models are:
• Fast
• Agreeable
• Always available
• Consistent
Real Users are:
• Messy
• Contradictory
• Hard to schedule
• Ask challenging questions
I see why AI is appealing, but it's lazy. Real users may not know what they want, but they're the only source of truth that matters. They don't hallucinate. They don't reinforce your bias. They show you where things break. They are the ones who will pay for your product/solution/service.
Innovation isn't about removing friction. It's about using it. When AI removes the friction and replaces the customer, the opportunity is lost.
AI can help if - if you let it
I've been using AI in my work every day. I don't reject AI. When it comes to code, it does great - and weird stuff. So, far from it. My view is like many things, when organised, thought about, applied logically, tested mentally, it's an amazing augmentation of the innovation process (and many other processes).
I'm not an expert in what all teams are doing, but I imagine the best teams aren't using AI to replace the customer—they're using it to better understand them. CPO's and senior product and tech leaders are guiding their teams to use AI for:
• Preparing sharper customer interviews
• Clustering insights that would otherwise be buried
• Generating divergent thinking and exploring non-obvious angles
AI accelerates the exploration, not the confirmation. Really, it's crap at confirmation. If you disagree, use it.
In these environments, AI isn't the "source of truth." It's a tool for better questions, not easier answers. Used this way, it doesn't bypass the human—it supports them.
Don't replace; refocus
Or course, innovation processes need to evolve, but not by removing the customer. By making space for more rigorous distinction between model insight and market truth. That means:
• Designing workflows that expect AI to be wrong sometimes
• Training teams to interrogate rather than accept what a machine says
• Fostering a culture where "real feedback" still means someone out there said it, felt it, needed it—and might PAY for it
Foundation matters: If your innovation foundation is weak—lacking clear purpose, customer anchor, or internal discipline—AI will expose and amplify that weakness. If the foundations are strong, AI can become an incredible force multiplier.
But AI doesn't decide that for you. You do. Your process does. Own it. Don't delegate it. Don't outsource it. There's enough of that already.
The core challenge of innovation hasn't changed: find out what matters, and build towards it. AI makes that faster—but only if you stay honest about where truth comes from.
Otherwise, you're just talking to yourself—faster. And that might feel like progress. But it isn't. Don't be the team that mistakes speed for substance. Stay grounded. Stay curious. Know your real user.
Author
Andrew Todd
These are my personal views based on practical experience, influenced by aspects of my professional life, work engagements and often curious discussions with those I see as software, technology, leadership and strategy experts.
Initially published on 27 May 2025