Submission
Privacy Policy
Code of Ethics
Newsletter

Hype or Value? The Real Measure of Generative AI – Part I.

Generative AI has received a great deal of attention in recent years, yet in practice a wide gap remains between its promises and its real-world impact on corporate applications. In fact, most projects fail to deliver lasting and measurable benefits, although under certain conditions opportunities for genuine value creation are already emerging. This duality makes it urgent to understand what separates failures from successes.

Tools like ChatGPT or Copilot have already become part of many people’s daily routines, helping them draft emails, collect ideas, or complete code. Because these tools are used so widely, it is easy to get the impression that generative AI is both ubiquitous and genuinely useful, as if it had already transformed how companies operate. In reality, this is only a surface-level impression, and the underlying picture is far more nuanced. Part of the reason is that the technology has received disproportionate media attention in recent years while at the same time becoming easily accessible to almost anyone. Many individuals use it effectively for personal purposes, but corporate environments impose entirely different demands. Here, it is not enough for a tool to suggest ideas or draft emails; it must integrate into regulated processes and guarantee secure data handling. This explains why such tools do not perform as well when companies attempt to implement them. Although adoption is spreading rapidly, corporate experience shows that most deployed systems do not in fact deliver the expected business results.

This picture is reinforced by a report published recently by MIT, NANDA, which found that the introduction of generative AI fails at most companies. The study, based on interviews and case analyses covering more than three hundred corporate projects, showed that 95 percent of such initiatives produced no measurable business benefit during the observed period. The authors also emphasized that these failures were not the result of technological flaws or legal barriers. The figure indicates that while the systems function, they are unable to create real value: the projects did run, but they did not evolve into solutions that measurably improved operations or increased revenue. As it turned out, such systems often perform well in demonstration settings, but once they need to be integrated into everyday operations, they repeatedly stall. The main reason is that they are difficult to align with existing corporate processes and often fail to provide adequate data security. As a result, they do not become an organic part of workflows and ultimately do not make a meaningful contribution to either performance or revenue growth.

To describe the current reality, the authors use the term “GenAI divide.” This refers to the situation where many people try AI tools, but only a few are able to use them in a lasting, profitable way that generates real returns. The researchers distinguished between two groups. One consists of those who experiment and launch pilots but never reach systems that are operational and useful. The other is a much smaller group that has managed to integrate generative AI into its processes with measurable results. The gap is significant: only one in ten companies was truly able to benefit from the technology, representing the small minority that did not fall into the previously mentioned 95 percent and could actually demonstrate business value and returns.

It is worth examining why most initiatives fail to produce genuine business value, while the few successful companies achieved dramatic efficiency gains and real structural change. According to the authors, most companies try solutions that, based on user experience, cannot adapt effectively. They fail to adjust to the processes they are supposed to support and generally do not evolve based on the operational experience that accumulates over time. So-called off-the-shelf solutions, such as chatbots without persistent memory, perform well in simple tasks like drafting emails or brainstorming. The problem begins when AI is expected not only to generate text but to become an active part of the workflow. At that point, it must fit into corporate structures, understand context, and respond to feedback. Yet today’s systems generally lack memory, cannot think in context, and do not learn from use. This clearly shows why the transition from individual experiments to corporate-level, sustainable, and profitable applications is so difficult.

According to the researchers, the biggest obstacle is not a lack of hardware or legal regulation but the fact that most generative AI tools cannot learn from user feedback. They do not remember previous work and do not fit well into workplace environments. This missing flexibility frustrates users, who often end up wasting time and eventually have to return to traditional methods. While many corporate projects remain stuck in pilot phases for months, employees continue to rely on their own trusted AI tools in daily work. The report calls this the “shadow AI economy”: as long as official systems fail to become part of operations, employees automate parts of their tasks on their own. They frequently use personal ChatGPT accounts, Claude subscriptions, or other public applications to draft emails, edit spreadsheets, create summaries, or produce customer letters. Although this often happens without the employer’s knowledge, in practice it still results in tangible time savings.

The GenAI divide is also clearly visible at the industry level. The study examined nine major sectors. Out of these, only two (technology and media) showed signs of genuine transformation, despite the fact that generative AI is often described as a disruptive, transformative technology. The potential is undoubtedly there, but in most industries it has yet to materialize in a concrete way. In these two sectors, new business models have emerged, workflows have shifted, and competitive conditions have changed. This is no coincidence: technology and media companies operate with larger data flows, abundant digital content, and a stronger willingness to experiment. In other industries, such as healthcare, finance, or energy, many attempts have been made, but so far, they have not produced significant results.

Despite frequent setbacks, many still believe that generative AI alone can transform how companies operate. The evidence, however, paints a more nuanced picture. The report identified five common misconceptions that often mislead decision-makers. The first is that AI will eliminate masses of jobs in the short term, although evidence for this is limited. The second is that generative AI has already fundamentally reshaped corporate operations, when in fact the changes so far remain modest. The third is that companies are slow to react to technological shifts, while in reality most have already launched some form of generative AI project. The fourth is that legal and technological barriers are the biggest obstacles, when in fact the real challenge lies in the inability of systems to evolve based on experience gained after deployment. The fifth is that in-house development is the most effective path, although the study shows these projects fail more often than those conducted with external partners.

The study concludes that companies remain on the wrong side of the GenAI divide not because they are unwilling to experiment but because they misunderstand what leads to genuine success. Many projects have started from the assumption that AI will deliver results on its own, without significant adaptation. Experience shows instead that current systems do not integrate seamlessly into organizational operations, do not evolve through use, and remain static as a result. Until this mindset changes, generative AI tools will remain impressive in demonstrations but will not generate lasting and measurable business value.

In the next section, we will look at how generative AI can nevertheless be applied successfully. We will examine which organizational, technological, and strategic approaches can lead to results and which pitfalls are worth avoiding.


István ÜVEGES, PhD is a Computational Linguist researcher and developer at MONTANA Knowledge Management Ltd. and a researcher at the HUN-REN Centre for Social Sciences. His main interests include the social impacts of Artificial Intelligence (Machine Learning), the nature of Legal Language (legalese), the Plain Language Movement, and sentiment- and emotion analysis.