AI-ready data is a power problem

Every AI programme eventually hits the same wall, and the wall is always described in engineering language. The data is not ready. In February 2025 Gartner forecast that through 2026, organisations would abandon 60% of AI projects unsupported by AI-ready data — a prediction, not a post-mortem. Read that as an engineering problem and you buy pipelines, catalogues and quality dashboards. Some projects, though, stall on something no pipeline touches: the question of who pays to prepare the data.

Part of data readiness really is technical: the data is missing, or the source system cannot release it. That leaves a case the language of engineering describes badly. When the data exists and can legally be shared, a project can still stall: the source team bears the cost of preparing it, and the benefit lands in another department. The silo then needs no bad faith and no lack of discipline, because people measured on something else are enough.

Your data quality problem has a job title and a bonus scheme.

That sentence is literally true when a named person carries the cost of preparing the data in their annual objectives and nobody has written down their share of the benefit. Where the data simply does not exist, job titles and bonuses explain nothing.

Read the numbers before you quote them

The 60% figure is usually quoted without the qualifiers that sit next to it in the source. Gartner's press release of 26 February 2025 says that "through 2026, organizations will abandon 60% of AI projects unsupported by AI-ready data." It is a forecast issued in early 2025 about a window ending in 2026, and nobody has published a check against outcomes. The denominator is projects unsupported by AI-ready data, not all AI projects. The popular version, "60% of AI projects will fail," is a different and false claim. And the release does not publish the model behind the prediction. Quote and figures as reported by Freevacy; the original was not verified.

The companion number is more interesting, and more often mangled. Gartner reports that 63% of organisations "either do not have or are unsure if they have the right data management practices for AI." That comes from a survey of 248 data management leaders in the third quarter of 2024. So: a self-assessment, from a narrow professional subpopulation, about their own organisations. Not an audit of practice, and not a view from the board.

And it welds together two states that are not the same thing. "We do not have the practices" is a resourcing answer. "We are not sure whether we have them" is an answer about who is accountable for knowing. Gartner does not split the two, and no percentage breakdown exists in the source. Reading the second half as a signal about accountability is my interpretation, not a Gartner finding. An organisation that cannot say whether it has data management practices is telling you something about who is accountable for knowing.

One definitional point sits underneath all of this, and it is mine. "Ready data" in general does not exist. Data is ready or not ready for a particular use case, against the requirements of a particular technique, and only until the use case changes. Readiness is a relation, not a property. That has an organisational consequence people skip: you cannot centralise accountability for a moving relation into one horizontal programme, because the programme has no stable object to manage. Whoever owns "data quality" in the abstract owns a noun that changes shape whenever someone else's project changes shape.

Which is roughly how a programme ends up owning a catalogue instead of an outcome.

Information was always the currency

The mechanism is not new, and the best description of it predates the current wave by more than three decades. Thomas Davenport, Robert Eccles and Laurence Prusak published "Information Politics" in MIT Sloan Management Review on 15 October 1992, based on two years of fieldwork across more than 25 companies. Their finding runs against the era's optimism. Information technology was supposed to free the flow of information and flatten the hierarchy. It did close to the opposite. Information became the currency of organisational life, too valuable for most managers to give away.

Their frame is a taxonomy of five information-political models: technocratic utopianism, anarchy, feudalism, monarchy and federalism. Feudalism is the familiar one, where business units run their own information estates with their own vocabularies. The authors describe the firm as a set of city-states, each with its own culture, leadership and language. Federalism, meaning negotiated common terms across units that retain real autonomy, is their realistic goal, not monarchy.

The model worth naming today is the first. Technocratic utopianism is the belief that a sufficiently good technical programme will dissolve an organisational problem. Build the warehouse, model the enterprise, standardise the definitions, and the politics will follow. In 1992 that meant enterprise data modelling. Today it means "AI-ready data" as a platform initiative with a delivery date.

The age of the source is an argument, not a weakness. The same behaviour survived the data warehouse, master data management, the data lake, the lakehouse and data mesh. Five technical answers, one unchanged organisational pattern. A constraint that outlives every solution aimed at it is probably not in the layer those solutions address.

If the tool keeps changing and the outcome does not, the tool was not the variable.

Why the silo is rational

Assume, for the rest of this section, that everyone involved is competent and acting in good faith. The silo still forms, and the reason shows up in who pays to prepare the data.

Bringing a source system to the quality another team's model needs costs that system's engineers hours and a slot on their roadmap. The benefit accrues to a project owned by a different director and measured on a metric in someone else's part of the P&L. Nothing in that arrangement is irrational on either side. Refusal or delay is the predictable output.

Take a hypothetical insurer. The claims department wants a model that flags suspicious claims early, and it needs policy history from the sales system. The data exists, and the legal team has confirmed it may be used. Preparing it, though, falls to the sales engineering team, which has to unify product codes and fill in missing contract dates. While they do that they will not ship changes to the purchase flow, which is exactly what the sales director is measured on in annual objectives, and the whole team's bonus depends on conversion. If the model works, lower claim payouts show up in the claims department's results, not in sales. Nobody here acts in bad faith, and the data preparation still waits in the queue behind work someone is evaluated on.

The more departments draw on the same data, the worse it gets. Mancur Olson showed in The Logic of Collective Action (1965) that when costs are concentrated and benefits diffuse, a collective good is chronically undersupplied, and more so the larger the group. Data quality funded by one producer for many consumers fits that description. Olson's remedy is a selective incentive: a benefit available only to the party bearing the cost.

Two other frames add one element each. In the strategic contingencies theory of power (Hickson and colleagues, 1971), a unit is stronger the harder it is to substitute, so data handed over in a form others can use without asking weakens the position of whoever hands it over. Michael Heller described the tragedy of the anticommons (1998): when many parties hold a veto and nobody holds an effective right of use, the resource sits idle. A governance council in which every domain must sign something can add further vetoes to a system already suffering from them.

Under those conditions, the silo is the organisation chart working.

Governance without decision rights is theatre

The definition of data governance sharp enough to detect theatre comes from the academic literature, not from a vendor. Vijay Khatri and Carol Brown, writing in Communications of the ACM in 2010, define governance as the allocation of decision rights and accountabilities for data-related processes. They separate governance (which decisions must be made, and who makes them) from management, which is who executes.

That gives a test you can apply in a single meeting. After the programme, does anyone decide something differently, or answer for something they did not answer for before? Governance does not have to move rights to another person to do that. It also works when it writes down rights that so far existed only by custom, when it starts enforcing obligations nobody was held to, or when it sets an escalation path for disputes between a data owner and a data consumer. If the programme does none of these and changes no line in anyone's annual objectives, it is management wearing governance as a costume. Catalogues and committees without such a change meet that description exactly.

It also explains the specific way these programmes decay. A catalogue is documentation of an asset someone else owns. Keeping an entry current is a private cost paid by the source team for a benefit spread across consumers they will never meet. That is Olson's structure again, one level down, at the granularity of a single table description. The undersupply is predicted by the same model that predicts the underfunding. One gap deserves flagging. I could not find a credible public study measuring how many enterprise catalogues fall out of maintenance; the claims that circulate come from firms selling catalogues. Treat the mechanism as reasoned, not measured.

The same test explains the data steward with no authority. The role is usually designed as accountability without decision rights: responsible for quality, unable to set a source system's roadmap or to escalate. That is a structural impossibility handed to a person.

One indirect indication of how that goes is a single number. Bean and Davenport reported in Harvard Business Review in 2021, as summarised by MIT Sloan the following year, that the average chief data officer tenure runs about two and a half years. The comparison figures are roughly seven years for a CEO and four and a half for a CFO or CIO. More recent industry surveys land in the same region, with over half of respondents reporting under three years. The reasons given centre on an unclear mandate and a role carved out of the CIO's territory, with transformation expected inside about eighteen months. Three cautions. These are self-report surveys of data leaders, not a registry of appointments. MIT Sloan does not name a lack of budget or formal authority as the principal cause, and I am not going to put that in their mouths. And tenure is a proxy for a hard job, not proof of a mechanism.

Gartner's own forecast on governance is the sharpest thing they have published on this, and it is rarely read closely. In February 2024 they predicted that by 2027, 80% of data and analytics governance initiatives will fail "due to a lack of a real or manufactured crisis." Saul Judah, a VP Analyst, is quoted as saying that a governance programme which does not enable prioritised business outcomes fails. The recommendation is to move away from a centre-out, command-and-control posture toward tangible business outcomes. Again: a forecast, not a measurement. Quote and figures as reported by BizTechReports; the original was not verified.

Sit with the word manufactured. An advisory firm is openly recommending that you produce a crisis, because governance without one does not take. That is an admission about what these programmes actually run on: attention and priority. Both are political currencies, allocated by people with the standing to allocate them. Gartner does not describe this as a power problem. That reading is mine.

Which returns us to the 63%, and to the half of it that says "we are not sure." An organisation gives that answer when nobody owns the question. Owning a question means being the person whose review includes the answer. Where no such person exists, the honest response to an auditor is a shrug, and the shrug gets recorded as a data management finding. It is really a finding about accountability.

A political diagnostic, run before the project

None of this is useful until it changes what you do in week one. So here is the procedure I use. It is my own synthesis, not a Gartner method or a theorem, and it should be read as a framework rather than a measurement.

Five questions, asked before anyone writes an integration ticket.

Who owns each data set the project needs? Not which system holds it — which named person controls the roadmap of the system that produces it. If you cannot name a human with a budget, you have found the first problem.

What does the handover cost them this year? In their engineering hours and their quarterly commitments. Get an estimate from them, not from your architect. The number is usually larger than the requesting team assumes, and that gap is where projects stall.

Who is worse off if this project succeeds? In scope, headcount, budget line, or in owning a metric they currently get to explain. Someone almost always is. Resistance from that quarter is information about the distribution of power, not evidence of stupidity.

Which decision right has to move or be clarified, and who signs that? Name the right: approving schema changes, or granting access without case-by-case review. If no right or obligation changes, expect no flow.

Is the legal and technical path already open? If permission exists, the interface exists, and the data still does not move, you have a signal to check. Ask who is accountable for the handover and who bears its cost, then look at what sits ahead of it in the priority queue. Only the answers tell you whether the block is political.

The table below is how I keep the two readings apart in practice.

Then three moves. They are deliberately small, because large ones require authority a project lead usually does not have.

Put the data quality budget inside the AI project budget, not beside it. This is the proportionality of costs and benefits, one of the design principles Elinor Ostrom derived from studying how shared resources are governed (1990), applied directly. A separate data quality programme asks a source team to supply a collective good for free. A line item inside the project budget converts it into a purchased service with a customer, a price and a delivery date. The same work, moved from charity to trade. It also puts an honest number on the project, which some sponsors will not enjoy.

Put the data owner inside the project team, with a share of the outcome. Not on a steering committee, which is a place where accountability goes to be diluted. Skin in the game means their objectives contain a piece of the project's result. This is Olson's selective incentive in its smallest usable form: a benefit reaching only the party bearing the cost.

Change or clarify exactly one decision right, and write it down. It can mean moving a right to another person, or writing down one that so far existed only by custom, together with an escalation path. One is enough to test whether the organisation is willing. If not even one can be written down, you have learned something important early. The programme will produce documentation and no flow, and you can decide whether to fund it on that basis.

The nearest thing to supporting evidence is a Gartner release from April 2026: organisations with self-reported AI success invest up to four times more, as a percentage of revenue, in foundations. Their list of foundations covers data quality, governance, AI-ready people and change management. The survey ran across 353 data and analytics and AI leaders in November and December 2025. In the same release, Rita Sallam is quoted saying only 39% of technology leaders are confident current AI investment will improve financial results. Success is self-assessed. Causality is unresolved: success can fund foundations as easily as the reverse. And "up to four times" is an upper bound, not an average. What I take from it is the composition of that list of foundations, which puts people and change management alongside data quality. Those are organisational line items, and they are sitting in the same sentence as the technical one.

The argument connects to a broader point about where AI value hides. The return from an AI system rarely sits in the model; it lives in the links around it. Data ownership is the least visible of those links, because it appears in no architecture diagram.

Where the thesis is wrong

A thesis that explains every failure explains none, and "it's a power problem" can be stretched to cover anything. That makes it comfortable, which is a reason to find the places where it stops working.

Sometimes the constraint really is that the data does not exist. RAND's 2024 report on the root causes of failure for AI projects lists insufficient data among its five. The organisation simply does not hold the data needed to train an effective model. Two rigour notes. The study is qualitative, based on 65 interviews with data scientists and engineers with at least five years of experience, so 65 is a count of people, not of projects. And the report's ">80% of AI projects fail" line is quoted from other estimates, not measured by RAND.

That root cause is a hard boundary for everything above. Data never captured at the required granularity will not appear because you fixed the incentives. Better incentives can start the capture today. That does nothing for a project needing twenty-four months of history, and no amount of stakeholder mapping shortens that wait.

Sometimes the constraint is tedious engineering that nobody wants to do. Sculley and colleagues made the case in "Hidden Technical Debt in Machine Learning Systems" at NIPS 2015, and it has aged well. The model code is a small fraction of a real production system; the rest is collection, feature extraction, serving and monitoring. Data dependencies cost more to maintain than code dependencies and receive less engineering rigour. Their CACE principle (changing anything changes everything) describes why these systems resist clean decomposition. That work is expensive and unglamorous. It is also exactly the work that gets relabelled "an organisational problem" by people who would rather run a workshop than a migration.

I want to be blunt about that failure mode, because this essay supplies the vocabulary for it. A political diagnosis is an excellent alibi. It converts a hard engineering backlog into someone else's character flaw, and it lets a team spend a quarter on stakeholder maps while the pipeline stays broken. If reading this makes you feel better about not doing the pipeline work, you have used it wrong.

Sometimes the money genuinely is not there. The UK National Audit Office reported in January 2025 on the state as of March 2024. Government departments ran around 228 significant legacy systems, of which 63 (28%) were red-rated for high likelihood and high impact of operational or security risk. For 120 of those 228, more than half, no fully funded remediation plan existed. Separately, the DSIT and Government Digital Service review of digital government, also published in January 2025, found that 28% of the central government technology estate is legacy. That is up from 26% in 2023, with a range of 10% to 60% across organisations, and 22% of legacy systems red-rated. Note the different denominators in the two red-rated figures. They are not the same measurement and should not be compared.

No vendor lock, missing interface or absent supplier support is fixed by an incentive change. Those are real and countable, and they will still be there after the workshop.

So where is the line? The same DSIT review contains the best answer I have found. Half of survey respondents indicated that when a budget exists for legacy system remediation, it frequently gets reallocated to other initiatives. Read that carefully. The problem is technical. The money was approved. It vanished into someone else's priority, repeatedly and by decision.

The same review adds two findings that point the same way. Only 27% of respondents believe their data infrastructure gives a full operational or transactional view, and 70% describe the data landscape as uncoordinated, lacking interoperability and without a single source of truth. And departments treat data as their own, which blocks aggregation even where the legal basis for sharing already exists under the Digital Economy Act 2017.

That gives three tests, and they are empirical rather than rhetorical. First: does the data physically exist, at the granularity the use case needs? If not, this is not politics, and no diagnostic of mine will help you. Second: are legal permission and technical capability already in place while nothing flows? Then politics is the likely explanation, and the gap between what is allowed and what happens tells you where to look for confirmation. Third: does a budget exist and get reallocated year after year? Then it is politics too, and the reallocation decision is where you should be looking.

Forget "technical versus political." The line runs between what cannot be done, what cannot be funded two years running, and what is permitted but not consented to by the person who holds it. Only the first is an engineering problem.

Closing

Before you approve another catalogue, check whether the data exists and may be shared. If it does, find the person whose budget pays for its quality and whose bonus does not depend on it. If you cannot name them, you do not have a data problem yet — you have an ownership vacancy, and the catalogue will document it beautifully.

Sources

1. primaryGartner, Lack of AI-Ready Data Puts AI Projects at Risk (press release, 26 February 2025) — the "through 2026 … abandon 60% of AI projects unsupported by AI-ready data" forecast, and the 63% figure from a Q3 2024 survey of 248 data management leaders. (Quote and figures as reported by Freevacy, 26 February 2025; the original was not verified.)
2. primaryGartner, Gartner Predicts 80% of D&A Governance Initiatives Will Fail by 2027, Due to a Lack of a Real or Manufactured Crisis (press release, 28 February 2024) — the forecast and the Saul Judah quote on prioritised business outcomes. (Quote and figures as reported by BizTechReports, 29 February 2024; the original was not verified.)
3. primaryGartner, Organizations with Successful AI Initiatives Invest Up to Four Times More in Data and Analytics Foundations (press release, 16 April 2026) — 353 D&A and AI leaders, November–December 2025; the "up to four times" upper bound; Rita Sallam on 39% confidence. (Quote and figures as reported by TechEdge AI, 20 April 2026; the original was not verified.)
4. primaryThomas H. Davenport, Robert G. Eccles & Laurence Prusak, "Information Politics," MIT Sloan Management Review (15 October 1992) — two years of fieldwork across more than 25 firms; information as organisational currency; the five models from technocratic utopianism to federalism.
5. primaryVijay Khatri & Carol V. Brown, "Designing Data Governance," Communications of the ACM 53(1):148–152 (2010) — governance as the allocation of decision rights and accountabilities, distinct from management.
6. primaryDavid J. Hickson, C. R. Hinings, C. A. Lee, R. E. Schneck & J. M. Pennings, "A Strategic Contingencies' Theory of Intraorganizational Power," Administrative Science Quarterly 16(2):216–229 (1971) — subunit power as coping with uncertainty, workflow centrality and non-substitutability.
7. primaryMichael A. Heller, "The Tragedy of the Anticommons: Property in the Transition from Marx to Markets," Harvard Law Review 111(3):621–688 (1998) — too many rights of exclusion and no effective privilege of use produce underuse; the observation of empty storefronts in Eastern European cities with kiosks multiplying in front of them. The opposite sign to the tragedy of the commons (Garrett Hardin, "The Tragedy of the Commons," Science 162(3859):1243–1248, 1968), meaning overuse of a resource without an owner; silos are often mistaken for that picture.
8. primaryElinor Ostrom, Governing the Commons: The Evolution of Institutions for Collective Action (Cambridge University Press, 1990) — self-governance of shared resources without privatisation or central authority; design principle two, proportionality between benefits and costs.
9. primaryMancur Olson, The Logic of Collective Action: Public Goods and the Theory of Groups (Harvard University Press, 1965) — concentrated costs and diffuse benefits produce chronic undersupply; selective incentives as the remedy.
10. primaryD. Sculley et al., "Hidden Technical Debt in Machine Learning Systems," NIPS 2015 — ML code as a small fraction of the system; data dependencies costlier than code dependencies; CACE.
11. primaryRAND Corporation (James Ryseff, Brandon De Bruhl, Sydne Newberry), The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed, RRA2680-1 (2024) — qualitative study, 65 interviews; insufficient data as one of five root causes; the ">80% fail" line is a quoted estimate, not a RAND measurement.
12. primaryNational Audit Office, Government cyber resilience (29 January 2025) — 228 significant legacy systems as of March 2024; 63 (28%) red-rated; 120 of 228 without fully funded remediation plans.
13. primaryDepartment for Science, Innovation and Technology / Government Digital Service, State of digital government review (21 January 2025) — 28% of the central government estate is legacy, range 10–60%, 22% red-rated; remediation budgets frequently reallocated; 27% report a full operational view; 70% describe an uncoordinated landscape; data treated as departmental property despite the Digital Economy Act 2017.
14. secondaryRandy Bean & Thomas H. Davenport, "Why Do Chief Data Officers Have Such Short Tenures?", Harvard Business Review (August 2021), as summarised by Brian Eastwood, "Chief data officers don't stay in their roles long. Here's why," MIT Sloan (1 September 2022) — average CDO tenure around 2.5 years against roughly 7 for CEOs and 4.5 for CFOs and CIOs; unclear mandate as a stated cause. Self-reported survey data, not a registry.