borys

AI · MBA

2026-10-03~20 minSolertio · esejWritten with AI assistance · edited by Borys Nowicki

Stop measuring development by what the model does for less

An analyst has been in the job for seven years and does it faultlessly. She gets a question, picks the databases, builds the query, sifts out the junk, hands back a memo with citations. Deadlines always met. Zero errors in the references. Her manager cannot remember the last time he had to correct anything of hers.

The person is invented. The task is not.

That task has its own row in the O*NET catalogue and its own share in the data. In the Anthropic Economic Index for May 2026, first place worldwide goes to the task the catalogue describes as searching electronic and manual sources for information. Almost five percent of sampled conversations. Right behind it sits searching standard reference materials, just under four percent. Further down the top ten: answering enquiries about how software operates, recommending products to customers, editing material prepared by other writers.

The analyst's skill is real. Its market price is another matter.

I claim that competency in its classic definition has stopped being what is worth buying from a human, because its definition doubles as an operating manual for the model. Competency is the correct execution of a known task under predictable conditions. Known and predictable. Those two words describe the area where the machine is faster and cheaper than a salary line. That is this essay's thesis. It is not a conclusion from cost data, because I have no such data here. Two things hang on whether you accept it: who you hire, and what you measure their development by after a year.

Competency is the correct execution of a known task

I will start from a definition that does not come out of the debate about artificial intelligence.

Robyn Phelps and Stewart Hase, together with Allan Ellis, wrote about competency-based training in 2004 in the British Journal of Educational Technology. They start from its origins. Competency-based training grew out of the behaviourist movement. Its adoption was driven by economic and social forces rather than educational ones; at that point they cite Velde (1999) and Mulcahy (2000). The movement set out to replace the question "what does this person know" with "what can this person do, and how will we check it". A sensible change, as long as you see its boundary condition. Their sentence about that condition reads: competencies deal with the known and the predictable rather than unknown contexts.

The sentence is over twenty years old. I repeat it because it reads differently now.

The same authors quote an older remark by Price from 1996: the problem with competency training is that it is always in danger of equipping the young for the performance of yesterday's jobs. Price was writing about competency-based training, not about language models. The same problem applies to any system that first standardises a task, then teaches the standard, and finally measures conformity with the standard; that conclusion is mine, not his. Such a system closes itself on the past, because a standard can only be built out of what somebody has already done and described.

The core sits in the word "describe". To turn something into a competency you have to describe it first: break it into observable behaviours, set the criteria, write the manual and the test. Only then can it be taught at scale and assessed. Description is a necessary condition.

Description is a necessary condition for the model as well. Not because the model reads the manual. Because it learns from the text people have already produced about that task. The better something has been described, the more examples of correct execution exist, the more reliably the machine will reproduce it. The condition that makes competency-based training possible is the same condition that makes automation possible.

That remark about the movement's origins has a hard budget consequence today. If the competency layer entered companies mainly because it could be counted and accounted for, then its defence rests on convenience of measurement. Convenience of measurement has just stopped being an argument, because the same criterion attracts automation.

So we did not build a competitor for the employee. We built a competitor for the manual they were trained from.

Capability, or the ability to act without instructions

On the other side of that line sits a concept which barely appears in the Polish debate about development, though it is nearly fifty years old and has a literature of its own.

Capability as a concept grew in Britain out of the Education for Capability initiative, launched by the Royal Society of Arts in 1978. John Stephenson became the main theorist of that current in higher education, first with Susan Weil in 1992, then with Mantz Yorke in 1998. Crediting Stephenson with the term itself is shorthand; he developed it, he did not coin it.

The most frequently cited compact definition came from Len Cairns in 2000; Phelps and Hase together with Ellis quote it, and that is where I take it from. Capability is having justified confidence in your ability to take appropriate and effective action to formulate and solve problems in both familiar and unfamiliar and changing settings.

Note two elements of that definition. Neither is decoration.

The first element is "formulate". Not "solve". Competency starts work from a task somebody has already set. Capability covers the earlier move: working out what the problem here actually is. Whoever formulates the question settles half the answer.

The second element is "justified confidence". Not self-assurance. Confidence with cover behind it. The three-part breakdown of capability came from Cairns and Stephenson in Capable Workplace Learning from 2009. I take it from the article by Ryder and co-authors in the Journal of Clinical Nursing from 2023: ability, self-efficacy and values.

The third item on that list is the most interesting one in a conversation about models. Values work here as the criterion of choice in a situation where two solutions are technically correct and what separates them is who bears the cost. The model will hand you both and describe the difference accurately. The choice itself is an act of assigning responsibility, so it has to be made by somebody who can be held to account.

Lisa Marie Blaschke, in a 2012 review in the International Review of Research in Open and Distributed Learning, sets this up as a difference of levels. Competency in her account is proven ability in acquiring knowledge and skills. Capability begins where the learner has confidence in their own competency.

Polish has no good single word for this. Zdolność is too broad, kompetencja is taken, and sprawczość means something else, which is why the Polish version of this essay leaves capability untranslated. I mean by it: the use of knowledge you already have in a situation nobody has described before.

How you know where the model's reach ends

Now the most important question in this essay. Does the line between competency and capability really coincide with the model's reach, or does it only sit neatly on top of it?

The answer can rest on data anyone can read for themselves.

The Anthropic Economic Index for May 2026 splits conversations into two patterns. Augmentation means the person stays inside the task and works together with the model. Automation means the person hands the model the whole job. The split comes out almost even: 51.4 percent to 48.6 percent. So in almost half of the classified conversations somebody hands over the entire task.

What matters more than the proportion is which tasks these are. The global top ten holds searching for information and answering enquiries about how software operates. Right next to them sits writing programs to meet customer requirements. Further on: advising clients on financial matters, recommending products to customers, answering students' questions, editing material prepared by other writers. Every one of these tasks can be written up as a procedure, and every one has left a trail behind it in the form of described, correct executions.

What is missing from that list? There is no decision about what not to do at all. There is no settling of a dispute between two departments with conflicting and defensible cases. There is no choice about which client to let go. There is no situation where the data is incomplete, the deadline passed yesterday, and responsibility for the outcome sits with one named person.

The mechanism is simple, though I give it as my own reasoning and not as a finding from the literature. The model reproduces well the distribution it has seen. A new situation is by definition what was not in that distribution. You can describe it to the model, at which point it stops being new and returns to the territory of competency. Somebody has to produce that description. It always costs, and a human always does it.

Take a case from a two-sided services marketplace. The example is constructed. An operator reports that the job was priced too low, and the client maintains that the scope grew after signing. Both accounts are internally consistent. The documentation is incomplete on both sides. The model will settle this dispute for you in two seconds and give you reasoning that sounds sensible. The thing is that the settlement is not the product here. The product is what the company does with that whole class of disputes over the next year: whether it changes the definition of scope in the form, whether it makes a photograph of the starting condition mandatory. You take that decision without a procedure, because the procedure has just turned out to have holes in it. Nobody described it beforehand, so there is nowhere to take it from.

Out of this comes a rule you can apply on Monday. A task you can describe well enough for a stranger to carry out, hand to the model and spot-check. A task you cannot describe shows you where your edge is and where a human has to stand. The test is cheap and reversible, because it costs one attempt at a description.

Hence the practical conclusion for a founder and for a manager. Value has moved from execution to formulating the task and to absorbing the consequence. If somebody brings you a well-posed question and you answer it correctly, you are on the cheap side. If you get a mess and have to extract the question from it, you are on the expensive side.

Here a third term comes in and gives the whole matter its name. Pedagogy describes learning led by a teacher. Andragogy, which Malcolm Knowles described in 1970, covers learning directed by the adult learner within limits set from outside. Heutagogy was introduced in 2000 by Stewart Hase and Chris Kenyon, who developed it seven years later in Complicity. It describes a situation in which the learner decides for themselves what they learn and how they will know they have learned it. I take that characterisation from the literature citing the original, because ultiBASE, the service where the 2000 paper was published, is no longer running. Three words for three different distributions of power over the curriculum. The distinction does one job in this essay: it names the condition under which development keeps up with a moving line at all. It will not come up again, because this essay is not about typology.

Development programmes measure the layer you no longer buy

Now to why companies do not see this, even though they have it within arm's reach.

A development department has to report. What gets reported is what can be counted: the number of hours, the completion rate, the test score, a competency map with levels from one to five. Every one of those measures is legitimate, and every one measures the same thing: conformity with a described standard. And the description of a standard is what the model reproduces most cheaply.

This is not organisational stupidity. It is the effect of an incentive.

Capability is hard to measure on a quarterly cycle, because it shows up at rare events. A person demonstrates it when a situation breaks out of the procedure, and such situations by definition do not run on a calendar. Competency you can measure with a test any Tuesday. So you measure competency, you report competency and you budget for competency, until the whole development machine starts optimising the one thing it can see.

Look at what such a matrix can hold. The one below is constructed, though modelled on a typical layout. Row: data analysis. Columns from level one to level five. Level three: prepares a report to an agreed template without supervision. Level five: designs their own analyses and trains others. Each of those descriptions assumes it is known which analysis is needed. The matrix has no row for recognising that the question was posed badly, nor for persuading the requester of that. Not because somebody forgot. Because such a row cannot be scored on a five-point scale without knowledge of the specific case, and the matrix is meant to work precisely without it.

The 2019 review by Nikoletta Agonács and João Filipe Matos, covering sixty-nine publications on heutagogy, shows the same phenomenon from the researchers' side. Among the dimensions of the theory, capability development came out as under-represented, together with the non-linearity of learning. Even people who write about self-determined learning rarely study that component. I suspect it is simply the hardest one to observe, but the review does not settle that.

Put this to your own company in one question. When did somebody last get promoted for a decision taken in a situation with no procedure for it, and is that decision recorded anywhere as the reason for the promotion. The answer "promoted on results" settles nothing here, because the result could have come out of the procedure alone.

Single-loop and double-loop learning in practice

There is one operation the model in its present form will not perform for you. Its name dates from the nineteen-seventies.

Chris Argyris and Donald Schön developed the distinction between two loops of learning in Theory in Practice from 1974 and in Organizational Learning from 1978. In the single loop a person sees a mismatch between intention and outcome and changes the action, leaving alone the values that govern that action. In the double loop they first examine and alter those values, and only then the action. Auqui-Cáceres and Furlan quote Argyris's compact 1999 formulation: mismatches are corrected by first examining and altering the governing variables and then the actions.

Now look at your own session with a model.

You write a prompt. You get an answer. You read it, judge it, accept it or ask for a revision. If the answer is weak, you change the prompt. You repeat until it works. This is the single loop in its textbook form: you correct the action to hit a target you never question once.

The double loop starts from a different question. Not "how do I phrase this instruction better". The question is whether I should want this at all. Whether the criterion I judge the answer by is the right criterion. Whether the problem I handed to the model is the problem I have.

The model can describe that second loop and can even simulate it on demand. Ask it to challenge your assumptions and you will get a list of challenges. What will you not get? You will not get the cost. Changing the governing variables in an organisation means somebody admits to a wrong assumption in front of people who built their projects and their positions on that assumption. Nobody does this willingly, because the bill lands on the person who has to make the admission. The model risks no loss of face, so it has nothing to give up. Its version of the double loop stays an exercise on paper.

This shows best on a question about a metric. Third constructed example. A team accounts for campaigns by customer acquisition cost and has been driving that cost down for several quarters. Every iteration is correct: better creative, cheaper channels. That is the single loop, and it can run for years, because each round shows progress. The double loop starts from the question of whether acquisition cost is the right metric for a product that only makes money on the third order. An affirmative answer costs two quarters of work and the reputation of the person who introduced the metric. The model will put that question in its first reply if you ask for it. It will bear none of those costs.

So what stays on your side is the bill. Thinking about assumptions you get free in every session, and the willingness to pay for changing them costs as much as your position in the company is worth.

When the opponent is right

The thesis of this essay breaks in four places, and I want to name them honestly before somebody else does.

Not every environment is unpredictable. In a regulated profession, competency is not a stage along the way. It is the end product and the legal basis. Somebody is meant to perform a known procedure under known conditions correctly every time, and an audit will call any deviation from it by its name. Janet O'Connell and Glenn Gardner, together with Fiona Coyer, put this plainly in the Journal of Advanced Nursing in 2014: competencies have been described as appropriate for practice in stable environments with familiar problems. Their article argues for a capability framework in advanced practice nursing, which is exactly why it makes a good witness here. Even there, capability is built on the regulator's competency standards rather than in place of them. The same logic governs every profession in which a regulator defines the performance standard and audits conformity with it. If your company works mainly under that kind of regime, moving the development budget from competency to capability is a mistake, because a deviation from the procedure costs more there than even the best idea of a single person can bring in.

Capability without a base of competency is a con. You do not adapt knowledge you do not have. Paul Kirschner and John Sweller, together with Richard Clark, argue in Educational Psychologist from 2006 that instruction with minimal guidance works poorly precisely with novices. Novices lack schemas in long-term memory, so cognitive load eats the gain from working on their own. Their article gathers other people's results and does not report an experiment of its own. Phelps and Hase together with Ellis, authors sympathetic to the concept of capability, write themselves that competency and capability share a common foundation in basic skills and knowledge. The slogan "teach yourself, straight at the meta level" produces people with strong opinions and no craft. You will recognise them by how well they judge other people's work and how badly they do their own.

The evidence for the effectiveness of heutagogy is thin. This is where I have to weaken my own thesis the most. Agonács and Matos, in their 2019 review of sixty-nine publications, report a predominance of qualitative methods, an absence of quantitative and longitudinal data, and a narrow geographical spread of researchers. The samples come mostly from formal adult education. Robert Moore collected thirty-three peer-reviewed publications from the years 2000 to 2019 in Distance Education in 2020. Both numbers, sixty-nine and thirty-three, follow from different inclusion criteria and do not measure the size of the field. They measure how many works met the review's criteria, and in both cases that is a handful. Mirjam Neelen and Paul Kirschner charged the heutagogy literature in 2021, on the blog 3-Star Learning Experiences, with passing over what the learning sciences already know about self-directed learning. The charge lands. From heutagogy you can take a conceptual frame today and a few case studies from higher education. You cannot take evidence that implementing it in a company improves results, because no such evidence exists.

The non-delegable remainder has thin implementation evidence too. Since I am hitting heutagogy, I have to hit Argyris in the same place. Milagros Auqui-Cáceres and Andrea Furlan reviewed one hundred and twenty-eight studies of the double loop from the years 1974 to 2021 in European Management Review in 2023. Their conclusion: the impact of the concept on the literature and on practice is noticeable but superficial. They give a twofold reason: the complexity of its definition and the difficulty of its implementation. In other words, the thing I am calling the remainder on the human side has for fifty years been hard to observe and harder still to put in place. That weakens my practical conclusion. The distinction itself holds.

One caveat about my data on top of that. The Anthropic Economic Index measures the content of conversations with Claude matched to the O*NET task catalogue. It does not measure employment or the labour market. It also says nothing about whether anybody lost a job because of it. The publishers state this plainly. I use these numbers solely to show which tasks people hand to the model when they can choose. That is a strong premise about where the line of easiness lies. It is not a premise about employment, and I will not pretend that it is.

What is left of the thesis after those four blows? The distinction is left and the direction is left. What is not left is a promise that implementing heutagogy in a company will produce a measurable return, because nobody has shown that so far.

Self-determination without infrastructure is a cost transfer

Now the part where I have the most objections to how the subject looks at development conferences.

"Take responsibility for your own development" sounds like emancipation. In practice it is sometimes a transfer of cost from the company's balance sheet into the employee's private time, wrapped in the language of freedom. The company saves on the training budget, and the employee studies in the evenings out of their own money and their own sleep. Calling it self-determination changes nothing except who gets to complain.

Capability does not come out of an attitude. It comes out of conditions. Four are necessary, and none of them can be replaced by a declaration.

Organisational memory with access to it. A person learns from cases they did not live through, if they have access to them. How many such cases has your company written down in a form somebody from another department will be able to read in two years? A deck from a project that ended in success will not do, because it leaves out everything that went differently. What is useful is a decision record: what we knew at the time, what we assumed, what turned out differently. Without that, everybody learns from zero on their own mistakes, and the company pays for the same mistake as many times as it has employees, and rarely sees that cost in any budget line.

Feedback out of real consequences. An annual review is not feedback, because it arrives too late and mixes several signals at once. Capability grows where a decision has a visible effect within a time shorter than the memory of it. A founder has this by nature, because the market answers directly and fast. An employee of a large company often has none of it, because the effect of their decision dissolves into the department's quarterly result.

Time written into the calendar. An hour a week for learning that nobody protects is the first thing to go under any urgent task. If that hour does not have the same status as a client meeting, it does not exist. A declaration about a learning culture costs zero and is worth that much.

The right to an attempt that misses. This is the most expensive condition and the rarest. Capability is by definition practised in situations where nobody knows what will work, so a share of the attempts has to come out badly. An organisation that punishes every attempt that misses teaches one skill: avoiding situations without a procedure. If the only rewarded path is the one where nothing can go wrong, people will start picking tasks by risk, and the company will see the effect only when it runs short of somebody brave. After a year it gets people who are perfectly competent and completely helpless outside the described area. It gets what it paid for.

If a company does not put these four things in place and still talks about self-determination in development, it is talking about a transfer of cost. Call it by its name at the next meeting where the slogan comes up.

What changes in hiring and in the measurement of development

Let us drop to the level of decisions you will take in the coming quarter.

In hiring, the question "can this person do it" stops working. The answer is more and more often "so what". Start asking about a situation where the candidate did not know what to do and had to decide. Not about a hard project, because a hard project is still a procedure, only a longer one. Ask about the moment when there was no procedure. Then ask about a second such moment. People with craft have several of them and remember the details. People with a story have one, polished smooth.

The second hiring question concerns the double loop and sounds innocent. When did you last change your mind on a professional matter, and what exactly convinced you. The answer "I am always learning" is the absence of an answer. The signal is an answer with a named opponent and a specific date. The strongest one also gives the price paid for the change of mind.

In the measurement of development, swap one metric. Instead of the percentage of completed training, start counting decisions taken without a procedure, together with their effect after six months, and read them in series, because a single outcome will not tell sound judgement from a lucky hit. It is a statistically worse metric and a rarer one. It will also be a source of arguments, because it does not settle itself. What it measures is what you pay for rather than what is easy to count. Start with one team and twelve months, because that move is reversible. Over a year you will collect a handful of such decisions and no distribution, and that has to be said out loud at the start.

In development policy, do one thing and do it first. Put a protected block for learning into the calendar and defend it for a quarter the way you defend a client meeting. Only then talk to people about responsibility for their own development. In the reverse order the same sentence means something else, and employees hear that meaning immediately.

The analyst from the opening does not need a course on operating the model. She needs the first task of her career in which nobody tells her what the question should be. If your company has no such tasks to hand out, it has nothing to measure either.

References

  1. primaryArgyris, C., Schön, D. A. (1974). Theory in Practice: Increasing Professional Effectiveness. San Francisco: Jossey-Bass.
  2. primaryArgyris, C., Schön, D. A. (1978). Organizational Learning: A Theory of Action Perspective. Reading: Addison-Wesley.
  3. primaryArgyris, C. (1999), p. 68 — the compact definition of correction in the double loop. Cited after Auqui-Cáceres and Furlan (2023); I have not verified the full publication details, so in the text the quotation is described as a citation at second hand.
  4. primaryCairns, L., Stephenson, J. (2009). Capable Workplace Learning. Rotterdam: Sense Publishers. The three-part breakdown of capability cited after Ryder, Smith and Furlong (2023).
  5. primaryHase, S., Kenyon, C. (2000). From andragogy to heutagogy. ultiBASE, RMIT University. The ultiBASE service is no longer running, so I report the content of this paper after the literature that cites it. Some databases give the year as 2001; the year 2000 is adopted after the Southern Cross University repository and after citations that agree with it.
  6. primaryHase, S., Kenyon, C. (2007). Heutagogy: a child of complexity theory. Complicity: An International Journal of Complexity and Education, 4(1), 111–118.
  7. primaryKirschner, P. A., Sweller, J., Clark, R. E. (2006). Why minimal guidance during instruction does not work. Educational Psychologist, 41(2), 75–86.
  8. primaryKnowles, M. S. (1970). The Modern Practice of Adult Education: Andragogy versus Pedagogy. New York: Association Press. Revised edition 1980.
  9. primaryStephenson, J., Weil, S. (eds.) (1992). Quality in Learning: A Capability Approach in Higher Education. London: Kogan Page.
  10. primaryStephenson, J., Yorke, M. (eds.) (1998). Capability and Quality in Higher Education. London: Kogan Page.
  11. primaryAnthropic (2026). Anthropic Economic Index, reporting period May 2026, accessed 11 September 2026. https://www.anthropic.com/economic-index
  12. secondaryAgonács, N., Matos, J. F. (2019). Heutagogy and self-determined learning: a review of the published literature on the application and implementation of the theory. Open Learning, 34(3), 223–240.
  13. secondaryAuqui-Cáceres, M., Furlan, A. (2023). Revitalizing double-loop learning in organizational contexts: a systematic review and research agenda. European Management Review, 20(4), 741–761.
  14. secondaryBlaschke, L. M. (2012). Heutagogy and lifelong learning: a review of heutagogical practice and self-determined learning. The International Review of Research in Open and Distributed Learning, 13(1), 56–71.
  15. secondaryBlaschke, L. M. (2021). The dynamic mix of heutagogy and technology: preparing learners for lifelong learning. British Journal of Educational Technology, 52(4), 1629–1645. Source of the characterisation of heutagogy reported here.
  16. secondaryMoore, R. L. (2020). Developing lifelong learning with heutagogy: contexts, critiques, and challenges. Distance Education, 41(3), 381–401.
  17. secondaryNeelen, M., Kirschner, P. A. (2021). The heutagogy hotchpotch. Blog post on 3-Star Learning Experiences, 16 February 2021.
  18. secondaryO'Connell, J., Gardner, G., Coyer, F. (2014). Beyond competencies: using a capability framework in developing practice standards for advanced practice nursing. Journal of Advanced Nursing, 70(12), 2728–2735.
  19. secondaryPhelps, R., Hase, S., Ellis, A. (2004). Competency, capability, complexity and computers: exploring a new model for conceptualising end-user computer education. British Journal of Educational Technology, 36(1), 67–84. Source of the second-hand citations of Cairns (2000), Price (1996), Velde (1999) and Mulcahy (2000); each of them is described in the text as a citation after this article. The year needs care: the DOI and volume 36(1) point to 2005, while the publisher's citation line gives 2004 (online publication December 2004).
  20. secondaryRyder, M., Smith, R., Furlong, E. (2023). Evaluation of a nurse practitioner clinical practicum module using a capability education framework: a case study design. Journal of Clinical Nursing, 32(13–14), 3775–3786.