Software companies promise a productivity boost from AI agents; SAP even speaks of the „autonomous enterprise“. Economic history suggests that the gains do not arrive when the technology is purchased, but only after the organisation has been rebuilt. With agents, this rebuilding touches a question that has so far been reserved for management alone: who gets to set the rules by which decisions are made in a company?

Christian Klein, chief executive of SAP since 2019, sees AI agents as the greatest growth opportunity in the history of the Walldorf-based software group. In May, at a customer conference in the United States, SAP presented the „autonomous enterprise“: agents anchored in business processes, data and corporate management that are meant to deliver results and cut costs. In an interview with the German news agency dpa, however, Klein added a qualification that matters more than the product name: in critical areas of a business, an agent will not act fully autonomously in the future either; in the end, a human being always bears responsibility.

This caveat is economically revealing. It touches on a question that is usually passed over in the debate on the productivity effects of artificial intelligence: how does the return on a new technology depend on the position it is given within the organisation that uses it?

The Long Lesson of Electrification

In his 1990 paper „The Dynamo and the Computer“, the economic historian Paul David drew attention to a parallel. The electrification of American industry showed up in the productivity statistics only decades after the electric motor had been introduced. The reason lay not in the technology but in the factory. As long as electric motors merely replaced the central steam engine and power continued to be distributed through line shafts, the gains remained small. Only when factories were redesigned, with individual drives on each machine and a layout organised around the flow of materials rather than the transmission of power, did the new technology realise its potential.

In 2021, the economists Erik Brynjolfsson, Daniel Rock and Chad Syverson generalised this finding under the heading of the productivity J-curve. General-purpose technologies require complementary, mostly intangible investments in processes, skills and organisational forms. These investments initially generate costs without being recorded as capital in the statistics, which is why measured productivity can even fall at first before it rises.

Applied to AI agents, this raises a more precise question than that of the models‘ capabilities: what organisational groundwork do agents require, and what does it depend on? The provisional answer is: on how deeply the agent reaches into the company’s body of rules.

Decisions About Decisions

A robust point of entry is provided by the sociologist Niklas Luhmann. In his book „Organisation und Entscheidung“ (English: „Organization and Decision“), he describes organisations as systems that consist of decisions and perpetuate themselves through decisions. Central to this is the concept of decision premises: stipulations that frame future decisions without anticipating them in detail. They include, above all, programmes (the rules by which decisions are made), communication channels (who decides what with whom) and personnel (who decides). Among programmes, Luhmann distinguishes conditional programmes, which follow the pattern „if A, then B“, from purpose programmes, which set a goal and leave the choice of means open.

This distinction yields three ways in which an agent can take part in operational decisions.

A rule-bound agent executes conditional programmes. It matches an invoice against the purchase order and the goods receipt, books it if they agree and passes it on if they do not. The rules come from outside.

A rule-intelligent agent works with purpose programmes. It knows the goal and the regulatory framework and decides for itself which rule applies in a given case, how an ambiguous provision is to be interpreted and how to deal with cases nobody anticipated.

A rule-designing agent decides on decision premises. It changes approval thresholds, restructures processes or formulates rules by which others will decide in future. In doing so, it enters the level that in organisations has traditionally been reserved for leadership.

Klein’s caveat places SAP’s offering essentially at the first level, with extensions into the second. The name „autonomous enterprise“, by contrast, points to the third. What separates the two is not primarily a technical but an organisational distance.

Level One: The Limit of Control

With rule-bound agents, the familiar division of labour remains in place: management sets the rules, the agent executes, the human checks. The organisational groundwork consists in making explicit the rules that previously resided in employees‘ experience, and in defining review thresholds.

The productivity gains are real but limited. In „The Theory of the Growth of the Firm“ (1959), the economist Edith Penrose argued that the rate at which a firm can grow is constrained by its available managerial capacity, because new activities have to be introduced and integrated by experienced managers. This so-called Penrose effect has a counterpart with rule-bound agents: an agent can only achieve as much more as the people who are responsible for its actions can follow and endorse. Control capacity becomes the bottleneck.

This entails a twofold risk. If control remains serious, it caps the return. If it becomes a formality, responsibility without understanding emerges. In principal-agent theory, as set out by Michael Jensen and William Meckling in their 1976 paper „Theory of the Firm“, monitoring costs are the price a principal pays for delegating tasks to an agent. If these costs are saved by merely signing off rather than checking, they do not disappear. They turn into undetected risks.

Level Two: The Shift of Judgement

In administrative processes, a large share of costs arises not in the standard case but in the exception: the invoice without a matching purchase order, the contract with an unusual clause, the expense claim no guideline foresaw. Rule-bound agents pass precisely these cases on to humans. Rule-intelligent agents handle them themselves. Their productivity potential is correspondingly greater.

At the same time, the nature of control changes. An outcome can be checked against a rule; an interpretation only against the purpose of the rule. Whoever reviews must be able to assess the same discretionary latitude the agent has used. The bottleneck shifts from control capacity to capacity for judgement. For the structure of employment, this would have a consequence that rarely features in the debate about job losses: fewer clerks would be needed, but more people capable of deciding questions of interpretation, and these are scarcer and more expensive.

There is also a less visible effect. Whoever repeatedly interprets rules in the same way creates precedents. The agent’s interpretation becomes established practice and thus a de facto rule, without ever having been adopted as one. There is much to suggest that rule-intelligent agents gradually turn into rule-designing ones in this way, but without the procedure that would be intended for setting rules.

In „A Behavioral Theory of the Firm“ (1963), Richard Cyert and James March described how firms rarely resolve goal conflicts between departments but merely settle them provisionally, for instance by attending to goals sequentially or cushioning them with reserves they called organisational slack. A rule-intelligent agent weighing conflicting requirements, such as cost reduction against a supplier relationship, takes over this settlement. It makes trade-offs that used to surface in negotiations between departments and thereby removes them from the view of those who are supposed to be accountable for them.

At this level, the organisational groundwork consists in systematically observing the agents‘ interpretive practice and deciding at regular intervals which interpretations are adopted as rules and which are corrected. This amounts to a form of judicial oversight inside the company, for which most organisations so far have neither procedures nor responsibilities.

Level Three: The Question of Legitimacy

At the third level, the agent decides how all cases of a given kind are to be handled in future. This is where the greatest productivity potential lies, because it is no longer individual transactions that are accelerated but the organisation itself that is rebuilt. To stay with the analogy of electrification: only at this level would the factory be redesigned rather than merely having its drive replaced.

Yet this is precisely where the economic point lies. In the early twentieth century, redesigning the factory was a task for engineers and entrepreneurs who were answerable for it. With rule-designing agents, this task would pass in part to the very system whose productivity is meant to be increased. In Luhmann’s understanding, however, deciding on decision premises is the core of what leadership consists of. An agent that enters this level no longer competes with clerks but with management itself.

This gives rise to the provisional thesis that at this level, governance can no longer function as after-the-fact control. A body reviewing every draft rule would need the same design competence as the agent and would quickly be overwhelmed at a high frequency of change. Governance has to begin earlier, in a kind of constitution: which rules may the agent change and which not? By what procedure do changes take effect? Which are reversible, which time-limited, which require the consent of which body?

In „The Practice of Management“ (1954), Peter Drucker described management as a distinct practice, different from mere administration. At the third level, a new form of this practice is emerging: the design of institutions before execution, rather than leadership and coordination during ongoing operations.

The bottleneck thus shifts a second time, from the capacity for judgement to the capacity for legitimation. The decisive question is no longer whether a rule is substantively sound, but who may put it into force and who is liable for it. Klein’s statement that a human always bears responsibility in the end becomes fragile here. For a rule he neither designed nor fully understands in its rationale, a human can at best vouch formally.

Measuring the return also reaches its limits. The effect of a changed rule often becomes apparent only after months, and in areas that seem to have nothing to do with the change. And an agent that designs rules may also help design the metrics by which its success is measured. The danger of self-referential optimisation is structurally built into this level.

The J-Curve of Agents

Placing the three levels side by side reveals a pattern that can so far be regarded as provisionally supported at best. With each level, productivity potential grows, because agents intervene more deeply in the organisation: first in routine, then in the exception, finally in the structure. At the same time, the bottleneck moves from control through judgement to legitimation, and the required organisational groundwork becomes more extensive: first making rules explicit, then overseeing their interpretation, finally a constitution for changing them.

This corresponds to the logic of the J-curve, with one peculiarity. The complementary investments that higher-level agents require concern not only processes and skills but the leadership structure of the company itself. They cannot be bought and cannot be delegated to a software vendor. Companies that fail to make them effectively remain with rule-bound agents, even if they deploy more capable ones. Or they get the consequences of the higher levels without the governance.

For expectations about the productivity impact of artificial intelligence, this leads to a sober conjecture. The large gains held out by software vendors are likely to materialise, if at all, only with considerable delay, and first in companies that deliberately recast their decision-making order. Electrification showed that decades can separate the availability of a technology from its effect. With agents, the length of this delay depends less on computing power than on the willingness to answer the question of who, in the company, writes the rules.

Ralf Keuper

Sources

Literature

• Brynjolfsson, Erik / Rock, Daniel / Syverson, Chad: The Productivity J-Curve: How Intangibles Complement General Purpose Technologies. In: American Economic Journal: Macroeconomics 13 (2021), No. 1, pp. 333–372.
• Cyert, Richard M. / March, James G.: A Behavioral Theory of the Firm. Englewood Cliffs 1963.
• David, Paul A.: The Dynamo and the Computer: An Historical Perspective on the Modern Productivity Paradox. In: American Economic Review 80 (1990), No. 2, pp. 355–361.
• Drucker, Peter F.: The Practice of Management. New York 1954.
• Jensen, Michael C. / Meckling, William H.: Theory of the Firm: Managerial Behavior, Agency Costs and Ownership Structure. In: Journal of Financial Economics 3 (1976), pp. 305–360.
• Luhmann, Niklas: Organisation und Entscheidung. Opladen/Wiesbaden 2000 (English: Organization and Decision. Cambridge 2018).
• Penrose, Edith: The Theory of the Growth of the Firm. Oxford 1959.

Web

• heise online: SAP-Chef sieht historische Wachstumschance durch KI-Produkte (in German) – https://www.heise.de/news/SAP-Chef-sieht-historische-Wachstumschance-durch-KI-Produkte-11467558.html
• Tagesspiegel (dpa): Softwarekonzern: SAP-Chef sieht historische Wachstumschance durch KI-Produkte (in German) – https://www.tagesspiegel.de/wirtschaft/dpa-softwarekonzern-sap-chef-sieht-historische-wachstumschance-durch-ki-produkte-16100394.html