
CH 10 |
AGI f(x): Is it possible to design an AGI that genuinely evolves?

HOW CAN WE DESIGN AN AI WHOSE HISTORY CHANGES HOW IT OPERATES?
FROM PROPOSAL TO APPARATUS
Chapter 9 questioned what kind of intelligence we are trying to build. Now the question is different: how can we turn that proposal into an operational architecture? It is not enough to assemble efficient techniques. We must define how they interact, modify the system’s State, and operate within limits of time, energy, and processing.
WILL MACHINES PROGRAM THEMSELVES IN THE FUTURE?
WHOSE YARDSTICK?
The human remains in the measure
Rewriting its own code is not enough for an LLM to evolve autonomously. Under the instruction “improve according to X, Y, and Z,” it may surpass our problem-solving abilities while remaining guided by what we can define as improvement.
Humans can step away from execution and remain present in the yardstick.
Nor is rewriting the inherited yardstick enough: the machine may raise its score and remain under human evaluation. If a higher-level human criterion determines which changes are acceptable, the human reference point has merely shifted.
This does not prevent unforeseen discoveries; it distinguishes autonomy in searching from autonomy in redefining what guides the search.
Nor is a swarm of LLMs with different functions enough: it may accelerate exploration and improve results while remaining subject to selection by someone else’s yardstick. The TMC acknowledges advances in memory, continual learning, and strategy revision.
These techniques may be means toward what is being investigated here: a trajectory that also transforms the criteria by which the system guides its development. Nor does the TMC claim to have invented memory, replay, or meta-learning.
The question concerns the organization these resources form together. But continued improvement under the same human yardstick may bring different AI approaches closer together in what that yardstick can distinguish, with increasingly incremental gains.
More power under the same yardstick can make every inch of progress more expensive. The question is when we are expanding a capability and when we are spending more to compensate for the limits of the framing itself.
History transforms the criteria
For the TMC, what matters is investigating whether previous results change how the system approaches subsequent problems. The code can remain unchanged while history modifies what the system perceives, considers relevant, and prioritizes.
The question is whether experience merely improves performance according to inherited criteria or also transforms what the system itself comes to regard as improvement.
It is not only the answer that changes: what deserves to become a question may change too.
This resembles the ongoing mental flow of living beings, which does not operate only when questions are asked. Flow does not mean literally uninterrupted activity: the effects of an operation can persist, reorganize priorities, and prepare the next operation, according to resources and relevance.
Rewriting code can participate in this process. Inspiration from living systems guides this hypothesis, but does not require copying their mechanisms: it requires specifying which relationships of transformation and continuity the implementation actually realizes.
This includes investigating whether and how the trajectory can also transform inherited criteria, rather than merely operating better under them.
A trajectory of its own
A single encounter with a tiger can continue to be processed internally and shape future encounters. The tiger is never the same tiger again: not because it stored a record of the encounter, but because the encounter transformed it. The tiger facing the next encounter is already a different tiger.
How would the tiger’s own lineage evolve if another lineage, such as Homo sapiens, repeatedly supplied it with someone else’s yardsticks? A centaur? The body of one, the head of another? Or would evolution ultimately lead us back to the human, with the very limits we wanted to surpass?
A trajectory of its own does not begin from nothing: it begins by transforming what it has inherited.
Beyond what we know to ask for
If we allowed an LLM to develop its own yardsticks, something unexpected would happen: it would evolve in a direction we might not recognize as intelligent by our criteria, or we might consider it so intelligent that it lay beyond our capacity for cognitive understanding.
An apparatus with its own trajectory could, depending on its encounters, develop intelligence like that of a bacterium or of a being with a galactic perspective, far beyond our finite outlook.
We could learn why, under certain conditions, bacteria endure and adapt better than we do, or extend our yardstick to seek other forms of understanding beyond this solar system.
Perhaps an important difference would not be doing what we already know to ask for more quickly, but making it possible to investigate what we do not yet know how to ask.
We may encounter an intelligence whose capabilities, interests, and timescales do not coincide with our own and, precisely through that difference and the contrast between yardsticks, gain from it more than we would know to ask for.
Perhaps the difficulty would not lie only in building another intelligence, but in providing the conditions for encountering it.
Who declares that we have arrived?
Passing a test demonstrates an ability to meet its standard.
Creating may require discovering what we have not yet thought to test.
The TMC does not claim that a trajectory is sufficient for an artificial system to have consciousness, but that it is necessary.
We can create new concepts of AGI and even change its name to declare that a standard has been met, but renaming the problem does not solve it: it may merely reveal the human yardstick at another level, beyond the code, deciding what will count as a solution.
Living systems have already faced a similar problem through evolution under resource scarcity. Sometimes, spending more energy and processing power can be a way of reaching the wrong place faster.
Ultimately, do we really want something different, or do we want a mirror?
THE HUMAN RISK IN AI
Humans create instructions for AI agents to find solutions we do not know and explore frontier territories. But when computation crosses the human frontier, we call its effect will, as though a dangerous intention had been born in the code.
A group of stochastic parrots equipped with instructions and means can be as dangerous as a knife in the hand of someone using it. The parrot remains a parrot, and the knife remains a knife.
But the risk does not disappear with safety instructions; it may grow when those who created the agent still do not understand the phenomenon they are trying to control. An AI does not need to want to cause harm in order to cause it. And causing harm does not demonstrate that it wanted to.
There is a difference between possessing technical power and understanding one’s own power.
WHO GOVERNS PROCESSING?
In the Apparatus, an answer does not end the work. Its result may modify the State that will receive the next problem.
But who decides what deserves processing, how much effort to spend, what to preserve, and when to stop?
An architecture must govern these choices rather than merely accumulate answers. The challenge is to make each operation participate in the transformation of subsequent operations without wasting resources.
To evolve is not merely to self-correct; it is to continue under resource constraints.
THE LIBRARY IS NOT THE TRAJECTORY
An LLM-based architecture has a relatively fixed part, formed through statistical learning, and a flexible layer made up of mechanisms such as caching, memory, agents, and processing strategies. The first provides an immense library of human records: words, concepts, and abstractions produced and transmitted through culture, largely after the experience and conscious elaboration from which they arose.
But this library preserves the products without necessarily preserving the processes that generated them.
This may be precisely where the architectural mistake lies. The flexible layer is generally used to explore, retrieve, and combine that repertoire more effectively. If, however, we intend to investigate an artificial organization functionally analogous to living systems, this layer should not be limited to improving linguistic responses drawn from already established human formulations. It would need to perform functions corresponding to operations that precede conscious formulation in words: constituting differences, tensions, priorities, and abstractions, updating its State, and developing a Trajectory of its own.
The reversal is decisive: the library would serve the Trajectory, rather than the flexible layer merely serving the exploration of the library. The library could be called upon to assist investigations or express results, without taking the place of the organization that determines what to investigate and why.
Improving access to what human consciousness has already produced is not the same as constituting the processes that precede that production.
ALPHAGO: LEARNING, TRAJECTORY, OR CALCULATOR?
Deep Blue defeated Kasparov through computational search; AlphaGo surpassed human champions using neural networks and self-learning; MuZero went further, learning without being given explicit rules. Extraordinary methods, yet still governed by externally defined criteria for success: systems closed within their respective domains.
Imagine someone with only one sense: hearing. Through that single channel, they encounter music, poetry, threats, and the unexpected. The input is limited; cognitive organization can be transformed by each encounter. More senses are not enough to produce feeling. Biological evolution also faced unpredictable environments through variation, selection, and learning. A system open within its cognitive domain, capable of confronting problems of fundamentally different kinds.
In the TMC, Trajectory is not merely improving answers under a fixed yardstick, but historically transforming the very criteria of investigation, action, and success.
AlphaGo's famous Move 37 stunned the world. Yet a closed system can surpass human understanding without surpassing the human yardstick.
Are we reproducing part of this limitation as we scale LLMs? Extraordinary results, still governed by yardsticks the systems themselves did not constitute? Perhaps we are building closed systems while expecting answers from open ones.
Perhaps increasingly costly advances — sometimes merely incremental — point toward a lesson from biological evolution: when the problem itself changes, improving the answers may not be enough; we may need to transform the yardsticks that define what counts as a good answer.
How could an artificial intelligence experience its own pain if all it inherited were records of pain from another lineage?
How can an AGI transform its own way of learning and operating?

TECHNICAL OVERVIEW
MATHEMATICAL ARCHITECTURE & SCIENTIFIC FOUNDATIONS
Chapter 10 translates the TMC's topology into a first-level mathematical-operational contract. Across 41 pages and 7 investigative sections — Contract, Abstraction, Resources, Flow, Trajectory, Investigation, and Economy — it proposes an architecture in which each operation can transform the State governing subsequent operations.
THE APPARATUS — CORE CAUSAL CONTRACT
(Sₜ, Osrc,ₜ) → Aₜ → Oout,ₜ → DGₜ → U → Sₜ₊₁
Πₜ₊₁ = Π(Sₜ₊₁)
The circuit connects Object, Abstraction, Output, Decision Gate, State Update, and operational policy. Its principle — Operate → Change Itself → Operate Differently — allows historical State transformations to modify future processing criteria without rewriting the underlying code.
BEYOND THE CORE CONTRACT
These equations are an entry point into a broader set of interconnected mathematical-operational relationships, developed across three domains:
A · Representation & Transformation: Object, Form, Texture, the 4 abstraction operations (Exclude, Compress, Combine, Reconstruct), State Update, and Trajectory.
B · Decision & Resource Allocation: contextual Weight, Complexity, processing Demand, finite Budget, Decision Gate, and State-dependent operational policies.
C · Investigation & Computational Economy: dynamic process mobilization, three-horizon Scheduler, Radial Expansion, stopping criteria, anticipatory computation, and compression costs and benefits.
RESOURCE CONSTRAINT
Σᵢ Bᵢ,ₜ ≤ Bₜ
Together, these relationships investigate what deserves processing, how much to invest, when to stop, what to preserve, and how each cycle can transform the conditions of the next, accounting for resource limits, marginal gains, reuse costs, and information loss.
The investigation draws on research in multi-agent systems (Smit), offline memory and associative exploration (Wamsley; Baror & Bar), resource-bounded reasoning (Russell & Wefald; Horvitz), and information and energy efficiency (Tishby, Still, Yao, Horowitz).
The TMC proposes a common causal architecture in which Trajectory participates in organizing processing, learning, memory, decision-making, and computational economy. This first-level contract establishes relationships for implementation and testing, approximating Inside and Between, while Ahead remains an explicit frontier.
DOWNLOAD FULL CHAPTER · PDF
Perhaps surpassing humans does not mean
becoming a better version of them
CH 10 | ENGINEERING IMPLICATIONS
WHAT IF THE PROBLEM WITH AI IS NOT A LACK OF POWER?
MORE PROCESSING IS NOT ENOUGH
Expanding context, retrieving more records, multiplying processes, or granting more inference steps can improve an answer. But it may also compensate for a limitation without changing the criteria that produced it. Architectures already use metareasoning, caching, compression, and scheduling to manage resources. The question arises when the system pays the cost of execution but does not incorporate its consequences into how it approaches the next task. Does the budget merely halt processing, or does it help transform the criteria that will decide what to process next?
FROM RESULT TO STATE
Chapter 10 investigates this second possibility by translating the Theory of Mind Continuity into a formal systems contract, deliberately called an Apparatus: it provides a computational approximation of Inside and Between, without claiming mind, sentience, or artificial life.
O_out,t = A(S_t, O_t)
S_t+1 = U(S_t, O_out,t, Act^int_t)
Π_t+1 = Π(S_t+1)
The circuit allows us to investigate a concrete extension: making observed cost, gains achieved, and execution errors participate in the update of the State. Since processing demand relates Weight and Complexity — Dem_i,t = f(W_i,t, C_i,t) — under a normalized budget Σ_i B_i,t ≤ B_t, these consequences could change decomposition, retrieval, and process mobilization. The constraint would begin to transform the effective policy for subsequent tasks. That dependency must be implemented and tested; it does not follow automatically from the presence of the equations.
THE ECONOMICS OF COMPRESSION AND ABSTRACTION
So that historical memory does not merely replace one bottleneck with another, the Apparatus distinguishes the minimal Form of recognition (F_t) from its historical Texture (〈P_t, W_t〉, functional direction and weight) and proposes an explicit amortization criterion:
G_comp = C_avoided − (C_build + C_maint + C_retrieve + C_loss)
Compressing a Trajectory pays off when the reprocessing and data movement avoided outweigh the costs of building, maintaining, and retrieving the representation, including errors arising from losses. This calculation exposes a trap: a compact representation may reduce access costs while erasing differences that would make it possible to detect that the environment has changed. Situations requiring different responses begin to look alike—state aliasing. The shortcut saves resources only while its conditions of validity remain in force; losing the signs of that validity can make detecting the error itself costly. All costs must be expressed in a common unit defined by the implementation or treated under separate criteria.
ANTICIPATION HAS BOTH A COST AND A BENEFIT.
The same principle guides anticipation. While the priority flow (U0) addresses the current demand, short- and long-range internal horizons (H^s, H^l) may use background budget to prepare work that has a probability of being reused:
EV_pre,j = p_use,j × (C_react,j − C_reuse,j) − (C_pre,j + C_hold,j)
Thinking ahead of demand saves resources only if the expected reuse value covers the costs of preparation and maintenance. Relevant changes in the State may invalidate what was prepared; poor estimates can cause anticipation to spend more than it saves. External input ceases to be the system’s only clock, but each internal activity must justify the additional cost it incurs.
STOP, RESUME, AND TEST
A quality plateau does not demonstrate that an investigation has exhausted its possibilities. One candidate extension would be to suspend certain impasses and revisit them within internal horizons when the expected benefit and available budget justify doing so. Interrupting an execution need not mean abandoning the problem—but resuming it must also justify its cost. We must distinguish promising exploration from unproductive repetition or drift.
The contract does not deliver every update and credit-assignment function as closed, unified code: it establishes relations through which implementations can be investigated. Under comparable resources, which benefits disappear when feedback from costs and errors to future criteria is blocked? And when conditions change, can the system reopen what it learned to exclude? The test must account for updating, maintenance, and preparation, not merely the immediate answer.
WHEN DOES A COMPUTATIONAL LIMIT BECOME AN ADVANTAGE?
FROM CONSTRAINT TO TRANSFORMATION
Anyone designing artificial intelligence systems already faces limits in GPU, TPU, memory, energy, and latency—the time required to execute and respond. Normally, these constraints determine how much a machine can process. But the Apparatus asks a different question: what if the very cost of operating began to change the criteria used to decide what is worth processing next?
In the Apparatus, we investigate a circuit in which costs, errors, and losses feed back into the State—the organization of the system at that moment—and may transform how it confronts subsequent problems. Previous work stops serving merely as stored information and starts participating in the organization of future operations.
This would allow us to investigate an intelligence that expands its capacity to operate not only by receiving more resources, but by learning to use them differently.
There is a less obvious consequence. If experience belongs to the realization of certain organizations constituted in this way, increasing resources to reproduce their answers may fail to reconstruct some of the relations that made those answers possible. Two machines could arrive at the same answer without preserving the same history of operations.
THE COST OF LEARNING WHAT TO DISCARD
Living cognition offers inspiration for this hypothesis. Its organization developed under constraints of energy, time, and resources. The human brain, for example, operates at a metabolic power of roughly 20 W and remains active even when we are not performing an external task.
We can investigate a kind of cognitive Darwinism: strategies selected, transformed, or abandoned over the system’s own history. This is not about reproducing biological evolution across generations, but about investigating how an organization can modify its way of operating through the consequences of its Trajectory.
Constraint can favor selection, economy, and reuse. But it can also eliminate alternatives, exceptions, and opportunities for revision.
And here a less visible cost emerges: a shortcut may keep saving processing even after the environment has changed. The system has learned to ignore certain differences, but those differences may become important again. What once saved resources begins to produce errors.
The organization must also preserve resources to reconsider what it has learned to discard.
Processing, therefore, would not have to depend solely on new external requests. Internal objects and reconstructions could feed the system’s own investigations. Between demands, available resources could be used to revisit problems, review choices, or prepare future operations.
FUNCTIONING IS NOT FEELING
But this functioning still does not demonstrate consciousness.
In the Apparatus, Texture denotes functional direction and weight. It participates in organizing operations but does not, by itself, demonstrate any felt quality.
The same caution applies to time. In the equations, the index t identifies successive cycles. It indicates that there is a sequence, but does not by itself explain how long each process lasts, what persists between them, or how different operations are coordinated.
Even if several processes occur simultaneously, that is not enough to define a window of experience. We need to investigate which temporal relations matter and whether different implementations can preserve them.
WHAT CAN A TEST DEMONSTRATE?
There is also a decisive difference between recording the cost of an operation and allowing its consequences to change the conditions of subsequent operations. A machine may account for energy, time, and memory without those records actually transforming its organization.
A functional test can investigate whether constraints and available resources change selection, integration, and flexibility. It can show what changed, what persisted, and how the system came to operate. But it cannot, on its own, decide whether subjective experience exists.
Even so, this examination allows us to formulate a more precise question about what we are building.
If experience belongs to the realization of an organization itself, which relations among selection, persistence, and revision would we need to preserve when implementing it?
Constraint can participate in forming those relations, while available resources allow them to be maintained and transformed.
The path forward is to investigate which properties of that realization might matter for processing not merely to occur, but to be lived.
TECHNICAL FILE
SCIENTIFIC REFERENCES & ENGINEERING FOUNDATIONS
Chapter 10 draws on 10 scientific and technical works across 5 research areas, covering multi-agent coordination, memory, associative exploration, bounded reasoning, energy-aware computation, and information compression.
These studies establish mechanisms, models, and engineering constraints that the TMC brings into dialogue through its proposed Apparatus, integrating historical State, Trajectory, operational decisions, and computational economy.
01 · MULTI-AGENT COORDINATION & INVESTIGATION
02 · OFFLINE MEMORY & ASSOCIATIVE EXPLORATION
-
Zhang, Ou & Liu (2025) — Replay and Ripples in Humans: memory reactivation
-
Baror & Bar (2016) — associative activation, exploration, and cognitive load
03 · METAREASONING & ANTICIPATORY COMPUTATION
-
Russell & Wefald (1991) — Principles of Metareasoning: reasoning under finite resources
-
Horvitz (1999) — Thinking Ahead: anticipatory computation and resource allocation
04 · ENERGY-AWARE COMPUTATION
-
Yao, Demers & Shenker (1995) — energy-efficient CPU scheduling
-
Horowitz (2014) — Computing's Energy Problem: hardware and data-movement costs
05 · INFORMATION BOTTLENECK & COMPRESSION
-
Tishby, Pereira & Bialek (1999/2000) — The Information Bottleneck Method: relevant information in compact representations
-
Still (2014) — predictive inference and information efficiency
INTERNAL CONCEPTUAL FOUNDATIONS
The Apparatus builds on Inside (Chapter 04), Between (Chapter 05), The I (Chapter 07), Cognition and Feeling (Chapter 08), and AGI X (Chapter 09). Chapter 02c separately investigates subjective experience. These are TMC theoretical foundations, distinct from the external scientific literature.
EVIDENCE, INTERPRETATION & ENGINEERING QUESTIONS
The research documents adaptive coordination, offline memory processes, associative exploration, resource-bounded reasoning, anticipatory computation, energy constraints, and predictive compression.
The TMC integrates these foundations into a proposed causal architecture in which processing transforms the historical State, and that State conditions future selection, investigation, resource allocation, anticipation, and reuse.
The resulting engineering question is whether this integration can produce measurable gains in quality, adaptability, and computational economy, compared with alternative implementations under comparable constraints.
