Next Step for AGI - High Order System

Background

Our recent great progress with large language models (LLMs) and other multimodal foundation models—now capable of superhuman performance in daily conversation as well as mathematical and coding reasoning—has hit a wall: the major body of the model weights is spent remembering the gigantic amount of training data rather than the general principles of learning. As a result, these systems fail to achieve general intelligence, especially the adaptivity to new tasks and environments. This wall becomes increasingly visible as realistic tasks grow more long-tailed and long-horizon, which makes the specific training data scarce and expensive to collect (so data scaling law breaks). Yet the human brain, which is never exposed to data on such a scale, still attains a comparable level of intelligence—a central mystery of intelligence that remains unresolved.

We argue that the missing piece is human-level sample and parameter efficiency, attained by leveraging higher-order principles of learning. Current systems directly learn the mapping $x\mapsto y$, for example in next-token prediction over text sequences, QA pairs and reasoning trajectories or visual reasoning in future frame generation; in this first-order manner, all task-relevant knowledge must be memorized inside the parameters, and the model never explicitly acquires meta-level, learning-to-learn capabilities. Such higher-order capability could substantially improve sample efficiency by generalizing knowledge acquisition and decomposing the problem space, instead of memorizing everything beforehand. The core assumption of this article is therefore that the higher-order system is the key to break the limit of training-data scarcity, improve sample efficiency by enabling decomposition of the problem space and achieve general intelligence.

Under this view, three seemingly distinct instances—the world model, the self-evolving agent, and self-modeling—are unified as special cases of a single higher-order functional formulation.

Core Idea

Definition (Higher-order functional system)

Let $\mathcal X$ be an input space, $\mathcal Y$ an output space, and $\mathcal H\subseteq\{f:\mathcal X\to\mathcal Y\}$ a hypothesis space of first-order predictors. Let $\mathcal D_m=(\mathcal X\times\mathcal Y)^m$ be the space of task evidence. A higher-order functional system is an operator

\mathcal F_\theta:\mathcal D_m\to\mathcal H.

Given evidence $D\in\mathcal D_m$ and a query $x\in\mathcal X$, its prediction is

\hat y = [\mathcal F_\theta(D)](x).

First-order systems directly model $x\mapsto y$. Higher-order systems model $D\mapsto f_D$ and then use $f_D$ to answer a new query.

Current System and Higher-Order Functional Modeling

Example: Self-evolving agent as higher-order learning. Here $\tau\sim\rho$ denotes a task sampled from a task distribution $\rho$, so all quantities subscripted by $\tau$ (evidence $D_\tau$, update $\delta f_\tau$, adapted predictor $f_\tau^{\mathrm{adapt}}$, meta-loss $\mathcal J_\tau$) are task-specific. The learned operator $\mathcal U_\theta$ outputs a functional update $\delta f_\tau$, while the meta-gradient $\nabla_\theta \mathcal J$ updates the updater itself.

First-order systems.

Let $\mathcal X$ be an input space and $\mathcal Y$ be an output space. A standard predictive system is a first-order function

\begin{equation} f:\mathcal X \rightarrow \mathcal Y, \end{equation}

which maps an input, context, or problem $x\in\mathcal X$ to an output, extension, or answer $y\in\mathcal Y$. For example, in language modeling, $x$ may denote the context and $y$ the continuation; in problem solving, $x$ may denote the problem statement and $y$ the answer.

A first-order model directly learns the relation

\begin{equation} x \mapsto y. \end{equation}

Therefore, all task-relevant knowledge must be represented implicitly inside the parameters of $f$. In this sense, a first-order system attempts to encode the solution map itself.

Higher-order systems.

A higher-order system does not only model the direct relation $x\mapsto y$. Instead, it models how an appropriate first-order function should be inferred from evidence.

Let

\begin{equation} \mathcal H \subseteq \{f:\mathcal X\rightarrow \mathcal Y\} \end{equation}

be a hypothesis space of first-order systems. Let

\begin{equation} D = \{(x_i,y_i)\}_{i=1}^m \in \mathcal D_m, \qquad \mathcal D_m := (\mathcal X\times\mathcal Y)^m, \end{equation}

denote a set of task-specific observations or evidence.

A higher-order system is an operator

\begin{equation} \mathcal F_\theta:\mathcal D_m \rightarrow \mathcal H, \end{equation}

which maps evidence $D$ to a task-adapted first-order function

\begin{equation} f_D = \mathcal F_\theta(D). \end{equation}

Given a new query $x'\in\mathcal X$, prediction is performed by the composition

\begin{equation} \hat y' = [\mathcal F_\theta(D)](x') = f_D(x'). \end{equation}

Thus, the higher-order system models

\begin{equation} D \mapsto f_D, \qquad f_D:x'\mapsto y', \end{equation}

rather than only

\begin{equation} x'\mapsto y'. \end{equation}

Equivalently, $\mathcal F_\theta$ is a learned functional or operator whose output is itself a function. A first-order system stores a direct solution map, whereas a higher-order system stores a rule for constructing solution maps.

Regularized formulation.

A classical way to define such an operator is through regularized empirical risk minimization:

\begin{equation} \mathcal F(D) = \operatorname*{argmin}_{f\in\mathcal H} \left[ \sum_{(x_i,y_i)\in D} \ell(f(x_i),y_i) + \Omega(f) \right], \end{equation}

where $\ell:\mathcal Y\times\mathcal Y\to\mathbb R_+$ is a loss function and $\Omega:\mathcal H\to\mathbb R_+$ is a regularizer. In this view, the higher-order system is the map

\begin{equation} D \mapsto \operatorname*{argmin}_{f\in\mathcal H} \left[ \sum_{(x_i,y_i)\in D} \ell(f(x_i),y_i) + \Omega(f) \right]. \end{equation}

The neural higher-order system $\mathcal F_\theta$ can be interpreted as a learned approximation to this inference operator.

Bayesian formulation.

Alternatively, suppose there is a prior distribution $p(f)$ over functions $f\in\mathcal H$ and a likelihood $p(D\mid f)$. Then the posterior over functions is

\begin{equation} p(f\mid D) = \frac{p(D\mid f)p(f)}{p(D)}. \end{equation}

In this case, the higher-order system may output either a posterior distribution

\begin{equation} \mathcal F(D)=p(f\mid D), \end{equation}

or a point estimate such as the maximum a posteriori function

\begin{equation} \mathcal F(D) = \operatorname*{argmax}_{f\in\mathcal H} p(f\mid D). \end{equation}

Thus, high-order modeling can be viewed as amortized inference over first-order functions.

Latent-task formulation.

Assume tasks are sampled from a task distribution

\begin{equation} \tau\sim\rho. \end{equation}

Each task $\tau$ has an unknown target function

\begin{equation} f^\star_\tau:\mathcal X\rightarrow\mathcal Y. \end{equation}

For task $\tau$, data are sampled from a task-specific distribution $P_\tau$:

\begin{equation} (x,y)\sim P_\tau, \qquad y = f^\star_\tau(x) + \epsilon, \end{equation}

where $\epsilon$ denotes observation noise.

A first-order model learns a single function

\begin{equation} f_\theta:\mathcal X\rightarrow\mathcal Y \end{equation}

by minimizing the average risk

\begin{equation} \min_\theta \; \mathbb E_{\tau\sim\rho} \mathbb E_{(x,y)\sim P_\tau} \left[ \ell(f_\theta(x),y) \right]. \end{equation}

This objective forces one global model $f_\theta$ to solve all tasks.

In contrast, a higher-order model learns an operator

\begin{equation} \mathcal F_\theta:\mathcal D_m\rightarrow\mathcal H \end{equation}

by minimizing the meta-risk

\begin{equation} \min_\theta \; \mathbb E_{\tau\sim\rho} \mathbb E_{D_\tau\sim P_\tau^m} \mathbb E_{(x',y')\sim P_\tau} \left[ \ell\left( [\mathcal F_\theta(D_\tau)](x'), y' \right) \right]. \end{equation}

The learned object is therefore not merely a predictor, but a task-conditioned predictor constructor.

Bayes-optimal high-order predictor.

The optimal higher-order system admits a clean characterization under squared loss.

Proposition (Bayes-optimal higher-order predictor)

Let the loss be squared error:

\begin{equation} \ell(\hat y,y)=\|\hat y-y\|_2^2. \end{equation}

Suppose that, conditioned on evidence $D$, the target function $f^\star$ is distributed according to $p(f^\star\mid D)$. Then the Bayes-optimal higher-order predictor satisfies

\begin{equation} [\mathcal F^\star(D)](x) = \mathbb E[f^\star(x)\mid D]. \end{equation}

Proof

For fixed $D$ and $x$, the Bayes-optimal prediction is

\begin{equation} g^\star(x) = \operatorname*{argmin}_{z\in\mathcal Y} \mathbb E \left[ \|z-f^\star(x)\|_2^2 \mid D \right]. \end{equation}

Expanding the square,

\begin{align} \mathbb E \left[ \|z-f^\star(x)\|_2^2 \mid D \right] &= \mathbb E \left[ \|z\|_2^2 -2\langle z,f^\star(x)\rangle +\|f^\star(x)\|_2^2 \mid D \right] \\ &= \|z\|_2^2 -2\left\langle z,\mathbb E[f^\star(x)\mid D]\right\rangle + \mathbb E \left[ \|f^\star(x)\|_2^2 \mid D \right]. \end{align}

Differentiating with respect to $z$ and setting the derivative to zero gives

\begin{equation} 2z - 2\mathbb E[f^\star(x)\mid D]=0. \end{equation}

Therefore,

\begin{equation} z = \mathbb E[f^\star(x)\mid D]. \end{equation}

Hence

\begin{equation} [\mathcal F^\star(D)](x) = \mathbb E[f^\star(x)\mid D]. \end{equation}

This proposition shows that the optimal high-order system is an inference operator from evidence $D$ to the posterior mean first-order function.

First-order systems as a special case.

A first-order model is a degenerate higher-order system that ignores the evidence $D$:

\begin{equation} \mathcal F_\theta(D)=f_\theta, \qquad \forall D\in\mathcal D_m. \end{equation}

Then prediction becomes

\begin{equation} [\mathcal F_\theta(D)](x)=f_\theta(x). \end{equation}

Therefore, higher-order modeling strictly generalizes first-order modeling. It can recover a fixed first-order predictor when evidence is uninformative, but it can also produce task-specific predictors when evidence contains information about the underlying task.

Instantiation

Shared abstraction and three instances of higher-order functional modeling. The top part shows the common form: evidence $D$ is mapped by a higher-order operator to a first-order function $f_D$. The bottom part instantiates this object as a world model $M_D$, a functional update $\delta f$, or a self-model $\hat f_c$. In the world-model branch, the inferred model $M_D$ is further reused as a simulator for policy optimization via rollouts (second-order usage); the optimized policy $\pi_{\phi^*}$ then collects fresh evidence in the real environment to update the operator parameters $\theta$, closing a self-improving outer loop.

Example I: world-model operator.

Let $\mathcal S$ be the state space and $\mathcal A$ the action or control space. A first-order world model is a dynamics model

\begin{equation} M:\mathcal S\times\mathcal A\rightarrow\mathcal S, \end{equation}

which predicts the next state

\begin{equation} \hat s_{t+1}=M(s_t,a_t). \end{equation}

This is a first-order transition predictor.

A higher-order world-modeling system instead learns an operator

\begin{equation} \mathcal F_\theta^{\mathrm{wm}}: \mathcal D_{\mathrm{env}} \rightarrow \mathcal M, \end{equation}

where

\begin{equation} \mathcal M \subseteq \{M:\mathcal S\times\mathcal A\rightarrow\mathcal S\} \end{equation}

is a class of dynamics models, and

\begin{equation} D_{\mathrm{env}} = \{(s_t,a_t,s_{t+1})\}_{t=1}^m \end{equation}

is evidence about the current environment.

The adapted world model is

\begin{equation} M_D = \mathcal F_\theta^{\mathrm{wm}}(D_{\mathrm{env}}), \end{equation}

and prediction is

\begin{equation} \hat s_{t+1} = M_D(s_t,a_t) = [\mathcal F_\theta^{\mathrm{wm}}(D_{\mathrm{env}})](s_t,a_t). \end{equation}

The learning objective can be written as

\begin{equation} \min_\theta \; \mathbb E_{\tau} \mathbb E_{D_{\mathrm{env}}\sim P_\tau^m} \mathbb E_{(s,a,s')\sim P_\tau} \left[ d_{\mathcal S} \left( [\mathcal F_\theta^{\mathrm{wm}}(D_{\mathrm{env}})](s,a), s' \right) \right], \end{equation}

where $d_{\mathcal S}$ is a state prediction loss. Thus, the higher-order world model learns how to infer the environment dynamics from experience, rather than learning only a single universal transition function.

Once the adapted world model $M_D$ is obtained, it serves as a first-order object that can be reused by a second-order procedure: policy optimization through rollouts inside the model. Let

\begin{equation} \pi_\phi: \mathcal S \rightarrow \Delta(\mathcal A) \end{equation}

be a policy with parameters $\phi$, so that actions are sampled as

\begin{equation} a_t\sim\pi_\phi(\cdot\mid s_t). \end{equation}

Starting from an initial state $s_0\sim\rho_0$, we generate an imagined trajectory entirely inside the learned world model by alternating

\begin{equation} a_t\sim\pi_\phi(\cdot\mid s_t), \qquad s_{t+1}=M_D(s_t,a_t), \end{equation}

without querying the real environment. The RL objective simply maximizes the expected return of the policy under these model rollouts:

\begin{equation} \max_\phi \; J(\pi_\phi;M_D), \end{equation}

where $J(\pi_\phi;M_D)$ denotes the expected return of $\pi_\phi$ evaluated inside the world model $M_D$. Here the world-model operator $\mathcal F_\theta^{\mathrm{wm}}$ is held fixed, and only the policy parameters $\phi$ are updated. This exhibits the second-order usage of the world model: the first-order system $M_D$ produced by the higher-order operator is turned into a differentiable or sampled simulator, so that learning a good policy no longer requires expensive interaction with the real environment but only rollouts within the inferred dynamics.

Finally, the optimized policy $\pi_{\phi^*}$ can be deployed back into the real environment to keep collecting fresh evidence. These new real-world transitions $(s_t,a_t,s_{t+1})$ are appended to $D_{\mathrm{env}}$ and used to update the world-model operator parameters $\theta$, closing an outer loop:

\begin{equation} \pi_{\phi^*} \xrightarrow{\text{act in real env}} D_{\mathrm{env}} \xrightarrow{\text{update }\theta} \mathcal F_\theta^{\mathrm{wm}} \xrightarrow{} M_D \xrightarrow{\text{rollouts}} \pi_{\phi^*}. \end{equation}

In this cycle, a better policy gathers more informative data, the additional evidence yields a more accurate inferred world model, and the improved model in turn enables a better policy. Data collection, world-model inference, and policy optimization are thereby coupled into a single self-improving loop.

Example II: self-evolving agent.

Let

\begin{equation} f_0:\mathcal X\rightarrow\mathcal Y \end{equation}

be a base first-order agent. A self-evolving higher-order system learns an update operator

\begin{equation} \mathcal U_\theta: \mathcal D\times\mathcal H \rightarrow T\mathcal H, \end{equation}

where $T\mathcal H$ denotes a tangent or update space over the function class $\mathcal H$.

Given evidence $D$ and the current system $f_0$, the update operator outputs a functional correction

\begin{equation} \delta f = \mathcal U_\theta(D,f_0). \end{equation}

The improved agent is

\begin{equation} f_1 = f_0 + \delta f. \end{equation}

Therefore, its prediction is

\begin{equation} \hat y = f_1(x) = \left(f_0+\mathcal U_\theta(D,f_0)\right)(x). \end{equation}

The corresponding objective is

\begin{equation} \min_\theta \; \mathbb E_{\tau,D_\tau,(x',y')} \left[ \ell \left( \left(f_0+\mathcal U_\theta(D_\tau,f_0)\right)(x'), y' \right) \right]. \end{equation}

This formalizes self-improvement as learning an operator that maps evidence and a current first-order function to a functional update.

Gradient-based special case.

A standard gradient-based update can be written as

\begin{equation} f_1 = f_0 - \alpha \nabla_f \mathcal L_D(f_0), \end{equation}

where

\begin{equation} \mathcal L_D(f) = \sum_{(x_i,y_i)\in D} \ell(f(x_i),y_i). \end{equation}

This is a special case of the update-operator formulation with

\begin{equation} \mathcal U(D,f_0) = - \alpha \nabla_f \mathcal L_D(f_0). \end{equation}

A learned self-evolving system generalizes this by learning

\begin{equation} \mathcal U_\theta(D,f_0) \end{equation}

instead of using a fixed hand-designed update rule.

This actually leads to high-order meta-gradient as explained in the Appendix.

Example III: self-modeling and reflective correction.

Let an agent's first-order behavior be represented by

\begin{equation} f:\mathcal X\rightarrow\mathcal Y. \end{equation}

Let $c\in\mathcal C$ denote internal context, such as hidden states, reasoning traces, memory states, uncertainty estimates, or previous outputs. A self-modeling system learns an operator

\begin{equation} \mathcal S_\theta: \mathcal C \rightarrow \widehat{\mathcal H}, \end{equation}

where $\widehat{\mathcal H}$ is a class of models over the agent's own behavior.

The self-model is

\begin{equation} \hat f_c = \mathcal S_\theta(c), \end{equation}

and it predicts the agent's own first-order behavior:

\begin{equation} \hat y = \hat f_c(x) \approx f(x). \end{equation}

A calibrated self-model satisfies

\begin{equation} d(\hat f_c,f)\leq \varepsilon \end{equation}

under a suitable functional distance $d$. For example, one may define

\begin{equation} d(\hat f_c,f) = \mathbb E_{x\sim P} \left[ \ell(\hat f_c(x),f(x)) \right]. \end{equation}

Such a self-model enables reflective correction. Suppose the self-model predicts that the first-order system will make an error on input $x$. Let the predicted error be

\begin{equation} e_c(x) = \hat f_c(x)-y^\star, \end{equation}

where $y^\star$ is the desired answer, target, or externally verified signal. A correction operator

\begin{equation} \mathcal C_\phi: \mathcal C\times\widehat{\mathcal H}\times\mathcal D \rightarrow T\mathcal H \end{equation}

then produces

\begin{equation} \delta f = \mathcal C_\phi(c,\hat f_c,D). \end{equation}

The final answer is generated by the corrected first-order system:

\begin{equation} \hat y = (f+\delta f)(x). \end{equation}

Thus, self-awareness can be formalized not as an informal psychological property, but as calibrated self-modeling: the agent maintains an internal model of its own first-order behavior and uses that model to improve or correct its outputs.

Summary

Compositional view.

The above examples share the same abstract structure:

\begin{equation} \text{evidence} \quad D \xrightarrow{\mathcal F_\theta} \quad \text{first-order function} \quad f_D \xrightarrow{\mathrm{eval\ at\ }x} \quad \text{answer} \quad y. \end{equation}

Equivalently,

\begin{equation} y = \operatorname{Eval}(\mathcal F_\theta(D),x), \end{equation}

where

\begin{equation} \operatorname{Eval}:\mathcal H\times\mathcal X\rightarrow\mathcal Y, \qquad \operatorname{Eval}(f,x)=f(x). \end{equation}

Therefore, the complete higher-order predictor is the composition

\begin{equation} G_\theta(D,x) = \operatorname{Eval}(\mathcal F_\theta(D),x) = [\mathcal F_\theta(D)](x). \end{equation}

Summary.

The distinction between first-order and higher-order systems can be expressed as

\begin{align} \text{First-order system:} \qquad & f:\mathcal X\rightarrow\mathcal Y, \\ \text{Higher-order system:} \qquad & \mathcal F:\mathcal D\rightarrow\mathcal H, \qquad [\mathcal F(D)](x)\rightarrow y. \end{align}

The first-order system learns a direct answer map. The higher-order system learns a rule for constructing answer maps. Hence, higher-order intelligence can be formulated as operator learning over first-order predictors:

\begin{equation} \boxed{ \mathcal F_\theta:D\mapsto f_D, \qquad f_D:x\mapsto y. } \end{equation}

Appendix

Why the update operator is higher-order.

To make the update-operator view explicit, parameterize the first-order system by weights $w\in\mathbb R^p$:

\begin{equation} f_w:\mathcal X\rightarrow\mathcal Y . \end{equation}

Given task evidence

\begin{equation} D = \{(x_i,y_i)\}_{i=1}^m , \end{equation}

define the task loss

\begin{equation} \mathcal L_D(w) = \sum_{(x_i,y_i)\in D} \ell(f_w(x_i),y_i). \end{equation}

The usual gradient-descent update is

\begin{equation} w_1 = w_0 - \alpha \nabla_w \mathcal L_D(w_0). \end{equation}

Equivalently, in function space, this update induces a new first-order function

\begin{equation} f_{w_1} = f_{w_0} + \delta f_D, \end{equation}

where

\begin{equation} \delta f_D := f_{w_0-\alpha\nabla_w\mathcal L_D(w_0)} - f_{w_0}. \end{equation}

Therefore, gradient descent itself can be interpreted as an operator that maps a dataset and a current first-order function to a new first-order function:

\begin{equation} \mathcal U_{\mathrm{GD}}: \mathcal D\times\mathcal H \rightarrow T\mathcal H, \end{equation}

with

\begin{equation} \mathcal U_{\mathrm{GD}}(D,f_{w_0}) = f_{w_0-\alpha\nabla_w\mathcal L_D(w_0)} - f_{w_0}. \end{equation}

The adapted system is then

\begin{equation} f_{w_1} = f_{w_0} + \mathcal U_{\mathrm{GD}}(D,f_{w_0}). \end{equation}

This is higher-order because $\mathcal U_{\mathrm{GD}}$ does not output an ordinary answer $y\in\mathcal Y$. Instead, it outputs a change to a function. That is, the object being transformed is itself a first-order predictor. The update acts at the level of functions:

\begin{equation} \text{data and current function} \quad (D,f) \quad \longmapsto \quad \text{functional update} \quad \delta f . \end{equation}

Learned update operator.

A self-evolving agent replaces the hand-designed gradient-descent update by a learned update operator

\begin{equation} \mathcal U_\theta: \mathcal D\times\mathcal H \rightarrow T\mathcal H, \end{equation}

where $\theta$ parameterizes the update rule itself. The adapted first-order system is

\begin{equation} f_1 = f_0+\mathcal U_\theta(D,f_0). \end{equation}

For a query $x'$, the answer after self-update is

\begin{equation} \hat y' = \left(f_0+\mathcal U_\theta(D,f_0)\right)(x'). \end{equation}

Thus, the prediction pipeline is

\begin{equation} (D,f_0) \xrightarrow{\mathcal U_\theta} \delta f \xrightarrow{+f_0} f_1 \xrightarrow{\mathrm{eval\ at\ }x'} \hat y' . \end{equation}

The learned object $\mathcal U_\theta$ is therefore not merely a predictor. It is a predictor-transformer. It learns how to modify a first-order predictor so that the modified predictor performs better.

Meta-learning objective.

To train the update operator, assume tasks are sampled as

\begin{equation} \tau\sim\rho. \end{equation}

For each task $\tau$, split the data into an adaptation set $D_\tau^{\mathrm{tr}}$ and an evaluation set $D_\tau^{\mathrm{val}}$. The update operator first produces an adapted model

\begin{equation} f_\tau^{\mathrm{adapt}} = f_0 + \mathcal U_\theta(D_\tau^{\mathrm{tr}},f_0). \end{equation}

The meta-objective evaluates the adapted model on held-out data from the same task:

\begin{equation} \min_\theta \; \mathcal J(\theta) := \mathbb E_{\tau\sim\rho} \left[ \mathcal L_{D_\tau^{\mathrm{val}}} \left( f_0+\mathcal U_\theta(D_\tau^{\mathrm{tr}},f_0) \right) \right]. \end{equation}

This is a meta-learning objective because $\theta$ is optimized not to solve the training examples directly, but to produce an update rule that yields a better adapted function on future examples.

Meta-gradient.

The gradient with respect to $\theta$ is a meta-gradient:

\begin{equation} \nabla_\theta \mathcal J(\theta) = \mathbb E_{\tau\sim\rho} \left[ \nabla_\theta \mathcal L_{D_\tau^{\mathrm{val}}} \left( f_0+\mathcal U_\theta(D_\tau^{\mathrm{tr}},f_0) \right) \right]. \end{equation}

By the chain rule,

\begin{equation} \nabla_\theta \mathcal J(\theta) = \mathbb E_{\tau\sim\rho} \left[ \left( \nabla_{f} \mathcal L_{D_\tau^{\mathrm{val}}}(f) \bigg|_{f=f_\tau^{\mathrm{adapt}}} \right) \cdot \nabla_\theta \mathcal U_\theta(D_\tau^{\mathrm{tr}},f_0) \right]. \end{equation}

This gradient updates the parameters of the updater, not merely the parameters of the first-order model. Hence, optimization of $\theta$ is learning how to update.

In parameter space, suppose the update operator outputs a weight update

\begin{equation} \Delta w_\theta = U_\theta(D_\tau^{\mathrm{tr}},w_0), \end{equation}

so that

\begin{equation} w_\tau' = w_0+\Delta w_\theta. \end{equation}

Then the meta-objective is

\begin{equation} \mathcal J(\theta) = \mathbb E_{\tau\sim\rho} \left[ \mathcal L_{D_\tau^{\mathrm{val}}}(w_\tau') \right] = \mathbb E_{\tau\sim\rho} \left[ \mathcal L_{D_\tau^{\mathrm{val}}} \left( w_0+U_\theta(D_\tau^{\mathrm{tr}},w_0) \right) \right]. \end{equation}

The corresponding meta-gradient is

\begin{equation} \nabla_\theta \mathcal J(\theta) = \mathbb E_{\tau\sim\rho} \left[ \nabla_{w'} \mathcal L_{D_\tau^{\mathrm{val}}}(w') \bigg|_{w'=w_\tau'} \; \nabla_\theta U_\theta(D_\tau^{\mathrm{tr}},w_0) \right]. \end{equation}

Therefore, $\theta$ receives credit according to whether its proposed update $U_\theta(D_\tau^{\mathrm{tr}},w_0)$ improves validation performance after adaptation. This is precisely the meta-learning signal.

Gradient descent as a special case.

Ordinary gradient descent is recovered when the update operator is fixed as

\begin{equation} U_{\mathrm{GD}}(D,w_0) = -\alpha\nabla_w\mathcal L_D(w_0). \end{equation}

A learned update operator generalizes this by replacing the fixed rule with

\begin{equation} U_\theta(D,w_0). \end{equation}

For example, it may learn task-dependent step sizes, coordinate-wise preconditioning, curvature corrections, memory-dependent updates, or entirely non-gradient update directions:

\begin{equation} w_1 = w_0 + U_\theta \left( D, w_0, \nabla_w\mathcal L_D(w_0), h_0 \right), \end{equation}

where $h_0$ denotes an optional optimizer state or memory.

Thus, standard learning optimizes the first-order parameters $w$:

\begin{equation} w \leftarrow w-\alpha\nabla_w\mathcal L_D(w), \end{equation}

whereas meta-learning optimizes the higher-order parameters $\theta$ of the update rule:

\begin{equation} \theta \leftarrow \theta-\beta\nabla_\theta\mathcal J(\theta). \end{equation}

The former learns a predictor; the latter learns how to produce a better predictor.

Citation

BibTeX:

@misc{ding2026highorder,
  author       = {Ding, Zihan},
  title        = {Next Step for {AGI}: High-Order System},
  year         = {2026},
  month        = {August},
  howpublished = {\url{https://quantumiracle.github.io/webpage/blogs/blog20260805.html}},
  note         = {Blog post}
}