Advances & challenges in foundation agents: Section 2.1.1 – A unified formulation of learning

This article is Section 2.1.1 of a series of articles featuring Liu and colleagues’ book Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems.

The previous chapter defined learning in the Foundation Agents loop by the update rule M_t = L(M_t−1,a_t−1,o_t). We now replace the black-box placeholder L with a single objective that subsumes supervised fitting, unsupervised representation learning, reinforcement learning, active inference, meta-learning, and continual adaptation. The formulation reveals that every algorithm negotiates three forces simultaneously: fidelity to experience, parsimony of internal representation, and foresight across multiple temporal horizons.

This equation instantiates L as a concrete optimisation problem inside the Foundation Agents framework. Figure 2.2 offers an intuitive visualisation of this framework. It shows how the agent’s mental state M_t evolves by balancing three fundamental forces: fidelity to experience, behavioural shaping via action cost, and preservation of accumulated knowledge through regularisation. These forces interact across multiple temporal scales, from immediate adaptation to lifelong consistency, forming the core dynamics of learning in both biological and artificial systems.

To illustrate the breadth and unifying power of the above formulation, Table 2.1 provides concrete instantiations across major learning paradigms. Each paradigm corresponds to a different way of instantiating the likelihood term, temporal weighting, behavioural cost, and complexity regularisation. Despite their diversity, they all conform to the same abstract objective, validating the generality of Definition 3.

Next part: Section 2.1.2 – Learning across mental state components.

Article source: Liu, B., Li, X., Zhang, J., Wang, J., He, T., Hong, S., … & Wu, C. (2025). Advances and challenges in foundation agents: From brain-inspired intelligence to evolutionary, collaborative, and safe systems. arXiv preprint arXiv:2504.01990. CC BY-NC-SA 4.0.

Header image: AI is Everywhere by Ariyana Ahmad & The Bigger Picture / Better Images of AI, CC BY 4.0.