Which theory of motivation says that animals have a desire to return to homeostasis?

1Group for Neural Theory, INSERM U960, Departément des Etudes Cognitives, Ecole Normale Supérieure, PSL Research University, Paris, France

2Gatsby Computational Neuroscience Unit, University College London, London, United Kingdom

Find articles by Mehdi Keramati

Boris Gutkin

1Group for Neural Theory, INSERM U960, Departément des Etudes Cognitives, Ecole Normale Supérieure, PSL Research University, Paris, France

3Center for Cognition and Decision Making, National Research University Higher School of Economics, Moscow, Russia

Find articles by Boris Gutkin

Eve Marder, Reviewing editor

Eve Marder, Brandeis University, United States;

Author information Article notes Copyright and License information Disclaimer

1Group for Neural Theory, INSERM U960, Departément des Etudes Cognitives, Ecole Normale Supérieure, PSL Research University, Paris, France

2Gatsby Computational Neuroscience Unit, University College London, London, United Kingdom

3Center for Cognition and Decision Making, National Research University Higher School of Economics, Moscow, Russia

*For correspondence: ku.ca.lcu.ybstag@idhem (MK);

Email: [email protected] (BG)

Received 2014 Sep 18; Accepted 2014 Nov 3.

Copyright © 2014, Keramati and Gutkin

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.

Associated Data

Figure 3—source data 1: Free parameters for the anticipatory responding simulation. DOI: http://dx.doi.org/10.7554/eLife.04811.006

elife04811s001.docx (41K)

DOI: 10.7554/eLife.04811.006

Figure 5—source data 1: Free parameters for the risk-aversion simulations. DOI: http://dx.doi.org/10.7554/eLife.04811.009

elife04811s002.docx (38K)

DOI: 10.7554/eLife.04811.009

Figure 6—source data 1: Free parameters for the simulations showing that the model avoids extreme homeostatic deviations. DOI: http://dx.doi.org/10.7554/eLife.04811.011

elife04811s003.docx (40K)

DOI: 10.7554/eLife.04811.011

Figure 8—source data 1: Free parameters for the reinforcing vs. satiation simulations. DOI: http://dx.doi.org/10.7554/eLife.04811.021

elife04811s004.docx (45K)

DOI: 10.7554/eLife.04811.021

Figure 10—source data 1: Free parameters for the over-eating simulations. DOI: http://dx.doi.org/10.7554/eLife.04811.026

elife04811s005.docx (42K)

DOI: 10.7554/eLife.04811.026

Figure 12—source data 1: Free parameters for the within-session dose-change simulation. DOI: http://dx.doi.org/10.7554/eLife.04811.029

elife04811s006.docx (50K)

DOI: 10.7554/eLife.04811.029

Abstract

Efficient regulation of internal homeostasis and defending it against perturbations requires adaptive behavioral strategies. However, the computational principles mediating the interaction between homeostatic and associative learning processes remain undefined. Here we use a definition of primary rewards, as outcomes fulfilling physiological needs, to build a normative theory showing how learning motivated behaviors may be modulated by internal states. Within this framework, we mathematically prove that seeking rewards is equivalent to the fundamental objective of physiological stability, defining the notion of physiological rationality of behavior. We further suggest a formal basis for temporal discounting of rewards by showing that discounting motivates animals to follow the shortest path in the space of physiological variables toward the desired setpoint. We also explain how animals learn to act predictively to preclude prospective homeostatic challenges, and several other behavioral patterns. Finally, we suggest a computational role for interaction between hypothalamus and the brain reward system.

DOI: http://dx.doi.org/10.7554/eLife.04811.001

Research organism: None

eLife digest

Our survival depends on our ability to maintain internal states, such as body temperature and blood sugar levels, within narrowly defined ranges, despite being subject to constantly changing external forces. This process, which is known as homeostasis, requires humans and other animals to carry out specific behaviors—such as seeking out warmth or food—to compensate for changes in their environment. Animals must also learn to prevent the potential impact of changes that can be anticipated.

A network that includes different regions of the brain allows animals to perform the behaviors that are needed to maintain homeostasis. However, this network is distinct from the network that supports the learning of new behaviors in general. These two systems must, therefore, interact so that animals can learn novel strategies to support their physiological stability, but it is not clear how animals do this.

Keramati and Gutkin have now devised a mathematical model that explains the nature of this interaction, and that can account for many behaviors seen among animals, even those that might otherwise appear irrational. There are two assumptions at the heart of the model. First, it is assumed that animals are capable of guessing the impact of the outcome of their behaviors on their internal state. Second, it is assumed that animals find a behavior rewarding if they believe that the predicted impact of its outcome will reduce the difference between a particular internal state and its ideal value. For example, a form of behavior for a human might be going to the kitchen, and an outcome might be eating chocolate.

Based on these two assumptions, the model shows that animals stabilize their internal state around its ideal value by simply learning to perform behaviors that lead to rewarding outcomes (such as going into the kitchen and eating chocolate). Their theory also explains the physiological importance of a type of behavior known as ‘delay discounting’. Animals displaying this form of behavior regard a positive outcome as less rewarding the longer they have to wait for it. The model proves mathematically that delay discounting is a logical way to optimize homeostasis.

In addition to making a number of predictions that could be tested in experiments, Keramati and Gutkin argue that their model can account for the failure of homeostasis to limit food consumption whenever foods loaded with salt, sugar or fat are freely available.

DOI: http://dx.doi.org/10.7554/eLife.04811.002

Introduction

Survival requires living organisms to maintain their physiological integrity within the environment. In other words, they must preserve homeostasis (e.g. body temperature, glucose level, etc.). Yet, how might an animal learn to structure its behavioral strategies to obtain the outcomes necessary to fulfill and even preclude homeostatic challenges? Such, efficient behavioral decisions surely should depend on two brain circuits working in concert: the hypothalamic homeostatic regulation (HR) system, and the cortico-basal ganglia reinforcement learning (RL) mechanism. However, the computational mechanisms underlying this obvious coupling remain poorly understood.

The previously developed classical negative feedback models of HR have tried to explain the hypothalamic function in behavioral sensitivity to the ‘internal’ state, by axiomatizing that animals minimize the deviation of some key physiological variables from their hypothetical setpoints (Marieb & Hoehn, 2012). To this end, a direct corrective response is triggered when a deviation from setpoint is sensed or anticipated (Sibly & McFarland, 1974; Sterling, 2012). A key lacuna in these models is how a simple corrective action (e.g. ‘go eat’) in response to a homeostatic deficit might be translated into a complex behavioral strategy for interacting with the dynamic and uncertain external world.

On the other hand, the computational theory of RL has proposed a viable computational account for the role of the cortico-basal ganglia system in behavioral adaptation to the ‘external’ environment, by exploiting experienced environmental contingencies and reward history (Sutton & Barto, 1998; Rangel et al., 2008). Critically, this theory is built upon one major axiom, namely, that the objective of behavior is to maximize reward acquisition. Yet, this suite of theoretical models does not resolve how the brain constructs the reward itself, and how the variability of the internal state impacts overt behavior.

Accumulating neurobiological evidence indicates intricate intercommunication between the hypothalamus and the reward-learning circuitry (Palmiter, 2007; Yeo & Heisler, 2012; Rangel, 2013). The integration of the two systems is also behaviorally manifest in the classical behavioral pattern of anticipatory responding in which, animals learn to act predictively to preclude prospective homeostatic challenges. Moreover, the ‘good regulator’ theoretical principle implies that ‘every good regulator of a system must be a model of that system’ (Conant & Ashby, 1970), accentuating the necessity of learning a model (either explicit or implicit) of the environment in order to regulate internal variables, and thus, the necessity of associative learning processes being involved in homeostatic regulation.

Given the apparent coupling of homeostatic and learning processes, here, we propose a formal hypothesis for the computations, at an algorithmic level, that may be performed in this biological integration of the two systems. More precisely, inspired by previous descriptive hypotheses on the interaction between motivation and learning (Hull, 1943; Spence, 1956; Mowrer, 1960), we suggest a principled model for how the rewarding value of outcomes is computed as a function of the animal's internal state, and of the approximated need-reduction ability of the outcome. The computed reward is then made available to RL systems that learn over a state-space including both internal and external states, resulting in approximate reinforcement of instrumental associations that reduce or prevent homeostatic imbalance.

The paper is structured as follows: After giving a heuristic sketch of the theory, we show several analytical, behavioral, and neurobiological results. On the basis of the proposed computational integration of the two systems, we prove analytically that reward-seeking and physiological stability are two sides of the same coin, and also provide a normative explanation for temporal discounting of reward. Behaviorally, the theory gives a plausible unified account for anticipatory responding and the rise-fall pattern of the response rate. We show that the interaction between the two systems is critical in these behavioral phenomena and thus, neither classical RL nor classical HR theories can account for them. Neurobiologically, we show that our model can shed light on recent findings on the interaction between the hypothalamus and the reward-learning circuitry, namely, the modulation of dopaminergic activity by hypothalamic signals. Furthermore, we show how orosensory information can be integrated with internal signals in a principled way, resulting in accounting for experimental results on consummatory behaviors, as well as the pathological condition of over-eating induced by hyperpalatability. Finally, we discuss limitations of the theory, compare it with other theoretical accounts of motivation and internal state regulation, and outline testable predictions and future directions.

Which theory of motivation is based on homeostasis?

What is arousal theory of motivation?

What is Humanistic theory of motivation?

What are the 4 motivational theories?