Link Search Menu Expand Document

Instrumental Actions: Goal-Directed and Habitual

An instrumental action is an action that happens in order to bring about an outcome. When you press a lever in order to retrieve a snack, or when you board a bus in order to travel home, you are performing an instrumental action. What grounds the relation between an instrumental action and the outcome it occurs in order to bring about? This section introduces a key distinction between two answers to this question, goal-directed and habitual.

This recording is also available on stream (no ads; search enabled). Or you can view just the slides (no audio or video).

If the video isn’t working you could also watch it on youtube. Or you can view just the slides (no audio or video).

If the slides are not working, or you prefer them full screen, please try this link.

The recording is available on stream and youtube.

Notes

Terminology

An instrumental action is an action that happens in order to bring about an outcome. We will say that the outcome is a goal of the action,[1] and that the action is directed to the outcome.

Main Question

What is the relation between an instrumental action and the outcome or outcomes to which it is directed?

A Standard Answer

One standard answer to this question involves intention. An intention specifies an outcome, coordinates your actions, and coordinates your actions in a way that would normally increase the probability of the specified outcome ocurring. So if an intention causes you to act, it follows that your action happens in order to bring about the outcome intended. And this implies that your action is instrumental.

What is an intention? Although there is much debate about this (Setiya, 2014), for our purposes only a widely agreed characteristic is necessary. Intentions are the upshot of beliefs and desires (or are identical to one or both of these). To illustrate:

desire: I fill Zak’s glass.

belief: If I pour, I will fill Zak’s glass.

intention: I pour to fill Zak’s glass.

This simplistic example captures a key idea. Behind an intention lie two things. There is a desire to bring an outcome about, and there is a belief about which action will bring the action about.[2]

If you would like more background on action and intention, see Lecture 10 of Mind and Reality.

Our Main Question is about the relation between an instrumental action and the outcome or outcomes to which it is directed. According to the Standard Answer, the relation involves belief, desire and intention:

Background Assumption: Instrumental actions are caused by intentions to bring outcomes about, which are the upshot of desires to bring outcomes about and beliefs that certain actions will bring them about.

Standard Answer: The outcome (or outcomes) to which an instrumental action is directed is that outcome (or outcomes) specified by the intention (or intentions) which caused it.

Does the Standard Answer involving intention provide a full answer to that question? Or are there things other than intentions which might link an instrumental action to an outcome?

A Clue from Animal Learning

According to Dickinson (2016, p. 177):

‘instrumental behavior is controlled by two dissociable processes: a goal-directed and an habitual process’

He goes on to specify what the ‘goal-directed process’ involves:

‘an action is goal-directed if it is mediated by the interaction of a representation of the causal relationship between the action and outcome and a representation of the current incentive value, or utility, of the outcome in a way that rationalizes the action as instrumental for attaining the goal’ Dickinson (2016, p. 177).

Dickinson’s ‘goal-directed process’ corresponds to the belief--desire model we just considered. The ‘representation of the causal relationship between the action and outcome’ could be a belief about which action will bring an outcome about (e.g. the belief that if I pour, I will fill Zak’s glass). And the ‘representation of the current incentive value, or utility, of the outcome’ could be a desire.

philosophy animal learning decision theory
belief representation of the causal relationship between the action and outcome subjective probability
desire representation of the current incentive value, or utility, of the outcome preference

Table: rough correspondence between terms used for modelling action across three disciplines.

So when Dickinson says that instrumental actions are ‘controlled by two dissociable processes’, he is implying that the Standard Answer about belief, desire and intention cannot fully explain the relation between an instrumental action and the outcome or outcomes to which it is directed. If he is right, we also have to consider something he calls ‘an habitual process’.

What Are Habitual Processes?

Habitual processes involve connections between stimuli and actions. For example, the presence of an empty glass (a stimulus) may be connected to the action of pouring. These connections are characterised by two features:

  1. When the action is performed in the presence of the simulus, the connection between action and stimulus is strengthened (or ‘reinforced’) if the action is rewarded.

  2. If the connection is strong enough, the presence of the stimulus will cause the action to occur.

This is another way of stating _Thorndyke’s Law of Effect_:

‘The presenta­tion of an effective [=rewarding] outcome following an action [...] rein­forces a connection between the stimuli present when the action is per­formed and the action itself so that subsequent presentations of these stimuli elicit the [...] action as a response’ (Dickinson, 1994, p. 48).

How do habitual processes differ from those involving belief, desire and intention? Two differences are important for our purposes:

  1. The effects of habitual processes do not depend on what you currently desire. This is because the strength of the stimulus--action connection depends only on what was rewarding for you in the past, not what is rewarding for you now.

  2. The effects of habitual processes do not depend on what you currently believe about which outcome the action will have. This is because the strength of the stimulus--action connection depends only on what outcomes the action had in the past, not on which outcomes it will have now.

Because habitual processes have these features, we can be sure that they are genuinely distinct from processes involving belief, desire and intention.

Habitual Processes and Instrumental Action

Our Main Question is, What is the relation between an instrumental action and the outcome or outcomes to which it is directed? This question can be answered by invoking habitual processes. For if an action is due to an habitual process, then there is a stimulus--action connection which caused it. This stimulus--action connection must have been strengthened in the past because, often enough, some (one or more) rewarding outcomes occurred when the action was performed in the presence of the stimulus. But since habitual processes exist to enable the agent repeatedly bring about such rewarding outcomes, it follows that the action occurs now in order to bring about these (one or more) rewarding outcomes. That is, the action is directed to the outcome; it is an instrumental action.

The Standard Answer therefore fails to provide a full answer to the Main Question about instrumental action. To fully answer it we need not only belief, desire and intention but, minimally, also the kind of stimulus--action connections involved in habitual processes.

So What?

After this section, you should understand what an instrumental action is, you should understand the Main Question, and you should understand how habitual processes and goal-directed processes differ.

The next step is to investigate possible consequences for philosophical theories of action.

Ask a Question

Your question will normally be answered in the question session of the next lecture.

More information about asking questions.

Glossary

directed : For an action to be directed to an outcome is for the action to happen in order to bring that outcome about.
goal : A goal of an action is an outcome to which it is directed.
goal-directed process : A process which involves ‘a representation of the causal relationship between the action and outcome and a representation of the current incentive value, or utility, of the outcome’ and which influences an action ‘in a way that rationalizes the action as instrumental for attaining the goal’ (Dickinson, 2016, p. 177).
goal-state : an intention or other state of an agent which links an action of hers to a particular goal to which it is directed.
habitual process : A process underpinning some instrumental actions which obeys Thorndyke’s Law of Effect: ‘The presenta­tion of an effective [=rewarding] outcome following an action [...] rein­forces a connection between the stimuli present when the action is per­formed and the action itself so that subsequent presentations of these stimuli elicit the [...] action as a response’ (Dickinson, 1994, p. 48).
instrumental action : An action is instrumental if it happens in order to bring about an outcome, as when you press a lever in order to obtain food. (In this case, obtaining food is the outcome, lever pressing is the action, and the action is instrumental because it occurs in order to bring it about that you obtain food.)
You may variations on this definition of instrumental in the literature. Dickinson (2016, p. 177) characterises instrumental actions differently: in place of the teleological ‘in order to bring about an outcome’, he stipulates that an instrumental action is one that is ‘controlled by the contingency between’ the action and an outcome. And de Wit & Dickinson (2009, p. 464) stipulate that ‘instrumental actions are learned’.
model : A model is a way some part or aspect of the world could be.
model based : A model-based process is one that relies on a model. This is usually thought to involve deriving predictions from representations of the model. Compare Dayan & Berridge (2014, p. 477): ‘A model-based strategy involves prospective cognition, formulating and pursuing explicit possible future scenarios based on internal representations of stimuli, situations, and environmental circumstances.’
model free : A model-free process is one that does not rely on a model. This term is often used for processes which exploit causal or statistical connections that are not represented.
outcome : An outcome of an action is a possible or actual state of affairs.
stimulus : A stiumlus is just a situation or event. Typically, ‘stimlus’ is used to label things which do, or might, prompt actions such as the presence of a lever or the flashing of a light.

References

Dayan, P., & Berridge, K. C. (2014). Model-based and model-free Pavlovian reward learning: Revaluation, revision, and revelation. Cognitive, Affective, & Behavioral Neuroscience, 14(2), 473–492. https://doi.org/10.3758/s13415-014-0277-8
de Wit, S., & Dickinson, A. (2009). Associative theories of goal-directed behaviour: A case for animalhuman translational models. Psychological Research PRPF, 73(4), 463–476. https://doi.org/10.1007/s00426-009-0230-6
Dickinson, A. (1994). Instrumental conditioning. In N. Mackintosh (Ed.), Animal learning and cognition. London: Academic Press.
Dickinson, A. (2016). Instrumental conditioning revisited: Updating dual-process theory. In J. B. Trobalon & V. D. Chamizo (Eds.), Associative learning and cognition (Vol. 51, pp. 177–195). Edicions Universitat Barcelona.
Setiya, K. (2014). Intention. In E. N. Zalta (Ed.), The stanford encyclopedia of philosophy (Spring 2014). Metaphysics Research Lab, Stanford University. Retrieved from http://plato.stanford.edu/archives/spr2014/entries/intention/

Endnotes

  1. Be careful not to confuse a goal with a goal-state, which is an intention or other state of an agent linking an action to a particular goal to which it is directed. (Some authors use the term ‘goal’ for goal-states rather than outcomes.) A goal is a possible or actual outcome (such as filling a glass with prosecco). A goal-state is a psychological attribute of an agent (such as an intention to fill a glass with prosecco). ↩︎

  2. We will see the same structure when we come to decision theory (in Expected Utility). Preferences correspond to desires and expected utilities to beliefs. ↩︎