20170821013819finalassignment_instructions__1_ 8pagesdue1am.zip
For this final assignment, you will write a paper in APA 6th edition styling. Your paper should have a title page, running head, references section, as well as a body section that is properly formatted. You can consult your APA manual and also look under “Files ► Course Documents” on Canvas for more information about formatting. A sample paper is provided for a visual demonstration. This paper should be no less than 8 pages long excluding the title page, abstract, and references page. You will need to conduct a t-test on the data to determine if the sample means are significantly different. After the data from Math Tasks Assignment is compiled, the data will be posted on Canvas.
Account info will be included after match.
Use scholarly
INTRODUCTION TO PSYCHOLOGICAL RESEARCH
– PAGE 1 OF 6 –
FFIINNAALL AASSSSIIGGNNMMEENNTT
For this final assignment, you will write a paper in APA 6th edition styling. Your paper
should have a title page, running head, references section, as well as a body section
that is properly formatted. You can consult your APA manual and also look under
“Files ► Course Documents” on Canvas for more information about formatting. A
sample paper is provided for a visual demonstration.
This paper should be no less than 8 pages long excluding the title page, abstract, and
references page.
You will need to conduct a t-test on the data to determine if the sample means are
significantly different. After the data from Math Tasks Assignment is compiled, the data
will be posted on Canvas.
Think of a test with 100 math problems…There are two types of problems:
addition and subtraction. Would you finish the test faster if you completed all of
the addition problems first and all of the subtraction problems second? Is it more
efficient to complete all of the addition problems first? If you switch back and
forth between addition and subtraction, will the mental effort of ‘task-switching’
actually slow you down? Or is simple math so easy that there will be no
difference?
The class experiment that we conducted intended to measure just that. We
used a math task that could be completed in an organized fashion to minimize
task switching (Task A), or a disorganized fashion (Task B) that required a task
switch for every problem attempted. If there is no difference between switching
tasks repeatedly, and completing the same problems in an organized fashion,
then there should be no difference in the mean (average) completion times of
our Task A and Task B samples.
T tests are used to determine if the difference between sample means is
significantly different or not. Using the class’s pooled data, you will conduct a t-
test to compare the sample means of Task A and Task B. Then, you will write a
paper that describes the experiment and analyzes the data.
OVERVIEW
1. Access the data in the excel file [DATASET.xlsx] → Pages section of Canvas…
2. You will need to find 3 journal articles that have relevance to our study, and cite
information from those articles using in-text citations (articles are provided for you
Canvas).
You need at least 2 relevant in-text citations for each article.
The in-text citations should be bold in the color red.
3. You need to be able to analyze the data using a t test, and analyze patterns by
comparing/reporting correlations.
4. Write a paper (APA 6th edition) that is at least 8 pages (10 including title/ref.)
INTRODUCTION TO PSYCHOLOGICAL RESEARCH
– PAGE 2 OF 6 –
––––––––––––TT TTEESSTT––––––––––––
When the data is compiled, you will use the completion times from each condition to
complete the t-test worksheet after you print it out [TtestWorksheet6_ ]. The
worksheet will walk you through a t-test with an alpha of .05 in one tail. This is because
the research predicts that the results will occur in one particular direction.
The data file [DATASET.xlsx] has a tab called “T-test Calculation” that saves you the
work of performing many computations. Be sure to check out this page before starting
your t-test.
In the Results section, please report the results of your t-test in the following
way. You can copy and paste the text into your paper, just replace the
results of your t-test where the #’s are.
M represents ‘Mean’; SD represents ‘Standard Deviation’. The t-test
tutorial [TtestWorksheet6_ ] will help you find these values.
RREEPPOORRTTIINNGG TTHHEE RREESSUULLTTSS OOFF AANN IINNDDEEPPEENNDDEENNTT‐‐MMEEAASSUURREESS TT TTEESSTT
WWhheenn tteesstteedd uussiinngg aa 5511‐‐iitteemm aasssseessssmmeenntt ccoonnttaaiinniinngg aaddddiittiioonn,,
ssuubbttrraaccttiioonn aanndd mmuullttiipplliiccaattiioonn pprroobblleemmss,, tthhee ttaasskk‐‐sswwiittcchhiinngg
ggrroouupp ccoommpplleetteedd pprroobblleemmss [[ffaasstteerr oorr sslloowweerr]] ((MM == ######..####,, SSDD ==
####..####)) tthhaann tthhee ggrroouupp tthhaatt ddiidd nnoott sswwiittcchh ttaasskkss rreeppeeaatteeddllyy ((MM ==
######..####,, SSDD == ####..####)).. TThhee rreessuullttss ooff aann iinnddeeppeennddeenntt‐‐mmeeaassuurreess tt‐‐
tteesstt iinnddiiccaattee tthhaatt tthhee ddiiffffeerreennccee bbeettwweeeenn ssaammppllee mmeeaannss [[wwaass
**OORR** wwaass nnoott ‐‐ ppiicckk oonnee]] ssiiggnniiffiiccaanntt,, tt((ddff)) == ##..#### ((rroouunnddeedd ttoo
ttwwoo ddeecciimmaall ppllaacceess)),, pp << ..0055,, oonnee‐‐ttaaiilleedd..
RED CIRCLE
FROM
T‐TEST
TUTORIAL ORANGE
CIRCLE FROM
T‐TEST
INTRODUCTION TO PSYCHOLOGICAL RESEARCH
– PAGE 3 OF 6 –
––––––––––––CCOORRRREELLAATTIIOONN AANNAALLYYSSIISS––––––––––––
Investigate and discuss patterns that are present in the data by looking at the
correlation between our variables. Use the Critical Values of the Correlation Coefficient
– Table R [In the back of your book] to find out whether the correlations are significant
based on the df (degrees of freedom). Use a two-tailed test with an alpha of .05.
Critical values are based on degrees of freedom. Use the “Correlation Analysis” tab of
the data file [DATASET.xlsx] to analyze the correlations.
Check out the following combinations of variables:
Age and completion time (does being older make people faster or slower?)
Age and # incorrect
Gender and completion time (are males faster than females)
Gender and # incorrect
Determine if the patterns present in our data match patterns present in the literature.
For example, you may cite Reaction Time Literature Review by Robert J. Kosinski, and
make a connection between factors affecting reaction time and whether those factors
may have had an effect in our experiment.
RREEPPOORRTTIINNGG TTHHEE RREESSUULLTTSS OOFF CCOORRRREELLAATTIIOONNSS ((EEXXAAMMPPLLEE))
FFoorr TTaasskk AA,, tthhee vvaarriiaabblleess aaggee aanndd ccoommpplleettiioonn ttiimmee wweerree nnoott
ssttoonnggllyy ccoorrrreellaatteedd rr((ddff))== ‐‐..003322,, pp<< ..0055..
NNoottee:: DDeeggrreeeess ooff ffrreeeeddoomm ffoorr ccoorrrreellaattiioonnss iiss ccaallccuullaatteedd ddiiffffeerreennttllyy.. IItt
iiss tthhee ttoottaall nnuummbbeerr ooff ppaaiirreedd ssccoorreess mmiinnuuss 22.. SSeeee yyoouurr tteexxttbbooookk ffoorr
mmoorree iinnffoorrmmaattiioonn……
INTRODUCTION TO PSYCHOLOGICAL RESEARCH
– PAGE 4 OF 6 –
Abstract
Create an abstract that concisely summarizes this study in 120 words or less.
Introduction (Literature Review)
Provide a brief background of the task-switching literature relevant to this study by
providing 12 in-text citations from the literature (2 citations for each article used). These
in-text citations may be 1 sentence long and should be written in the color red (you do
not necessarily have to read all of the articles in entirety from start to finish! Pull a
relevant citation from each article). Emphasis should be placed on what they did, and
their results; a connection should be made to our study in a predictive manner if
necessary. Some keywords that may help you find relevant articles are:
Task switching
Cognitive Efficiency
Multi-tasking
Selective Attention
Divided Attention
Use the link below to access the journal article databases:
http://library.montclair.edu/articlesdatabases/index.php?View=Subject&Subject=Psych
ology%2FSociology
Explain the hypothesis of this study. Explain what is expected to occur in this experiment
based on the background literature.
Clearly explain what the independent variable is, and what the dependent variable is.
Make sure you include all necessary information for a thorough introduction.
Method
Participants
In the first paragraph, explain the sampling process and how the entire class provided
their own subjects for this experiment. What is the average age of all of the participants
for each condition? How many males and females participated in each condition?
Were these people randomly selected?
In a second paragraph, describe YOUR subjects age and gender for each condition.
Describe the testing environments. In what rooms did the tests take place? What
times? Describe any other information you see fit.
Materials and Procedure
In one paragraph, describe the materials used for this experiment: pen/pencil, printed
addition tasks, etc. Describe Task A and Task B to the reader, explaining the differences
between the two, and similarities between the two.
In another paragraph, explain your procedure. Where did you administer Task A and
Task B? What time did you administer them? Did everything go smoothly, or did you
INTRODUCTION TO PSYCHOLOGICAL RESEARCH
– PAGE 5 OF 6 –
have to deviate slightly from the procedure? Did the person interact with you during
the test or did they sit quietly and remain focused? Did you stay in the room with the
person during the test? Did anything unexpected occur? Did your subjects understand
the directions?
In another paragraph, explain 2 experimental controls that we utilized (For example, the
same 51 mixed-math problems appear in Task A and Task B; by using the same 51
problems we can eliminate any alternative explanations that differences in the difficulty
of the problems are causing a longer completion time).
In yet another paragraph, explain if anything occurred that would threaten the validity
of the test? Describe anything that happened that was not ideal and could have had
an effect on the results. Why is it important for everyone to follow the same procedure?
What would happen if everyone followed their own procedure, how could this affect
the results?
Results
In one titled paragraph , discuss and compare the sample means for Task A and Task B.
Is there a difference; is one higher than the other? Is the difference between means
significant? Discuss the results of the t test, and what the conclusion of this study is.
What was the critical t value? What was the obtained t value? Are the results
significant?
In a second titled paragraph, explain the results of your correlation analysis describing
the information from above. Discuss the relationships between the variables and
modify the following phrase to report each one:
The variables Age and # Incorrect were [not/strongly] correlated, r(ddff) = .##, p<.05.
Discussion
In a final summative paragraph, discuss the implications of these results. Summarize the
findings (Task switching had a negative effect on completion time because
participants required more time to complete the math problems…etc.). Do you think
practice would reduce the effect? Why? Provide a direction for new research, do you
think task switching would have an effect on reaction time in any other task?
Explain the background reasoning of why you think there is an effect.
If there was no effect: Discuss reasons why you think there was no effect? What are
some things we could change about the experiment to get stronger results? Are there
any additional precautions we should take next time? Was the test too easy or too
hard? (If there WAS an effect, then omit this section).
Discuss the limitations of the study. Think of alternative explanations, confounding and
third variables. Problems of the method/procedure, qualities of the sample, etc.
INTRODUCTION TO PSYCHOLOGICAL RESEARCH
– PAGE 6 OF 6 –
References
Properly cite all six references; some articles are provided for you online.
¡¡GGOOOODD LLUUCCKK!!
20170821013819finalassignment_instructions__1_
INTRODUCTION TO PSYCHOLOGICAL RESEARCH
– PAGE 1 OF 6 –
FFIINNAALL AASSSSIIGGNNMMEENNTT
For this final assignment, you will write a paper in APA 6th edition styling. Your paper
should have a title page, running head, references section, as well as a body section
that is properly formatted. You can consult your APA manual and also look under
“Files ► Course Documents” on Canvas for more information about formatting. A
sample paper is provided for a visual demonstration.
This paper should be no less than 8 pages long excluding the title page, abstract, and
references page.
You will need to conduct a t-test on the data to determine if the sample means are
significantly different. After the data from Math Tasks Assignment is compiled, the data
will be posted on Canvas.
Think of a test with 100 math problems…There are two types of problems:
addition and subtraction. Would you finish the test faster if you completed all of
the addition problems first and all of the subtraction problems second? Is it more
efficient to complete all of the addition problems first? If you switch back and
forth between addition and subtraction, will the mental effort of ‘task-switching’
actually slow you down? Or is simple math so easy that there will be no
difference?
The class experiment that we conducted intended to measure just that. We
used a math task that could be completed in an organized fashion to minimize
task switching (Task A), or a disorganized fashion (Task B) that required a task
switch for every problem attempted. If there is no difference between switching
tasks repeatedly, and completing the same problems in an organized fashion,
then there should be no difference in the mean (average) completion times of
our Task A and Task B samples.
T tests are used to determine if the difference between sample means is
significantly different or not. Using the class’s pooled data, you will conduct a t-
test to compare the sample means of Task A and Task B. Then, you will write a
paper that describes the experiment and analyzes the data.
OVERVIEW
1. Access the data in the excel file [DATASET.xlsx] → Pages section of Canvas…
2. You will need to find 3 journal articles that have relevance to our study, and cite
information from those articles using in-text citations (articles are provided for you
Canvas).
You need at least 2 relevant in-text citations for each article.
The in-text citations should be bold in the color red.
3. You need to be able to analyze the data using a t test, and analyze patterns by
comparing/reporting correlations.
4. Write a paper (APA 6th edition) that is at least 8 pages (10 including title/ref.)
INTRODUCTION TO PSYCHOLOGICAL RESEARCH
– PAGE 2 OF 6 –
––––––––––––TT TTEESSTT––––––––––––
When the data is compiled, you will use the completion times from each condition to
complete the t-test worksheet after you print it out [TtestWorksheet6_ ]. The
worksheet will walk you through a t-test with an alpha of .05 in one tail. This is because
the research predicts that the results will occur in one particular direction.
The data file [DATASET.xlsx] has a tab called “T-test Calculation” that saves you the
work of performing many computations. Be sure to check out this page before starting
your t-test.
In the Results section, please report the results of your t-test in the following
way. You can copy and paste the text into your paper, just replace the
results of your t-test where the #’s are.
M represents ‘Mean’; SD represents ‘Standard Deviation’. The t-test
tutorial [TtestWorksheet6_ ] will help you find these values.
RREEPPOORRTTIINNGG TTHHEE RREESSUULLTTSS OOFF AANN IINNDDEEPPEENNDDEENNTT‐‐MMEEAASSUURREESS TT TTEESSTT
WWhheenn tteesstteedd uussiinngg aa 5511‐‐iitteemm aasssseessssmmeenntt ccoonnttaaiinniinngg aaddddiittiioonn,,
ssuubbttrraaccttiioonn aanndd mmuullttiipplliiccaattiioonn pprroobblleemmss,, tthhee ttaasskk‐‐sswwiittcchhiinngg
ggrroouupp ccoommpplleetteedd pprroobblleemmss [[ffaasstteerr oorr sslloowweerr]] ((MM == ######..####,, SSDD ==
####..####)) tthhaann tthhee ggrroouupp tthhaatt ddiidd nnoott sswwiittcchh ttaasskkss rreeppeeaatteeddllyy ((MM ==
######..####,, SSDD == ####..####)).. TThhee rreessuullttss ooff aann iinnddeeppeennddeenntt‐‐mmeeaassuurreess tt‐‐
tteesstt iinnddiiccaattee tthhaatt tthhee ddiiffffeerreennccee bbeettwweeeenn ssaammppllee mmeeaannss [[wwaass
**OORR** wwaass nnoott ‐‐ ppiicckk oonnee]] ssiiggnniiffiiccaanntt,, tt((ddff)) == ##..#### ((rroouunnddeedd ttoo
ttwwoo ddeecciimmaall ppllaacceess)),, pp << ..0055,, oonnee‐‐ttaaiilleedd..
RED CIRCLE
FROM T‐TEST
TUTORIAL ORANGE
CIRCLE FROM
T‐TEST
INTRODUCTION TO PSYCHOLOGICAL RESEARCH
- PAGE 3 OF 6 -
––––––––––––CCOORRRREELLAATTIIOONN AANNAALLYYSSIISS––––––––––––
Investigate and discuss patterns that are present in the data by looking at the
correlation between our variables. Use the Critical Values of the Correlation Coefficient
- Table R [In the back of your book] to find out whether the correlations are significant
based on the df (degrees of freedom). Use a two-tailed test with an alpha of .05.
Critical values are based on degrees of freedom. Use the “Correlation Analysis” tab of
the data file [DATASET.xlsx] to analyze the correlations.
Check out the following combinations of variables:
Age and completion time (does being older make people faster or slower?)
Age and # incorrect
Gender and completion time (are males faster than females)
Gender and # incorrect
Determine if the patterns present in our data match patterns present in the literature.
For example, you may cite Reaction Time Literature Review by Robert J. Kosinski, and
make a connection between factors affecting reaction time and whether those factors
may have had an effect in our experiment.
RREEPPOORRTTIINNGG TTHHEE RREESSUULLTTSS OOFF CCOORRRREELLAATTIIOONNSS ((EEXXAAMMPPLLEE))
FFoorr TTaasskk AA,, tthhee vvaarriiaabblleess aaggee aanndd ccoommpplleettiioonn ttiimmee wweerree nnoott
ssttoonnggllyy ccoorrrreellaatteedd rr((ddff))== ‐‐..003322,, pp<< ..0055..
NNoottee:: DDeeggrreeeess ooff ffrreeeeddoomm ffoorr ccoorrrreellaattiioonnss iiss ccaallccuullaatteedd ddiiffffeerreennttllyy.. IItt
iiss tthhee ttoottaall nnuummbbeerr ooff ppaaiirreedd ssccoorreess mmiinnuuss 22.. SSeeee yyoouurr tteexxttbbooookk ffoorr
mmoorree iinnffoorrmmaattiioonn……
INTRODUCTION TO PSYCHOLOGICAL RESEARCH
- PAGE 4 OF 6 -
Abstract
Create an abstract that concisely summarizes this study in 120 words or less.
Introduction (Literature Review)
Provide a brief background of the task-switching literature relevant to this study by
providing 12 in-text citations from the literature (2 citations for each article used). These
in-text citations may be 1 sentence long and should be written in the color red (you do
not necessarily have to read all of the articles in entirety from start to finish! Pull a
relevant citation from each article). Emphasis should be placed on what they did, and
their results; a connection should be made to our study in a predictive manner if
necessary. Some keywords that may help you find relevant articles are:
Task switching
Cognitive Efficiency
Multi-tasking
Selective Attention
Divided Attention
Use the link below to access the journal article databases:
http://library.montclair.edu/articlesdatabases/index.php?View=Subject&Subject=Psych
ology%2FSociology
Explain the hypothesis of this study. Explain what is expected to occur in this experiment
based on the background literature.
Clearly explain what the independent variable is, and what the dependent variable is.
Make sure you include all necessary information for a thorough introduction.
Method
Participants
In the first paragraph, explain the sampling process and how the entire class provided
their own subjects for this experiment. What is the average age of all of the participants
for each condition? How many males and females participated in each condition?
Were these people randomly selected?
In a second paragraph, describe YOUR subjects age and gender for each condition.
Describe the testing environments. In what rooms did the tests take place? What
times? Describe any other information you see fit.
Materials and Procedure
In one paragraph, describe the materials used for this experiment: pen/pencil, printed
addition tasks, etc. Describe Task A and Task B to the reader, explaining the differences
between the two, and similarities between the two.
In another paragraph, explain your procedure. Where did you administer Task A and
Task B? What time did you administer them? Did everything go smoothly, or did you
INTRODUCTION TO PSYCHOLOGICAL RESEARCH
- PAGE 5 OF 6 -
have to deviate slightly from the procedure? Did the person interact with you during
the test or did they sit quietly and remain focused? Did you stay in the room with the
person during the test? Did anything unexpected occur? Did your subjects understand
the directions?
In another paragraph, explain 2 experimental controls that we utilized (For example, the
same 51 mixed-math problems appear in Task A and Task B; by using the same 51
problems we can eliminate any alternative explanations that differences in the difficulty
of the problems are causing a longer completion time).
In yet another paragraph, explain if anything occurred that would threaten the validity
of the test? Describe anything that happened that was not ideal and could have had
an effect on the results. Why is it important for everyone to follow the same procedure?
What would happen if everyone followed their own procedure, how could this affect
the results?
Results
In one titled paragraph , discuss and compare the sample means for Task A and Task B.
Is there a difference; is one higher than the other? Is the difference between means
significant? Discuss the results of the t test, and what the conclusion of this study is.
What was the critical t value? What was the obtained t value? Are the results
significant?
In a second titled paragraph, explain the results of your correlation analysis describing
the information from above. Discuss the relationships between the variables and
modify the following phrase to report each one:
The variables Age and # Incorrect were [not/strongly] correlated, r(ddff) = .##, p<.05.
Discussion
In a final summative paragraph, discuss the implications of these results. Summarize the
findings (Task switching had a negative effect on completion time because
participants required more time to complete the math problems…etc.). Do you think
practice would reduce the effect? Why? Provide a direction for new research, do you
think task switching would have an effect on reaction time in any other task?
Explain the background reasoning of why you think there is an effect.
If there was no effect: Discuss reasons why you think there was no effect? What are
some things we could change about the experiment to get stronger results? Are there
any additional precautions we should take next time? Was the test too easy or too
hard? (If there WAS an effect, then omit this section).
Discuss the limitations of the study. Think of alternative explanations, confounding and
third variables. Problems of the method/procedure, qualities of the sample, etc.
INTRODUCTION TO PSYCHOLOGICAL RESEARCH
- PAGE 6 OF 6 -
References
Properly cite all six references; some articles are provided for you online.
¡¡GGOOOODD LLUUCCKK!!
20170821013952ttestworksheet6___1_ (1)
State the hypotheses & select an alpha level
H0: µA - µB = 0
H1: µA - µB ≠ 0
For the population there is NO difference between
the groups.
For the population there IS a difference between
the groups.
Identify the critical region. for the independent measures
t statistic, Degrees of freedom are determined by:
df = nA + nB - 2 = = df
Consult the t - distribution table for a One-tailed test with:
The critical t values are: t =
Find the Pooled Variance
+ - 2 =
=
+
=
= S
²=
SSA + SSB
dfp
=
nA
+
s(xA- xB)=
S² p S
² p
nB
+
=
+
= =
t =
(XA - XB) - (µA - µB )
s(x
A
- x
B
)
=
( - ) - ( 0 )
=
=
Make a decision about H0 using the obtained t value...
If the Absolute Value of the obtained t value is more than the critical value, Reject the
null hypothesis and conclude that There is a significant effect...
Compare the RED circle and the PURPLE circle. Remove the negative/minus sign. If the # in
the red circle is bigger than the number in the Purple circle, then your sample means are
significantly different and there is an effect. If the purple circle is bigger than the red
circle, then there is no effect!
α = .05
nA nB
Find the estimated standard error
Compute the test statistic...
t Test tutorial
for two independent samples
MMXIV©MRSIII
PAGE 2
Sample A Sample B
nA =
ΣX = ΣX² =
XA=
ΣX
n
=
SSA= ΣX²
(ΣX)²
n
SSA =
s2=
SSA
n - 1
=
s2= =
sA
= =s2
XA=
Find the mean for each sample
Find the Sum of Squares (SS)
FInd the Standard Deviation (SD)
¡Round to the Thousandths place!
=
(
SSA
)²
=
SSA
=
SSA
nB =
ΣX = ΣX² =
XB=
ΣX
n
=
SSB= ΣX²
(ΣX)²
n
SSB =
s2=
SSB
n - 1
=
s2= =
sB
= =s2
XB=
Find the mean for each sample
Find the Sum of Squares (SS)
FInd the Standard Deviation (SD)
¡Round to the Thousandths place!
=
(
SSB
)²
=
SSB
=
SSB
t Test tutorial
MMXIV©MRSIII
PAGE 1
literature_review_reaction_time_1
A Literature Review on Reaction Time
by Robert J. Kosinski
Clemson University
Reaction time has a been a favorite subject of experimental psychologists since the middle of the nineteenth
century. However, most studies ask questions about the organization of the brain, so the authors spend a lot of
time trying to determine if the results conform to some mathematical model of brain activity. This makes these
papers hard to understand for the beginning student. In this review, I have ignored these brain organization
questions and summarized the major literature conclusions that are applicable to undergraduate laboratories
using my Reaction Time software. I hope this review helps you write a good report on your reaction time
experiment. I also apologize to reaction time researchers for omissions and oversimplifications.
Kinds of Reaction Time Experiments
Psychologists have named three basic kinds of reaction time experiments (Luce, 1986; Welford, 1980):
In simple reaction time experiments, there is only one stimulus and one response. 'X at a known location,' 'spot
the dot,' and 'reaction to sound' all measure simple reaction time.
In recognition reaction time experiments, there are some stimuli that should be responded to (the 'memory set'),
and others that should get no response (the 'distractor set'). There is still only one correct response. 'Symbol
recognition' and 'tone recognition' are both recognition experiments.
In choice reaction time experiments, the user must give a response that corresponds to the stimulus, such as
pressing a key corresponding to a letter if the letter appears on the screen. The Reaction Time program does not
use this type of experiment because the response is always pressing the spacebar.
By the way, professional psychologists doing these experiments typically employ about 20 people doing 100-200
reaction times each...per treatment (Luce, 1986, Ch. 6)! Sanders (1998, p. 23) recommends an adequate period of
practice, and then collection of 300 reaction times per person. Our experiments of 3 or 4 people doing 10 reaction
times each are very small. Whelan (2008) has an extensive series of recommendations on how to analzye reaction
time data.
Mean Reaction Times
For about 120 years, the accepted figures for mean simple reaction times for college-age individuals have been
about 190 ms (0.19 sec) for light stimuli and about 160 ms for sound stimuli (Galton, 1899; Fieandt et al., 1956;
Welford, 1980; Brebner and Welford, 1980).
Simple vs. Recognition vs. Choice Reaction Times
The pioneer reaction time study was that of Donders (1868). He showed that a simple reaction time is shorter than
a recognition reaction time, and that the choice reaction time is longest of all. Laming (1968) concluded that
simple reaction times averaged 220 msec but recognition reaction times averaged 384 msec. This is in line with
many studies concluding that a complex stimulus (e.g., several letters in symbol recognition vs. one letter) elicits a
slower reaction time (Brebner and Welford, 1980; Teichner and Krebs, 1974; Luce, 1986). An example very much
like our experiment was reported by Surwillo (1973), in which reaction was faster when a single tone sounded
than when either a high or a low tone sounded and the subject was supposed to react only when the high tone
sounded.
Miller and Low (2001) determined that the time for motor preparation (e.g., tensing muscles) and motor response
(in this case, pressing the spacebar) was the same in all three types of reaction time test, implying that the
differences in reaction time are due to processing time.
Numer of possible valid stimuli. Several investigators have looked at the effect of increasing the number of
possible stimuli in recognition and choice experiments. Hick (1952) found that in choice reaction time
experiments, response was proportional to log(N), where N is the number of different possible stimuli. In other
words, reaction time rises with N, but once N gets large, reaction time no longer increases so much as when N
was small. This relationship is called "Hick's Law." Sternberg (1969) maintained that in recognition experiments,
as the number of items in the memory set increases, the reaction time rises proportionately (that is, proportional
to N, not to log N). Reaction times ranged from 420 msec for 1 valid stimulus (such as one letter in symbol
recognition) to 630 msec for 6 valid stimuli, increasing by about 40 msec every time another item was added to
the memory set. Nickerson (1972) reviewed several recognition studies and agreed with these results.
Type of Stimulus
Many researchers have confirmed that reaction to sound is faster than reaction to light, with mean auditory
reaction times being 140-160 msec and visual reaction times being 180-200 msec (Galton, 1899; Woodworth and
Schlosberg, 1954; Fieandt et al., 1956; Welford, 1980; Brebner and Welford, 1980). Perhaps this is because an
auditory stimulus only takes 8-10 msec to reach the brain (Kemp et al., 1973), but a visual stimulus takes 20-40
msec (Marshall et al., 1943). Reaction time to touch is intermediate, at 155 msec (Robinson, 1934). Differences in
reaction time between these types of stimuli persist whether the subject is asked to make a simple response or a
complex response (Sanders, 1998, p. 114).
Stimulus Intensity
Froeberg (1907) found that visual stimuli that are longer in duration elicit faster reaction times, and Wells (1913)
got the same result for auditory stimuli.
Pi=E9ron (1920) and Luce (1986) reported that the weaker the stimulus (such as a very faint light) is, the longer
the reaction time is. However, after the stimulus gets to a certain strength, reaction time becomes constant. In
other words, the relationship is:
Hsieh et al. (2007) found that simulated vibration of a computer monitor increased reaction times to stimuli
presented on the monitor, worsened error rates, and caused more visual fatigue. In an application to Web site
design, Tuch et al. (2009) found that visually complex Web sites increased user arousal (and stress), but slowed
reaction times. Kohfeld (1971) found that the difference between reaction time to light and sound could be
eliminated if a sufficiently high stimulus intensity was used.
Other Factors Influencing Reaction Time
If variation caused by the type of reaction time experiment, type of stimulus, and stimulus intensity are ignored,
there are still many factors affecting reaction time.
Arousal. One of the most investigated factors affecting reaction time is 'arousal' or state of attention, including
muscular tension. Reaction time is fastest with an intermediate level of arousal, and deteriorates when the subject
is either too relaxed or too tense (Welford, 1980; Broadbent, 1971; Freeman, 1933). That is, reaction time responds
to arousal as follows:
Etnyre and Kinugasa (2002) found that subjects who had to react to an auditory stimulus by extending their leg
had faster reaction times if they performed a 3 second isometric contraction of the leg muscles prior to the
stimulus. You might expect that the muscle contraction itself would be faster (because the muscle was warmed
up, etc.), but what was surprising was that the precontraction part of the reaction time was shorter too. It was as
if the isometric contraction allowed the brain to work faster. The same conclusion was reached by Masanobu and
Choshi (2006). They found that moderate muscular tension (10% of maximum) shortened the precontraction
reaction times of subjects who were asked to extend either their left or right leg in a choice reaction time task.
Again, it seemed that muscular tension allowed the brain to work faster. Ironically, muscular tension did not
affect movement time. Davranche et al. (2006) also concluded that exercise improved reaction time by increasing
arousal. VaezMousavi et al. (2009) measured arousal in a continuous performance task by skin conductance, and
found that while some subjets showed the traditional pattern in the graph above, others showed the opposite
trend. In general, reaction time tended to improve as arousal increased.
Age. Simple reaction time shortens from infancy into the late 20s, then increases slowly until the 50s and 60s, and
then lengthens faster as the person gets into his 70s and beyond (Welford, 1977; Jevas and Yan, 2001; Luchies et
al., 2002; Rose et al., 2002; Der and Deary, 2006). Luchies et al.(2002) also reported that this age effect was more
marked for complex reaction time tasks, and Der and Deary (2006) concurred. Reaction time also becomes more
variable with age (Hultsch et al., 2002; Gorus et al., 2008) and with Alzheimer's disease (Gorus et al., 2008).
MacDonald et al. (2008) found that reaction time variability in older adults was usually associated with slower
reaction times and worse recognition of stimuli, and suggested that variability might be a useful measure of
general neural integrity. Welford (1980) speculates on the reason for slowing reaction time with age. It is not just
simple mechanical factors like the speed of nervous conduction. It may be the tendency of older people to be more
careful and monitor their responses more thoroughly (Botwinick, 1966). When troubled by a distraction, older
people also tend to devote their exclusive attention to one stimulus, and ignore another stimulus, more completely
than younger people (Redfern et al., 2002). Myerson et al. (2007) found that older adults were as adept as younger
people at assimilating information, but they did take longer to react. Lajoie and Gallagher (2004) found that old
people who tend to fall in nursing homes had a significantly slower reaction time than those that did not tend to
fall. An early study (Galton, 1899) reported that for teenagers (15-19) mean reaction times were 187 msec for light
stimuli and 158 ms for sound stimuli. Reaction times may be getting slower, because we hardly ever see a
Clemson freshman (or professor) who is that fast.
Gender. At the risk of being politically incorrect, in almost every age group, males have faster reaction times
than females, and female disadvantage is not reduced by practice (Noble et al., 1964; Welford, 1980; Adam et al.,
1999; Dane and Erzurumlugoglu, 2003; Der and Deary, 2006). The last study is remarkable because it included
over 7400 subjects. Bellis (1933) reported that mean time to press a key in response to a light was 220 msec for
males and 260 msec for females; for sound the difference was 190 msec (males) to 200 msec (females). In
comparison, Engel (1972) reported a reaction time to sound of 227 msec (male) to 242 msec (female). However,
things may be changing--Silverman (2006) reported evidence that the male advantage in visual reaction time is
getting smaller (especially outside the US), possibly because more women are participating in driving and fast-
action sports. Botwinick and Thompson (1966) found that almost all of the male-female difference was accounted
for by the lag between the presentation of the stimulus and the beginning of muscle contraction. Muscle
contraction times were the same for males and females. In a surprising finding, Szinnai et al. (2005) found that
gradual dehydration (loss of 2.6% of body weight over a 7-day period) caused females to have lengthened choice
reaction time, but males to have shortened choice reaction times. Adam et al. (1999) reported that males use a
more complex strategy than females. Barral and Debu (2004) found that while men were faster than women at
aiming at a target, the women were more accurate. Jevas and Yan (2001) reported that age-related deterioration in
reaction time was the same in men and women.
Left vs. right hand. The hemispheres of the cerebrum are specialized for different tasks. The left hemisphere is
regarded as the verbal and logical brain, and the right hemisphere is thought to govern creativity, spatial
relations, face recognition, and emotions, among other things. Also, the right hemisphere controls the left hand,
and the left hemisphere controls the right hand. This has made researchers think that the left hand should be
faster at reaction times involving spatial relationships (such as pointing at a target). The results of Boulinquez and
Bart=E9l=E9my (2000) and Bart=E9l=E9my and Boulinquez (2001 and 2002) all supported this idea. Dane and
Erzurumluoglu (2003) found that in handball players, the left-handed people were faster than right-handed people
when the test involved the left hand, but there was no difference between the reaction times of the right and left
handers when using the right hand. Finally, although right-handed male handball players had faster reaction
times than right-handed women, there was no such sexual difference between left-handed men and women. The
authors concluded that left-handed people have an inherent reaction time advantage. In an experiment using a
computer mouse, Peters and Ivanoff (1999) found that right-handed people were faster with their right hand (as
expected), but left-handed people were equally fast with both hands. The preferred hand was generally faster.
However, the reaction time advantage of the preferred over the non-preferred hands was so small that they
recommended alternating hands when using a mouse. Derakhshan (2006 and 2009) cautions that preferred hand is
not always a good guide to the dominant hemisphere. In most people, a dominant (and faster) right hand implies
a dominant left hemisphere. However, he found that a minority (20%-25%) of right-handed people actually had a
dominant right hemisphere, and that reaction time on the right side of the body was slower in these people
because commands had to originate in the right hemisphere and then cross over to the left hemisphere, and then
get to the right hand. In other words, the side of the body with the longer reaction time (not always the side with
the nonpreferred hand) is the side with the dominant hemisphere. Bryden (2002), using right-handed people only,
found that task difficulty did not affect the reaction time difference between the left and right hands. Miller and
Van Nes (2007) found that responses involving both hands were faster when the stimulus was presented to both
hemispheres of the brain simultaneously. Because the right (emotional) hemisphere is supplied with input by the
left eye, it might be suspected that the left visual field would be the fastest at identifying emotions. Alves et al.
(2009) confirmed that faces showing happiness or fear were identified most rapidly when presented to the left
visual field (e.g., and examined by the right hemisphere), and that neutral expressions were detected most rapidly
by the right visual field. Muscians appear to have hemispheres that are more equally capable of paying attention
to stimuli than non-muscians, and to have faster reaction times as well (Patston et al., 2007).
Direct vs. Peripheral Vision. Brebner and Welford (1980) cite literature that shows that visual stimuli perceived
by different portions of the eye produce different reaction times. The fastest reaction time comes when a stimulus
is seen by the cones (when the person is looking right at the stimulus). If the stimulus is picked up by rods
(around the edge of the eye), the reaction is slower. Ando et al., 2002 found that practice on a visual stimulus in
central vision shortened the reaction time to a stimulus in peripheral vision, and vice versa.
Practice and Errors. Sanders (1998, p. 21) cited studies showing that when subjects are new to a reaction time
task, their reaction times are less consistent than when they've had an adequate amount of practice. Also, if a
subject makes an error (like pressing the spacebar before the stimulus is presented), subsequent reaction times are
slower, as if the subject is being more cautious. Koehn et al. (2008) also found that "accusing" subjects of making
an error slowed their processing of the next stimulus more than indicating that they had made a correct choice.
Ando et al. (2002) found that reaction time to a visual stimulus decreased with three weeks of practice, and the
same research team (2004) reported that the effects of practice last for at least three weeks. Fontani et al. (2006)
showed that in karate, more experienced practitioners had shorter reaction times, but in volleyball, the
inexperienced players had shorter reaction times (and made more errors too). Visser et al. (2007) found that
training on a complex task both shortened reaction time and improved accuracy. Rogers et al. (2003) found that
training older people to resist falls by stepping out to stabilize themselves did improve their reaction time.
Fatigue. Welford (1968, 1980) found that reaction time gets slower when the subject is fatigued. Singleton (1953)
observed that this deterioration due to fatigue is more marked when the reaction time task is complicated than
when it is simple. Mental fatigue, especially sleepiness, has the greatest effect. Kroll (1973) found no effect of
purely muscular fatigue on reaction time. Philip et al. (2004) found that 24 hours of sleep deprivation lengthened
the reaction times of 20-25 year old subjects, but had no effect on the reaction times of 52-63 year old subjects.
Van den Berg and Neely (2006) found that sleep deprivation caused subjects to have slower reaction times and to
miss stimuli over a test period that lasted two hours. Cote et al. (2009) had the same conclusions about two days
of restricted sleep, and also found that the more restricted sleep was, the worse the deterioration in reaction time,
and the subjects seemed to be compensating for this by more mental effort (measured by high-frequency EEG
waves). Takahashi et al. (2004) studied workers who were allowed to take a short nap on the job, and found that
although the workers thought the nap had improved their alertness, there was no effect on choice reaction time.
Fasting. Three days without food does not decrease reaction time, although it does impair capacity to do work
(Gutierrez et al., 2001). These results were confirmed by Cheatham et al. (2009) found that six months of calorie-
limited diets with either high and low carbohydrates did not affect reaction time or any other cognitive measure.
Diets high in carbohydrates did result in depressed mood.
Distraction. Welford (1980) and Broadbent (1971) reviewed studies showing that distractions increase reaction
time. Trimmel and Poelzl (2006) found that background noise lengthened reaction time by inhibiting parts of the
cerebral cortex. Richard et al. (2002) and Lee et al. (2001) found that college students given a simulated driving
task had longer reaction times when given a simultaneous auditory task. They drew conclusions about the safety
effects of driving while using a cellular phone or voice-based e-mail. Horrey and Wickens (2006) and Hendrick
and Switzer (2007) had similar conclusions about cell phone use while driving, and said that hands-free phones
did not improve reaction time performance. Reaction time suffered more than tasks like keeping in the right lane.
Redfern et al. (2002) found that subjects strapped to a platform that periodically changed orientation had slowed
reaction time before and during platform movement. The reaction time to auditory stimuli was more affected
than response to visual stimuli. Hsieh et al. (2007) found that simulated vibration of a computer monitor increased
reaction times to stimuli presented on the monitor, worsened error rates, and caused more visual fatigue. The
effect of distraction may depend on emotional state and prior experiences. Reed and Antonova (2007) frustrated
some subjects by giving them unsolvable problems, and then tested the reaction times of all the subjects with
distraction. Subjects who had been given the difficult problems were more slowed and distracted than subjects
who had not been frustrated before the reaction time measurement. Similar results were cited by Gerdes et al.
(2008), who found that subjects who were phobic about spiders had their reaction time slowed more by
distracting pictures of spiders than by distracting pictures of objects like flowers and mushrooms. This was caused
by the phobic subjects' failure to look away from the spider pictures as fast as they looked away from the other
pictures.
Warnings of Impending Stimuli. Brebner and Welford (1980) report that reaction times are faster when the
subject has been warned that a stimulus will arrive soon. In the Reaction Time program, the delay is never more
than about 3 sec, but these authors report that even giving 5 minutes of warning helps. Bertelson (1967) found
that as long as the warning was longer than about 0.2 sec., the shorter the warning was, the faster reaction time
was. This effect probably occurs because attention and muscular tension cannot be maintained at a high level for
more than a few seconds (Gottsdanker, 1975). Jakobs et al. (2009) found that stimuli that were predictable elicited
faster reaction times, probably because of decreased computational load on the brain. Also, warning of the
stimulus can increase the number of erroneous responses given before the stimulus (O'Neill and Brown, 2007).
However, Perruchet et al. (2006) said that when two events are associated with one another, conscious
expectation of the second event may actually slow reaction to it. They considered this evidence that expectation
of an event and reaction to it are independent processes.
Alcohol. Moskowitz and Fiorentino (2000) review the imparing effects of alcohol on reaction time. Kruisselbrink
et al. (2006) found that adult females who drank from one to six cans of beer did not suffer delayed reaction times
the next morning, although they made more errors on a choice reaction time task. Hernandez et al. (2007) found
that the slowing of reaction time by alcohol was due to a slowing of muscle activation, not muscle action.
Fillmore and Blackburn (2002) found that subjects who had drunk an impairing dose of alcohol reacted faster
when they were warned that this was enough alcohol to slow their reaction time. Unwarned subjects who drank
suffered more decreased reaction times. However, the warned subjects were also less inhibited and careful in their
responses. Even subjects who drank some nonalcoholic beverage and then were warned (falsely) about
impairment by alcohol reacted faster than unwarned subjects who drank the same beverage.
Order of Presentation. Welford (1980), Laming (1968) and Sanders (1998) observed that when there are several
types of stimuli, reaction time will be faster where there is a 'run' of several identical stimuli than when the
different types of stimuli appear in mixed order. This is called the "sequential effect." Hsieh (2002) found that the
shifting of attention between two different types of tasks caused an increase in reaction time to both tasks.
Breathing Cycle. Buchsbaum and Calloway (1965) found that reaction time was faster when the stimulus
occurred during expiration than during inspiration.
Finger Tremors. Brebner and Welford (1980) report that fingers tremble up and down at the rate of 8-10
cycles/sec, and reaction times are faster if the reaction occurs when the finger is already on the 'downswing' part
of the tremor.
Personality Type. Brebner (1980) found that extroverted personality types had faster reaction times, and Welford
(1980) and Nettelbeck (1973) said that anxious personality types had faster reaction times. Lenzenweger (2001)
found that the reaction times of schizophrenics was slower than those of normal people, but their error rates were
the same. Robinson and Tamir (2005) found that neurotic college students had more variable reaction times than
their more stable peers.
Exercise. Exercise can affect reaction time. Welford (1980) found that physically fit subjects had faster reaction
times, and both Levitt and Gutin (1971) and Sjoberg (1975) showed that subjects had the fastest reaction times
when they were exercising sufficiently to produce a heartrate of 115 beats per minute. Kashihara and Nakahara
(2005) found that vigorous exercise did improve choice reaction time, but only for the first 8 minutes after
exercise. Exercise had no effect on the percent of correct choices the subjects made. Nakamoto and Mori (2008)
found that college students who played basketball and baseball had faster reaction times than sedentary students.
At least for baseball, the more sports experience the students had, the faster their reaction times were to baseball-
specific stimuli. Davranche et al. (2006) concluded that exercise on a stationary bicycle improved reaction times.
On the other hand, McMorris et al. (2000) found no effect of exercise on reaction time in a test of soccer skill, and
Lemmink and Visscher (2005) found that choice reaction time and error rate in soccer players were not affected
by exercise on a stationary bicycle. Pesce et al. (2007) concurred that exercise did not improve the reaction time of
soccer players. Collardeau et al. (2001) found no post-exercise effect in runners, but did find that exercise
improved reaction time during the exercise. They attributed this to increased arousal during the exercise. See the
"Arousal" section for effect of exercise also. Lord et al. (2006) found that water exercise over a period of 22 weeks
did not improve the reaction times of elderly people. The effects of exercise were reviewed by McMorris and
Grayden (2000) and Tomporowski (2003).
Punishment, Stress, and Threats . Shocking a subject when he reacts slowly does shorten reaction time
(Johanson, 1922; Weiss, 1965). Simply making the subject feel anxious about his performance has the same effect,
at least on simple reaction time tasks (Panayiotou, 2004). Mogg et al. (2008) found that it might be hard to
disentangle the effects of threat-induced anxiety from the simple distraction that the threat was causing. In other
words, even a non-threatening stimulus can cause distraction and slow reaction time, but not by causing anxiety.
Verlasting (2006) found that deployment to Iraq caused soldiers to have shorter reaction times, but also increased
tension and reduced proficiency at tasks requiring memory and attention.
Stimulant Drugs. Caffeine has often been studied in connection with reaction time. Lorist and Snel (1997) found
that moderate doses of caffeine decreased the time it took subjects to find a target stimulus and to prepare a
response for a complex reaction time task. Durlach et al. (2002) found that the amount of caffeine in one cup of
coffee did reduce reaction time and increase ability to resist distraction, and did so within minutes after
consumption. McLellan et al. (2005) found that soldiers in simulated urban combat maintained their
marksmanship skills and their reaction times through a prolonged period without sleep better when given
caffeine. Liguori et al. (2001) found that caffeine can reduce the slowing effect of alcohol on reaction time, but
can't prevent other effects such as body sway. On the other hand, Linder (2001), using our software and a "Spot-
the-Dot" test, found that drinking one can of either a caffeinated or a caffeine-free cola had no detectable effect
on reaction time. Froeliger et al. (2009) found that smokers who were abstaining from cigarettes had faster
reaction times on a recognition reaction time task when they were wearing a nicotine patch, and even
nonsmokers had increased accuracy (implying better memory) when they were wearing nicotine patches.
Kleemeier et al. (1956) found that administering an amphetamine-like drug to a group of elderly men did not
make their reaction times faster, although it did make their physical responses more vigorous. On the other hand,
O'Neill and Brown (2007) found that amphetamine and a drug called KW-6002 speeded reaction times and also
increased the frequency of erroneous responses before the stimulus in the hyper-alert participants (rats).
Methylphenidate is a stimulant drug that is used in treatment of attention deficit hyperactivity disorder (ADHD).
If children with ADHD were given methylphenidate (which reduces lapses in attention), their times on a
recognition reaction time task were both shorter and less variable (Spencer et al., 2009).
Intelligence. The tenuous link between intelligence and reaction time is reviewed in Deary et al. (2001). Serious
mental retardation produces slower and more variable reaction times. Among people of normal intelligence, there
is a slight tendency for more intelligent people to have faster reaction times, but there is much variation between
people of similar intelligence (Nettelbeck, 1980). The speed advantage of more intelligent people is greatest on
tests requiring complex responses (Schweitzer, 2001).
Learning Disorders. Miller and Poll (2009) found that college students with a history of language and/or reading
difficulties had slower reaction times. Within the affected group of students, better language skills were associated
with faster reaction times.
Brain Injury. As might be expected, brain injury slows reaction time, but different types of responses are slowed
to different degrees (reviewed in Bashore and Ridderinkhof, 2002). Collins et al. (2003) found that high school
athletes with concussions and headache a week after injury had worse performance on reaction time and memory
tests than athletes with concussions but no headache a week after injury. However, Kaminski et al. (2008) found
that hitting the ball with the head in soccer (and possibly suffering injury from it) had no significant effect on the
reaction time of female soccer players.
Illness. Minor upper respiratory tract infections slow reaction time, make mood more negative, and cause
disturbance of sleep (Smith et al., 2004).
Bibliography
Adam, J., F. Paas, M. Buekers, I. Wuyts, W. Spijkers and P. Wallmeyer. 1999. Gender differences in choice reaction
time: evidence for differential strategies. Ergonomics 42: 327.
Alves, N. T., J. A. Aznar-Casanova, and S. S. Fukusima. 2009. Patterns of brain asymmetry in the perception of
positive and negative facial expressions. Laterality 14(3): 256-272.
Ando, S., N. Kida and S. Oda. 2002. Practice effects on reaction time for peripheral and central visual fields.
Perceptual and Motor Skills 95(3): 747-752.
Ando, S, N. Kida and S Oda. 2004. Retention of practice effects on simple reaction time for peripheral and central
visual fields. Perceptual and Motor Skills 98(3): 897-900.
Barral, J. and B. Debu. 2004. Aiming in adults: Sex and laterality effects. Laterality: Assymmetries of Body, Brain
and Cognition 9(3): 299-312.
Barth=E9l=E9my, S., and P. Boulinguez. 2001. Manual reaction time asymmetries in human subjects: the role of
movement planning and attention. Neuroscience Letters 315(1): 41-44.
Barth=E9l=E9my, S., and P. Boulinguez. 2002. Orienting visuospatial attention generates manual reaction time
asymmetries in target detection and pointing. Behavioral Brain Research 133(1): 109-116.
Bashore, T. R. and K. R. Ridderinkhof. 2002. Older age, traumatic brain injury, and cognitive slowing: some
convergent and divergent findings. Psychological Bulletin 128(1): 151.
Bellis, C. J. 1933. Reaction time and chronological age. Proceedings of the Society for Experimental Biology and
Medicine 30: 801.
Bertelson, P. 1967. The time course of preparation. Quarterly Journal of Experimental Psychology 19: 272-279.
Botwinick, J. 1966. Cautiousness in advanced age. Journal of Gerontology 21: 347-353.
Botwinick, J. and L. W. Thompson. 1966. Components of reaction time in relation to age and sex. Journal of
Genetic Psychology 108: 175-183.
Boulinguez. P. and S. Barth=E9l=E9my. 2000. Influence of the movement parameter to be controlled on manual
RT asymmetries in right-handers. Brain and Cognition 44(3): 653-661.
Brebner, J. T. 1980. Reaction time in personality theory. In A. T. Welford (Ed.), Reaction Times. Academic Press,
New York, pp. 309-320.
Brebner, J. T. and A. T. Welford. 1980. Introduction: an historical background sketch. In A. T. Welford (Ed.),
Reaction Times. Academic Press, New York, pp. 1-23.
Broadbent, D. E. 1971. Decision and Stress. Academic Press, London.
Bryden, P. 2002. Pushing the limits of task difficulty for the right and left hands in manual aiming. Brain and
Cognition 48(2-3): 287-291.
Buchsbaum, M. and E. Callaway. 1965. Influence of respiratory cycle on simple RT. Perceptual and Motor Skills
20: 961-966.
Cheatham, R. A., S. B. Roberts, S. K. Das, C. H. Gilhooly, J. K. Golden, R. Hyatt, D. Lerner, E. Saltzman, and H. R.
Lieberman. 2009. Long-term effects of provided low- and high-glycemic-load low-energy diets on mood and
cognition. Physiology and Behavior 98(3): 374-379.
Collardeau, M., J. Brisswalter, and M. Audiffren. 2001. Effects of a prolonged run on simple reaction time of well-
trained runners. Perceptual and Motor Skills 93(3): 679.
Collins, M. W., M. field, M. R. Lovell, G. Iverson, K. M. Johnston, J. Maroon, and F. H. Fu. 2003. Relationship
between postconcussion headache and neuropsychological test performance in high school athletes. The
American Journal of Sports Medicine (31(2): 168-174.
Cote, K. A., C. E. Milner, B. A. Smith, A. J. Aubin, T. A. Greason, B. P. Cuthbert, S. Wiebe, and S. E. G. Duffus.
2009. CNS arousal and neurobehavioral performance in a short-term sleep restriction paradigm. Journal of Sleep
Research18(3): 291-303.
Dane, S. and A. Erzurumluoglu. 2003. Sex and handedness differences in eye-hand visual reaction times in
handball players. International Journal of Neuroscience 113(7): 923-929.
Davranche, K., M. Audiffren, and A. Denjean. 2006. A distributional analysis of the effect of physical exercise on
a choice reaction time task. Journal of Sports Sciences 24(3): 323-330.
Deary, I. J., G. Der, and G. Ford. 2001. Reaction times and intelligence differences: A population-based cohort
study. Intelligence 29(5): 389.
Der, G., and I. J. Deary. 2006. Age and sex differences in reaction time in adulthood: Results from the United
Kingdom health and lifestyle survey. Psychology and Aging 21(1): 62-73.
Derakhshan, I. 2006. Crossed-uncrossed difference (CUD) in a new light: anatomy of the negative CUD in
Poffenberger's paradigm. Acta Neurologica Scandinavica 113(3): 203-208.
Derakhshan, I. 2009. Right sided weakness with right subdural hematoma: Motor deafferentation of left
hemisphere resulted in paralysis of the right side. Brain Injury 23(9): 770-774.
Donders, F. C. 1868. On the speed of mental processes. Translated by W. G. Koster, 1969. Acta Psychologica 30:
412-431.
Durlach, P. J., R. Edmunds, L. Howard, and S. P. Tipper. 2002. A rapid effect of daffeinated beverages on two
choice reaction time tasks. Nutritional Neuroscience 5(6): 433-442.
Engel, B. T., P. R. Thorne, and R. E. Quilter. 1972. On the relationship among sex, age, response mode, cardiac
cycle phase, breathing cycle phase, and simple reaction time. Journal of Gerontology 27: 456-460.
Etnyre, B. and T. Kinugasa. 2002. Postcontraction influences on reaction time (motor control and learning).
Research Quaterly for Exerciseand Sport 73(3): 271-282.
Fieandt, K. von, A. Huhtala, P. Kullberg, and K. Saarl. 1956. Personal tempo and phenomenal time at different age
levels. Reports from the Psychological Institute, No. 2, University of Helsinki.
Fillmore, M. T. and J. Blackburn. 2002. Compensating for alcohol-induced impairment: alcohol expectancies and
behavioral disinhibition. Journal of Studies on Alcohol 63(2): 237.
Fontani, G., L. Lodi, A. Felici, S. Migliorini and F. Corradeschi. 2006. Attention in athletes of high and low
experience enganged in different open skill sports. Perceptual and Motor Skills 102(3): 791-816.
Freeman, G. L. 1933. The facilitative and inhibitory effects of muscular tension upon performance. American
Journal of Psychology 26: 602-608.
Froeberg, S. 1907. The relation between the magnitude of stimulus and the time of reaction. Archives of
Psychology, No. 8.
Froeliger, B., D. G. Gilbert, and F. J. McClernon. 2009. Effects of nicotine on novelty detection and memory
recognition performance: double-blind, placebo-controlled studies of smokers and nonsmokers.
Psychopharmacology 205(4): 625-633
Galton, F. 1899. On instruments for (1) testing perception of differences of tint and for (2) determining reaction
time. Journal of the Anthropological Institute 19: 27-29.
Gerdes, A. B. M., G. W. Alpers and P. Pauli. 2008. When spiders appear suddenly: spider-photic patients are
distracted by task-irrelevant spiders. Behavior Research and Therapy 46(2): 174-188.
Gorus, E., R. De Raedt, M. Lambert, J. Lemper and T. Mets. 2008. Reaction times and performance variability in
normal aging, mild cognitive impairment, and Alzheimer's disease. Journal of Geriatric Psychiatry and Neurology
21(3): 204-219.
Gottsdanker, R. 1975. The attaining and maintaining of preparation. Pages 33-49 in P. M. A. Rabbitt and S. Dornic
(Eds.), Attention and Performance, Vol. 5. London, Academic Press.
Gutierrez, A., M. Gonzalez-Gross, M. Delgado, and M. J. Castillo. 2001. Three days fast in sportsmen decrease
physical work capacity but not strength or perception-reaction time. International Journal of Sport Nutrition and
Exercise Metabolism 11(4): 420.
Hendrick, J. L. and J. R. Switzer. 2007. Hands-free versus hand-held cell phone conversation on a braking response
by young drivers. Perceptual and Motor Skills (105(2): 514-523.
Hernandez, O. H., M. Vogel-Sprott, and V. I. Ke-Aznar. 2007. Alcohol impairs the cognitive component of reaction
time to an omitted stimulus: a replication and an extension. Journal of Studies on Alcohol and Drugs 68(2): 276-
282.
Hick, W. E. 1952. On the rate of gain of information. Quarterly Journal of Experimental Psychology 4: 11-26.
Horrey, W. J., and C. D. Wickens. 2006. Examining the impact of cell phone conversations on driving using meta-
analytic techniques. Human Factors 48(1): 196.
Hsieh, S. 2002. Tasking shifting in dual-task settings. Perceptual and Motor Skills 94(2): 407.
Hsieh, Y., C. J. Lin and H. Chen. 2007. Effect of vibration on visual display terminal work performance. Perceptual
and Motor Skills 105(3): 1055-1059.
Hultsch, D. F., S. W. MacDonald and R. A. Dixon. 2002. Variability in reaction time performance of younger and
older adults. The Journals of Gerontology, Series B 57(2): 101.
Jevas, S. and J. H. Yan. 2001. The effect of aging on cognitive function: a preliminary quantitative review.
Research Quarterly for Exercise and Sport 72: A-49.
Johanson, A. M. 1922. The influence of incentive and punishment upon reaction-time. Archives of Psychology, No.
54.
Jakobs, O., L. E. Wang, M. Dafotakis, C. Grefkes, K. Zilles, and S. B. Eickhoff. 2009. Effects of timing and
movement uncertainty implicate the temporo-parietal junction in the prediction of forthcoming motor actions.
NeuroImage 47(2): 667-677.
Kaminski, T. W., E. S. Cousino and J. J. Glutting. 2008. Examining the relationship between purposeful heading in
soccer and computerized neuropsychological test performance. Research Quaterly for Exercise and Sport 79(2):
235-245.
Kashihara, K. and Y. Nakahara. 2005. Short-term effect of physical exercise at lactate threshold on choice reaction
time. Perceptual and Motor Skills 100(2): 275-281.
Kemp, B. J. 1973. Reaction time of young and elderly subjects in relation to perceptual deprivation and signal-on
versus signal-off condition. Developmental Psychology 8: 268-272.
Kleemeier, R. W., T. A. Rich, and W. A. Justiss. 1956. The effects of alpha-(2-piperidyl) benzhydrol hydrochloride
(Meratran) on psychomotor performance in a group of aged males. Journal of Gerontology 11: 165-170.
Koehn, J. D., J. Dickenson, and D. Goodman. 2008. Cognitive demands of error processing. Psychological Reports
102(2): 532-539.
Kohfeld, D. L. 1971. Simple reaction time as a function of stimulus intensity in decibels of light and sound.
Journal of Experimental Psychology 88: 251-257.
Kroll, W. 1973. Effects of local muscular fatigue due to isotonic and isometric exercise upon fractionated reaction
time components. Journal of Motor Behavior 5: 81-93.
Kruisselbrink, L. D., K. L. Martin, M. Megeney, J. R. Fowles, and R. J. L. Murphy. 2006. Physical and psychomotor
functioning of females the morning after consuming low to moderate quantities of beer. Journal of Studies on
Alcohol 67(3): 416-421.
Lajoie, Y. and S. P. Gallagher. 2004. Predicting falls within the elderly community: comparison of postural sway,
reaction time, the Berg balance scale and the Activities-specific Balance Confidence (ABC) scale for comparing
fallers and non-fallers. Archives of Gerontology and Geriatrics 38(1): 11-25.
Laming, D. R. J. 1968. Information Theory of Choice-Reaction Times. Academic Press, London.
Lee, J. D., B. Caven, S. Haake, and T. L. Brown. 2001. Speech-based interaction with in-vehicle computers: The
effect of speech-based e-mail on drivers' attention to the roadway. Human Factors 43(4): 631.
Lemmink, K. and C. Visscher. 2005. Effect of intermittent exercise on multiple-choice reaction times of soccer
players. Perceptual and Motor Skills 100(1): 85-95.
Lenzenweger, M. F. 2001. Reaction time slowing during high-load, sustained-attention task performance in
relation to psychometrically identified schizotypy. Journal of Abnormal Psychology 110: 290.
Liguori, A. and J. H. Robinson. 2001. Caffeine anatagonism of alcohol-induced driving impairment. Drug and
Alcohol Dependence 63(2): 123-129.
Linder, G. N. 2001. The effect of caffeine consumption on reaction time. Bulletin of the South Carolina Academy
of Science, Annual 2001: 42.
Lorist, M. M. and J. Snel. 1997. Caffeine effects on perceptual and motor processes. Electroencephalography and
Clinical Neurophysiology 102(5): 401-414.
Levitt, S. and B. Gutin. 1971. Multiple choice reaction time and movement time during physical exertion.
Research Quarterly 42: 405-410.
Lord, S., R, B. Matters, R. St George, M. Thomas, J. Bindon, K. Chan, A. Collings, and L. Haren. 2006. The effects
of water exercise on physical functioning in older people. Australasian Journal on Ageing 25(1): 36-42.
Luce, R. D. 1986. Response Times: Their Role in Inferring Elementary Mental Organization. Oxford University
Press, New York.
Luchies, C. W., J. Schiffman, L. G. Richards, M. R. Thompson, D. Bazuin, and A. J. DeYoung. 2002. Effects of age,
step direction, and reaction condition on the ability to step quickly. The Journals of Gerontology, Series A 57(4):
M246.
MacDonald, S. W. S., L. Nyberg, J. Sandblom, H. Fischer, and L. Backman. 2008. Increased response-time
variability is associated with reduced inferior parietal activation during episodic recognition in aging. Journal of
Cognitive Neuroscience 20(5): 779-787.
Marshall, W. H., S. A. Talbot, and H. W. Ades. 1943. Cortical response of the anaesthesized cat to gross photic and
electrical afferent stimulation. Journal of Nerophysiology 6: 1-15.
Masanobu, A. and K. Choshi. 2006. Contingent muscular tension during a choice reaction task. Perceptual and
Motor Skills 102(3) (June 2006): 736-747.
McLellan, T. M., G. H. Kamimori, D. G. Bell, I. F. Smith, D. Johnson, and G. Belenky. 2005.Caffeine maintains
vigilance and marksmanship in simulated urban operations with sleep deprivation. Aviation, Space, and
Environmental Medicine 76(1): 39-45.
McMorris, T., and Graydon, J. 2000. The effect of incremental exercise on cognitive performance. International
Journal of Sport Psychology 31: 66-81.
McMorris, T., J. Sproule, S. Draper, and R. Child. 2000. Performance of a psychomotor skill following rest, exercise
at the plasma epinephrine threshold and maximal intensity exercise. Perceptual and Motor Skills 91(2): 553-563.
Miller, C. A. and G. H. Poll. 2009. Response time in adults with a history of language difficulties. Journal of
Communication Disorders 42(5): 365-379.
Miller, J. O. and K. Low. 2001. Motor processes in simple, go/no-go, and choice reaction time tasks: a
psychophysiological analysis. Journal of Experimental Psychology: Human Perception and Performance 27: 266.
Miller, J. and F. Van Nes. 2007. Effects of response task and accessory stimuli on reduncancy gain: tests of the
hemispheric coactivation model. Journal of Experimental Psychology: Human Perception and Performance 33(4):
829-845.
Mogg, K., A. Holmes, M. Garner, and B. P. Bradley. 2008. Effects of threat cues on attentional shifting,
disengagement and response slowing in anxious individuals. Behavior Research and Therapy 46(5): 656-558.
Moskowitz, H. and Fiorentino, D. 2000. A Review of the Literature on the Effects of Low Doses of Alcohol on
Driving-Related Skills, Report DOT HS 809 028, Washington: National Highway Traffic Safety Administration,
Department of Transportation.
Myerson, J. S. Robertson, and S. Hale. 2007. Aging and intraindiviual variability in performance: Analysis of
respone time distributions. Journal of the Experimental Anlysis of Behavior 88(3): 319-337.
Nakamoto, H. and S. Mori. 2008. Sport-specific decision-making in a go/no go reaction task: difference among
nonathletes and baseball and basketball players. Perceptual and Motor Skills 106(1): 163-171.
Nettelbeck, T. 1973. Individual differences in noise and associated perceptual indices of performance. Perception 2:
11-21.
Nettelbeck, T. 1980. Factors affecting reaction time: Mental retardation, brain damage, and other
psychopathologies. In A. T. Welford (Ed.), Reaction Times. Academic Press, New York, pp. 355-401.
Nickerson, R. S. 1972. Binary-classification reaction times: A review of some studies of human information-
processing capabilities. Psychonomic Monograph Supplements 4: 275-318.
Noble, C. E., B. L. Baker, and T. A. Jones. 1964. Age and sex parameters in psychomotor learning. Perceptual and
Motor Skills 19: 935-945.
O'Neill, M. and V. J. Brown. 2007. Amphetamine and the adenosine A2A antagonist KW-6002 enahance the
effects of conditional temporal probability of a stimulus in rats. Behavioral Neuroscience 121(3): 535-543.
Panayiotou, G. and S. R. Vrana. 2004. The role of self-focus, task difficulty, task self-relevance, and evaluation
anxiety in reaction time performance. Motivation and Emotion 28(2): 171-196.
Patston, L. M., S. L. Hogg, and L. J. Tippett. 2007. Attention in muscians is more bilateral than in non-muscians.
Laterality 12(3): 262-272.
Perruchet, P., A. Cleeremans and A. Destrebecqz. 2006. Dissociating the effects of automatic activation and
explicit expectancy on reaction times in a simple associative learning task. Journal of Experimental Psychology:
Learning, Memory and Cognition 32.5 (Sept 2006): 955-966.
Pesce, C., A. Tessitore, R. Casella, M. Pirritano and L. Capranica. 2007. Focusing on visual attention at rest and
during physical exericise in soccer players. Journal of Sports Sciences 25(11): 1259-1271.
Peters, M. and J. Ivanoff. 1999. Performance asymmetries in computer mouse control of right-handers, and left
handers with left- and right-handed mouse experience. Journal of Motor Behavior 31(1): 86-94.
Philip, P., J. Taillard, P. Sagaspe, C. Valtat, M. Sanchez-Ortuno, N. Moore, A. Charles, and B. Bioulac. 2004. Age,
performance, and sleep deprivation. Journal of Sleep Research 13(2): 105-110.
Pi=E9ron, H. 1920. Nouvelles recherches sur l'analyse du temps de latence sensorielle et sur la loi qui relie ce
temps a l'intensit=E9 de l'excitation. Ann=E9e Psychologique 22: 58-142.
Redfern, M. S., M. Muller, J. R. Jennings, J. M. Furman. 2002. Attentional dynamics in postural control during
perturbations in young and older adults. The Journals of Gerontology, Series A 57(8): B298.
Reed, P. and M. Antonova. 2007. Interference with judgments of control and attentional shift as a result of prior
exposure to controllable and uncontrollable feedback. Learning and Motivation 38(3): 229-242.
Richard, C. M., R. D. Wright, C. Ee, S. L. Prime, U. Shimizu, and J. Vavrik. 2002. Effect of a concurrent auditory
task on visual search performance in a driving-related image-flicker task. Human Factors44(2): 108.
Robinson, E. S. 1934. Work of the integrated organism. In C. Murchison (Ed.), Handbook of General Experimental
Psychology, Clark University Press, Worcester, MA.
Robinson, M. C. and M. Tamir. 2005. Neuroticism as mental noise: a relation between neuroticism and reaction
time standard deviations. Journal of Personality and Social Psychology 89(1): 107-115.
Rogers, M. W., M. E. Johnson, K. M. Martinez, M-L Mille, and L. D. Hedman. 2003. Step training improves the
speed of voluntary step initiation in aging. The Journals of Gerontology, Series A 58(1): 46-52.
Rose, S. A., J. F. Feldman, J. J. Jankowski, and D. M. Caro. 2002. A longitudinal study of visual expectation and
reaction time in the first year of life. Child Development 73(1): 47.
Sanders, A. F. 1998. Elements of Human Performance: Reaction Processes and Attention in Human Skill.
Lawrence Erlbaum Associates, Publishers, Mahwah, New Jersey. 575 pages.
Schweitzer, K. 2001. Preattentive processing and cognitive ability. Intelligence 29 i2: p. 169.
Silverman, I. W. 2006. Sex differences in simple visual reaction time: a historical meta-analysis (sports events). Sex
Roles: A Journal of Research 54(1-2): 57-69.
Singleton, W. T. 1953. Deterioration of performance on a short-term perceptual-motor task. In W. F. Floyd and A.
T. Welford (Eds.), Symposium on Fatigue. H. K. Lewis and Co., London, pp. 163-172.
Sjoberg, H. 1975. Relations between heart rate, reaction speed, and subjective effort at different work loads on a
bicycle ergometer. Journal of Human Stress 1: 21-27.
Smith, A., C. Brice, A. Leach, M. Tilley, and S. Williamson. 2004. Effects of upper respiratory tract illnesses in a
working population. Ergonomics 47(4): 363-369.
Spencer, S. V., L. W. Hawk, Jr., J. B. Richards, K. Shiels, W. E. Pelham, Jr., and J. G. Waxmonsky. 2009. Stimulant
treatment reduces lapses in attention among children with ADHD: The effects of methylphenidate on intra-
individual response time distributions. Journal of Abnormal Child Psychology 37(6): 805-816.
Sternberg, S. 1969. Memory scanning: Mental processes revealed by reaction time experiments. American Scientist
57: 421-457.
Surwillo, W. W. 1973. Choice reaction time and speed of information processing in old age. Perceptual and Motor
Skills 36: 321-322.
Szinnai, G. H. Schachinger, M. J. Arnaud, L. Linder, and U. Keller. 2005. Effect of water deprivation on cognitive-
motor performance in healthy men and women. The American Journal of Physiology 289(1): R275-280.
Takahashi, M., A. Nakata, T. Haratani, Y. Ogawa, and H. Arito. 2004. Post-lunch nap as a worksite intervention to
promote alertness on the job. Ergonomics 47(9) 1003-1013.
Teichner, W. H. and M. J. Krebs. 1974. Laws of visual choice reaction time. Psychological Review 81: 75-98.
Tomporowski, P. D. 2003. Effects of acute bouts of exercise on cognition. Acta Psychologica 112: 297-324.
Trimmel, M., and G. Poelzl. 2006. Impact of background noise on reaction time and brain DC potential changes of
VDT-based spatial attention. Ergonomics 49(2): 202-209.
Tuch, A. N., J. A. Bargas-Avila, K. Opwis, and F. H. Wilhelm. 2009. Visual complexity of websites: Effects on
users' experience, physiology, performance, and memory. International Journal of Human-Computer Studies
67(9): 703-715.
Vasterling, J. J. 2006. Neuropsychological outcomes of Army personnel following deployment to the Iraq War.
JAMA, The Journal of the American Medical Association 296(5): 519-530.
van den Berg, J., and G. Neely. 2006. Performance on a simple reaction time task while sleep-deprived. Perceptual
and Motor Skills 102(2): 589-6
VaevMousavi, S. M., R. J. Barry, and A. R. Clarke. 2009. Individual differences in task-related activation and
performance. Physiology and Behavior 98(3): 326-330.
Visser, I., M. E. J. Raijmakers, and P. C. M. Molenaar. 2007. Characterizing sequence knowledge using online
measures and hidden Markov models. Memory and Cognition 35(6): 1502-1518.
Weiss, A. D. 1965. The locus of reaction time change with set, motivation, and age. Journal of Gerontology 20: 60-
64.
Welford, A. T. 1968. Fundamentals of Skill. Methuen, London.
Welford, A. T. 1977. Motor performance. In J. E. Birren and K. W. Schaie (Eds.), Handbook of the Psychology of
Aging. Van Nostrand Reinhold, New York, pp. 450-496.
Welford, A. T. 1980. Choice reaction time: Basic concepts. In A. T. Welford (Ed.), Reaction Times. Academic Press,
New York, pp. 73-128.
Wells, G. R. 1913. The influence of stimulus duration on RT. Psychological Monographs 15: 1066.
Whelan, R. 2008. Effective analysis of reaction time data. The Psychological Record 58(3): 475-483.
Woodworth, R. S. and H. Schlosberg. 1954. Experimental Psychology. Henry Holt, New York.
control_of_cognitive_processes_pages_267_276_
Oct 2000
ISBN 0262133679
698 pp.
122 illus.
$95.00 ( (hardback)
$76.00 ( (hardback)
Control of Cognitive Processes
Stephen Monsell Jon Driver
One of the most challenging problems facing cognitive
psychology and cognitive neuroscience is to explain how mental
processes are voluntarily controlled, allowing the computational
resources of the brain to be selected flexibly and deployed to
achieve changing goals. The eighteenth of the celebrated
international symposia on Attention and Performance focused on
this problem, seeking to banish or at least deconstruct the
"homunculus": that conveniently intelligent but opaque agent still
lurking within many theories, under the guise of a central
executive or supervisory attentional system assumed to direct
processes that are not "automatic."
The thirty-two contributions discuss evidence from psychological
experiments with healthy and brain-damaged subjects, functional
imaging, electrophysiology, and computational modeling. Four
sections focus on specific forms of control: of visual attention, of
perception-action coupling, of task-switching and dual-task
performance, and of multistep tasks. The other three sections
extend the interdisciplinary approach, with chapters on the neural
substrate of control, studies of control disorders, and
computational simulations. The progress achieved in
fractionating, localizing, and modeling control functions, and in
understanding the interaction between stimulus-driven and
voluntary control, takes research on control in the mind/brain to a
new level of sophistication.
T A B L E O F C O N T E N T S
ACKNOWLEDGMENTS
T H E ATTENTION AND PERFORMANCE SYMPOSIA
PARTICIPANTS
G R O U P PHOTO
INTRODUCTION
1 B A N I S H I N G THE CONTROL HOMUNCULUS
BY STEPHEN MONSELL AND J O N DRIVER
ASSOCIATION LECTURE
TASK SWITCHING, STIMULUS-RESPONSE BINDINGS, AND NEGATIVE
PRIMING
BY ALAN ALLPORT AND G L E N N W Y L I E
CONTROL OF VISUAL ATTENTION
GOAL-DIRECTED AND STIMULUS-DRIVEN DETERMINANTS OF
ATTENTIONAL CONTROL (TUTORIAL)
BY STEVEN YANTIS
O N THE T I M E COURSE O F T O P - D O W N AND B O T T O M - U P CONTROL O F
VISUAL ATTENTION
BY J A N THEEUWES, P A U L ATCHLEY AND ARTHUR F. KRAMER
B Y STEVEN P . TIPPER, LOUISE A . HOWARD AND GEORGE HOUGHTON
11 T H E PREPARED REFLEX: AUTOMATICITY AND CONTROL IN
STIMULUS-RESPONSE TRANSLATION (TUTORIAL)
BY BERNHARD HOMMEL
I I I TASK SWITCHING AND MULTITASK PERFORMANCE
12 TASK SWITCHING AND MULTITASK PERFORMANCE (TUTORIAL)
BY HAROLD PASHLER
13 MULTITASKING PERFORMANCE DEFICITS: FORGING LINKS BETWEEN
THE ATTENTIONAL BLINK AND THE PSYCHOLOGICAL REFRACTORY
PERIOD
BY PIERRE JOLICOEUR, ROBERTO D E L L ACQUA AND JACQUELYN
CREBOLDER
14 INTENTIONAL RECONFIGURATION AND INVOLUNTARY PERSISTENCE IN
TASK S E T SWITCHING
BY THOMAS GOSCHKE
15 AN INTENTION-ACTIVATION ACCOUNT OF RESIDUAL SWITCH COSTS
B Y RITSKE D E J O N G
16 RECONFIGURATION OF STIMULUS T A S K S E T S AND RESPONSE T A S K
SETS DURING TASK SWITCHING
BY NACHSHON MEIRAN
17 TASK SWITCHING IN A CALLOSOTOMY PATIENT AND IN NORMAL
PARTICIPANTS: EVIDENCE FOR RESPONSE-RELATED SOURCES OF
INTERFERENCE
BY RICHARD B. IVRY AND ELIOT HAZELTINE
I V CONTROL O F MULTISTEP TASKS
18 T H E ORGANIZATION OF SEQUENTIAL ACTIONS
B Y G L Y N W . HUMPHREYS, E M E R M . E . FORDE AND D A W N FRANCIS
19 COGNITIVE CONTROL OF MULTISTEP ROUTINES: INFORMATION
PROCESSING AND CONSCIOUS INTENTIONS
B Y RICHARD A . CARLSON AND M Y E O N G - H O S O H N
20 REAL-WORLD MULTITASKING FROM A COGNITIVE NEUROSCIENCE
PERSPECTIVE
B Y P A U L W . BURGESS
V T H E NEURAL SUBSTRATE OF CONTROL
21 FUNCTIONING OF FRONTOSTRIATAL ANATOMICAL LOOPS IN
MECHANISMS OF COGNITIVE CONTROL (TUTORIAL)
B Y TREVOR W . ROBBINS AND ROBERT D . ROGERS
22 T H E NEURAL BASIS OF T O P - D O W N CONTROL OF VISUAL ATTENTION IN
PREFRONTAL CORTEX
B Y EARL K . MILLER
23 MIDDORSOLATERAL AND MIDVENTROLATERAL PREFRONTAL CORTEX:
T W O LEVELS OF EXECUTIVE CONTROL FOR THE PROCESSING OF
MNEMONIC INFORMATION
BY MICHAEL PETRIDES
24 T H E R O L E OF DORSOLATERAL PREFRONTAL CORTEX IN THE
SELECTION OF ACTION AS REVEALED BY FUNCTIONAL IMAGING
BY C H R I S FRITH
25 DISSOCIATIVE METHODS IN THE STUDY OF FRONTAL L O B E FUNCTION
(COMMENTARY)
B Y J O H N DUNCAN AND ADRIAN M . O W E N
V I DISORDERS O F CONTROL
26 NEURAL CORRELATES OF PROCESSES CONTRIBUTING TO
WORKING-MEMORY FUNCTION: EVIDENCE FROM
NEUROPSYCHOLOGICAL AND PHARMACOLOGICAL STUDIES
BY MARK D ESPOSITO AND BRADLEY R. POSTLE
REGULATING PREFRONTAL FUNCTION AND WORKING MEMORY
B Y T O D D S . BRAVER AND JONATHAN D . C O H E N
IS THERE AN INHIBITORY MODULE IN THE PREFRONTAL CORTEX?
WORKING MEMORY AND THE MECHANISMS UNDERLYING COGNITIVE
CONTROL (COMMENTARY)
B Y DANIEL Y . KIMBERG AND MARTHA J . FARAH
AUTHOR INDEX
SUBJECT INDEX
1 Banishing the Control Homunculus
Stephen Monsell a n d Jon Driver
ABSTRACT We define the problem addressed at the eighteenth Attention and Performance
symposium as that of explaining h o w voluntary control is exerted over the organization
a n d activation of cognitive processes in accordance with current goals, without appealing
to an all-powerful but ill-defined “executive” or controlling “homunculus.” We provide
background to the issues a n d approaches represented in the seven parts of the volume a n d
review each chapter, mentioning also some other contributions made at the symposium. We
identify themes a n d controversies that recur through the volume: the multiplicity of control
functions that must be invoked to explain performance even of simple tasks, the limits of
endogenous control in interaction with exogenous influences and habits, the emergence of
control through top-down “sculpting” of reflexive procedures, the debate between struc-
tural and strategic accounts of capacity limits, the roles of inhibition and working memory,
the fertile interactions between functional a n d neural levels of analysis. We identify impor-
tant control issues omitted from the symposium. We argue that progress is at last being
m a d e in banishing—or fractionating—the control homunculus.
When we invited the psychologists and neuroscientists whose contribu-
tions constitute this volume to speak at the Eighteenth International
Symposium on Attention a n d Performance, we declared the theme of the
meeting to be “Control of Cognitive Processes: Banishing the H o m u n -
culus.” We took the provocative subtitle from a contribution by the late
Alan Newell to the eighth symposium:
A major item on the agenda of cognitive psychology is to banish the
homunculus (i.e., the assumption of an intelligent agent (little man) resid-
ing elsewhere in the system, usually off stage, w h o does all the marvelous
things that need to be done actually to generate the total behavior of
the subject). It is the homunculus that actually performs the control pro-
cesses in Atkinson and Shiffrin’s (1968) famous memory model, w h o still
does all the controlled processing (including determining the strategies)
in the more recent proposal of Shiffrin and Schneider (1977), w h o makes
all the confidence judgments, w h o analyses all the payoff matrices a n d
adjusts the behavior appropriately, w h o is renamed the “executive” in
many models (clearly a promotion).... (Newell 1980, 715)
The eighteenth symposium, twenty years later, seemed a suitable occa-
sion to take stock of progress on this agenda item. On the one hand, it is
our impression that, far from leading the furtive life of a fugitive, the
homunculus has continued to parade about in broad daylight, its powers
largely intact a n d indeed dignified by even grander titles—not merely
the “executive” but the “central executive,” or the “supervisory attention
system,” or the “anterior attention system”—and flagrantly laying claim
to prime real estate in the frontal lobes. On the other hand, there has been
a substantial increase in research by neuroscientists, neuropsychologists,
a n d experimental psychologists on “executive” functions, a n d on inter-
actions between endogenous (voluntary) a n d exogenous (stimulus-
driven) control of cognitive processes. We may n o w have a sufficient
database for a serious attack on the problem to which the control homun-
culus has been the default solution.
1.1 THE PROBLEM OF VOLUNTARY CONTROL
Viewed from a subjective perspective, the problem of control is as old as
philosophical speculation about the nature of our mental faculties. We
feel able to exercise voluntary control over our thoughts and behavior, yet
we also experience limitations to that control: we sometimes feel at the
mercy of habits, impulses, compulsions, obsessions, or reflexes; we do
things we apparently did not intend to do a n d leave u n d o n e things we
intended to d o . Can the seemingly voluntary aspects of our mental life be
given the same sort of mechanistic causal explanation that we happily
apply to the more reflexive aspects? If they can—if our exercise of vol-
untary control is no less determined than our reflexes—then in what
sense do we have free will?
Posed in these subjective terms, the problem of control carries with it
considerable philosophical, theological, a n d moral baggage. The compu-
tational theory of mind that n o w underlies cognitive psychology a n d
neuroscience provides us with a more objective perspective from which
to pose the problem. The m i n d / b r a i n evidently contains many represen-
tations of information (perceptual, semantic, motoric, etc.), a n d proce-
dures for translating between and transforming those representations.
The performance of any one cognitive skill involves only a subset of these
resources, which must be organized in a particular fashion for the task at
hand, and defended against disruption by other influences. Although
some processes (such as the earliest stages of sensory processing) may be
triggered by appropriate input in an inflexible manner regardless of cur-
rent goals, others m a y not. These other processes m a y h a v e to be flex-
ibly enabled or disabled, connected or disconnected, tuned, organized,
directed, scheduled, and monitored (or some subset of these) to accom-
plish particular goals. The goals often change from moment to moment.
The problem of voluntary control is thus: H o w are goal-relevant orga-
nizations of particular mental resources created or activated, a n d h o w
are goal-appropriate processes triggered, when they are appropriate, a n d
suppressed w h e n they are inappropriate? And what constrains the flexi-
bility of this deployment?
Monsell and Driver
Although the problem of control is inherent in virtually every task that
people can perform, it is often overlooked. As an example, consider the
much studied task of reading a fixated word aloud. Typically, successive
levels of abstraction from the retinal input are thought to result in a rep-
resentation of letter identities a n d their sequence: an abstract ortho-
graphic representation. This is followed by translation into an abstract
phonological sequence by several parallel pathways—perhaps two or
three, depending on h o w one views proposed separations between
assembling an abstract description of the pronunciation through letter-
sound “rules,” versus retrieving the learned pronunciation of a recog-
nized letter string. Discrepancies between assembled a n d retrieved pro-
nunciation are somehow reconciled, and the resulting abstract phonolog-
ical description is then translated into articulatory form a n d executed,
again via several substages. The literature contains many fine-grained
analyses of this general sequence.
If you examine recent volumes of, say, the Journal of Experimental
Psychology or the Journal of Cognitive Neuroscience, you cannot but be
impressed by the wealth of data, the detailed models, and the critical
debate about the overall architecture a n d substages involved in this sin-
gle skill of word reading. But ask a slightly different, a n d rather simple,
question, a n d you will scan the pages in vain. Why d i d the subjects in all
these experiments speak each word aloud, as instructed? At a whim,
they could instead have elected to perform innumerable other tasks in re-
sponse to each word, using some parts of the same mental machinery
but other resources as well. Given the same written input, they could
readily have performed semantic categorization, letter counting, pho-
neme counting, free association, translation into French, a n d numerous
other tasks (on many of which there is also an experimental literature).
Given current mechanistic models of word reading, w h y is it that skilled
readers do not always say a fixated word aloud, and h o w can they flexi-
bly choose which task to perform on a given word? The same questions
can be asked about all the other tasks psychologists have studied inten-
sively. In each case, we may have detailed analyses of the components
underlying performance of a given task, but little understanding of h o w
that task rather than another comes to be performed.
Of course, there is an extensive literature on one aspect of the control
of reading, concerning the difficulty of control in situations where the
conversion from print to phonology seems to h a p p e n even though not
required, as when naming the ink color of a printed color word (MacLeod
1991; Stroop 1935). That we can have difficulty not reading is, however,
only part of the problem; that we can perform innumerable alternative
tasks at will is just as important. Moreover, in comparison to the sophis-
ticated analysis of, say, the translation from orthography to phonology,
theoretical analyses of control seem quite crude, even for cases of control
failure. Essentially, these boil d o w n to the dichotomy between “con-
Banishing the Control Homunculus
trolled” and “automatic” processes, as in the influential Shiffrin a n d
Schneider (1977) paper to which Newell referred, and its numerous
precedents (e.g., James 1890). Like most dichotomies, this has been soft-
ened by use, so that “automaticity” may n o w be seen more as a matter of
degree than as an all-or-none state. Nonetheless, the important question
that the dichotomy begs, about exactly h o w any “controlling” is done,
still tends to be neglected. Indeed, most theoretical claims in existing
work have primarily concerned what is controlled (or cannot be con-
trolled), rather than how that control is exercised.
1.2 HOMUNCULITIS
To the extent that control problems have been explicitly considered in
psychology and neuroscience, until recently it seems to have been
assumed that if control is exercised, then there must be a controller.
Another common assumption is that the controller responsible for one
“controlled” process (rehearsal, say) is likely to be the very same con-
troller that controls another process (rotation of mental images, say, or
direction of visual attention). Doubtless the readiness with which this
assumption is m a d e has deep roots in our culture, in the Cartesian doc-
trine of the soul as (singular) director of the material and mechanical
brain, a n d its evolution into the “Will” of nineteenth-century philosophy
a n d psychology (e.g., James 1890). The assumption may also reflect our
familiarity with the pyramidal control structure of many social organiza-
tions, such as schools, armies, or governments. Even within modern
information-processing approaches to the mind, the powerful metaphor
of the “operating system” that directs—or at least schedules—program-
specific processes in the standard computer (Johnson-Laird 1983) has
proved very seductive. Yet the wide world also contains m a n y examples
of complex systems that are flexibly controlled without containing any-
thing identifiable as a singular controller (e.g., termite communities). The
notion of distributed control also has a long and respectable history in
computer science, and some have already m a d e the speculative extrapo-
lation to the computational architecture of the h u m a n mind (e.g., Minsky
1985).
In defense of many theorists w h o talk of the “executive,” it might be
said that they are often not so much advocating a singular controller as
declaring the problem of control to be extrinsic to their current concerns,
which lie elsewhere, in the processes being controlled, or the limits of
control. Talk of the “executive,” then, is just a placeholder for mecha-
nisms u n k n o w n . In one influential example, Baddeley a n d Hitch (1974)
proposed a tripartite model of working memory, placing a “central exec-
utive” at center stage, flanked by two subsidiary systems, the “articula-
tory loop” a n d the “visuospatial sketchpad.” Baddeley (1990, 117) later
reflected on complaints that the central executive had remained under-
Monsell and Driver
specified: “Most of the research in the working memory tradition has
tended to concentrate on the subsidiary systems, principally on the
grounds that they offer more tractable problems than the central execu-
tive, which from time to time has tended to become something of a rag-
bag for consigning such important but difficult questions as h o w infor-
mation from the various slave systems is combined, and how strategies
are selected a n d operated.” This may be a sensible strategy for dealing
with complexity. We cannot understand every component of the system
at once, a n d everyone is entitled to a ragbag. It is unfortunate, however,
that language enforces a choice between singular a n d plural referential
terms: a diagram with a big oval at the center labeled the “central ex-
ecutive” may seem more assertively homuncular than a cloud labeled
“ u n k n o w n executive functions.”
Even when we address issues of control directly, to invoke homuncu-
lus-like entities may still be a productive strategy if used with sufficient
caution. Consider Daniel Dennett’s remarks (1978, 124) on the modeling
of intentional systems (“intentional” in the philosophical sense of repre-
senting beliefs, goals, etc.) in artificial intelligence:
Homunculus talk is ubiquitous in AI, a n d almost always illuminating. AI
homunculi talk to each other, wrest control from each other, volunteer,
subcontract, supervise, a n d even kill. Homunculi are bogeymen only if
they duplicate entirely the talents they are rung in to explain.... If one can
get a team or committee of relatively ignorant, narrow-minded, blind
homunculi to produce the intelligent behavior of the whole, this is
progress. A flow chart is typically the organizational chart of a committee
of homunculi (investigators, librarians, accountants, executives); each
box specifies a homunculus by prescribing a function without saying how
it is to be accomplished (one says, in effect: put a little m a n in there to do the
job). If we then look closer at the individual boxes we see that the func-
tion of each is accomplished by subdividing it via another flow chart into
still smaller, more stupid homunculi. Eventually this nesting of boxes
within boxes lands you with homunculi so stupid . . . that they can be, as
one says, “replaced by a machine.” One discharges fancy homunculi from
one’s scheme by organizing armies of such idiots to do the work.
Although Dennett w a s addressing a somewhat different issue, this may
prove a good blueprint for analyzing voluntary control over mental
processes. Perhaps our slogan should be, not “Banish the homunculus!”,
but “Dissolve, deconstruct, or fractionate, the executive! Let a h u n d r e d
idiots flourish!” Of course, there may still be those w h o will defend an
indissoluble, unitary controller at the heart of the system, against the
suggested “army” (or platoon) of “idiots.” If however, their argument is
based solely on a desire for parsimony in the number of agents proposed,
then that parsimony must be fully costed against the concomitant extrav-
agance of attributing multiple powers to a singular controller.
One common motive for proposing a central agency with plenipoten-
tiary powers has been to provide a seat for “consciousness,” in deference
Banishing the Control Homunculus
to the supposedly unitary nature of awareness (see, for example, Carlson
a n d Sohn, chap. 19, this volume). Although processes associated with
conscious awareness may well turn out to play an important functional
role in cognitive control, we regard questions about the nature, unity, a n d
substrate of consciousness as logically separable from those about the
functional architecture a n d neural substrate of control, a n d have tried to
keep this volume focused on the latter issues. The problem of control
seems hard enough without confounding it with an even greater mystery.
It is perhaps better first to model control functions from the “outside,”
a n d only then to worry about h o w they relate to what control or lack of
control feels like from the “inside.”
1.3 THE COMING OF AGE OF RESEARCH ON CONTROL
Theoretical developments often need decisive data, a n d vice versa. One
reason this symposium seemed timely is that a sufficiently rich set of data
on control functions is at last beginning to accumulate, across several dif-
ferent areas a n d disciplines. Research on normal h u m a n performance has
increasingly concerned itself with issues of control, not only in familiar
paradigms (such as the Stroop effect, visual search, and the psychological
refractory period), but also in newly invented or rediscovered paradigms,
such as task switching and the antisaccade task. As noted earlier, much
psychological work has failed to address the control problem directly
because it has been concerned primarily with just a single task, such as
reading. Work on task switching specifically aims to determine h o w peo-
ple reconfigure their cognitive resources, in accordance with arbitrary
goals, to deal with stimuli that can afford several possible tasks in the
experimental context. The antisaccade task provides an example of a par-
adigm that artificially a n d deliberately pits endogenous control against
exogenous control to explore their behavioral a n d neural correlates.
Another powerful engine driving research on control is neuropsycho-
logical work on brain-damaged patients. As with the psychological
analysis of individual tasks, many neuropsychological studies have over-
looked control issues, focusing on h o w individual cognitive skills (e.g.,
reading, recognizing objects, reaching, etc.) are affected by brain injury.
Recently, however, following observations of “dysexecutive” behavior
after damage to the frontal lobes and associated structures (Luria 1966;
Shallice 1988), impairments to cognitive control per se have become the
focus of much neuropsychological work. Although the behavior of
patients suffering such brain damage may be unimpaired on tests of
specific perceptual, linguistic, spatial, or motoric functions, their behav-
ior in daily life is often chaotically disorganized a n d often captured a n d
diverted d o w n task-irrelevant routes by a potent stimulus. Unlike other
neuropsychological syndromes such as acquired dyslexia, these diffi-
culties cannot be explained by damage to the standard components of
Monsell and Driver
models for particular tasks. Instead, they seem to suggest damage to
mechanisms that coordinate these components, though not necessarily
to a single central executive.
In the past, such patients have often been studied using batteries
of complex clinical tests for “frontal impairment,” such as Wisconsin
card sorting, that involve many cognitive components, only some of
which relate to control. Increasingly, however, paradigms adapted from
the “normal” experimental laboratory, together with further custom-
designed tests, are being used to isolate particular control d e m a n d s .
Moreover, these tests are being applied to patient groups with increas-
ingly specific types a n d regions of frontal damage, a n d to those with
lesions in other parts of interconnected neurotransmitter networks, lead-
ing to a neuropsychology of control with the potential to document the
neural structures associated with particular control deficits.
In neuroscience more generally, there has been a substantial growth of
interest in control processes, as part of a shift toward studying higher-
level function. One impetus for this was the development of single-cell
recording in awake rather than anesthetized animals, making it possible
to study the effect of current goal state on neural activity. A further spur
has been the increasing sophistication of tracing methods and pharmaco-
logical blockades for understanding interactions between “higher” a n d
“lower” areas in network terms. But perhaps the biggest methodological
advance has been the development of new technologies for measuring
brain activity in h u m a n s . Functional neuroimaging can reveal neural
activity as people perform any cognitive task, including tasks that exer-
cise control functions. Unlike animals, people can be instructed to per-
form almost any arbitrary task “at will,” with very little practice. By
contrast, massive training is often required to get animals to perform
tasks of the necessary complexity and contingency; in such cases, there is
a danger of observations being restricted to overlearned skills, thus miss-
ing the heart of the control problem. Functional neuroimaging has
already been used with h u m a n s in an effort to pin d o w n specific func-
tions for areas of prefrontal cortex and to characterize their interactions
with other cortical and subcortical regions. These developments in neu-
roscience have led to increasing recognition of the plurality of control
functions and the wide distribution of their neural substrate.
These trends in experimental psychology, neuropsychology a n d neuro-
science are amply represented in this volume, which also demonstrates
the considerable scope for mutual education on the control theme
between these research traditions. As Newell (1973) complained in an-
other celebrated paper, it is the besetting sin of experimental psychol-
ogy, including the chronometric tradition represented in past Attention
a n d Performance volumes, to become “phenomenon driven”—trapped
in minute exploration of paradigm-specific effects. We firmly believe that
the way to keep sight of the big picture a n d thus avoid the trap of
Banishing the Control Homunculus
paradigm-bound research is discourse a n d interaction with others using
very different approaches to tackle related problems. We also take it as
self-evident that experimental psychologists whose primary interest is at
the functional level can learn much from appropriate study of the neural
substrate. But the relationship between psychological a n d neuroscientific
research must be reciprocal. As the focus in neuroscience shifts from
cortical a n d subcortical regions close to the sensory a n d motor periphery,
to brain activation in so-called association cortex during performance
of complex tasks, neuroscience surely needs the sophistication in task
analysis—specifying the functional components—that has been devel-
oped by several decades of h u m a n information-processing research.
Where better to promote this two-way interaction than at an Attention
a n d Performance symposium?
Data are not enough, of course, no matter h o w many different methods
are used to collect them. What we would all like is a theory of control, or
at least a theoretical framework, at a level above the specific behavioral
paradigm or brain region. Although the problem of voluntary control has
long been recognized, there have been few theories of control. The most
influential of these, proposed by Norman and Shallice (1980, 1986) a n d
further developed by Shallice (1988), w a s motivated largely by observa-
tions of action errors in everyday life (e.g., driving straight to work rather
than taking the intended unusual detour for an errand), and of the more
extreme but similar behavior seen in “dysexecutive” patients. Such errors
seem to result from a stimulus “seizing” control of behavior, against cur-
rent intention, by evoking a well-established habit or an action schema
recently associated with the stimulus. To account for this, Norman a n d
Shallice followed the theory-building strategy ( r e c o m m e n d e d by
Dennett) of hiving off from the control homunculus an additional (dumb)
layer of control, conceived in production system terms, which they called
“contention scheduling.” The organization of components of a familiar
task was attributed to stored “schemata” activated by appropriate input.
Competition between different schemata that might simultaneously be
activated by current input, a n d prohibition of mutually incompatible
actions, were mediated by the d u m b “contention-scheduling” level of
control. Based on competition at this level alone, recently or frequently
exercised schemata w o u l d tend to dominate (as in action errors or the
dysexecutive syndrome) d u e to their greater competitive strength. For
less well established or less recently used schemata to win the competi-
tion (as required in relatively novel situations and some experimental
tasks), input from a superordinate layer of control—the “supervisory
attention system” (SAS)—was assumed to modulate activation levels of
schemata according to current goals.
Recognizing that “higher” control processes do not direct domain-
specific resources in a hands-on, omniscient manner, but merely m o d u -
late or “sculpt” the activation of lower-level schemata organizing those
Monsell and Driver
resources, is an important step in our understanding of control mecha-
nisms. It gives d u e weight to the role in our mental life of relatively auto-
matic routines that can be exogenously triggered by stimuli. Indeed, the
theme that controlled behavior may arise from subtle “sculpting” of more
automatic response tendencies runs through this volume. Nevertheless,
it must be acknowledged that the SAS as originally proposed was a
homunculus only marginally reduced in powers. It retained sufficient
omniscience to set activations so that lower-level contention scheduling
would generally achieve the right outcome, a n d it w a s somehow clever
enough to assemble and schedule the elements of a novel task (i.e., one
for which lower level schemata do not yet exist), to troubleshoot when
things went wrong, and to overcome temptation (Shallice 1988). Clearly,
further deconstruction of the SAS and of the interaction of its parts with
lower-level processes is required, ideally in explicitly computational
terms that can be tested in simulations. This seems at last to be happen-
ing, a n d the volume includes illustrative contributions from both pro-
duction system and connectionist traditions of computational modeling.
1.4 OVERVIEW OF VOLUME
Parts I–IV focus on specific forms of control in particular cognitive
domains: control of visual attention (part I), translation between percep-
tion and action in the face of competing response tendencies (part II),
coordination of simultaneous or closely successive performance of differ-
ent tasks (part III), a n d management of successive elements in multistep
tasks (part IV). Parts V–VII, although they speak to the particular forms
of control described in parts I–IV, are organized around methodologies.
Part V illustrates work on control functions using the techniques of neu-
roscience—anatomy, single-unit recording, lesions, a n d functional neuro-
imaging—and focusing on the functions of particular brain regions or
circuits; part VI illustrates work on pathological control in neurological
a n d developmental populations; and part VII, work on computational
modeling of control functions, with the phenomena modeled ranging
from reaction time data to the effects of neurotransmitters. Of course, the
interplay among the various methodologies for studying control mecha-
nisms is already sufficiently advanced that these divisions are somewhat
arbitrary. For example, experiments with neuropsychological patients
also appear in parts I–IV, a n d electrophysiological and functional neu-
roimaging research is discussed in part I.
Most contributors to this volume were invited to present their o w n
recent research, a n d all were encouraged to consider “how” control oper-
ates, not merely “what” is controlled. A few agreed to contribute tutorial
reviews rather than focus on their own research. Each group of papers
presented at the meeting led to a discussion, initiated by a discussant.
Several discussants agreed to contribute short commentaries on the field
Banishing the Control Homunculus
covered by the papers in their group. Some participants w h o did not
present papers at the meeting presented posters or described new data in
extended discussion sessions. Without trying to be exhaustive, we have
included some mentions of these valuable contributions to the meeting.
At every Attention a n d Performance symposium it is customary to
honor an eminent researcher’s distinguished contribution by an invita-
tion to give the Association Lecture. We were fortunate to have as associ-
ation lecturer Alan Allport, w h o has both posed a n d challenged many of
the critical questions about control and attention (e.g., Allport 1980, 1989,
1993; Allport, Styles, and Hsieh 1994). In chapter 2, he describes his recent
research on task switching, which relates most closely to part III; we shall
discuss it under that heading.
Although there are many points of contact among the chapters, some
overarching themes are apparent, including
. the “limits” theme—the deliberate exploration of cases where our exer-
cise of control is limited, typically through stimuli tending to drive pro-
cessing irrespective of intentions, or in opposition to them;
. the “sculpting” theme—seeing control as top-down modulation of
lower-level reflexlike circuitry a n d “reflexes” as potential building blocks
rather than the enemy of control;
. the “no simple dichotomy” theme—general dissatisfaction with a n d
superseding of the dichotomy between “controlled” and “automatic”
processes;
. the “multiple control functions” theme—identifying and distinguishing
between distinct control functions: some recruits to the “army of idiots”;
. the “working memory” theme—recognizing that goal-appropriate pro-
cessing requires short-term maintenance both of procedural “instruc-
tions” and of the information operated on; and
. the “interdisciplinary convergence” theme—recognizing that the function-
al and neural levels of the description of control functions should be
complementary.
The papers, posters, a n d discussions at the meeting also highlighted
running controversies about theory or methodology that cut across the
topics:
. Is inhibition necessary? Do we need inhibitory processes to prevent
undesired processes from occurring, or is it sufficient that the appro-
priate procedure or representation be the most activated, in a purely
facilitatory manner?
. Structural versus strategic bottlenecks. Are apparent limits on informa-
tion processing the result of immutable structural constraints on the
architecture of the m i n d / b r a i n , or of strategic choices about h o w best to
deploy a n d coordinate available resources, or even of motivational limi-
tations?
Monsell and Driver
. How apt is the operating system metaphor? Control problems that are
trivial for computers may be more challenging for brains, a n d vice versa.
. Complex versus simple tasks. Will we discover more about control by
studying performance in complex situations that challenge many control
functions or in simplified paradigms that seek to isolate specific control
functions.
. Is prefrontal cortex the control center? To what extent should control func-
tions be attributed to subcortical centers or regions of cortex other than
prefrontal cortex? Indeed, is a search for discrete control “centers” mis-
guided? Are the extensive network circuits that connect them a more
appropriate level for analyzing the neural substrate?
. Explicit versus emergent control? Is it appropriate to see control systems,
whether in prefrontal cortex or elsewhere, as “higher” mechanisms mod-
ulating dumber “lower” mechanisms, or is control better seen as an emer-
gent property of interactions between equally d u m b domain-specialist
modules a n d organization-specialist modules?
We n o w provide a brief overview of each section, highlighting the over-
arching themes a n d controversies where space permits.
Part I: Control of Visual Attention
Visual attention seems a good model system for introducing our confron-
tation of control issues. Much is n o w known about “what” is controlled
in this domain, but rather less about “how” such control is exerted. The
early stages of vision are well characterized both in psychophysical a n d
in neural terms, a n d there is good evidence that even these early stages
of perception can be modulated to some extent by voluntary attention, in
both people and animals. Overt eye movements can be dissociated from
covert attention, although these are usually coordinated. The overarching
themes a n d controversies of the volume are evident for both forms of
attention. Limits in voluntary control are apparent: u n d e r some situa-
tions, and in some pathological states, salient stimuli attract attention,
gaze, or both, regardless of intention. Inhibitory mechanisms of control
have often been invoked to explain phenomena such as inhibition of
return, antisaccades, or negative priming. Moreover, there is a long con-
troversy over whether the limits of attentional capacity reflect an inflexi-
ble bottleneck or strategic filtering. Finally, the neurophysiology a n d
anatomy of vision and eye movements are perhaps better understood
than any other part of the system, and many attention researchers are
already combining the research tools of h u m a n performance with those
of neuroscience.
Yantis (chapter 3) presents a tutorial review of the limits on voluntary
visual attention, describing h o w goal states interact with stimulus factors
Banishing the Control Homunculus
to determine what will be attended. He reviews the controversy over
whether salient features that “ p o p out” when deliberately searched for
likewise attract attention even when task irrelevant, or whether even this
apparently early segregation of the visual field is subject to top-down
modulation. Theeuwes, Atchley, and Kramer (chapter 4) take up this
theme, with a fine-grained analysis of the time course of the interaction
between endogenous and exogenous factors, suggesting that initial pro-
cessing is driven solely by stimulus salience, with top-down modulation
developing only later. A poster presented at the meeting by Kramer,
Theeuwes, H a h n and Irwin provided further data on attentional capture
by irrelevant but salient distractor stimuli: interestingly, while subjects
often fixated the distractor and were sometimes aware of its presence,
they were sure they never fixated it when attempting a deliberate saccade
to the target. The role of strategy in visual search was also addressed in a
poster by Müller, Krummenacher, a n d Heller, on situations where the tar-
get could be defined in predictable or unpredictable dimensions (e.g.,
color or orientation). Evidence for top-down dimension weighting was
found, but also for limits in control in the form of a bias toward recently
experienced target dimensions.
Rafal, Ro, Ingle, and Machado (chapter 6) focus on saccade preparation
a n d the mechanisms that allow us to modulate the primitive fixation
reflex to achieve voluntary control over our visual orienting by appropri-
ate “sculpting” of reflexes. They discuss the neural substrates of these
mechanisms a n d the deficits in eye movement control that can follow
neurological damage. Klein a n d Shore’s commentary (chapter 8) com-
pares and contrasts exogenous a n d endogenous mechanisms for both
overt and covert visual orienting in an integrative review.
Hopfinger, Jha, Handy, a n d Mangun (chapter 5) show that combining
the temporal precision of event-related potentials (ERPs) with the spatial
precision of functional imaging can reveal h o w early in the visual system
top-down attentional modulation can penetrate: top-down gain modula-
tion is found in extrastriate cortex. (In a poster, Worden a n d Schneider
reported fMRI data suggesting attentional modulation even earlier, in
striate cortex.) In addition to their detailed look at “what” is controlled,
Hopfinger et al. also provide some preliminary data on the possible con-
trol structures. Finally, Lavie (chapter 7) proposes psychological bound-
ary conditions for when such modulation of early sensory processing by
top-down attention is possible. She provides a novel answer to the clas-
sic controversy of early versus late selection. In her view, perceptual
categorization of irrelevant stimuli cannot be prevented unless the pro-
cessing of relevant stimuli exhausts perceptual capacity. Early selection,
as revealed by immunity to irrelevant distractors, is therefore apparent
only under conditions of high perceptual load in relevant processing.
Perceptual capacity may shrink with aging, with the paradoxical effect
that, under some circumstances, the elderly can be less susceptible to dis-
Monsell and Driver
tractor effects than the young. Lavie argues that perceptual load should
not be equated with task difficulty: a task that loads working memory
(hence control functions) rather than perceptual capacity may lead to
more distraction rather than less.
Part II: Control of Perception-Action Coupling
One of the most familiar manifestations of “control difficulty” in the
h u m a n performance laboratory arises in cases where the required re-
sponse differs from the most natural response to the stimulus, as in the
Stroop effect. In some cases, the more compatible, though currently u n d e -
sired, response (e.g., reading the word) is in some sense just as arbitrary
as the required response (color naming), but has been massively over-
learned. In other cases, to respond according to the compatible mapping
is not only well practiced, but is also assisted by phylogenetically ancient
action systems (e.g., those guiding looking or reaching towards an
object). In his commentary (chapter 9), Milner reviews evidence for the
multiplicity of such systems (a veritable platoon of “idiots”) that can
transform visuospatial input directly into natural actions, bypassing
pathways responsible for perception in the traditional sense. He con-
siders the problems of coordination a n d integration posed by all these
systems. In discussion, Rossetti supplemented the evidence mentioned
by Milner, describing striking dissociations in both patients and normal
subjects between immediate pointing (under “direct” control?) a n d
somewhat delayed pointing (controlled by considered perception?) to
tactile or visual targets.
The impact of direct affordances for action from visuospatial input are
studied by Tipper, Howard, a n d Houghton (chapter 10), w h o describe
findings from a paradigm in which subjects must move eye or h a n d to
a visual target, while ignoring a concurrent visual distractor. Taking a
strong position on the disputed need for inhibition in control, they argue
that kinematic properties of the eye and hand trajectory reveal not only
competition between representations of the actions directly evoked by
the two stimuli, but also inhibition of the unwanted action.
Hommel (chapter 11) provides a tutorial review of results from choice
reaction time situations in which interference is caused by irrelevant
stimuli (as in the “flanker” effects) or stimulus properties (as in the Simon
a n d Stroop effects) when associated with a competing response. His sur-
vey integrates these phenomena with stimulus-response compatibility
effects. All have been interpreted as indicating difficulty in suppressing
activation of an inappropriate response via a relatively direct a n d auto-
matic pathway. (A different kind of theory was represented in a poster by
Stevens and Kornblum, w h o presented their connectionist model, which
locates the interference observed in the flanker paradigm at the stimulus
identification level.) Hommel takes to task theorists w h o hold that these
Banishing the Control Homunculus
interference effects arise from competition between concurrent “inten-
tional” and “automatic” translation processes. In a revival of Exner’s
late-nineteenth-century “prepared reflex” concept, he argues that the
intentional and automatic components of processing operate at different
points in time, with the intentional process (prior to the stimulus) setting
the stage for automatic translation when the stimulus arrives—a clear
example of the “sculpting” theme.
Part III: Task Switching and Multitask Performance
Dual-task performance has been a frequent theme at Attention a n d
Performance meetings. In one of the most popular paradigms, subjects
are required to perform two different reaction time tasks, with the stimuli
so close in time that the second stimulus often occurs before a response to
the first. The delay in response to the second stimulus when the interval
between the stimuli is very short—the “psychological refractory period”
(PRP) effect—has traditionally been attributed to a bottleneck in pro-
cessing: the second task must wait until some critical processing stage of
the first is completed (see Pashler 1993 for review). Meyer a n d Kieras
(1997) have argued that the PRP effect may arise instead as the result of
strategic control processes: a voluntary organization of processing pri-
orities to ensure that the first stimulus is responded to first. In a poster
presented at the symposium, Schumacher, Seymour, Glass, Lauber, a n d
Meyer displayed their evidence that, w h e n given appropriate instruc-
tions, subjects achieve almost perfect time-sharing (i.e., no PRP effect)
with certain combinations of audiovocal a n d visuomanual tasks a n d a
moderate amount of practice. But when these subjects are given different
instructions about task priorities, a PRP effect appears. Thus the PRP par-
adigm has recently become particularly relevant to the controversy over
strategic a n d structural bottlenecks.
At the same time, there has been a sudden flurry of research using vari-
ants of a “task-switching” paradigm in which subjects perform just one
reaction time task at any time for each of a sequence of stimuli, but with
the task frequently changing (either predictably or signaled by a cue). The
focus of interest is the increased reaction time a n d error rate on the trial
following a switch of task. This “switch cost” might seem to offer an
index of the control processes involved in reconnecting and reconfiguring
the various modules in our brains, so as to perform one task rather than
another given the same input (e.g., naming an object aloud versus classi-
fying or grasping it). It may thus provide a point of attack on the control
problem traditionally referred to as “task set.”
Both PRP a n d task-switching paradigms typically involve two choice
reaction time tasks a n d thus require subjects to keep two task sets avail-
able. In the PRP case, the tasks may overlap in time, whereas in the task-
switching paradigm the task sets must be enabled successively. In the
Monsell and Driver
belief that there may be at least some theoretical commonality between
these apparently similar domains, we solicited several contributions
under a common heading. In his tutorial review (chapter 12), Pashler
takes on the difficult integrative task of surveying both paradigms a n d
exploring possible commonalties between the processing limitations they
reveal. He argues that the PRP effect cannot be attributed to strategic lim-
itations a n d that there is a structural bottleneck associated with response
selection, speculating also that this may be a special case of a more gen-
eral principle: only one memory retrieval operation can be carried out at
a time. He considers but rejects the notion that the same difficulty in
maintaining more than one task-set (or stimulus-response mapping) in
an active state is responsible for both the PRP effect and switch costs.
Jolicoeur, Dell’Acqua, and Crebolder (chapter 13) perform a detailed
experimental comparison of the PRP effect a n d a seemingly similar phe-
nomenon known as the “attentional blink” (AB): the decline in the
ability to detect a second target in a very rapid stream of stimuli for half
a second or so after a first target is detected. Here, too, there have been
suggestions, especially by Potter (who presented a poster on the AB at the
meeting) and her colleagues, that some instances of the limitation may
be d u e to the need to change task sets. However, Jolicoeur, Dell’Acqua,
a n d Crebolder argue that the PRP effect and the attentional blink reflect
similar “bottlenecks” in processing, of structural rather than strategic
origin. Ivry and Hazeltine (chapter 17) present experiments following
up earlier work with a split-brain patient, which had suggested that
despite still exhibiting a PRP effect, the divided brain is not subject to the
same response selection bottleneck as an intact brain. Their new experi-
ments, which combine the PRP a n d task-switching paradigms, suggest
that, unlike normal subjects, the commisurotomy patient can main-
tain two S-R mappings for the same stimuli simultaneously, in separate
hemispheres.
Pashler’s review stresses one type of account of the switch cost—that
it reflects the duration of control processes needed for reconfiguring task
set, although some aspects of the reconfiguration may not be possible
until after the stimulus. This latter idea provides an account of the “resid-
ual cost” (Rogers a n d Monsell 1995) observed even when subjects have
ample time to prepare for a change of tasks. Alan Allport’s Association
Lecture (chapter 2), coauthored by Glenn Wylie, presents a development
of Allport, Styles, and Hsieh’s very different theory (1994). The residual
cost is attributed to proactive interference with task-specific processing, a
positive priming of the now-irrelevant task set through its recent associ-
ation with the same stimulus or class of stimuli. For certain task pairs, it
may also reflect carryover from an earlier trial of inhibition needed then
to suppress the now-appropriate task set. Thus associations among stim-
uli, responses, and task can constrain the efficiency of task switching,
even when ample preparation time for the switch is provided.
Banishing the Control Homunculus
Other types of evidence that apparently inhibitory priming may con-
tribute to residual task switch costs were presented at the symposium.
For example, Goschke (chapter 14), combines the two prevailing views of
switch cost, showing that switch costs arise in part from an active
preparatory control process, which may be disrupted by certain concur-
rent tasks. But they may also arise in part when a stimulus affords com-
peting responses, so that inhibition is applied to the irrelevant dimension
or stimulus-response (S-R) mapping, a n d this carries over to the next
trial. A poster by Mayr a n d Keele showed that when subjects must switch
from judging dimension A to judging dimension B a n d then back to judg-
ing dimension A, performance is slowed relative to a C-B-A sequence,
suggesting that inhibition is applied to a task set (e.g., “Attend to A”)
when it is abandoned a n d can persist for at least a few trials.
Another poster, by Monsell, Azuma, Eimer, Le Pelley, and Strafford,
while acknowledging that priming from previous trials can slow per-
formance on post-switch trials, showed some data comparing lateralized
readiness potentials on switch a n d nonswitch trials, suggesting that, for
at least one task pair, response selection was postponed, rather than
merely prolonged, by the need to switch tasks. This implies that the resid-
ual cost was in part d u e to the insertion of an extra (control) process on
switch trials. Meiran (chapter 16) partitions switch costs into component
processes reflecting separate reconfiguration of a stimulus task set and a
response task set.
Hence as data on task-switching costs accumulate, their causation
begins to look far from simple. Some data suggest that the duration of
active control processes forms one component of switch costs. Other data
demonstrate the contribution of passive priming—both positive a n d neg-
ative priming—at the levels of both task sets and individual responses.
By the standards of many reaction time (RT) “effects,” the switch cost can
be substantial (hundreds rather than tens of milliseconds). Thus we
should not be surprised if this total is composed of several elements.
Logically, too, most instances of task switching seem to require most
of the following distinct functions: reorienting perceptual attention;
resetting the criteria for classification; readying a response mode, a
set of responses within it, or both; enabling or disabling S-R mappings;
adjusting criteria for response initiation to balance speed and accuracy
appropriately.
Most authors distinguish between a component of the cost of task
switching that can be overcome by anticipatory preparation (if time a n d
opportunity permit) a n d a component that cannot. This distinction is
challenged, however, by De Jong (chapter 15), w h o presents evidence that
RT distributions on switch a n d nonswitch trials can be fit by a model in
which costs are attributed to a single “intention-activation” process, but
that even with time to prepare, most subjects succeed in engaging this
process before the stimulus only on a proportion of trials, d u e in part to
Monsell and Driver
the cognitive effort required. He discusses the necessary compromise
between minimizing control effort a n d maximizing task performance,
a n d shows that the balance can to some extent be manipulated experi-
mentally. Results from the task-switching paradigm in neuropsycho-
logical patients are also reported in later parts, by Robbins a n d Rogers
(chapter 21), a n d by Keele and Rafal (chapter 28).
Clearly, both the classic PRP paradigm a n d the newer task-switching
paradigm are producing research that addresses many of the themes
highlighted earlier: the limits to control, the role of inhibition, structural
versus strategic bottlenecks, a n d the multiplicity of control functions. On
the other hand, later in the volume, Burgess (chapter 20), a n d Kieras et al.
(chapter 30) argue that the task-switching paradigm p u t s only a minor
load on control processes compared to many multitasking situations in
daily life, which require multiple goals to be fulfilled in tasks interleaved
over a much longer time span. This may be so. The value of the task-
switching paradigm, as for a number of the other paradigms surveyed in
this volume, may precisely be that it offers the possibility of isolating for
study a small subset of the controlling “army of idiots,” such as those
specifically responsible for reconfiguration of S-R mappings. Other para-
digms are needed to tap planning, decision making, monitoring, trou-
bleshooting, managing a goal-subgoal task structure, and a host of other
potential control functions, some of which are considered in part IV.
Part IV: Control of Multistep Tasks
Much of the research u n d e r the previous three headings concerned sim-
ple tasks requiring discrete speeded responses to single events (e.g., clas-
sifying an object, or reaching for a target). More complex multistep tasks
in daily life (such as cooking a meal or finding a route to a destination)
may require additional layers of control. Subgoals need to be established
a n d prioritized, triggers set in prospective memory to initiate subtasks
when the conditions for them become ripe, transitions between subtasks
managed to avoid capture of behavior by habitual transitions, and so on.
The outcomes of each processing step may be have to be matched to
intended outcomes, so that troubleshooting can be initiated if sufficient
divergence from the goal or subgoal is detected or anticipated.
The natural history of “action slips” m a d e by people in daily life
(Norman 1981; Reason 1984) has suggested a number of different kinds of
failure in multistep tasks, and the more frequent and pathological slips of
“dysexecutive” patients have proved equally illuminating. Schwartz a n d
colleagues (e.g., 1991) pioneered the detailed analysis of errors in famil-
iar multistep tasks, such as making a cup of coffee, by patients with
frontal brain damage. Humphreys, Forde, a n d Francis (chapter 18)
describe neuropsychological research in this tradition, a n d extend it to
the performance of normal subjects under dual-task conditions.
Banishing the Control Homunculus
The performance of multistep tasks typically places considerable load
on “working memory” to maintain representations both of the operations
to be performed a n d of the information to be operated on, raising issues
of how external instructions about the structure and content of complex
tasks may most readily be assimilated. Carlson a n d Sohn (chapter 19)
present research in which subjects perform multistep numerical a n d spa-
tial tasks whose sequence is determined by the experimenter. Examining
whether it is better for subjects to know the operator or operand in
advance, they interpret their data within a “procedural frame” hypothe-
sis derived from a more general theory of cognitive control.
In his commentary (chapter 20), Burgess points to the m a n y control
d e m a n d s of real-life multitasking—the planning a n d interleaved execu-
tion of several multistep tasks—a d e m a n d familiar to the busy parent no
less than to the fighter pilot or astronaut. Burgess argues that such com-
plex situations may be more amenable to experimentation than is widely
supposed, and may tax surprisingly specific brain areas. With Shallice
a n d other colleagues, he has pioneered the study of frontal patients per-
forming everyday tasks of real-world complexity, such as carrying out a
series of errands in a busy shopping center. He has also developed sim-
plified laboratory analogues that have considerable diagnostic utility.
Burgess reports that if one studies a large range of control-dependent
tasks in frontal patients, clusters of associated symptoms emerge, which
suggest a particular fractionation of control functions that can be m a p p e d
to specific brain regions.
Part V: The Neural Substrate of Control
While focusing on brain mechanisms of control, especially in prefrontal
cortex (PFC) and related areas, part V also emphasizes psychological
function wherever possible. Robbins and Rogers (chapter 21) present a
tutorial review of the anatomy, physiology, and function of “cortico-
striatal loops” linking frontal cortex to the striatum a n d associated sub-
cortical structures. They make it abundantly clear that, contrary to many
textbook summaries, PFC cannot be considered in isolation with regard
to executive function. They also present convergent evidence from lesion
effects in h u m a n s a n d animals and from functional imaging on the role of
various structures in the formation, maintenance, and shifting of cogni-
tive set, in new paradigms that isolate specific components of the tradi-
tional Wisconsin card-sorting task, a n d in the task-switching paradigm.
Miller (chapter 22) describes research on single-unit activity in monkey
PFC for tasks requiring control of visual attention a n d task set, analogous
to some of the h u m a n tasks discussed in earlier parts. When the animals
are cued to attend to an object or location in a subsequent display of
several objects, prefrontal neurons show activity specific to anticipated
objects a n d locations, maintaining this activity over the interval following
the cue. Unlike activity in inferotemporal neurons, PFC activity is main-
20 Monsell and Driver
tained in the face of distractors occurring during the interval. These PFC
neurons appear to be functioning as (part of) a procedural working mem-
ory, maintaining a representation of where to attend, what to attend to,
a n d what to do with the attended information. One intriguing question is
h o w much of this PFC activity d e p e n d s on extensive training of the ani-
mals, although considerable flexibility is nevertheless shown.
Chapters 23 a n d 24 identify specific control functions of lateral regions
of PFC. Petrides (chapter 23) reviews his hypothesis, based on lesion
effects in monkeys and humans, a n d on functional imaging of normal
humans, that dorsal and ventral regions are specialized for different
working memory functions. He sees dorsolateral PFC as responsible for
“monitoring and manipulating” information in working memory, while
ventrolateral PFC is specialized for active retrieval of information stored
in posterior cortical association regions. In a poster, Owen described
fMRI activation during forward a n d backward digit span tasks that s u p -
ported a similar contrast between these two lateral frontal regions. Frith
(chapter 24) attributes a somewhat different role to dorsolateral PFC. On
the basis of functional imaging data indicating activation of this region
during tasks requiring subjects to select from among response alterna-
tives, he suggests that dorsolateral PFC selects responses, or response
sets, in situations where these responses are otherwise underconstrained,
by means of a top-down biasing of populations of cells in more posterior
regions that represent particular responses. The recurring theme of con-
trol as a “sculpting” process is particularly explicit here. Frith attempts to
reconcile his o w n perspective with that of Petrides, a n d both agree that
many different control processes may be subsumed under the general
heading of “working memory,” a point to which we return below.
In their commentary (chapter 25), Duncan a n d Owen sound a caution
on the inferences currently being d r a w n from functional imaging a n d
from comparisons of lesion groups about specialization of function with-
in PFC. They point out that the full double-dissociation design is rarely
used, a n d that inferences in neuroimaging must guard against over-
interpreting the locus of the “most active” voxel in particular tasks, when
in fact very broad regions of lateral PFC a n d dorsal anterior cingulate are
often activated by several types of increase in cognitive d e m a n d . They
suggest that the present data justify only rather crude functional distinc-
tions, for example, between the aforementioned regions, on the one hand,
a n d medial and orbital frontal cortex, on the other, the latter being asso-
ciated with affective a n d motivational processes.
Part VI: Disorders of Control
Although deficits in control following brain injury or disruption crop
up throughout the volume, they form the central theme of part VI.
D’Esposito and Postle (chapter 26) provide a further perspective on the
role of PFC in working memory, reporting a meta-analysis of studies
21 Banishing the Control Homunculus
where patients with focal PFC lesions performed short-term memory
tasks; a further behavioral study of patients with head injury a n d frontal
involvement, or Parkinson’s disease; a n d a pharmacological study of
brain-injured a n d normal subjects. They argue for a functional a n d
anatomical dissociation between tasks that require only passive main-
tenance of information in short-term memory a n d tasks that require
rehearsal and other control processes, attributing the latter to PFC (cf.
Petrides, chap. 23, this volume).
Riddoch, Humphreys, a n d Edwards (chapter 27) present data from
patients w h o have difficulty in suppressing actions triggered via the
“direct” pathways between perceptual affordances a n d motor control
discussed in part II by Milner (chapter 9) and by Tipper, Howard, a n d
Houghton (chapter 10). Such patients, w h o typically have frontal damage
or disconnection, may exhibit behaviors such as “anarchic h a n d syn-
drome” (where one h a n d performs object-appropriate actions against the
intention of the patient, w h o may use the other h a n d to try to suppress
this action) or “utilization behavior” (where patients pick up and use
the objects before them in schematic ways, such as lighting a match or
cutting paper with scissors, even when such actions are quite inappro-
priate in the current context). Riddoch, Humphreys, and Edwards illus-
trate h o w such deficits in control of “afforded actions,” which have
hitherto been described mainly in informal clinical terms, can be studied
experimentally.
Keele a n d Rafal (chapter 28) present data from patients with damage to
left or right PFC in a task-switching paradigm similar to those discussed
in part III. They find a deficit in patients with left frontal damage, but
unlike Rogers et al. (1998), w h o found an exaggerated switch cost in
patients with left frontal damage in a related but subtly different para-
digm, they find that the abnormality remains apparent several trials after
a switch. These patients seem to be showing abnormally large proactive
interference effects of the type documented in normals by Allport a n d
Wylie (chap. 2, this volume). Keele and Rafal speculate that this is d u e to
deficient inhibition.
Whereas chapters 25–28 concern the effects of acquired lesions, Logan,
Schachar, and Tannock (chapter 29) discuss research on a developmental
disorder of control—attention deficit hyperactivity disorder. Although
the impulsivity, hyperactivity, a n d inattentiveness of such children may
be all too apparent in the classroom a n d at home, it has been hard to pin-
point the underlying functional deficits. Logan, Schachar, a n d Tannock
describe the development a n d application to this group of a particular
experimental test—the stop signal paradigm—which appears to provide
a relatively pure measure of impulse control. Illustrating research on yet
another patient group increasingly seen as manifesting control impair-
ments, a poster by Fuentes described abnormalities in inhibition of return
a n d negative priming in schizophrenic patients.
Monsell and Driver
Leading the discussion on part VI, Stuss reviewed several examples of
functional dissociations from his long-term study of patients with focal
lesions of frontal lobe using variants of traditional clinical tests such as
the fluency and Wisconsin card-sorting tests. For example, patients with
right dorsolateral lesions were impaired in the fluency test, but those
with inferior medial lesions were not. Inferior medial patients showed a
tendency to lose set in a variant of the Wisconsin Card-Sorting Test
(WCST), when told the relevant dimensions and that the rule would
change, while superior medial patients d i d not. The latter patients, but
not the former, showed marked perseveration on the classical version of
the test, where they h a d to detect a change of rule for themselves.
Parts V a n d VI, together with a few of the earlier chapters, clearly illus-
trate the developing complementarity between behavioral and neurosci-
entific approaches to control, as well as revisiting many of the recurring
themes a n d controversies. The presentations led to a lively discussion at
the meeting of whether PFC plays the cardinal role in control. The emerg-
ing consensus was that although this large brain region clearly plays
many vital roles, many other cortical and subcortical structures with
which it interacts must also be considered.
Part VII: Computational Modeling of Control
As we noted earlier, a major need is for further development of a theo-
retical framework within which specific control functions can be mod-
eled. The final part illustrates approaches to modeling control in explicit
computational terms. General computational models of cognition have
been developed within the production system tradition pioneered by
Newell and colleagues, as developed in their SOAR project (Newell 1990;
Newell, Rosenbloom, a n d Laird 1989) a n d by Anderson in the various
generations of ACT* (Anderson 1983). Being global systems that pursue
goals, these systems have of necessity to address important control prob-
lems, especially in problem-solving contexts—for example, h o w to es-
cape from an impasse when the achievement of a particular subgoal is
blocked. Such models, however, have generally not been aimed at fine-
grained modeling of the temporal structure of h u m a n information pro-
cessing studied in the Attention and Performance tradition (see Shallice
1994 for a further critique of SOAR as a model of control).
Kieras a n d Meyer have recently engaged in an ambitious project to
develop a production system architecture they call “executive process
interactive control” (EPIC). Its purpose is explicitly to model executive
control processes, task-specific processes, a n d their interaction, and in so
doing to account for the detailed chronometry of performance in para-
digms like the PRP (Meyer a n d Kieras 1997) as well as more complex
“real-life” multitasking situations such as those of the telephone operator
or fighter pilot (Meyer a n d Kieras 1999). These theorists have taken a
Banishing the Control Homunculus
strong position on the structural versus strategic bottleneck, with strate-
gic factors being to the fore in their interpretation of the PRP effect.
Kieras, Meyer, Ballas, and Lauber (chapter 30) illustrate EPIC modeling
with applications to the task-switching paradigm, the PRP effect, a n d
more complex combinations of two continuous tasks. Based on an analy-
sis of general operating system principles from computer science, they
also propose the next step in their project. Hitherto, the achievement has
been to model control processes explicitly and to show that this can
account for objective performance data, as in the combination of two par-
ticular tasks. But thus far, the control processes have been hand-crafted
for each paradigm. N o w the challenge is to model control processes that
are more general in their application, so that they can coordinate and con-
trol a number of different task pairs.
Kieras et al. suggest that, as we learn to coordinate a particular pair of
tasks, the improvement with practice reflects in part an evolution from
control by general-purpose executive routines, to control by a learned set
of executive procedures specialized for that particular coordination prob-
lem. In essence, they propose to model explicitly, within the EPIC frame-
work, the contents of Norman a n d Shallice’s SAS a n d schemata (1986),
respectively. They argue convincingly that operating system principles
from computer science can shed light on many psychological issues,
although it remains unclear h o w literally the parallel should be taken. For
example, it turns out that task switching is a relatively trivial operation
for most computer operating systems, even though it produces very sub-
stantial costs in h u m a n performance, including proactive interference
effects from previous tasks (cf. Allport and Wylie, chap. 2, a n d Keele a n d
Rafal, chap. 28, this volume) that w o u l d never arise in standard com-
puter architectures.
Braver a n d Cohen’s contribution (chapter 31) comes from a connec-
tionist tradition that seeks to make computational models more brain-
like. Their approach has grown out of Cohen, Dunbar, a n d McClelland’s
model (1990) of the Stroop effect a n d Cohen and Servan-Schreiber’s
attempt (1992) to ground elements of the model in particular brain
regions and neurotransmitter systems. In the model, activation by context
of a representation of the current task biases processing, so as to achieve
information transmission via the appropriate set of S-R associations. The
problems addressed by Braver a n d Cohen are (1) h o w this task represen-
tation can be maintained in the face of other input to prevent irrelevant
information from overwriting the short-term memory representation of
the task context; a n d (2) how the system can learn what elements of the
context to respond to as task cues. The computational solution is a gating
mechanism they identify with interactions between prefrontal cortex neu-
rons a n d the dopamine system. The recurring themes of control as a
sculpting process, and of a critical role for working memory representa-
tions of the current task, resonate through this chapter.
Monsell and Driver
Kimberg and Farah’s commentary (chapter 32), which closes part VII,
makes a single but crucial point on the controversial role of inhibition in
modeling control functions. As we have seen in many of the previous
chapters, there are numerous phenomena suggesting that neurological
patients (as well as subjects with immature brains or developmental dis-
orders, a n d normal subjects under load or distraction) may lack the abil-
ity to overcome the effects of a prepotent response tendency or procedure.
The immediate temptation is to model this as impairment of an inhibi-
tory mechanism, often thought to be located in PFC. But, such behavioral
“disinhibition” can just as readily be modeled by loss of facilitatory acti-
vation of the “working memory” representation of the intended action as
by loss of inhibition of the habitual action. Applying Occam’s razor, we
should deploy an inhibitory mechanism to explain behavioral disinhibi-
tion only when there is positive evidence for it.
1.5 SOME OMISSIONS
Although the range of research areas addressed within a symposium
must necessarily be limited, we should acknowledge certain omissions.
First, like most previous Attention and Performance symposia, ours
focused on cognitive processes lasting between a fraction of a second a n d
several seconds, in tasks that are speedily executed, rather than on tasks
that fulfill goals over days or years. Only part IV explicitly considers
multistep tasks. Moreover, although many of the authors refer to the role
of memory for what to do, it is usually memory for what to do when the
next stimulus of a particular kind appears within a few trials (i.e., proce-
dural working memory), not what to do tomorrow, or by the end of next
week. There is n o w a substantial body of research on “prospective mem-
ory” over these longer time spans (see Brandimonte, Einstein, a n d
McDaniel 1996). The equally extended process of “automation” of a cog-
nitive skill, or combinations of skills, through substantial practice like-
wise receives rather little analysis here (though see Allport and Wylie,
chapter 2; and part VII, this volume)
We have also neglected some important control functions that operate
at our chosen timescale. Although there were contributions on rehearsal
or “monitoring” in working memory, the meeting did not address the
important distinction between “automatic” and “intentional” compo-
nents of retrieval from long-term memory (see Jacoby 1994), and strate-
gies of retrieval (see Barnes et al. 1999). Another important set of control
functions, as Newell (1980) p u t it in the quotation with which we open
this chapter, “make all the confidence judgments, analyze all the payoff
matrices and adjust the behavior appropriately.” That is, there is the need,
in addition to arranging cognitive resources suitably to accomplish a
given task, to evaluate performance, detect errors, assess efficiency, a n d
adjust decision a n d response criteria as appropriate. Relevant research
Banishing the Control Homunculus
includes that on reaction times following errors, a n d on the “blunder
blip”— error-related negativity in the evoked potential (Gehring et al.
1993), a n d its possible localization in the anterior cingulate (Holroyd,
Dien, a n d Coles 1998). As Robertson pointed out in discussion, the sym-
posium addressed neither sustained attention nor the interactions
between alerting and control (Robertson and Manly forthcoming). H o w
is it, for example, that by “making an effort” we can prevent ourselves,
for at least a while, from dropping asleep at the steering wheel when
driving at night?
Another major research domain that clearly involves aspects of control,
a n d on which we would have liked to include more is the planning a n d
conduct of complex problem solving, a favorite domain for production
system modeling (e.g., Newell 1980). There have been a number of neu-
ropsychological (e.g., Shallice 1988) and neuroimaging (e.g., Baker et al.
1996) studies of problem-solving tasks, such as the “Tower of Hanoi” a n d
the “Tower of London,” as well as the beginnings of a mental chronome-
try of such tasks (Ward a n d Allport 1997)
Further disorders of control for which we h a d no space include delu-
sions of control and auditory hallucinations in schizophrenia (analyzed
by Frith 1996 as d u e to loss of the signal conveying intention to act or
speak), intrusive thoughts in obsessive-compulsive disorders, and neu-
rodevelopmental conditions such as Tourette’s syndrome (see Georgiou,
Bradshaw, a n d Chiu 1996). We largely neglected the effects of aging on
cognitive control (see Kramer et al. 1999; Rabbitt 1997) a n d the normal
development of frontal control mechanisms (see Diamond 1990). We also
largely neglected individual differences in the ability of normal adults to
maintain goals and coordinate multiple tasks, a n d the relation of these
abilities to measures of intelligence (see Duncan, Emslie, a n d Williams
1996).
Perhaps our most fundamental omission is that while we have tried to
focus on h o w the deployment of cognitive resources is controlled by
“goals,” little is said in this volume about the source of those goals in the
interface between affective and cognitive systems (but see Robbins a n d
Rogers, chap. 21, this volume). Typically, goals are simply provided by
experimental instructions or training in laboratory studies, but they pre-
sumably derive from motivational states and reward values in the natu-
ral world. There has been some recent progress on this neglected topic,
including neuropsychological work on the association between loss of
affect and inappropriate decision-making in patients with orbitofrontal
damage (e.g., Bechara et al. 1998; Damasio, 1996); comparative work
on possible motivational bases for “perseverative” errors in different
species of monkey (Hauser 1999); a n d research showing activation
of orbitofrontal cortex in evaluative decision making (Rogers et al.
forthcoming). Nevertheless, the interface between cognitive control a n d
motivation remains a challenging issue for future research.
Monsell and Driver
1.6 CONCLUSIONS
Would the present volume lead Alan Newell or a like-minded skeptic to
think that some progress w a s at last being m a d e in banishing the control
homunculus? We think so. Although the contributions are varied in their
m o d e and level of analysis, a number of encouraging general trends are
apparent.
First, there is relatively little sign in these pages of any simple dichot-
omy between opposed “controlled” and “automatic” processes, save for
some nailing d o w n of its coffin lid. There is, instead, gratifying elabora-
tion of the fundamental insight, captured in the Norman a n d Shallice
(1980, 1986) model, of the complex and delicate interactions that are
found between endogenous and exogenous control wherever we look,
plus some explicit modeling of the functional and neural architecture of
these interactions in specific domains, such as control of eye movements.
In many cases, reflexes are no longer seen as the defining opposite of con-
trol, but as the fundamental building blocks from which controlled cog-
nition can be built, given suitable top-down modulation.
Second, there is evident appreciation of the multiplicity of control func-
tions. Even for a control problem considered relatively simple by some
of our theorists (Burgess, Kieras et al.), namely, reconfiguring “task set”
between two alternatives, we seem to need to invoke several sub-
functions. Researchers are developing experimental paradigms that can
dissect and isolate the contribution of these multiple control functions to
performance. A similar growing sophistication is apparent in neuroscien-
tific analyses of control, and the potential for combining psychological
a n d neural analyses seems enormous. Although we have nothing yet as
formal as a taxonomy of control processes a n d related neural substrates,
it is beginning to seem possible that one could be compiled.
The multiplicity of control functions does not of itself entail a multi-
plicity of controlling mechanisms (after all, the single central processing
unit of a standard computer has many functions). Nevertheless, the pro-
gressive fractionation a n d localization of control subfunctions, through
the combination of chronometric performance analyses, neuropsychol-
ogy, functional imaging, electrophysiology, and neuropharmacology, is
surely making the traditional view of a singular controller at the apex of
the system hard to sustain. It remains to be seen whether, in d u e course,
the control homunuculus will turn out to have been merely fractionated
or completely dissolved. That is, despite the progressive fractionation of
executive function, it may still turn out to be appropriate to postulate an
executive system (with interdependent a n d interacting parts) distinct
from the domain-specific resources controlled. Alternatively, it may end
up no more appropriate to ascribe functional coherence to all “control”
functions (and their neural substrates) than to mechanisms as diverse as
those that compute binocular stereopsis a n d segmentation of speech
Banishing the Control Homunculus
input into words. It is too early to tell. Either way, a basic lesson from
much psychology and neuroscience is that our intuitive notion of a uni-
tary self is largely illusory; we are composites of interacting subsystems,
a n d this seems no less true for our experience of “free will” than for other
aspects of mental life. Although the picture remains murky, each new
result a d d s a little light, a n d we are beginning to discern the identities of
some recruits to the army of control “idiots.” At the same time, compu-
tational modelers writing explicit code to get control jobs done in their
simulations are discovering what may be needed to do these jobs, a n d
hence what we should look for in the emerging scene.
Although the picture has many complex details, some simple patterns
a n d generalities are also apparent. Most basically, our capacity for volun-
tary control over mental processes is not absolute. In many cases, pro-
cesses are driven in part (or, more rarely, entirely) by salient stimuli, past
associations, or both, instead of by our intentions. Moreover, it is n o w
self-evident that to overcome such exogenous triggering, cognitive con-
trol requires further “input” to be a d d e d endogenously to the computa-
tions, in the form of activating some representation of current task goals.
This has become apparent for many different situations where a prepo-
tent response tendency has to be overcome. In the chapters of this vol-
ume, these situations range from making antisaccades, dissociating
covert attention from fixation, Stroop- and Simon-like interference effects,
selective reaching, negative priming of concepts, responses, or S-R rela-
tionships, to explaining the anarchic h a n d a n d utilization behavior in
frontal patients, the “A not B” error committed by babies toward the end
of their first year, a n d stop signal failures in children with attention dis-
orders. The evidence of dissociable deficits in these different situations
a n d pathologies suggests that each may involve some unique neural
structures at a fine-grained level of analysis. From a broader theoretical
perspective, however, all these situations have in common the need to
overcome prepotent response tendencies, and control for each may be
implemented in computationally similar ways. Indeed, it is quite striking
h o w many independent researchers in this volume propose that activat-
ing some form of “working memory” for current task goals may be the
critical step.
Of course, numerous traps still lie on the path of progress. We must, for
example, be wary of using “working memory” as an explanatory catchall.
Clearly, many forms of control require short-term maintenance of proce-
dural directives: where to orient, what the current contingencies are
between cue a n d S-R mapping, what the current operators are, what the
current goals a n d subgoals are in a multistep task, a n d so on. The PFC
neurons studied in monkeys by Miller, a n d the regions of h u m a n PFC
studied by Petrides or Frith with functional imaging, or by Rafal and col-
leagues and by D’Esposito and Postle in lesioned patients, are clearly
doing something that might be broadly described under the banner head-
Monsell and Driver
line of “working memory.” However, note that this is “procedural” work-
ing memory (i.e., of what to do), rather than the more commonly studied
“declarative” working memory for phonological sequences a n d spatial
patterns, n o w thought to be held in posterior cortical regions. Note also
that, just as declarative working memory has multiple levels a n d compo-
nents even for language input a n d output (Monsell 1984), so procedural
working memory may also comprise many components. Having chided
Baddeley a n d colleagues for labeling the ragbag at the center of their
working memory model the “central executive,” we should not place
a similarly singular rag bag at the center of a model of voluntary con-
trol, a n d label it “procedural working memory,” as if that explained
everything.
In a show-stopping dramatic monologue on the final evening of our
meeting, Ian Robertson suggested that we could n o w declare the control
homunculus extinct, with the few remaining examples of the species hav-
ing been slain by the heroic efforts of those present. In reality, we suspect
that the species will linger on in the pages of some learned journals a n d
in the minds of their writers and readers, if only because its pelt provides
such a convenient ragbag. Nevertheless, we hope readers of this volume
will agree that the control homunculus is n o w an endangered species,
a n d that a variegated genus of control “idiots” is beginning to colonize
the vacated niches.
NOTE
We thank Tim Shallice for his comments on an earlier draft of this chapter.
REFERENCES
Allport, A. (1980). Attention and performance. In G. Claxton (Ed.), Cognitive psychology: New
directions, p p . 26–64. London: Routledge and Kegan Paul.
Allport, D. A. (1989). Visual attention. In M. I. Posner (Ed.), Foundations of cognitive science,
p p . 631–682. Cambridge, MA: MIT Press.
Allport, D. A. (1993). Attention a n d control: Have we been asking the wrong questions? A
critical review of twenty-five years. In D. E. Meyer and S. Kornblum (Eds.), Attention and
Performance XIV: Synergies in experimental psychology, artificial intelligence and cognitive neuro-
science, p p . 183–218. Cambridge, MA: MIT Press.
Allport, D. A., Styles, E. A., a n d Hsieh, S. (1994). Shifting intentional set: Exploring the
dynamic control of tasks. In C. Umiltà and M. Moscovitch (Eds.), Attention and Performance
XV: Conscious and nonconscious information processing, p p . 421–452. Cambridge, MA: MIT
Press.
Anderson, J. R. (1983). The architecture of cognition. Cambridge, MA: Harvard University
Press.
Atkinson, R. C., a n d Shiffrin, R. M. (1968). H u m a n memory: A proposed system a n d its con-
trol processes. In K. W. Spence and J. T. Spence (Eds.), The psychology of learning and motiva-
tion, vol. 2, p p . 89–195. London: Academic Press.
Banishing the Control Homunculus
Baddeley, A. D. (1990). Human memory: Theory and practice. Hove, U.K.: Erlbaum.
Baddeley, A. D., a n d Hitch, G. (1974). Working memory. In G. A. Bower (Ed.), Recent
advances in learning and motivation, vol. 8, p p . 47–90. New York: Academic Press.
Baker, S. C., Rogers, R. D., Owen, A. M., Frith, C. D., Dolan, R. J., Frackowiack, R. S. J., a n d
Robbins, T. W. (1996). Neural systems engaged by planning: A PET study of the Tower of
London task. Neuropsychologia, 34, 515–526.
Barnes, A. E., Nelson, T. O., Dunlosky, J., Mazzoni, G., and Narens, L. (1999). An integrative
system of metamemory components involved in retrieval. In D. Gopher and A. Koriat
(Eds.), Attention and Performance XVII: Cognitive regulation of performance: Interaction of theory
and application, p p . 288–311. Cambridge, MA: MIT Press.
Bechara, A., Damasio, H., Tranel, D., a n d Anderson, S. W. (1998). Dissociation of working
memory from decision making within h u m a n prefrontal cortex. Journal of Neuroscience, 18,
428–437.
Brandimonte, M., Einstein, G. O., a n d McDaniel, M., Eds. (1996). Prospective memory: theory
and applications. Mahwah, NJ: Erlbaum.
Cohen, J. D., Dunbar, K., a n d McClelland, J. L. (1990). On the control of automatic pro-
cesses: A parallel distributed processing account of the Stroop effect. Psychological Review, 97,
332–361.
Cohen, J. D., and Servan-Schreiber, D. (1992). Context, cortex and dopamine: A connection-
ist approach to behavior a n d biology in schizophrenia. Psychological Review, 99, 45–77.
Damasio, A. R. (1996). The somatic marker hypothesis a n d the possible functions of the pre-
frontal cortex. Philosophical Transactions of the Royal Society of London, B351, 1413–1420.
Dennett, D. (1978). Brainstorms: Philosophical essays on mind and psychology. Cambridge, MA:
MIT Press.
Diamond, A. (1990). Developmental time course in h u m a n infants a n d infant monkeys, a n d
the neural bases of, inhibitory control in reaching. Annals of the New York Academy of Sciences,
608, 637–676.
Duncan, J., Emslie, H., a n d Williams, P. (1996). Intelligence a n d the frontal lobe: Goal selec-
tion in the active control of behavior. Cognitive Psychology, 30, 257–303.
Frith, C. (1996). The role of the prefrontal cortex in self-consciousness: The case of auditory
hallucinations. Philosophical transactions of the Royal Society of London, B351, 1505–1512.
Gehring, W. J., Goss, B., Coles, M. G. H., Meyer, D. E., and Donchin, E. (1993). A neural sys-
tem for error detection and compensation. Psychological Science, 4, 385–390.
Georgiou, N., Bradshaw, J. L., and Chiu, E. (1996). The effect of Huntington’s disease a n d
Gilles de la Tourette’s syndrome on the ability to hold a n d shift attention. Neuropsychologia,
34, 843–851.
Hauser, M. D. (1999). Perseveration, inhibition a n d the prefrontal cortex: A new look.
Current Opinion in Neurobiology, 9, 214–222.
Holroyd, C. B., Dien, J., and Coles, M. G. H. (1998). Error-related scalp potentials elicited by
h a n d a n d foot movements: Evidence for an output-independent error-processing system in
h u m a n s . Neuroscience Letters, 242, 65–68.
Jacoby, L. (1994). Measuring recollection: Strategic versus automatic influences of associa-
tive context. In C. Umiltà & M. Moskovitch (Eds.), Attention and Performance XV: Conscious
and unconscious information processing, p p . 661–679. Cambridge, MA: MIT Press.
Monsell and Driver
James, W. (1890). Principles of psychology. N e w York: Holt.
Johnson-Laird, P. N. (1983). Mental models. Cambridge: Cambridge University Press.
Kramer, A. F., Larish, J. L., Weber, T. A., a n d Bardell, L. (1999). Training for executive con-
trol: Task coordination strategies and aging. In D. Gopher and A. Koriat (Eds.), Attention and
Performance XVII: Cognitive regulation of performance: Interaction of theory and application, p p .
617–652. Cambridge, MA: MIT Press.
Luria, A. R. (1966). Higher cortical functions in man. N e w York: Basic Books.
MacLeod, C. M. (1991). Half a century of research on the Stroop effect: An integrative
review. Psychological Bulletin, 109, 163–203.
Meyer, D. E., and Kieras, D. E. (1997). A computational theory of executive control proceses
a n d h u m a n multiple task performance: 1. Basic mechanisms. Psychological Review, 104, 3–65.
Meyer, D. E., a n d Kieras, D. E. (1999). Précis to a practical unified theory of cognition a n d
action: Some lessons from EPIC computational models of h u m a n multiple-task perform-
ance. In D. Gopher and A. Koriat (Eds.), Attention and Performance XVII: Cognitive regulation
of performance: Interaction of theory and application, p p . 17–88. Cambridge, MA: MIT Press.
Minsky, M. (1985). The society of mind. New York: Simon and Schuster.
Monsell, S. (1984). Components of working memory underlying verbal skills: A “distributed
capacities” view. In H. Bouma a n d D. G. Bouwhuis (Eds.), Attention and Performance X:
Control of language processes, p p . 327–350. London: Erlbaum.
Newell, A. (1973). You can’t play twenty questions with nature a n d win. In W. A. Chase
(Ed.), Visual information processing, p p . 283–308. New York: Academic Press.
Newell, A. (1980). Reasoning, problem-solving a n d decision processes. In R. Nickerson
(Ed.), Attention and Performance VIII, p p . 693–718. Hillsdale, NJ: Erlbaum.
Newell, A. (1990). Unified theories of cognition. Cambridge, MA: Harvard University Press.
Newell, A., Rosenbloom, P. S., and Laird, J. E. (1989). Symbolic architectures for cognition.
In M. I. Posner (Ed.), Foundations of cognitive science, p p . 93–131. Cambridge, MA: MIT Press.
Norman, D. A. (1981). Categorization of action slips. Psychological Review, 88, 1–15.
Norman, D. A., and Shallice, T. (1980). Attention to action: Willed and automatic control of
behaviour. Technical Report no. 99. San Diego, CA: University of San Diego, Centre for
H u m a n Information Processing.
Norman, D. A., and Shallice, T. (1986). Attention to action: Willed and automatic control of
behaviour. In R. J. Davidson, G. E. Schwartz, and D. Shapiro (Eds.), Consciousness and self-
regulation, vol 4, p p . 1–18. New York: Plenum Press.
Pashler, H. (1993). Dual-task interference a n d elementary mental mechanisms. In D. E.
Meyer and S. Kornblum (Eds.), Attention and performance XIV, p p . 245–264. Cambridge, MA:
MIT Press.
Rabbitt, P. , Ed. (1997). Methodology of frontal and executive function. Hove, U.K.: Psychology
Press.
Reason, J. T. (1984). Lapses of attention in everyday life. In R. Parasuraman a n d R. D. Davies
(Eds.), Varieties of attention, p p . 515–549. Orlando, FL: Academic Press.
Robertson, I., a n d Manly, T. (Forthcoming). Sustaining attention in time a n d space. In G. W.
Humphreys, J. Duncan, and A. M. Treisman (Eds.), Attention, space and action: Studies in cog-
nitive neuroscience, p p . Oxford: Oxford University Press.
Banishing the Control Homunculus
Rogers, R., and Monsell, S. (1995). The costs of a predictable switch between simple cogni-
tive tasks. Journal of Experimental Psychology: General, 124, 207–231.
Rogers, R. D., Owen, A. M., Middleton, H. C., Williams, E. J., Pickard, B. J., Sahakian, T. W.,
a n d Robbins, T. W. (1999). Choosing between small, likely rewards a n d large, unlikely
rewards activates inferior a n d orbital prefrontal cortex. Journal of Neuroscience, 19,
9029–9038.
Rogers, R. D., Sahakian, B. J., Hodges, J. R., Polkey, C. E., Kennard, C., and Robbins, T. W.
(1998). Dissociating executive mechanisms of task control following frontal lobe damage
a n d Parkinson’s disease. Brain, 121, 815–842.
Schwartz, M. F., Reed, E. S., Montgomery, M., Palmer, C., and Mayer, N. H. (1991). The
quantitative description of action disorganisation after brain damage: A case study.
Cognitive Neuropsychology, 8, 381–414.
Shallice, T. (1988). From neuropsychology to mental Structure. Cambridge: Cambridge
University Press.
Shallice, T. (1994). Multiple levels of control processes. In C. Umiltà and M. Moscovitch
(Eds.), Attention and Performance XV, p p . 395–420. Cambridge, MA: MIT Press.
Shiffrin, R. M., and Schneider, W. (1977). Controlled and automatic h u m a n information pro-
cessing: 2. Perceptual Learning, automatic attending, and a general theory. Psychological
Review, 84, 127–190.
Stroop, J. R. (1935). Studies of interference in serial verbal reactions. Journal of Experimental
Psychology, 18, 643–662.
Ward, G., and Allport, A. (1997). Planning a n d problem solving using the five-disc Tower of
London task. Quarterly Journal of Experimental Psychology, 50A, 49–78.
32 Monsell and Driver
2 Task Switching, Stimulus-Response Bindings, and Negative Priming
Alan Allport a n d Glenn Wylie
ABSTRACT This chapter is about the effects of successively shifting between conflicting
stimulus-response (S-R) mappings in speeded selective response tasks. Even after some
time to prepare for a shift of task, there can still be a large reaction time (RT) cost on the first
trial of the shifted task, generally referred to as a “residual switch cost.’’ In five experiments,
subjects performed Stroop color naming (in response to incongruent combinations of color
a n d a distractor color word) and word reading. The word-reading task was in response to
both “Stroop’’ and “neutral’’ word stimuli.
Our results show that at least a large component of the so-called switch costs results from
a form of negative priming—or negative transfer of learning—arising from earlier per-
formance of the competing selection task (Stroop color naming), interfering with the execu-
tion of the current task (word reading). The competing task need not have been performed
on the immediately preceding trial to generate these effects. Hence these interference effects
cannot be d u e to a time-consuming “switch of set’’ on the current trial.
The data also point to the special status of the first trial, in any run of speeded RT trials,
even without any shift of task. In our experiments, the first trial of each block of speeded-
response trials was consistently slower (and more accurate) than later trials. (We refer to this
as the “restart’’ effect.) Following the Stroop color-naming task, however, word-reading RT
was hugely increased, not only on the first trial of the next word-reading block (i.e., the
“switch’’ trial), but also on the first trial of later (pure task) blocks of word reading without
any switch of task. Some of the negative priming—or negative transfer—from the Stroop
color-naming task to subsequent word reading turns out to be stimulus specific, depending
on the occurrence of the same individual stimulus items (as distractors, in one task; as tar-
get stimuli in the other), rather than on competing, abstract “task sets.’’ The results are inter-
preted in terms of a process of stimulus-response (S-R) binding in selection-for-action. Later
S-R events can trigger retrieval of previously formed (conflicting or consistent) S-R bind-
ings, resulting in positive or negative priming.
2.1 TASK SET AND TASK SWITCHING
The term task switching seems to suggest to many people the operation of
some kind of a control switch, which shunts the processing system from
one configuration of task readiness to another. These control operations
presumably take time, a n d so should be detectable in performance data,
in the form of reaction time (RT) “switching costs.’’ Some part of the con-
trol operation may also require triggering by an imperative task stimulus
(on a “switch’’ trial) for its completion. Several recent models of task
switching appear to suggest a general view of this kind (e.g., De Jong
1996, chap. 15, this volume; Meiran 1996, chap. 16, this volume; Rogers
a n d Monsell 1995; Rubinstein, Meyer, a n d Evans forthcoming). Although
the models differ in certain respects, they share t w o fundamental
assumptions: (1) “task set’’ corresponds to a certain configuration of the
processing pathways: in effect, facilitation of some task-relevant pro-
cesses a n d (at least partial) unenabling or “disengagement’’ of competing
pathways—crucially, task set configuration directly determines the level
of “task readiness’’; and (2) the processing system is essentially a finite-
state machine: once it has been “switched’’ into a given task configura-
tion, it should stay that way until it is “switched’’ again. The processing
system should thus remain in the same state of readiness for subsequent
“nonswitch’’ trials at the same task.1 From these assumptions it is infer-
red that the difference between switch a n d nonswitch RTs may be taken
as a measure of (or at least include) the time needed to complete the rel-
evant control operation. If these assumptions are correct, the mea-
surement of behavioral switching costs should thus provide a valuable
w i n d o w into the control operations themselves.
This conception of task set a n d the associated metaphor of a control
switch are attractive, not least because of their intuitive simplicity. How-
ever, these simple ideas are not easy to reconcile with the performance
data, as we shall try to show. In fact, the data lead us to question both
assumptions 1 a n d 2 above. Let us be clear. We do not doubt that there is
endogenous control of task set, in the sense of controlling which task is
performed. However, as we shall argue (following Fagot 1994), “task set’’
in this sense should not be simply equated with “task readiness’’—where
“readiness’’ is measured by speed of performance.
2.2 A LOOK AHEAD
In this chapter, we investigate speed of performance (task readiness) as a
function of certain other tasks that subjects have previously h a d to per-
form. Our experiments focus on the origin of the performance (RT)
costs—usually referred to as “switching costs’’—when first one, then
another, competing stimulus-response (S-R) mapping is executed, in
response to the same “bivalent’’ stimuli (see Pashler, chap. 12, this vol-
ume). The stimuli we used include Stroop color-word combinations, for
example a printed color name (e.g., “GREEN’’) presented in a different or
“incongruent’’ color (e.g., blue); response may be based either on the
color or the shape of the stimulus (Stroop 1935; MacLeod 1991).
Most current models of attention a n d control represent an essentially
“memory-less’’ (finite-state) processing system; implicit learning (or
“priming’’) effects from earlier S-R processing operations typically play
little or no part in such models. To the contrary, we shall argue that the
RT switching costs include a large component of (long-term) negative
Allport and Wylie
priming—or better, negative transfer—resulting from learning processes
that occurred in the prior, competing task(s). These priming effects can be
long-lasting; they appear to depend on the retrieval of conflicting S-R
associations formed in earlier processing episodes, their retrieval being
triggered by the same (bivalent) task stimuli.
To introduce our experiments, section 2.3 first outlines an earlier ver-
sion of this idea, linking switching costs and S-R priming; section 2.4 then
recaps some of the available evidence in its support; finally, section 2.5
briefly reviews current ideas on the mechanisms of both short- a n d long-
term priming, with emphasis on memory-based retrieval accounts.
2.3 AN EARLIER MODEL OF TASK SWITCHING AND PRIMING:
“TASK SET INERTIA’’
Allport, Styles, a n d Hsieh (1994) postulated that the task set (or task
readiness) for a given task is liable to persist, involuntarily, over succes-
sive trials, as a form of higher-order priming of competing S-R mappings
(see also Allport a n d Wylie 1999; Meuter and Allport 1999). This priming,
they supposed, took the form of persisting facilitation of the previously
task-relevant S-R mappings or processing pathways, a n d persisting s u p -
pression of the previously competing (but n o w task-relevant) pathways.
The result: negative priming of the current task, and “competitor prim-
ing’’ of the other (no longer intended) task. Allport a n d colleagues
referred to this as “task set inertia’’ (TSI). Their conjecture w a s that posi-
tive a n d negative priming of this kind underlay the performance costs, in
RT and errors, of switching between competing tasks, cued by the same,
bivalent stimuli.
Note that the TSI hypothesis, as formulated by Allport, Styles, a n d
Hsieh (1994), was consistent with—it certainly did not deny the existence
of—some active or endogenous control operation (goal setting?), that
determines which task does in fact get performed, a n d which may also be
able to reduce the performance costs of task switching, at least in part,
when the upcoming task is cued in advance.2 It denied only that the time
cost of task switching (i.e., the RT difference between switch a n d non-
switch trials) directly reflects the time needed to complete a shift of task
readiness (“task set reconfiguration’’), prior to executing the shifted task.
If the latter were the case, they argued, a switch cost of, say, 200 msec
(in terms of mean RT) should be eliminated simply by allowing an ad-
vance preparation interval of this order or longer. To the contrary, several
studies (e.g., Allport, Styles, a n d Hsieh 1994; De Jong 1996, chap. 15, this
volume; Fagot 1994; Goschke, chap. 14, this volume; Meiran 1996; Meiran
et al. forthcoming; Rogers and Monsell 1995; Sudevan a n d Taylor 1987)
have found that a preparation interval even of several seconds still left
a large “residual’’ switch cost. Clearly, the performance costs of task
switching can be at least partially offset by some process of task prepara-
Task Switching and Negative Priming
tion, or goal setting, in advance of the imperative stimulus for the shifted
task. This is not in dispute. In all of these cases, however, the reduction in
switch cost (or the benefit of endogenous task preparation) is very much
smaller (sometimes an order of magnitude smaller) than the length of the
preparation interval needed. Thus the endogenous component of the RT
switch cost, that is, the difference between the RT switch cost at zero a n d
at long precue intervals (Meiran 1996), does not correspond, in any direct
way, to the time needed for this preparation process (whatever it may be)
before the task stimulus.
However, the nature of this preparation remains unclear. Fagot (1994)
suggested the need to distinguish task “setting’’ a n d task “readiness.’’3
According to Fagot, “setting’’ determines which task is in fact performed
(the task goal) a n d can be executed during a preparation interval, where-
as “readiness’’ determines the speed or efficiency with which the task is
performed; according to him, it depends on the preceding trial and is
unaffected by any intentional preparation. In Fagot’s formulation, a sub-
ject can thus be “set’’ for one task but “ready’’ for another one. Clearly,
this formulation is closely related to the TSI hypothesis, although TSI
may have effects that last over many intervening trials (e.g., Allport,
Styles, a n d Hsieh 1994, exp. 4). We return to this issue in sections 2.4
a n d 2.6.
2.4 LONG-TERM NEGATIVE PRIMING AND TASK SWITCHING
Allport, Styles, and Hsieh (1994) offered a number of empirical argu-
ments for their interpretation of task-switching costs in terms of involun-
tary S-R priming (TSI) over an intended shift of tasks. We recapitulate
two of these arguments here.
Earlier studies of task switching suggested that alternation between
tasks resulted in substantial performance costs (relative to “pure’’ tasks)
only if the alternation was between potentially competing or divergent
S-R mappings, in response to the same stimulus set (Jersild 1927; Spector
a n d Biederman 1976). However, Allport, Styles, and Hsieh (1994, exp. 4)
showed that even tasks using dissimilar a n d entirely nonoverlapping
stimuli and responses could exhibit large alternation costs, relative to
pure task performance, if these task stimuli had previously—in an
earlier experimental condition—been involved in different (competing)
S-R mappings to those currently specified. These priming effects of the
previous, competing S-R mappings (as they interpreted them) declined
over successive runs at the new tasks, but were still detectable after more
than 100 responses with the new S-R mappings. Clearly, the time course
of TSI effects can be long-lasting, favoring stimulus-driven retrieval,
rather than simply persisting facilitation or suppression of S-R pathways.
We are not aware of any explanation of these results, to date, in terms of
the time taken by a postulated switch operation. This experiment
Allport and Wylie
(Allport, Styles, and Hsieh 1994, exp. 4) shows that task-switching costs
can be the product of varied S-R mappings that occurred, not just on the
preceding trial, b u t even in a prior experimental session. We present sev-
eral further examples of this point in our experiments 2 - 5 (section 2.6).
Switching from Stroop color naming to word reading resulted in
another, equally striking effect: word reading now showed large (—140
msec) “reverse Stroop’’ interference from the incongruent color, even
after a preparation interval of over a second (Allport, Styles, and Hsieh
1994, exp. 5). As is well known, in the Stroop color word and picture
word tasks, the interference is, typically, strongly asymmetrical (Mac-
Leod 1991; Smith and Magee 1980). Thus, with an incongruent Stroop
stimulus, color naming suffers interference from the word, but word
reading normally shows no interference from the incongruent color
(interference from color to word is known as the “reverse Stroop’’ effect).
This asymmetry has been attributed to differences in the long-term, rela-
tive “strength’’ of the competing pathways (MacLeod and Dunbar 1988;
Cohen, Dunbar, and McClelland 1990).
The reverse Stroop interference found by Allport, Styles, and Hsieh
(1994) seems difficult to account for if there were some kind of control
switch, before word processing (or indeed before response selection),
capable of disengaging or unenabling the processing pathways for color
naming, and selectively facilitating the processing pathways for word
reading. On the contrary, the interference seems to provide rather direct
evidence that the S-R bindings needed for (Stroop) color naming (and the
suppression of word reading) either simply persist or, as we shall argue,
are strongly reelicited, on a subsequent, intended switch to word reading,
in response to the same type of (multivalent) task stimuli. In the new
experiments to be described in section 2.6, we attempt to exploit these
rather dramatic, reverse Stroop interference effects further, to explore the
origins of switching costs, and the negative transfer between successive,
competing tasks.
2.5 VARIETIES OF (LONG-TERM) PRIMING
As is well known, selective attention (and selective response) to a target
stimulus can show persisting aftereffects in the form of item-specific,
positive and negative priming. Thus attending to a particular stimu-
lus attribute (in a “pop-out’’ search task) can facilitate a later selective
response to the same target value, over a number of intervening trials
(Maljkovic and Nakayama 1994; Nakayama and Joseph 1997). Moreover,
a previously ignored distractor, now presented as a subsequent atten-
tional target, can show a negative priming effect, over short lags, that has
been attributed to persisting distractor inhibition (Houghton and Tipper
1994; Milliken and Tipper 1998). Longer-lasting negative priming, it is
argued, reflects associative retrieval, based on prior, conflicting process-
Task Switching and Negative Priming
ing episodes, rather than persisting inhibition (e.g., Allport, Tipper, a n d
Chmiel 1985; De Schepper and Treisman 1996; Lowe 1998; Neill et al.
1992; Park and Kanwisher 1994). There is an ongoing debate about the
extent to which positive and negative priming effects (both short- a n d
long-term) are d u e to the retrieval of associative bindings—formed in
prior, congruent or conflicting processing episodes—rather than merely
persisting activation or inhibition (see, for example, Becker et al. 1997;
Fox 1995; Hommel 1998; Kane et al. 1997; Lowe 1998; Milliken and Tipper
1998; Neill 1997). Stimulus-driven retrieval of prior processing epis-
odes (or rather, of composite “echoes’’ of those episodes) is the basis also
of some models of long-term schema abstraction a n d automatization
(Goldinger 1998; Hintzman 1986; Logan 1988; Logan a n d Etherton 1994).
Involuntary priming effects are not restricted to item-specific stimulus
representations. Rabbitt and Vyas (1973, 1979) established the existence of
RT facilitation effects when the same, abstract S-R mapping rule was re-
peated, independent of the repetition of individual stimulus or response
items. Long-term semantic priming, where the stimuli are related only in
terms of higher-order categories, has also been demonstrated (Joordens
a n d Becker 1997; Becker et al. 1997). Each of these priming mechanisms—
positive a n d negative, item-specific and higher-order effects; temporary
activation or inhibition mechanisms; a n d long-term competitive retrieval
processes—may, in principle, contribute to task set inertia effects follow-
ing a switch of tasks. As we shall see, memory-based retrieval effects
appear to play a major role.
2.6 STROOP COLOR NAMING AND WORD READING:
EXPERIMENTS
In all of these experiments, we used the same pair of tasks: color naming
a n d word reading. The stimuli for both color-naming and word-reading
responses included “Stroop’’ (incongruently colored color words, that is,
bivalent stimuli) and “neutral’’ (univalent) task stimuli, which afford only
one or other of these tasks. We shall focus primarily on the effects of a
shift from color naming to word reading. Word-reading latencies for a
familiar word set have the advantage that they are exceptionally stable,
with a very compact RT distribution and low error rate.
Experiments 1 and 2 used the “alternating-runs’’ method of Rogers a n d
Monsell (1995). Experiments 3–5 used a different experimental paradigm,
in which the probe task (word reading) was performed in “pure task’’
conditions, following a shift from color naming. In all five experiments,
subjects responded by orally naming the target stimulus as fast as possi-
ble, a n d their RT was recorded by means of a voice key.
As a systematic constraint on stimulus sequences, the color or word
identities (concepts) on trial n were not allowed to occur, either as color
Allport and Wylie
or word, on trial n + 1; thus positive and negative item priming across
immediately successive trials was excluded. This constraint applied to all
the experiments reported in this chapter.
Finally, because we aimed to study “residual’’ switch costs, we pro-
vided relatively long (1.0 to 2.0 sec) preparation intervals before each
switch of task (0.5 sec in experiment 2); the subjects were encouraged to
do their best to prepare in advance, on each trial, for the upcoming task.
(Experiments 1 and 2 are described in greater detail in Wylie and Allport
forthcoming.)
Experiment 1: Task Alternation Costs on “Nonswitch’’ Trials and
Effects of S-R Mappings in the “Other’’ Task
The goal of this experiment was to assess to what extent the cost of alter-
nation between mutually competing tasks (color naming, word reading)
depends on the specific S-R mappings in the prior, competing task, or on
the control d e m a n d s of the current task. Subjects switched between color
naming and word reading, with three different pairings of Stroop and
neutral stimuli. Our prediction was that switching performance would
depend primarily on the status (Stroop versus neutral) of the “other’’
task, namely, the task switched from.
For this experiment, as also for experiment 2, we used the alternating-
runs method introduced by Rogers and Monsell (1995). Subjects saw a
large black cross, which divided the screen into four quadrants. On suc-
cessive trials, the task stimulus was presented successively in adjacent
quadrants, in continuous, clockwise rotation: top left, top right, bottom
right, bottom left, top left, and so on. (In experiment 1 and all the follow-
ing experiments, the stimulus remained on until the subjects’ response.)
Half the subjects were instructed to name the color of the stimuli in the
top two quadrants and to name the words appearing in the bottom two
quadrants; for the remaining subjects, this instruction was reversed. For
all subjects, therefore, responses to stimuli appearing in the top left and
bottom right quadrants represent switch trials, whereas responses to
stimuli in the other two quadrants are nonswitch or “repeat’’ trials.
Rogers and Monsell (1995) proposed that the time cost of task switching
can be appropriately measured as the difference between switch and
repeat trial RTs, in the alternating-runs paradigm. We follow this conven-
tion, initially.
The stimuli were the six color words, “red,’’ “green,’’ “blue,’’ “yellow,’’
“pink,’’ and “brown,’’ and the corresponding six colors. Words could
appear in any one of the colors except the color named by the word; that
is, these were incongruent Stroop stimuli. We also presented neutral stim-
uli, designed to afford the execution of only one of the two tasks (see
details below). The intertrial interval was approximately 1.2 to 1.5 sec,
Task Switching and Negative Priming
Figure 2.1 Reaction times (RT) a n d error rates (ER) for word reading a n d color naming in
experiment 1. Error bars show 95% within-subject confidence intervals (Loftus a n d Masson
1994). Subjects alternated every second trial between naming colors a n d reading words, in
three successive stimulus conditions.
varying randomly from trial to trial,4 a n d subjects were encouraged to
use this interval to prepare for the upcoming task. This w a s designed to
permit asymptotic, “endogenous’’ task preparation between trials; hence
there should be “residual’’ switch costs only (Meiran 1996; Meiran et al.
forthcoming; Rogers a n d Monsell 1995).
The experiment was divided into three successive blocks (of 120 trials
each), in a fixed order. In block 1, the “all-neutral’’ condition, the stimu-
lus for color naming was a row of colored Xs; for word reading, it was a
neutral (black) word. In block 2, the “color-neutral/word-Stroop’’ condi-
tion, the stimulus for color naming was a row of colored Xs, as before,
whereas for word reading, it w a s an incongruent Stroop stimulus. Finally,
in block 3, the “all-Stroop’’ condition, the stimuli for both tasks were in-
congruent Stroop stimuli. In block 1 (all-neutral), the respective stimulus
types afforded only one of the two tasks, word reading or color naming,
whereas in block 3 all stimuli were bivalent, affording both word-reading
a n d color-naming responses. In block 2—the critical color-neutral/word-
Stroop condition—the stimuli for the word-reading task were bivalent, as
in block 3, but the stimuli for the color-naming task were univalent. Each
experimental block of 120 trials w a s preceded by 30 trials of practice with
the n e w stimulus conditions.
The results of experiment 1 are illustrated in figure 2.1. Consider first
the results for blocks 1 a n d 3, all-neutral a n d all-Stroop. As described
above, Allport, Styles, a n d Hsieh (1994, exp. 5) studied task switching
Allport a n d Wylie
between color naming and word reading with the same stimulus types
as in blocks 1 and 3 (all-neutral, all-Stroop). The present results generally
replicate their findings: in the all-neutral condition, switch costs (defined
as the RT difference between switch a n d repeat trials) were small (about
20 msec) a n d symmetrical; mean color-naming RT was about 110 msec
slower than word reading. In the all-Stroop condition, switch costs were
larger, a n d markedly asymmetrical (about 30 msec for color naming a n d
over three times this value for word reading). The same asymmetry in the
switch costs between (all-Stroop) color naming a n d word reading is
found in all five experiments reported here. (The theoretical interpreta-
tion of this—at first sight—counterintuitive result is discussed in detail,
with reference to the TSI hypothesis, by Allport and Wylie (1999.)
The intended focus of the experiment, however, w a s on condition 2—
color-neutral/word-Stroop—and the word-reading task in particular. We
argued that, if alternation costs depended on the characteristics (e.g.,
bivalent task stimuli) of the task to which a switch is made, then the cost
of shifting to the word-reading task should be about the same in this
condition as in the all-Stroop condition because both conditions require
responses to the same set of incongruent Stroop stimuli. In contrast, if
switching costs depended on priming effects from the prior, competing
task, as postulated by the TSI hypothesis, then the cost of shifting to the
word-reading task in condition 2 should be about the same as in the all-
neutral condition because both conditions have neutral color naming as
the competing task. The results are extremely clear. As predicted by the
TSI account, the cost of switching to the word-reading task was practi-
cally identical in conditions 1 and 2, and significantly larger (p<0.0005)
in condition 3. That is, the switch cost here appears to be a function of the
complementary task set ( from which the switch is m a d e , in this case),
rather than depending on the intrinsic d e m a n d s of the task to which the
switch is m a d e .
Comparison of the color-naming performance in conditions 2 a n d 3 is
also relevant. Predictably, color-naming RTs to Stroop stimuli (condition
3) are much longer than to the neutral color stimuli in condition 2: a clas-
sic Stroop effect of about 180 msec. If switch costs reflected the control
d e m a n d s of the task set to which the switch is m a d e , we should presum-
ably expect a much larger switch cost for the color-naming task in condi-
tion 3, in response to Stroop stimuli, than in condition 2, with neutral
color stimuli. However, this is clearly not the case. Switch costs for color
naming did not differ significantly between the two conditions.
Returning to the word-reading task, we may also consider possible
reverse Stroop interference effects. Taking condition 1 as the available
baseline for reading neutral word stimuli, it is clear that condition 2, with
Stroop word stimuli, shows essentially no such reverse Stroop effect.
Word-reading performance in conditions 1 a n d 2 is practically iden-
Task Switching and Negative Priming
tical. In contrast, comparing word-reading RTs in condition 3—also in
response to Stroop word stimuli—against the neutral baseline of condi-
tion 1 appears to show a large (—200 msec) reverse Stroop effect on switch
trials (p< 0.0001), and a still very substantial (—120 msec) performance
cost on repeat trials (p < 0.0001). Given that, in most experimental condi-
tions (that do not involve switching between color and word), word read-
ing shows no interference from an incongruent stimulus color, the
appearance of reverse Stroop effects in the all-Stroop condition is strong
evidence that some components of the task set, a n d / o r the specific S-R
mappings, for color naming were still active (or were reactivated) during
the word-reading trials. We note that these task interference effects
occurred after a relatively long preparation interval (over 1 sec) between
trials. Even more strikingly, a large performance cost for word reading
was still present on repeat trials. In other words, readiness for the word-
reading task (including effective disengagement from the complementary
color-naming task) appears to be very far from complete on the repeat
trials of condition 3. This observation undermines a widespread assump-
tion of the alternating-runs method, namely, that task set reconfiguration
can be assumed to be complete on nonswitch trials after a single switch
trial, as several students of task switching have proposed (e.g., De Jong
1996, chap. 15, this volume; Rogers and Monsell 1995). Further discussion
is deferred until after experiment 2.
Experiment 2: Time Course Effects of Priming between Competing
S-R Mappings
Experiment 1 demonstrated large task interference effects from color
naming to word reading, in the all-Stroop condition, not only on switch
trials but also in the subsequent nonswitch or repeat trial RTs. Allport and
Wylie (1999) interpreted these effects as a form of task priming (or task set
inertia) resulting from the Stroop color-naming task. We may now ask:
H o w long do these priming effects persist? This is clearly an important
empirical question, both for the design of future studies of task switching
and for the interpretation of existing data. For example, consider the
word-reading performance in experiment 1, specifically, in conditions 2
and 3. (Recall that, in both conditions, word reading was in response to
Stroop stimuli. The conditions differed only in the type of stimuli pre-
sented for color naming: Stroop stimuli in condition 3 and neutral stim-
uli in condition 2). Suppose that, after performing condition 3 for some
time, the color task stimuli changed abruptly from Stroop to neutral, that
is, to condition 2, while subjects continued to perform both word-reading
and color-naming tasks in alternating runs. Prior to the stimulus change,
performance in condition 3 might be expected to resemble that observed
in experiment 1 for the same condition. H o w many trials (or how many
iterated cycles of alternating runs) will it take, with neutral color-naming
Allport and Wylie
stimuli, before the Stroop word-reading performance approaches that
observed in condition 2? This is the question that we attempted to answer
in experiment 2.
For this purpose, subjects successively performed the all-Stroop a n d
the color-neutral/word-Stroop condition (“color-neutral,’’ for short) of
experiment 1, in continuously repeating “miniblocks’’ of 6 cycles in each
condition (a “cycle’’ is four trials in the alternating-runs paradigm, with
double alternation—two color-naming trials and two word-reading
trials; a “miniblock’’ was 6 successive cycles). Stimuli appeared in suc-
cessive screen locations, without a break, between successive all-Stroop
a n d color-neutral miniblocks. (We had no way of knowing, in advance,
h o w many cycles of the color-neutral/word-Stroop condition w o u l d be
needed to track the decline of priming by the preceding all-Stroop color-
naming task, on the word-reading RTs. Six cycles (24 trials) was arbi-
trarily chosen as long enough, we hoped, to show a substantial—and
possibly complete—transition, after the change to color-neutral stim-
uli, to the no-interference pattern in word-reading RTs found in experi-
ment 1.)
Two modifications to the neutral stimulus displays of experiment 1
were introduced in experiment 2. First, instead of being presented in
solid print, as in experiment 1, the neutral word stimuli were presented
in o u t l i n e print. The letter outlines were in black, but they were not filled
with any color, thus appearing “transparent’’ to the screen background.
(This format was also used for the neutral word stimuli in all subsequent
experiments.) For the incongruent Stroop stimuli, the same outline char-
acters were filled in with the appropriate color. Second, instead of a string
of colored Xs, in experiment 2 the neutral color stimuli were filled, col-
ored rectangles of the relevant color, occupying approximately the area of
a five-character word. The w o r ds (and colors) used in experiment 2 were
“red,’’ “green,’’ “blue,’’ “purple,’’ “pink,’’ and “orange.’’ The response-
stimulus interval (RSI) was fixed at 500 msec.
Subjects completed a total of 30 alternating, 24-trial cycles of all-Stroop
a n d color-neutral stimulus conditions. They had 3 practice blocks of 30
trials each, with all-Stroop stimuli, immediately before the main experi-
ment. This began with 6 cycles (24 trials) with all-Stroop stimuli, followed
by 6 cycles (24 trials) of color-neutral stimuli, followed without a break by
a further 6 cycles of all-Stroop stimuli, a n d so on. Subjects were allowed
a rest pause after every 120 trials. Data from the first miniblock after a rest
pause were excluded from analysis. A cycle always began with the two
color-naming trials. The start of each cycle was also redundantly cued by
a high (800 Hz) tone, for all-Stroop cycles, a n d a low (220 Hz) tone for
color-neutral cycles, immediately before the first color trial of each cycle.
Nine subjects from the Oxford University subject panel participated in
the experiment, four men and five women, mean age 38 years. (For fur-
ther experimental details, see Wylie and Allport forthcoming.)
Task Switching and Negative Priming
750
700
650
R T 600
(msec)
550
500
ER
450
.3
.2
.1
0
WORD READING TASK
SwR
\ ^
All-Stroop
(Expt 1&2)
Sw R Sw R Sw R
1-2 3-4 5-6
Cycle Number
Colour-Neutral
(Experiment 2)
{ ^
P—o
S w R
Colour-
Neutral
(Expt 1)
Figure 2.2 Reaction times (RT) a n d error rates (ER) for word reading in experiment 2
(filled symbols). Error bars show 95% within-subject confidence intervals. Subjects succes-
sively performed six cycles (24 trials) with “all-Stroop’’ stimuli, followed without a break by
six cycles of “color-Neutral,’’ a n d so on. The only difference between all-Stroop and color-
Neutral conditions was in the stimuli presented for color naming. All word-reading RTs
were in response to incongruent “Stroop’’ stimuli. Data from experiment 1 (open symbols)
are shown for comparison.
The resulting mean RTs a n d error rates for the word-reading task are
shown in figure 2.2 and for the color-naming task in figure 2.3. For com-
parison, we also include the results of the same two stimulus conditions
from experiment 1. To track performance d u r i n g the color-neutral
miniblocks, the data were collapsed across cycles 1 and 2, 3 a n d 4, and 5
a n d 6, respectively, to give 24 observations per subject per cell.
The main focus of interest is the color-neutral condition. As expected,
word-reading RTs (figure 2.2) showed a progressive reduction over suc-
cessive cycles (p < 0.005), affecting both switch a n d repeat trials. The
further away (either in time or number of trials) from the preceding all-
Stroop miniblock, the smaller the task interference from the preceding
Stroop color naming appears to be. Switch costs—defined as the dif-
ference between switch a n d repeat trial RTs—also diminished progres-
sively over successive cycles (p < 0.01). However, as figure 2.2 shows,
even after 6 cycles (24 trials) of the color-neutral condition, word-reading
performance on switch trials h a d still not come d o w n to the level of per-
formance obtained in the color-neutral condition of experiment 1. Switch
costs in cycles 5–6 were still larger (p < 0.05) than in the color-neutral
condition of experiment 1, where subjects had, so far, done no Stroop
color naming.
46 Allport and Wylie
RT
(msec)
900
850
800
750
700
650
600 -
550
COLOUR NAMING TASK
ER
SwR
\SH
All-Stroop
(Expt 1&2)
S w R S w R S w R
1-2 3-4 5-6
Cycle Number
Colour-Neutral
(Experiment 2)
s
o—o
S w R
Colour-
Neutral
(Expt 1)
Figure 2.3 Reaction times (RT) and error rates (ER) for color naming in experiment 2 (filled
symbols). Error bars show 95% within-subject confidence intervals. Data from experiment 1
(open symbols) are shown for comparison.
These data thus provide a clear, but incomplete answer to the question
to which the experiment was addressed: H o w long do task-priming (i.e.,
interference) effects between color naming and word reading (generated
in the all-Stroop conditions) persist, after the color-naming task shifts
from Stroop to neutral stimuli? The incomplete answer is, evidently,
longer than 24 trials, or 6 cycles.
RTs and error rates in the color-naming task are shown in figure 2.3.
Data from the comparable conditions in experiment 1 are again included
for comparison. As expected, there w a s a large difference in the speed of
color naming in response to Stroop a n d neutral stimuli (p < 0.0001) a n d
between switch and repeat trials (p < 0.001). Error rates for color naming
were also significantly higher, as usual, in the all-Stroop condition. Color-
naming RTs in the all-Stroop miniblock were similar to those in the cor-
responding all-Stroop condition of experiment 1. In the color-neutral
condition, repeat trial performance was broadly similar to the equivalent
(color-neutral) repeat trials in experiment 1; this w a s so already in the
first two cycles, a n d showed no significant change on subsequent cycles.
Switch trial RTs, on the contrary, decreased significantly (p < 0.0005)
from cycles 1–2 to cycles 5–6. This combination resulted in a progressive
reduction in the nominal switch costs (switch trial RT minus repeat trial
RT) for color naming, as also in the word-reading task, over successive
cycles of the color-neutral condition.
47 Task Switching and Negative Priming
Discussion of Experiments 1 and 2
It may be useful to discuss experiments 1 a n d 2 together. Both experi-
ments used the alternating-runs method (Rogers and Monsell 1995), with
a fully predictable switch of task every second trial and a comparatively
long intertrial interval. In these conditions, it has been argued, any
anticipatory or endogenous task preparation is likely to be more or less
asymptotic. In these conditions, as several authors have postulated, the
residual switch costs (defined as the difference between switch a n d repeat
RT at these longer intervals) are taken to reflect the time cost of a control
operation (task set reconfiguration) executed during the course of the
switch trial (e.g., Meiran 1996, chap. 16, this volume; Rogers and Monsell
1995; Rubinstein, Meyer, and Evans forthcoming).
Experiment 1 suggested that the switch cost, measured in this way, is a
function primarily of the task requirements on the complementary, pre-
ceding task. However, the same experiment also demonstrated that the
repeat (or nonswitch) trials, used as the baseline for this assessment of
switch costs, by no means represent a fully or optimally prepared state of
task readiness. Word reading on repeat trials, in the all-Stroop condition,
still showed very large interference effects from the preceding color-
naming task. This finding seems inconsistent with a simple model of an
(exogenous) control switch that shunts the processing system from one
discrete task configuration to another on a single switch trial, to leave the
system fully prepared (“reconfigured’’) for the new task on subsequent
trials.
Experiment 2 provided even more problematic results for such a
conception. According to a simple executive switch model, in the color-
neutral condition, task reconfiguration to word reading should be com-
pleted on the first cycle (indeed, on the first switch trial to word reading).
It should then presumably remain in that state throughout the following
five cycles because the alternating, complementary task was n o w cued by
univalent stimuli (colored rectangles) that do not in any way afford word
reading. Consequently, after the first trial of word reading, color-neutral
performance should resemble that in the color-neutral condition of exper-
iment 1, where switch costs for word reading amounted to no more than
20 msec. However, contrary to these expectations, in experiment 2 (color-
neutral), we found switch costs for the word-reading task of between two
a n d four times this size, decreasing slowly over successive cycles. The
critical difference between the two experiments, we suggest, w a s that in
experiment 2, but not in experiment 1, subjects h a d also recently been
required to perform the Stroop color-naming task, in response to the
same set of bivalent stimuli. It seems clear that any account of these
results will need to refer to the priming effects of previous, competing
tasks—up to at least some 24 trials earlier—cued by the same, bivalent
stimuli.
Allport and Wylie
Experiment 2 provided results that also seem inconsistent with Allport,
Styles, a n d Hsieh’s interpretation (1994) of task priming (task set inertia):
that is, simply as the persisting facilitation or suppression of competing
processing pathways. If the cost of performing a previous, divergent S-R
mapping simply reflected persisting pathway activation or suppression,
then, without further priming, such performance costs should presum-
ably decrease monotonically over successive trials—they should cer-
tainly not rebound on the next switch trial. This, however, is precisely what
we observed over successive cycles of the color-neutral condition in
experiment 2, in both word-reading and color-naming RTs: in each case,
a relatively fast repeat trial was followed by a slower switch trial on the
next cycle or cycles (see figures 2.2–2.3). Task set inertia, interpreted sim-
ply as persisting pathway activation a n d inhibition, is not easily recon-
ciled with this pattern of results.
On the other hand, this pattern of results could be consistent with a
retrieval account of S-R priming by the prior, competing task. Suppose
that a Stroop stimulus, previously associated with color naming, triggers
the reactivation of the same S-R associative links (“bindings’’), previously
associated with those same stimulus attributes. These S-R bindings might
be postulated to include both “positive’’ links between the (previously)
task-relevant stimulus attributes a n d their associated responses, a n d
also “negative’’ links between (what were previously) distractor attrib-
utes and “do not respond’’ (or “nonresponse’’) action codes (cf. Allport,
Tipper, a n d Chmiel 1985; Hommel 1998; Lowe 1998; Neill et al. 1992; Stoet
a n d Hommel forthcoming). To account for the rebound effect on succes-
sive switch trials, however, the postulated retrieval of competing S-R
bindings would have to be in some way more effective, or to trigger a
greater interfering effect, at the start of each new run of trials, that is (in
these experiments), on the switch trials.
As we shall demonstrate in the following experiments, the RT inter-
ference from color naming to word reading (and from word reading to
color naming) is greatly enhanced on the first trial of each run of trials.
Moreover, a similar, massive rebound of RT interference from a prior task
occurs also on the first trial of a run, with no explicit switch of task. Indeed,
it turns out that even a brief interruption (as brief as two seconds) in a
regular series of speeded response trials, a n d subsequent restart of the
same task, is liable to trigger renewed task interference from earlier, com-
peting S-R mappings, executed in response to the same stimuli. The per-
formance costs on a task switch trial may thus include (or be a special
case of) a much more general phenomenon of competing, reevoked
S-R mappings (both “positive’’ and “negative’’ associative bindings), trig-
gered by the onset of a new r u n of trials.
Switching between “Pure’’ Tasks Experiments 1 a n d 2, using the
alternating-runs paradigm, found very large task interference (reverse
Task Switching and Negative Priming
Stroop) effects on word reading in the all-Stroop condition, including on
the nonswitch or repeat trials. This interference was still detectable up to
6 cycles, or 24 trials, after the requirement to switch between competing
S-R mappings—in response to the same, bivalent stimulus set—had been
lifted. These results suggest that, as a measure of switch costs, the dif-
ference in RTs between switch a n d repeat trials in the alternating-runs
paradigm may not represent a clean or appropriate contrast between
an unprepared (“not-yet-reconfigured’’) a n d a completely prepared
(“reconfigured’’) state.
This rather discouraging observation prompted us to search for other
possible procedures for studying the costs of task switching. Rogers a n d
Monsell (1995) argued forcefully that the procedure pioneered by Jersild
(1927) of comparing performance in alternating and fixed (pure) tasks
confounded the requirement to shift tasks a n d the requirement to “hold
in mind’’ two tasks versus just one. This argument w a s of critical impor-
tance in motivating the measurement of switch a n d repeat trials within
the same switching block (see also Meiran 1996). On the other hand, the
results of experiments 1 a n d 2 raise serious doubts about this procedure,
too, as a method for measuring straightforwardly interpretable switch
costs. However, we might still escape the postulated confound between
switching a n d task memory load if we could somehow probe the effects
of task alternation within pure task blocks. An experiment reported in the
much-cited landmark paper Stroop 1935 in fact suggests such a possible
method.
Stroop (1935) reported three experiments. His experiment 3 described
the following sequence of events, in three main stages. Subjects were first
asked to read aloud lists of printed color names, both in neutral lists
(words printed in black) and in lists of incongruently colored Stroop stim-
uli, 50 words to a sheet, to provide a baseline measure of word-reading
performance. Their mean list completion times corresponded to an aver-
age time per item of 388 msec for Stroop stimuli a n d 382 msec for neutral
word stimuli. Apparently, there w a s little or no reverse Stroop interfer-
ence effect here. In stage 2 of the experiment, the subjects practiced color-
naming similar lists of incongruent Stroop stimuli, again 50 items per list,
4 lists per session, for 8 successive days. Finally, in stage 3, after an inter-
vening session of naming neutral color patches, they returned to their
original task of word reading, though n o w only in response to incongru-
ent Stroop stimuli. They again read aloud 50-word lists, 4 lists per ses-
sion, on 2 successive days. Their list completion times in (postcolor) stage
3 corresponded to an average time per item of 696 msec for Stroop word
reading on day 1, a n d 440 msec on day 2. Word-reading performance on
postcolor day 1 thus revealed a mean cost of 308 msec per item, averaged
over the first 200 trials of postcolor word reading. This w a s with Stroop
stimuli. It is to be regretted that Stroop (1935) did not also include a con-
dition of neutral postcolor word reading to assess the possible presence
Allport and Wylie
of reverse Stroop interference. Neither did he report list-reading times
separately for successive lists in stage 3, to provide an indication of the
possible decline in the word-reading performance costs over the session.
Even on postcolor day 2, however, averaged over all 200 trials, there was
evidently still some 50 msec per item performance cost on Stroop word
reading.
These rather dramatic results are not often referred to (but see Mac-
Leod 1991, 164–165). They appear to represent a particularly powerful
a n d long-lasting demonstration of task set inertia. They might perhaps
also be described as the “long-term costs of task alternation, observed
in pure task conditions.’’ It w o u l d be interesting to know whether simi-
lar, though perhaps more transitory, effects could be generated by a
very much briefer induction phase than Stroop’s eight days (1935) of
color naming. It would be of interest also to track the time course of such
effects, trial by trial, using discrete RTs. To what extent is the first trial of
a run (as in a switch trial) differentially affected by long-term priming of
a competing S-R mapping?
Experiment 3: The “Before and After’’ Paradigm
Our first explorations of these questions (described in Allport a n d Wylie
1999) used just 30 trials of Stroop color naming, sandwiched between
an initial, baseline phase of both Stroop and neutral word reading, a n d a
following postcolor phase of word reading, again in response to both
Stroop and neutral stimuli. Allport a n d Wylie referred to this as the
“before and after’’ paradigm. For half the subjects, stimuli for color
naming appeared in the u p p e r half of the screen, above a horizontal line,
a n d stimuli for word reading appeared in the lower half of the screen. For
the remaining subjects, this arrangement was reversed. All word-reading
trials were performed under pure task conditions. Thus, in phase 2 (the
color-naming phase), after 10 practice color trials, subjects were instructed
that they w o u l d perform a further, single block of 20 color-naming trials;
there would then be a 2 sec pause, with instructions on the monitor
screen to return to the earlier word-reading task. Thereafter, they were
assured, there w o u l d be no further color-naming trials.
The stimuli for word reading, in both the baseline (phase 1) a n d post-
color phase (phase 3) of the experiment, occurred in successive blocks of
Stroop and neutral stimuli (10 trials per block). There was a 2 sec pause
between blocks, during which the instruction “Read words’’ appeared on
the screen. RSI within a block w a s fixed at 300 msec. In phase 1, all sub-
jects performed ten 10-trial blocks of word reading, with alternate blocks
of Stroop a n d neutral stimuli. The first 3 blocks of each type, in phase 1,
were treated as practice. In phase 3, one group of subjects saw Stroop
stimuli in postcolor block 1 a n d neutral stimuli in postcolor block 2
(“Stroop-first’’ subjects); the other experimental group saw neutral stim-
Task Switching and Negative Priming
uli in postcolor block 1, a n d Stroop stimuli in postcolor block 2 (“Neutral-
first’’ subjects). The same order of Stroop a n d neutral blocks was also
used in phase 1. Experiment 3 also included a control group w h o com-
pleted the same phase 1 a n d phase 3 word-reading tasks (in neutral-first
order), but simply rested during phase 2 (Stroop color naming). There
were 10 subjects in each group.
The results are illustrated in figure 2.4. The initial interference effects on
postcolor word reading, recorded in discrete reaction times, were even
larger than the mean effects on overall list completion times reported by
Stroop (1935), but lasted a very much shorter time. After a total of just 30
trials of Stroop color naming, the first trial of Stroop word reading (the
nominal switch trial) showed an RT cost of over 450 msec, compared to
the control group or to the experimental subjects’ baseline (phase 1) first-
trial performance. (In a partial replication experiment, we found an even
larger cost, of approximately 600 msec; see Allport a n d Wylie 1999.)
Errors on the first (postcolor) Stroop word-reading trial also increased
sharply, to over 35%. Subsequent nonswitch Stroop trials, in postcolor
block 1, also continued to show large, but rapidly diminishing, perfor-
mance costs. Thus immediate postcolor trials 2–5 (all of them nonswitch
trials) showed a mean Stroop word-reading cost relative to controls of
over 200 msec. Postcolor interference was still present throughout the
rest of this block (trials 6–10), with a mean RT cost of 135 msec. Subjects
w h o read neutral words in postcolor block 1 (“neutral-first’’ subjects) also
exhibited significant performance costs, though very much smaller than
for Stroop word reading: well over 100 msec on trial 1, and around 20–
30 ms over the remainder of the block. Comparison between postcolor
Stroop and neutral word-reading performance indicates a reverse Stroop
interference effect in immediate postcolor word reading on the order of
350 msec on trial 1, diminishing to around 180 and 100 msec over trials
2–5 a n d 6–10, respectively.
The most revealing feature of these results, however, was found in
postcolor word-reading block 2. Between postcolor blocks 1 a n d 2 there
was simply a 2 sec pause (a 1 sec screen prompt to continue to “Read
words,’’ followed by a 1 sec blank interval). Despite there being no switch
of task from color naming to word reading between blocks 1 a n d 2, word-
reading performance in postcolor block 2 again showed massive task
interference effects on trial 1. That is—over and above the first-trial RT
increment seen in the control subjects (who h a d not performed the pre-
vious color-naming task)—Stroop word reading on trial 1 of postcolor
Block 2 showed an additional RT cost of over 300 msec, whereas neutral
word reading showed an additional RT cost of around 150 msec (see
figure 2.4). These large a n d highly significant task interference costs were
only seen on trial 1 (the restart trial) of block 2, and not on any later trials
in the block. Restart trials in later word-reading blocks also continued to
show significant, but very much smaller reverse Stroop interference (i.e.,
Allport and Wylie
Figure 2.4 Reaction times (RT) a n d error rates (ER) in experiment 3, using the “before and
after’’ paradigm. All subjects first performed a baseline condition of word reading (phase
1) and, later on, a further series of “pure task’’ word-reading blocks (phase 3). Between
phases 1 a n d 3 the experimental groups performed a short period of Stroop color naming
(phase 2). The control group performed phases 1 a n d 3 (in neutral-first order), but rested
during phase 2.
compared to neutral w o r d reading) over a number of subsequent blocks
of w o r d reading (not shown in figure 2.4).
However, first-trial or restart effects were not confined to the postcolor
phase of word reading. As figure 2.4 shows, the baseline word-reading
performance, before any mention to the subjects of a color-naming task,
also showed a consistent (p < 0.0001) first-trial RT cost, on the order of
100 m s , accompanied by an equally consistent (p < 0.0005) reduction in
errors. A similar pattern can also be seen in the color-naming task. (Note
that trial 1 of this block of 20 color-naming trials was preceded by 10 pre-
vious color-naming trials, thus is also a restart trial, not a switch trial.)
The control subjects, w h o simply rested during phase 2, showed a simi-
lar RT cost, on the first trial of each word-reading block, both in phase 3
a n d in their previous baseline data.
Very similar RT costs on the first trial of a run, also in a fixed-task con-
dition, have been reported by De Jong et al. (forthcoming, exp. 3); the
Task Switching a n d Negative Priming
first-trial RT cost w a s very much larger in old than in young subjects.
Error rate w a s not reported. A possibly related effect has been studied by
Gopher a n d colleagues (Gopher, Greenshpan, and Armony 1996; Gopher,
Armony, a n d Greenshpan forthcoming). In their experiments, a run of RT
trials was briefly interrupted by an instruction cue, which requested the
subject either to shift tasks (“switch’’) or to continue as before with the
same task (“reconsider’’). The first trial following both “switch’’ a n d
“reconsider’’ instructions showed a large RT increment, the latter nearly
as large as the former in some conditions.
It seems evident that the initial trial of a r u n of successive, speeded-
response trials, even without any requirement to switch tasks, presents
some additional processing demand, relative to all subsequent trials in
the run. In the baseline performance (and in the control subjects through-
out), the data clearly show a shift in speed-accuracy criterion toward
greater caution, on the first trial of a run. Moreover, when the task stim-
ulus on the first trial of a run is of a type that has been associated
recently with a competing S-R mapping, the conflict latent in these
divergent S-R mappings appears to be strongly reevoked, even though
previous repeat trials in a preceding r u n may have exhibited apparently
reduced conflict effects. The possibility arises, therefore, that RT switch
costs, confined to the first trial of a r u n of alternating tasks, may reflect in
large measure the same conjunction of effects. (Further discussion is
deferred until after experiment 4.)
Experiment 4: “Restart’’ Costs and Repeated Task Switching
Experiment 4 represents a modified version of the “before and after’’
paradigm. There were several modifications. The principal difference was
that, after the baseline word-reading phase (which was unchanged), the
sequence of a short block of incongruent Stroop color-naming trials fol-
lowed by two postcolor blocks of word reading was iterated in successive
cycles throughout the experiment. (As in experiment 3, for half the sub-
jects, stimuli for color naming appeared in the u p p e r half of the screen,
above a horizontal line, a n d stimuli for word reading appeared in the
lower half of the screen. For the remaining subjects, this arrangement was
reversed.) The control group, instead of performing the color-naming
task, on each cycle performed what w a s intended to be (as far as pos-
sible) an unrelated RT task (size and luminance comparisons, with
two-alternative keypress responses) followed by the two blocks of word
reading.
Each Stroop color-naming block consisted of just 10 trials. At the end of
this color-naming block (and at the end of the keypress block, for the con-
trol subjects), the instruction “Read words’’ appeared on the screen for 1
sec, followed by a horizontal line on a blank screen for 1 sec, followed by
(postcolor) block 1 of word reading, consisting of 20 trials. The instruction
Allport and Wylie
“Read words’’ then appeared on the screen again for 1 sec, followed by a
blank screen (with the horizontal line) for 1 sec, immediately followed by
(postcolor) block 2 of word reading, again consisting of 20 trials. (RSI
within a block was fixed at 300 msec, as in Experiment 3. After a short rest
pause, the sequence then recommenced with the next block of color nam-
ing (or keypress), then two blocks of word reading, a n d so on, through-
out the remainder of the experiment. Subjects were encouraged to do
whatever they could to prepare for the next word-reading block, during
each 2 sec preparation interval.
Each block of 20 word-reading trials consisted of either 10 trials of
Stroop stimuli followed (without a break) by 10 trials of neutral word
stimuli, or the reverse sequence. Thus the color-naming (or keypress)
block could be followed immediately by either Stroop or neutral word
stimuli. Further, if postcolor block 1 consisted of 10 Stroop stimuli fol-
lowed by 10 neutral stimuli, block 2 contained the reverse sequence. In
this way, the break between postcolor blocks 1 a n d 2 never involved a
change either of task or of stimulus type. There were thus three different
types of transition to word-reading trials that might trigger a possible
restart effect: (1) a 2 sec task interrupt with renewed instructions and also
with a switch of tasks, at the start of postcolor block 1; (2) a 2 sec task
interrupt with renewed instructions but without a switch of task or a
change of stimulus type, at the start of postcolor block 2; a n d (3) a change
of stimulus type, but without a task interrupt or a switch of task, at the
transition from the first to the second 10 trials of each block.
The results are illustrated in figure 2.5. As in experiment 3, the first
trial of each block, in each of the experimental conditions (baseline word
reading; color naming; postcolor word reading, block 1; postcolor word
reading, block 2), showed a highly consistent (p < 0.0001) increase in RT,
relative to trials 2–10, a n d a reduction in errors (p < 0.001). In addition,
on the first trial of postcolor block 1 (i.e., on the switch trial immediately
following the color-naming block, for the experimental subjects) this
restart RT effect appears massively enhanced, w h e n compared either to
the control group or to the precolor baseline; the effect (i.e., the RT differ-
ence betwen trial 1 a n d all subsequent trials in the block) was also signi-
ficantly (p < 0.0001) larger for Stroop than for neutral trials. The control
group showed no additional performance cost (switching cost) on shift-
ing from the keypress task back to word reading, relative to their first-
trial baseline performance where there was no shift of task.
Unlike experiment 3, however, the performance costs on postcolor
word reading, for the experimental group, appear to be confined entirely
to the first trial of the run. The large performance costs on later postcolor
word-reading trials, found in experiment 3 a n d also (on a much longer
timescale) in Stroop 1935, are absent on the later repeat trials of experi-
ment 4. One factor that varies considerably between these different
experiments is the ratio of (Stroop) color-naming to word-reading trials.
Task Switching and Negative Priming
Figure 2.5 Reaction times (RT) a n d error rates (ER) in experiment 4. Error bars show 95%
within-subject confidence intervals. All subjects first performed a baseline condition of
word reading. They then performed repeated, successive blocks of either Stroop color nam-
ing (for the experimental groups) or an unrelated (keypress) RT task (for the control group),
followed immediately by t w o blocks of (Stroop a n d neutral) word reading.
In Stroop 1935, subjects began the postcolor word-reading phase with a
massive preponderance of color naming, in response to Stroop stimuli, in
their recent experience. Experiment 4—the only experiment in this series
where repeat trials showed no between-task interference—also h a d the
lowest ratio (1:4) of color-naming to word-reading trials. Wylie a n d All-
port (forthcoming) provide further evidence suggesting that the (recency-
weighted) ratio of color-naming to word-reading trials, in response to the
same set of bivalent Stroop stimuli, massively affects switch costs, as well
as repeat trial RTs.
Color-naming RTs also showed a small effect of the immediately pre-
ceding word-reading condition. Trial 1 of color-naming was some 45
msec slower, on average, when the last ten trials of block 2, in the pre-
ceding word-reading cycle, consisted of ten Stroop, rather than ten neu-
tral, word-reading trials.
However, experiment 4 was designed primarily to investigate the pos-
sible effects of three different types of restart trials on postcolor word-
Allport a n d Wylie
reading performance (the effects on the switch trials (i.e., block 1, trial 1)
have already been discussed). The second type of possible restart trial
was simply a change of stimulus type, with no temporal interrupt and no
switch of task (i.e., trial 11 of postcolor blocks 1 and 2). As figure 2.5
shows, this manipulation had relatively little effect, besides a transient
increase in accuracy, though there was a hint of an RT cost (trial 11 versus
later trials in the block; p = 0.077) on the change from neutral to Stroop
stimuli in postcolor block 1.
The third type of restart trial was at the start of postcolor block 2. The
first trial of block 2 followed a 2 sec interrupt with renewed instructions
but without a switch of task or a change of stimulus type. At this point,
the experimental subjects had last engaged in Stroop color naming 20
trials before. Nevertheless, here again their RTs showed a significantly
(p < 0.025) enhanced restart cost, relative to the control subjects’ first-trial
RT, analogous to the renewed task interference found previously in
experiment 3 (in trial 1 of postcolor block 2). Unlike experiment 3, how-
ever, there was no sign of a differential cost for Stroop and neutral word
stimuli. In experiment 3, subjects who read incongruent Stroop words
in postcolor block 2 had only responded to neutral words in block 1
(“neutral-first’’ subjects). By contrast, in the present experiment all sub-
jects had read both Stroop and neutral words in the preceding block 1. It
seems plausible that this difference in prior exposure to bivalent stimuli,
during postcolor word reading, may be responsible for this difference
between experiments.
In summary experiment 4 confirms and extends three major findings
from experiment 3:
1. The first trial of a run of speeded-response trials shows a substantial
RT cost—the restart cost—generally (but not always) associated with a
reduction in errors. (In experiment 3, there was a marked increase in
errors on the first (postcolor) word trials.) This restart effect is found on
the first trial of a run, without any switch of task (cf. also De Jong et al.
forthcoming; Gopher, Greenshpan, and Armony 1996; Gopher, Armony,
and Greenshpan forthcoming);
2. Prior performance of divergent S-R mappings (e.g., Stroop color nam-
ing) in response to the same (or related) stimuli as the current task (e.g.,
word reading), greatly amplifies or enhances the basic RT cost on restart
trials (e.g., in the first postcolor block), relative to control subjects who
have not been exposed to the competing, divergent task;
3. An enhanced RT cost (relative to the basic first-trial RT pattern seen in
control subjects) occurs also on the first trial of subsequent trial blocks,
many trials later (postcolor block 2). The effect looks like a rebound of the
earlier—so-called—“switch cost’’, except that, in this case, there was no
switch of task from the preceding trials.
Task Switching and Negative Priming
Discussion of Experiments 3 and 4
Together, these findings raise a number of provocative a n d important
questions. First, there is the restart RT cost itself. What is the causal rela-
tion (if any) between this effect and the RT switch cost, typically also
found only on the first trial of a run (cf. Rogers and Monsell, 1995)?
Second, what is the relation between either of these phenomena a n d
finding 3 above, namely, the rebound of enhanced RT costs (over a n d
above the basic restart effect seen in the control subjects’ RT) on the first
trial of later, nonswitch runs of word reading? Clearly, this is a rebound
of task interference, resulting from earlier performance of the Stroop
color-naming task because the enhanced RT cost is defined precisely by
comparison with the first-trial RTs of the control subjects, w h o had not en-
countered the color-naming task. That such interference can be reelicited
in later pure task blocks, with no intervening trials of the competing
color-naming task—hence with no intervening switch of set from color
naming to word reading—strongly favors some kind of learning or
memory-based account, whereby the task stimulus (at the start of a
new r u n of trials) triggers retrieval of the prior (conflicting) S-R bindings.
The rebound phenomenon appears inconsistent with Allport, Styles, a n d
Hsieh’s interpretation (1994) of task priming (or task set inertia), purely
in terms of the persisting activation or inhibition of task-relevant pro-
cessing pathways. It w o u l d be consistent, however, with current models
of long-term negative (and positive) priming, as the product of associa-
tive learning (S-R and S-S bindings), formed in the course of previous
processing episodes (e.g., Becker et al. 1997; Lowe 1998; Neill 1997; cf.
also Goldinger 1998). We suggest that the retrieval of conflicting S-R m a p -
pings further delays the system from settling to an internally consistent
set of stimulus-to-task or stimulus-to-response bindings—consistent also
with the currently activated task “goals.’’ In terms of this speculative
account of the results, the fact that neutral word stimuli also showed mas-
sive, first-trial rebound interference, as a result of prior Stroop color nam-
ing, implies that the S-R bindings formed during the course of the Stroop
color task must have included associative bindings between the distractor
(word) stimuli and some inhibitory (“do not respond’’) action codes, thus
generating long-term negative priming—or negative transfer to the
word-reading task (cf. Allport, Tipper, and Chmiel 1985; Lowe 1998; Neill
et al. 1992).
These rebound interference effects at the start of later nonswitch trial
blocks appear strikingly similar to the RT costs found on “true’’ switch
trials, referred to generally, hitherto, as “residual switching costs.’’ The
apparent similarity of these two effects inevitably raises the question
whether the same causal process may be responsible for both. (Two
entirely separate mechanisms would seem uneconomical, to say the
least.) Because the rebound cost occurs with no immediate switch of set
Allport and Wylie
between the two competing tasks, the resemblance of this phenomenon
to the RT costs on immediate switch trials thus calls in question whether
“residual switching costs’’ are appropriately so named. Of course, there
may be some additional processing cost on immediate switch trials, not
present in the rebound RT costs; if so, however, the results of experiment
3 suggest that this additional component may contribute only a small
part of the residual switch cost, at least in some conditions; further dis-
cussion will be postponed until after experiment 5.
Experiment 5: Item-Specific Priming, S-R Bindings, and Task
Switching
An important issue we have not yet addressed is the extent to which the
priming of competing S-R mappings applies to processing pathways as a
whole, namely, in these experiments, separable pathways for color nam-
ing or word reading in general, for example, the “grapheme-phoneme cor-
respondence’’ (GPC) system (Coltheart 1985), a n d the extent to which
these priming effects might be item specific, pertaining to individual S-R
mappings. The distinction is fundamental (cf. Monsell, Taylor, a n d
Murphy forthcoming). Positive or negative priming of a postulated pro-
cessing pathway, as a whole, can be thought of as a possible mechanism
of task readiness, or task set (e.g., Cohen, Dunbar, and McClelland 1990),
whereas positive or negative priming of individual, item-specific S-R
mappings cannot.
In our experiments 1–4, as in most other studies of task switching, we
used a fixed set of stimuli and responses, each of which occurred many
times in the course of the experiment. Moreover, in the “all-Stroop’’ con-
ditions, which resulted in by far the largest RT interference costs, subjects
encountered the identical, incongruent conjunctions of color and color
word in both color-naming and word-reading tasks, with complete over-
lap of stimulus sets. What would happen if we reduced this stimulus
overlap, even in part? To begin to address this question, we designed an
experiment in which we probed subjects’ postcolor word-reading per-
formance (1) on w o rd s that h a d been presented as distractors in the
Stroop color-naming task, as in previous experiments; a n d (2) on words
that subjects had never encountered in the color-naming task.
The experimental rationale is as follows. Insofar as long-term negative
priming, across a switch of task from Stroop color naming to word read-
ing, is item specific, this effect should apply only to the particular subset of
distractor words, word-color conjunctions, or both, encountered during
Stroop color naming. On the other hand, insofar as the negative priming
mechanism applies to the word-processing pathway as a whole, Stroop
color naming should result in equal performance costs, for word reading,
in response to all word stimuli, regardless of whether they h a d occurred
as distractors during the prior Stroop color naming or not.
Task Switching and Negative Priming
In experiment 5, subjects again alternated between short runs of color
naming and word reading. All color naming was in response to incon-
gruent Stroop stimuli; word reading was probed in response to both
Stroop and neutral word stimuli. We used a set of eight possible colors
(red, green, blue, purple, pink, orange, brown, and yellow) and the cor-
responding eight color words. The specific manipulation of stimulus
overlap in experiment 5 was as follows. For the Stroop color-naming task,
subjects saw, and named all eight colors; however, these were presented
in conjunction with only four of the possible color words as distractors,
resulting in just 28 (4 X 7) possible incongruent conjunctions of color and
word, which occurred equiprobably (Different subsets of four distractor
words were presented to different subjects.) For the Stroop word-reading
task, in contrast, all eight colors and color words occurred equiprobably,
in each of the 56 possible incongruent conjunctions of color and word.
Similarly, for neutral word reading, all eight color words were presented.
(As in previous experiments, neutral words appeared in the form of out-
line letters, appearing “transparent’’ to the gray color of the screen back-
ground. Stroop stimuli used the same outline letters, but incongruently
“colored in.’’)
Recall that, as in the previous four experiments, the color or word iden-
tities (concepts) presented on trial n could not occur, either as color or
word, on trial n + 1. Thus “negative priming’’ across immediately succes-
sive trials, either within or between tasks, was excluded.
Experiment 5 began with 30 trials of practice at the word-reading task,
in response to both Stroop and neutral stimuli, followed by 30 test runs
of word reading (3 trials per run, alternate runs of Stroop and neutral), to
provide a baseline of pure task word performance. In the baseline condi-
tion, as in later parts of the experiment, each run of word reading was
preceded by a 2 sec precue interval (see “Task Cuing’’ below). After the
baseline word reading, for the remainder of the experiment, subjects
alternately and repeatedly performed short runs of color naming fol-
lowed by word reading (as in experiment 4), for a total of 60 cycles.
Unlike experiment 4, however, in each cycle there were seven trials of the
Stroop color-naming task followed by just three trials of word reading.
(Thus, in the repeating cycles, the ratio of color naming to word reading
trials was 7:3, in contrast to the 1:4 ratio in experiment 4.)
Task Cuing The monitor screen was bisected by a bold horizontal line.
For half the subjects, stimuli for the color-naming task appeared in the
top half of the screen, 2 cm above the horizontal line, and stimuli for word
reading appeared 2 cm below the line; for the other subjects, this arrange-
ment was reversed. (To ensure that subjects did not forget this rule, the
word “WORD’’ remained present, as a reminder, at the top (or bottom)
edge of the screen, respectively, and a bar of eight colors at the bottom (or
top) edge, throughout the alternating runs.) The stimulus location (and
Allport and Wylie
850
800 j
750
700
650 j
RT 6 0 0
( m S e c ) 550
500
450
400
Word Reading
""*" Stroop N p
" * " Neutral
• Stroop TT p
- ° ~ Neutral u *
' ^ N e S Baseline
Color Naming
" • " Neg. Primed
~°~ Not Primed
fi
k
Y*
r4
Trials
Baseline
Word
Reading
(Before)
2-4 5-7
Trials
Color
Naming
Trials
Primed and
Unprimed
Word Reading
(After)
Figure 2.6 Reaction times (RT) a n d error rates (ER) in experiment 5. All subjects first per-
formed a baseline condition of word reading. As in experiment 4, they then performed
repeated, successive blocks of Stroop color naming a n d word reading. One set of words
presented for word reading h a d appeared also as the distractors in the Stroop color-naming
trials (NP or “negatively primed’’ items); another set of words appeared only in the word-
reading task (UP or “unprimed’’ items).
hence the task) was precued by the appearance of a lighter gray rectan-
gle, outlined in black, on the darker gray screen, in the location where the
next color-naming or word-reading stimulus would appear. The light
gray rectangle then remained present during the remaining trials in the
r u n . Each cycle (starting with the seven color-naming trials) w a s initiated
by the subject, by pressing a key when ready. The first color-naming stim-
ulus then appeared after a delay of 600 msec, a n d remained on until the
subject’s response. Within a run, RSI between successive color-naming
trials w a s fixed at 300 msec. After subjects h a d responded to the last
color-naming trial, there w a s a blank interval of 800 msec; then the light
gray rectangle reappeared, in the w o r d location, surrounded by a bold
black outline, for 600 msec; the black outline w a s then removed, leaving
61 Task Switching a n d Negative Priming
the light gray rectangle for 600 msec before the first word-reading stimu-
lus appeared. The RSI between color-naming a n d word-reading runs was
thus 2.0 sec. There were 8 subjects, of w h o m 6 were female (mean age 37
years).
The results are illustrated in figure 2.6. Baseline word reading again
showed a highly reliable first-trial RT cost of about 80 msec (relative to
trials 2 a n d 3), combined with a significant d r o p in the error rate. There
was no reverse Stroop interference in the baseline condition. In postcolor
word reading, the first-trial RT cost increased from 80 msec (baseline) to
about 140 msec for unprimed stimuli, and to 220 msec for the negatively
primed Stroop stimuli. The difference between the first trial RT to the neg-
atively primed Stroop stimuli a n d the first trial RTs in the other three
postcolor conditions was highly reliable (p < 0.0001 in each case). On the
nonswitch trials 2 and 3, by contrast, the only reliable differences in word-
reading RT were between the baseline a n d all other (postcolor) word-
reading conditions (trial 2, p < 0.0005; Trial 3, p < 0.0001); the postcolor
performance cost, relative to baseline, on these nonswitch trials was
50–70 msec in mean word-reading RT.
The color-naming task also showed a substantial first-trial effect both
in RTs (p < 0.01) a n d errors (p < 0.005), though with a tendency for RTs
to increase again later in the run. The color-naming task was included
primarily to induce negative priming in word reading. However, the
manipulation of presenting only half of the word set as distractors, in the
color-naming task, means that long-term (within-task) negative priming
can also be tested for in the color-naming RTs. Consider: four of the color
name responses, in the color task, were also potentially elicited (on other
color-naming trials) by the corresponding word distractors; hence (on
most accounts of the Stroop color-naming task) these color names would
have had to be actively suppressed, w h e n they occurred as potential
responses to the word distractors. There were four other color name
responses, however, that were never evoked by their corresponding word
distractors, because these distractor items were not presented in the
color-naming task. The first set thus includes (long-term) distractor-to-
target repetition; the second does not. Note that distractor-to-target repe-
tition on immediately successive trials was excluded by the experimental
design. Comparison of the color-naming RTs to these two stimulus sub-
sets should thus provide an index of (long-term) negative priming with-
in the color-naming task. This comparison resulted in a highly reliable
negative priming effect (p < 0.0005) on color-naming RTs, which did not
interact reliably with trial position.
Discussion of Experiment 5
The five principal results of this experiment can be summarized as
follows:
Allport and Wylie
1. All postcolor word reading showed a substantial performance cost,
relative to the prior baseline performance. All stimulus types also showed
a further, enhanced performance cost on postcolor trial 1, relative to trial 1
in the baseline (precolor) performance.
2. On trials 2 a n d 3, the postcolor performance cost was the same, regard-
less of whether the individual wor ds h a d occurred as distractors in the
color-naming task—and hence (nonresponse to) these stimulus items
could have been, individually, negatively primed—or not. In other words,
on trials 2 a n d 3, there w a s interference—long-term “negative prim-
ing’’—affecting (some element of) the word-reading task or the word-
processing pathway as a whole, independent of any item-specific priming.
This assertion receives its most compelling support from the observation
that postcolor word-reading RTs, in response to (“primed’’ or “unprimed’’)
neutral words, were consistently slower than in the baseline condition
(with the identical set of stimuli). Neutral word stimuli, as such, were of
course never encountered in the color-naming task, although half of these
word stimuli were presented, in the color-naming task, as distractors in a
color-word conjunction; the other half were not. This latter manipulation
h a d no effect whatever, either on trial 1 or on later trials. The question of
whether these postcolor performance costs on nonswitch trials apply to
all word reading, or specifically to the reading of English color names, or
words in the same typeface, or words sharing other contextual features
with the stimuli (or responses) in the color task, is beyond the scope of
this experiment. Clearly, these are key questions to be resolved by future
research.
3. In addition, however, postcolor word-reading RTs to negatively
primed (NP) Stroop stimuli showed an enhanced performance cost, on
trial 1 only, that appears to be strongly item specific. Interestingly, this is
the only condition in which word-reading responses were m a d e to the
same conjunctions of color a n d word that had been presented previously
in the color-naming task. This rather surprising pattern of results—a
large first-trial cost for NP Stroop stimuli; no additional first-trial cost for
NP neutral w o r d s — w o u l d thus be consistent with the possibility that
S-R bindings or connection weights, formed in the Stroop color-naming
task, might be specific to the individual conjunctions of task-relevant a n d
-irrelevant stimulus attributes. An alternative possibility, also consistent
with these results, w o u l d be that this component of the negative priming
from the color-naming task was specific to individual words, as distrac-
tors, but encoded simply as being “colored in’’ (in any colour?). Although
these intriguing conjectures remain to be established, the theoretically
crucial point is that a substantial component of the first-trial switch
cost, with repeated stimuli, is apparently item specific.
4. Unprimed (UP) word stimuli (i.e., items not presented as distractors in
the color-naming task) do not appear to show any performance differ-
Task Switching and Negative Priming
ences between Stroop a n d neutral words, even on postcolor trial 1.5 This
w o u l d suggest that the large, first-trial, reverse Stroop effects on post-
color word-reading RTs found in all our previous experiments may also
reflect item-specific priming from the prior color-naming task. This ques-
tion, also, clearly invites further experiment.
5. The color-naming task provided clear evidence of within-task negative
priming (distractor-to-target repetition costs) across stimulus domain,
that is, from the word distractors to later color-naming response; cf. Neill
1977; Neill a n d Westberry 1987; Tipper a n d Driver 1988). Recall, how-
ever, that distractor-to-target concept repetition over immediately succes-
sive trials was excluded in experiment 5, as in the previous experiments.
Analyses of lag effects, between the occurrence of an individual distractor
a n d its re-presentation as a target probe, are beyond the scope of the pres-
ent chapter; suffice to say that the within-task negative priming in color
naming is relatively long-lasting, consistent with our account of the
implicit retrieval of earlier S-R bindings that link specific distractors a n d
“do not respond’’ codes. Analyses of distance (lag) effects in the between-
task negative priming from color naming to word reading on postcolor
trial 1 failed to find any reliable effect of the number of intervening trials
between the most recent occurrence of an item, in the color-naming task,
a n d its being probed on a switch to Stroop word reading. Again, further
experiments, specifically designed to examine these issues, are required.
2.7 CONCLUSIONS
What have we learned from all this, as regards selection-for-action
(Allport 1980, 1987, 1989) in Stroop-like tasks, a n d the effects of task alter-
nation? (We note that our conclusions may—or may not—turn out to be
confined to task switching in response to incongruent Stroop stimuli.
Only further research can tell.)
Priming versus “Switching’’ Costs
Negatively, the results of each of our five experiments challenge what has
appeared to many people as the intuitively obvious interpretation for the
residual switch costs (switch minus repeat RTs, at long RSIs), namely as
the time cost of an interpolated control operation that shunts the pro-
cessing system from one configuration of task readiness to another. These
arguments have been presented at various points in the chapter, and will
not be laboured further here.
More positively, we have shown that negative priming (or negative
transfer) from prior, divergent S-R mappings can have massive a n d long-
lasting interference effects on the speed and accuracy of response to the
same or overlapping stimuli, following a shift (or reversal) of those m a p -
pings. This between-task proactive interference can be observed on both
Allport and Wylie
switch a n d nonswitch trials, and even in pure task conditions. The prior,
competing task may have been last performed some considerable time
before (cf. Stroop 1935, exp. 3; Allport, Styles, a n d Hsieh 1994, exp. 4; a n d
experiments 2, 3, and 4 in this chapter). Between-experiment compari-
sons suggest that the relative frequency a n d recency of the competing
S-R mappings strongly affect the size of the RT interference costs (see
also Wylie a n d Allport forthcoming). Related to this, Lowe (1998) has
reported evidence that within-task negative priming (over a 5 min delay)
increased with the number of times that an item had been previously
ignored.
Strikingly, however, the present between-task proactive interference—
that is, long-term negative priming or negative transfer resulting from
prior execution of competing S-R mappings—has by far its greatest
effects on the first trial of a run of speeded RT trials. It seems clear, there-
fore, that such proactive interference forms a major component of what
have hitherto been referred to as the “residual switch costs,’’ obtained by
subtracting nonswitch trial RTs from switch trial (i.e., first-trial) RTs, at
long RSIs, following a shift of tasks. Critically, for the interpretation of
switch costs, we have shown that there is no need for the source of this
proactive interference to be the execution of the competing task on the
immediately preceding trial. RT interference effects from competing S-R
mappings can also be reevoked on the first trial of later trial blocks in
pure task conditions. They can also be triggered by alternation between
intrinsically noncompeting tasks, with no overlap of either stimuli or
responses, hence no need, in principle, to disengage one task set and re-
engage another (Allport, Styles, and Hsieh 1994, exp. 4).
Moreover, even in the absence of any obvious source of negative prim-
ing by competing tasks (e.g., neutral word reading in the precolor, base-
line conditions) the first trial of a run of speeded RT trials appears to be
characteristically slow (and accurate): the restart effect (see also De Jong
et al. forthcoming; Gopher, Armony, a n d Greenshpan forthcoming). The
combination of the processes underlying these two effects, we are
tempted to speculate, may be responsible in large part for the so-called
“residual switch costs’’ in the cueing and alternating runs paradigms.
Finally, we have shown that the negative priming from prior, com-
peting S-R mappings includes a substantial, item-specific component.
Evidence for this component was confined to the switch trial itself in the
present data (experiment 5). It seems plausible to infer that item-specific
priming may also have contributed substantially to the observed differ-
ences between switch a n d repeat trial RTs (switch costs) in other ex-
periments on task switching that similarly used the same, repeated set of
stimulus items in the pre- and postswitch tasks. It is important to note
that item-specific RT costs cannot be explained by (i.e., are logically
beyond the scope of) models of task switching that postulate a discrete,
stagelike control operation (task set reconfiguration) that precedes stimu-
Task Switching and Negative Priming
lus identification. Nor, for that matter, can they be explained by task set
inertia—in which “task set’’ is conceived of as a control state that affects
the efficiency of different tasks (or task processing pathways) as a whole.
A Tentative Model of Goal Setting and Selection for Action
As already suggested by several authors, it seems clear that alternation
between competing S-R tasks (typically, tasks with divergent S-R m a p -
pings, in response to overlapping stimulus sets) involves a number of dif-
ferent processes a n d effects. We identify at least three. Following Fagot
(1994), we earlier distinguished “goal setting’’ (or goal activation, includ-
ing presumably the deactivation of other, competing goals) a n d “per-
formance readiness’’ (i.e., the time needed for the system to “settle’’ to a
u n i q u e response). Performance readiness, we tentatively p r o p o s e ,
d e p e n d s on at least three further factors: (1) the prior acquisition of both
congruent a n d conflicting S-R bindings, learned in the course of earlier
processing interactions, and giving rise to the negative—and positive—
transfer (priming) effects we have attempted to illustrate here; (2) the cue-
dependent activation of task-relevant (or -irrelevant) subsystems (e.g.,
subsystems involved in the coding of cue-related stimulus attributes,
response attributes, or both); a n d (3) suppression or inhibition of subsys-
tems that encode competing (distractor-related) attribute domains.
The process of goal activation—not directly studied in these experi-
ments—can presumably be triggered in advance of an imperative task
stimulus by appropriate externally or internally generated cues. Task pre-
cues (like task stimuli themselves) may also evoke activation or suppres-
sion of appropriate (or inappropriate) stimulus attribute domains, in
advance of the task stimulus (cf. Chelazzi et al. 1993; Luck 1998; Miller
1999).6 In contrast, neither of these processes (temporary goal activation;
preactivation of domain-specific subsystems) should have any effect on
the potentially conflicting, learned S-R connection weights, which may
simply not be susceptible to direct modification by “control processes.’’
On the other hand, our results lead us to believe, S-R connection weights
are indeed subject to continuous (and very substantial) modification,
through learning, in the course of trial-by-trial sensory-motor processing.
We find it helpful to think of attention a n d “control’’ issues in terms of
the integrated competition (IC) hypothesis, as put forward by Duncan
a n d colleagues (Duncan 1996; Duncan, Humphreys, a n d Ward 1997; see
also Phaf, van der Heijden, and H u d s o n 1990). Ward (1999) has described
a simple model of selection-for-action that illustrates some of the basic
assumptions of IC, in the form of a multimodule, interactive activation
a n d competition (IAC) network (McClelland and Rumelhart 1981), simi-
lar to that put forward by Phaf, van der Heijden, a n d H u d s o n (1990). In
this model, if one “goal node’’ is strongly activated (i.e., clamped on), a n d
other competing nodes inactivated, the IAC network can only settle to
states consistent with the activated goal. In other words, goal activation
66 Allport and Wylie
determines (constrains) which task is performed.7 The processing time
(number of cycles) needed to settle to a unique response, on the other
hand, will d e p e n d on the amount of conflict in the network. Associations
(connection weights) formed in the execution of a prior, competing task,
we suggest, can contribute massively to such conflict.
NOTES
Work on this chapter was funded by a studentship to Glenn Wylie from the McDonnell-
Pew Foundation, through the Oxford Center for Cognitive Neuroscience. We gratefully
acknowledge this support.
1. Of course, over a longer timescale there may be loss of arousal, or of “task activation,’’
but over immediately successive trials, in motivated subjects, any such effects should not
normally be expected to play a substantial role.
2. At the time that Allport, Styles, and Hsieh (1994) put forward the TSI hypothesis, there
was little evidence available that the RT cost of a switch of tasks could be reduced by pre-
cuing or anticipatory preparation. Since then, a number of studies (e.g., De Jong, chap. 15,
this volume; Meiran 1996; Rogers a n d Monsell 1995) have shown clear evidence of RT
benefits of these manipulations.
3. A related distinction between “goal setting’’ and S-R “rule activation’’ is an important
feature also of the model by Rubinstein, Meyer, and Evans (forthcoming).
4. The intertrial interval (ITI) varied in experiment 1 only, depending on the time needed
by the experimenter to code the subject’s response, via a keypress. In experiments 2–5, ITI
was computer controlled.
5. Analysis by items in experiment 5 showed a small RT advantage for neutral words in
postcolor word reading, but the contrast was not reliable.
6. After extended practice, more abstract task cues may come to do the same (see the strik-
ing data of Sudevan and Taylor 1987, fig. 3). Learned (and, in principle, arbitrary) cue-to-
attribute-domain activation links may be an important component of the “endogenous’’
task preparation effects that many authors have reported.
7. Goal setting will be effective to the extent that the goal has been sufficiently strongly acti-
vated, that is, is effectively “clamped on,’’ a n d competing goals inactivated. It is a matter of
degree. In h u m a n subjects many factors (motivation, practice, etc.) are liable to affect this
(Goschke and Kuhl 1993).
REFERENCES
Allport, A. (1980). Attention and performance. In G. Claxton (Ed.), Cognitive psychology: New
Directions, p p . 112–153. London: Routledge and Kegan Paul.
Allport, A. (1987). Selection-for-action: Some behavioural and physiological considerations
of attention and action. In H. Heuer and A. F. Sanders (Eds.), Perspectives on perception and
action, p p . 395–419. Hillsdale, NJ: Erlbaum.
Allport, A. (1989). Visual attention. In M. I. Posner (Ed.), Foundations of cognitive science,
p p . 631–682. Cambridge, MA: MIT Press.
Allport, A., Styles, E. A., and Hsieh, S. (1994). Shifting intentional set: Exploring the dy-
namic control of tasks. In C. Umiltà and M. Moscovitch (Eds.), Attention and Performance
XV, p p . 421–452. Cambridge MA: MIT Press.
Task Switching and Negative Priming
Allport, A., Tipper, S. P., and Chmiel, N. R. J. (1985). Perceptual integration a n d postcate-
gorical filtering. In M. I. Posner a n d O. S. M. Marin (Eds.), Attention and Performance, XI,
p p . 107–132. Hillsdale, NJ: Erlbaum.
Allport, A., and Wylie, G. (1999). Task-switching: Positive a n d negative priming of task-set.
In G. W. Humphreys, J. Duncan, and A. M. Treisman (Eds.), Attention, space and action:
Studies in cognitive neuroscience, p p . 273–296. Oxford: Oxford University Press.
Becker, S., Moscovitch, M., Behrmann, M., and Joordens, S. (1997). Long-term semantic
priming: A computational account a n d empirical evidence. Journal of Experimental
Psychology: Learning Memory and Cognition, 23, 1059–1082.
Chelazzi, L., Miller, E. K., Duncan, J., and Desimone, R. (1993). A neural basis for visual
search in inferior temporal cortex. Nature, 363, 245–347.
Cohen, J. D., Dunbar, K., a n d McClelland, J. L. (1990). On the control of automatic pro-
cesses: A parallel distributed processing account of the Stroop effect. Psychological Review,
97, 332–361.
Coltheart, M. (1985). Cognitive neuropsychology a n d the study of reading. In M. I. Posner
a n d O. S. M. Marin (Eds.), Attention and Performance, XI, p p . 3–37. Hillsdale, NJ: Erlbaum.
De Jong, R. (1996). Cognitive a n d motivational determinants of switching costs in the
task-switching paradigm. Paper presented at the Ninth ESCOP Conference, Würzburg,
Germany, September.
De Jong, R., Emans, B., Eenshuistra, R., and Wagenmakers, E.-J. (Forthcoming). Strategies
a n d intrinsic limitations in intentional task control.
De Schepper, B., and Treisman, A. (1996). Visual memory for novel shapes: Implicit coding
without attention. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22,
27–47.
Duncan, J. (1996). Cooperating brain systems in selective perception and action. In T. Inui
a n d J. L. McClelland (Eds.), Attention and Performance, XVI, p p . 549–578. Cambridge, MA:
MIT Press.
Duncan, J., Humphreys, G. W., a n d Ward, R. (1997). Competitive brain activity in visual
attention. Current Opinion in Neurobiology, 7, 255–261.
Fagot, C. (1994). Chronometric investigations of task switching. Ph.D. diss., University of
California, San Diego.
Fox, E. (1995). Negative priming from ignored distractors in visual selection: A review.
Psychonomic Bulletin and Review, 2, 145–173.
Goldinger, S. D. (1998). Echoes of echoes? An episodic theory of lexical access. Psychological
Review, 105, 251–279.
Gopher, G., Greenshpan, Y., a n d Armony, L. (1996). Switching attention between tasks:
Exploration of the components of executive control and their development with training. In
Proceedings of the Annual Meeting of the Human Factors and Ergonomics Society.
Gopher, G., Armony, L., a n d Greenshpan, Y. (Forthcoming). Switching tasks and attention
policies and the ability to prepare for such shifts. Journal of Experimental Psychology: General.
Goschke, T., and Kuhl, J. (1993). Representation of intentions: Persisting activation in mem-
ory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 19, 1211–1226.
H i n t z m a n , D. L. (1986). “Schema abstraction’’ in a multiple-trace m e m o r y model.
Psychological Review, 93, 411–428.
Allport and Wylie
Hommel, B. (1998). Event files: Evidence for automatic integration of stimulus-response
episodes. Visual Cognition, 5, 183–216.
Houghton, G., and Tipper, S. P. (1994). A model of inhibitory mechanisms in selective atten-
tion. In D. Dagenbach a n d T. Carr (Eds.), Inhibitory processes in attention, memory and lan-
guage, p p . 53–112. San Diego, CA: Academic Press.
Jersild, A. T. (1927). Mental set a n d shift. Archives of Psychology, no. 89.
Joordens, S., a n d Becker, S. (1997). The long and the short of semantic priming effects in
lexical decision. Journal of Experimental Psychology: Learning, Memory, and Cognition, 23,
1083–1105.
Kane. M. J., May. C. P., Hasher, L., Rahhal, T., and Stoltzfus, E. R. (1997). Journal of
Experimental Psychology: Human Perception and Performance, 23, 632–650.
Loftus, G. R., a n d Masson, M. E. J. (1994). Using confidence intervals in within-subject
designs. Psychonomic Bulletin and Review, 1, 476–490.
Logan, G. D. (1988). Toward an instance theory of automatization. Psychological Review, 95,
492–527.
Logan, G. D., and Etherton, J. L. (1994). What is learned during automatization? The role of
attention in constructing an instance. Journal of Experimental Psychology: Learning, Memory,
and Cognition, 20, 1022–1050.
Lowe, D. (1998). Long-term positive a n d negative identity priming: Evidence for episodic
retrieval. Memory and Cognition, 26, 435–443.
Luck, S. J. (1998). Neurophysiology of selective attention. In H. Pashler (Ed.), Attention,
p p . 257–295. Hove, U.K.: Psychology Press.
MacLeod, C. M. (1991). Half a century of research on the Stroop effect: An integrative
review. Psychological Bulletin, 109, 163–203.
MacLeod, C. M., and Dunbar, K. (1988). Training and Stroop-like interference: Evidence for
a continuum of automaticity. Journal of Experimental Psychology: Learning, Memory, and
Cognition, 14, 126–135.
Maljkovic, V., and Nakayama, K. (1994). Priming of popout: 1. Role of features. Memory and
Cognition, 22, 657–672.
McClelland, J. L., a n d Rumelhart, D. E. (1981). An interactive activation model of context
effects in letter perception: 1. An account of basic findings. Psychological Review, 88, 375–407.
Meiran, N. (1996). Reconfiguration of processing mode prior to task performance. Journal of
Experimental Psychology: Learning, Memory, and Cognition, 22, 1423–1442.
Meiran, N., Chorev, Z., and Sapir, A. (Forthcoming). Component processes in task-shifting.
Cognitive Psychology.
Meuter, R. F. I., and Allport, A. (1999). Bilingual language switching a n d naming: Asym-
metrical costs of language selection. Journal of Memory and Language, 40, 25–40.
Miller, E. K. (1999). Prefrontal cortex a n d the neural basis of executive functions. In G. W.
Humphreys, J. Duncan, and A. M. Treisman (Eds.), Attention, space and action: Studies in cog-
nitive neuroscience, p p . 251–272. Oxford: Oxford University Press.
Milliken, B., and Tipper, S. P. (1998). Attention and inhibition. In H. Pashler (Ed.), Attention,
p p . 191–221. Hove, U.K.: Psychology Press.
Monsell, S., Taylor, T. J., a n d Murphy, K. (Forthcoming). Naming the colour of a word: Is it
responses or task-sets that compete? Memory and Cognition.
Task Switching and Negative Priming
Nakayama, K., and Joseph, J. S. (1997). Attention, pattern recognition, and popout in visual
search. In R. Parasusaman (Ed.), The attentive brain. Cambridge, MA: MIT Press.
Neill, W. T. (1977). Inhibitory a n d facilitatory processes in attention. Journal of Experimental
Psychology: Human Perception and Performance, 3, 444–450.
Neill, W. T. (1997). Episodic retrieval in negative priming and repetition priming. Journal of
Experimental Psychology: Learning, Memory, and Cognition, 23, 1291–1305.
Neill, W. T., Valdes., Terry. K. M., and Gorfein, D. S. (1992). Persistence of negative priming:
2. Evidence for episodic trace retrieval. Journal of Experimental Psychology: Learning, Memory,
and Cognition, 18, 993–1000.
Neill, W. T., a n d Westberry, R. L. (1987). Selective attention and the suppression of cognitive
noise. Journal of Experimental Psychology: Learning, Memory, and Cognition, 13, 327–334.
Park, J., a n d Kanwisher, N. (1994). Negative priming for spatial locations: Identity mis-
matching, not distractor inhibition. Journal of Experimental Psychology: Human Perception and
Performance, 20, 613–623.
Phaf, R. H., van der Heijden, A. H. C., and Hudson, P. T. W. (1990). SLAM: A connectionist
model for attention in visual selection tasks. Cognitive Psychology, 22, 273–341.
Rabbitt, P. M. A., and Vyas, S. (1973). What is repeated in the “repetition effect’’? In S.
Kornblum (Ed.), Attention and Performance IV, p p . 327–342. London and N e w York:
Academic Press.
Rabbitt, P. M. A., and Vyas, S. (1979). Memory a n d data-driven control of selective attention
in continuous tasks. Canadian Journal of Psychology, 33, 71–87.
Rogers, R. D., and Monsell, S. (1995). The cost of a predictable switch between simple cog-
nitive tasks. Journal of Experimental Psychology: General, 124, 207–231.
Rubinstein, J., Meyer, D. E., a n d Evans, J. E. (Forthcoming). Executive control of cognitive
processes in task switching.
Smith, M. C., and Magee, L. E. (1980). Tracing the time course of picture-word processing.
Journal of Experimental Psychology: General, 109, 373–392.
Spector, A., and Biederman, I. (1976). Mental set and shift revisited. American Journal of
Psychology, 89, 669–679.
Stoet, G., and Hommel, B. (Forthcoming). Action planning and the temporal binding of
response codes. Journal of Experimental Psychology: Human Perception and Performance.
Stroop, J. R. (1935). Studies of interference in serial verbal reactions. Journal of Experimental
Psychology, 18, 643–662.
Sudevan, P., a n d Taylor, D. A. (1987). The cuing a n d priming of cognitive operations. Journal
of Experimental Psychology: Human Perception and Performance, 13, 89–103.
Tipper, S. P., a n d Driver, J. (1988). Negative priming between pictures and words: Evidence
for semantic analysis of ignored stimuli. Memory and Cognition, 16, 64–70.
Ward, R. (1999). Interactions between perception and action systems: A model for selective
action. In G. W. Humphreys, J. Duncan, and A. M. Treisman (Eds.), Attention, space and
action: Studies in cognitive neuroscience, p p . 311–332. Oxford: Oxford University Press.
Wylie, G., and Allport, A. (Forthcoming). Task switching and the measurement of “switch
costs.’’ Psychological Research.
Allport and Wylie
3 Goal-Directed and Stimulus-Driven Determinants of Attentional Control
Steven Yantis
ABSTRACT Selective visual attention to objects a n d locations depends both on deliberate
behavioral goals that regulate even early visual representations (goal-directed influences)
a n d on autonomous neural responses to sensory input (stimulus-driven influences). In this
chapter, I argue that deliberate goal-directed attentional strategies are always constrained
by involuntary, “hard-wired’’ computations, and that an appropriate research strategy is to
delineate the nature of the interactions imposed by these constraints. To illustrate the inter-
action between goal-directed and stimulus-driven attentional control, four domains of
visual selection are reviewed. First, selection by location is both spatially and temporally
limited, reflecting in part early visual representations of the scene. Second, selection by
feature is an available attentional strategy, but it appears to be mediated by location, a n d
feature salience alone does not govern the deployment of attention. Third, early visual seg-
mentation processes that parse a scene into perceptual object representations enable
object-based selection, but they also enforce selection of entire objects, and not just isolated
features. And fourth, the appearance of a new perceptual object captures attention in a
stimulus-driven fashion, but even this is subject to some top-down attentional control.
Possible mechanisms for the interaction between bottom-up a n d top-down control are
discussed.
People are perceptually selective: they subjectively experience a n d
respond to only a subset of the sensory signals evoked by objects a n d
events in the local environment. The psychological a n d neural mecha-
nisms that mediate perceptual selectivity are collectively termed atten-
tion. Although often used to refer to other psychological phenomena (e.g.,
the ability to perform two or more tasks at the same time, or the ability to
remain alert for long periods or time), for the purposes of this chapter,
“attention’’ shall refer exclusively to perceptual selectivity, a n d the exam-
ples will concern visual selection in particular.
Like any function of the brain, attention is adaptive: it supports behav-
ior that achieves goals a n d ultimately promotes survival. Visual selection
comprises an exquisite interaction between two mutually constraining
factors. First, current behavioral goals can modulate processing of sen-
sory input (top-down or goal-directed influences on selection). Second,
properties of the stimulus and “hard-wired’’ architectural properties of
the brain (i.e., properties that do not change with task set) together con-
strain the implementation of attentional goals (bottom-up or stimulus-
driven influences on selection).
Mosaic
Occlusion
Targets
r A
r
j
Distracters
• •
• •
Figure 3.1 Displays from Rensink a n d Enns 1998. Viewing displays of zero or one target
among varying numbers of distractors, subjects were to indicate whether the target was
present by pressing one of two buttons as quickly as possible. To p . In the “mosaic’’ condi-
tion, targets consisted of fragmented squares and distractors were complete squares. Here,
visual search for the target was highly efficient, with search slopes of 7 msec/element.
Bottom. In the “occlusion’’ condition, targets now abutted the circles to give the impression
of partly occluded squares. Performance was inefficient, with search slopes of 36 m s e c /
element in the target-present case, and 66 msec/element in the target-absent case. Visual
search apparently required a deliberate deployment of selective attention to one item at a
time. This suggests that the completion of the partly occluded square occurred automati-
cally before visual selection could operate.
Because “attention’’ is most often used in everyday language to refer to
an intentional a n d deliberate mental process, the autonomous bottom-up
influences on selection are sometimes overlooked. They arise as a result
of brain mechanisms that perform certain types of computation effi-
ciently a n d automatically; these are often referred to as “preattentive
processes.’’ For example, some forms of perceptual organization (e.g.,
figure-ground segregation a n d perceptual grouping) occur without a
deliberate intent on the part of the perceiver, although these computa-
tions can sometimes be modulated to some extent by task goals. These
autonomous computations presumably evolved to speed identification
a n d ensure rapid responses to threatening events, and to free computa-
tional resources for higher tasks such as decision making a n d planning.
Thus w h e n a perceiver with a particular goal encounters a scene, cer-
tain early, hard-wired visual computations will occur whether or not they
are consistent with the goal. An example of this sort of conflict is reported
in Rensink a n d Enns 1998. Observers were asked to search for a notched
square in an array of multiple complete disks a n d squares (see figure 3.1,
top). This was an easy task, a n d search w a s highly efficient, suggesting
that the target’s unique shape could be used to guide search. If, however,
the target was placed so that it appeared to be partly occluded by a disk
(figure 3.1, bottom), then the task was very difficult. Rensink and Enns
concluded that the partly occluded square was perceptually completed
by early vision without any deliberate intent to do so; additional scrutiny
was required to recover the “real’’ (proximal) shape of each item in order
to detect the target shape. Although this sort of perceptual completion is
74 Yantis
normally desirable, in this case, it interfered with the perceptual goal, a n d
thereby revealed h o w an early automatic process can constrain top-down
visual selection.
Although the goal-directed a n d stimulus-driven aspects of attentional
control are typically treated as separate a n d distinct, with most empirical
studies focusing on only one of the two factors, it has become increas-
ingly clear that this distinction is untenable. Every episode of selection
necessarily manifests both types of influence. The observer always occu-
pies some sort of goal state, a n d of course the stimulus and its represen-
tation in the brain always exert an influence. The question thus becomes
not whether or when attention is controlled in a bottom-up or top-down
fashion, but h o w autonomous stimulus-driven influences constrain
attentional goals in any given situation.
In this chapter, I review four domains of visual selection with an
emphasis on h o w stimulus-driven factors constrain deliberate attentional
deployment. These domains are neither mutually exclusive nor exhaus-
tive; they merely provide a convenient framework for organizing the
principles of stimulus-driven constraints on selection.
3.1 SELECTION BY LOCATION
Among the earliest ideas concerning the mechanisms of visual attention
was that one can attend to a restricted region of space (e.g., Helmholtz
1866, 455). We have all had the experience of turning our heads a n d eyes
when we are told, “Look over there!’’ Less obvious, however, is whether
one can selectively attend to one spatial location in a scene containing
many objects that are all equally visible (e.g., objects that are all equidis-
tant from the center of gaze) without moving one’s head or eyes (i.e.,
attend covertly). H o w rapidly can selection by location be accomplished,
a n d h o w efficiently does one reject information to be ignored?
Among the earliest empirical demonstrations that covert selection by
location is possible was Sperling’s observation (1960) that observers can
direct their attention to a specified region of a persisting visual memory
of a display. An array of letters was briefly flashed on a screen, a n d very
shortly after the array disappeared, a tone signaled the part of the display
to be reported, a n d hence to be attended (e.g., “If the tone is high, report
the items in the top row of the display’’). Because the letters themselves
were physically absent from the display by the time the tone sounded,
overt eye movements to fixate the indicated row were not possible; atten-
tion w a s instead directed to a spatial location through covert movements
of “the mind’s eye.’’
A vast body of work carried out in the last four decades has revealed
the spatial and temporal limitations of covert spatial selection. Eriksen
a n d colleagues (e.g., Colegate, Hoffman, a n d Eriksen 1973; Erkisen a n d
Hoffman 1972, 1973) used a cuing paradigm in which a circular array of
Determinants of Attentional Control
letters (centered on fixation) was presented, a n d a small bar marker (the
cue) appeared next to one of the letters. Subjects were to identify the cued
target letter as rapidly as possible. Performance improved as the distance
between the cued letter and its neighbors increased, suggesting a limita-
tion in the spatial precision of attention. Similarly, performance improved
as the duration between the onset of the cue a n d the onset of the letter
array increased, suggesting that about 100–300 msec was required to
focus attention at the cued location. Subjects were instructed not to move
their eyes (in some cases, adherence to this instruction was verified by
monitoring eye position) to ensure that covert attentional deployments,
a n d not overt eye movements, were being measured.
Selective attention, as the name implies, entails selection of attended
items a n d rejection of unattended ones; the efficiency of nontarget rejec-
tion w a s the focus of the work by Eriksen a n d colleagues. In addition,
however, there is evidence that attention can speed detection and iden-
tification of single targets. Posner and his colleagues (e.g., Posner 1978,
1980; Posner, Snyder, and Davidson 1980) conducted a series of experi-
ments varying the predictive validity of a spatial cue. For example, with-
in a block of trials, the cue might indicate the target location on 80% of the
trials (valid cues) and a nontarget location on 20% of the trials (invalid
cues); participants were always informed of this contingency. These
experiments revealed both benefits for valid cues and costs for invalid
cues, relative to “neutral’’ cues that indicated no particular location.
Eriksen and colleagues originally estimated that nontarget rejection
was efficient (i.e., that the identity of adjacent nontargets failed to affect
response time and accuracy) as long as the stimuli were at least 1 degree
of visual angle apart. Later, LaBerge and colleagues (LaBerge 1983;
LaBerge a n d Brown 1986; LaBerge et al. 1991, 1997) a n d Downing a n d
Pinker (1985) measured the spatial distribution of attention by cuing
attention to a location likely to contain a target element, and then pre-
senting a probe stimulus at other locations in space (see also Engel 1971;
Hoffman and Nelson 1981). They generally reported a smooth gradient of
selection surrounding the attended location for several degrees of visual
angle, rather than a sharp boundary separating attended and unattended
regions. More recently, other investigators have refined these techniques
to explore the two-dimensional (e.g., Egly a n d Homa 1984; Eriksen a n d
Yeh 1985; Henderson and Macquistan 1993; Kim a n d Cave 1995; Usai,
Umiltà, and Nicoletti 1995) and three-dimensional (e.g., Atchley et al.
1997; Ghirardelli and Folk 1996; Iavecchia a n d Folk 1995) profiles of spa-
tial selection. For example, Bahcall a n d Kowler (1998) have found that
attended locations are surrounded by a local inhibitory region, analogous
to a center-surround receptive field, which causes the attended target
to stand out perceptually against its immediate background in crowded
displays.
Yantis
The intuitive conception of attention as a deliberate, strategic process
led early investigators to consider spatial cues that were highly task rele-
vant (e.g., they indicated the likely location of the upcoming target), a n d
often indirect and symbolic (e.g., an arrow appearing in the center of the
display pointing to a peripheral location, or a digit indicating a labeled
location). In these cases, of course, the observer h a d an incentive to inter-
pret the cue a n d actively use it to select the content of the cued location;
the emphasis was on the efficiency of goal-directed, controlled deploy-
ments of attention.
Many studies have since investigated the extent to which certain stim-
ulus events may be said to capture attention despite either contrary or
“neutral’’ intentions. Jonides (1980, 1981) drew a distinction between
peripheral, direct cues (i.e., cues near the impending stimulus locations),
a n d central, symbolic cues (i.e., cues that indicated a location other than
the one they occupied and therefore required some translation before the
cued location could be decoded). He found that direct cues d r a w atten-
tion even when they are known to be uninformative and should be
ignored. In contrast, symbolic cues affected performance only when task
instructions required that they be used to direct attention. Evidently, the
visual system is hard-wired to select peripheral abrupt onsets, with little
need for top-down control (see section 3.4 for a more detailed discussion
of this issue).
Investigations of the time course of selection produced by indirect cen-
tral cues versus direct peripheral cues revealed distinct and characteristic
patterns of performance for the two cases (e.g., Cheal and Lyon 1991;
Koshino, Warner, a n d Juola 1992; Müller a n d Rabbitt 1989; Nakayama
a n d Mackeben 1989). Symbolic cues (e.g., a central arrowhead that points
to a likely target location) produce relatively sluggish a n d sustained
attentional effects at the cued location, but only w h e n the cue is task rel-
evant, suggesting that voluntary control is necessary. Direct peripheral
cues, in contrast, produce transient performance advantages for cued
targets relative to uncued ones within as little as 100 msec after the
cue, although these effects dissipate rapidly. Furthermore, the effects of
peripheral cues appear subject to little voluntary control. For example, a
peripheral cue that observers knew would never appear in the target
location, and which should therefore be ignored, nevertheless slowed
target identification by drawing attention automatically (Remington,
Johnston, a n d Yantis 1992).
The overall picture that has emerged from these studies is that when
attention is directed to a location in space, a spatial gradient is established
around the attended location such that items near it are processed more
efficiently than comparably visible items elsewhere. The time course of
selection by location d e p e n d s on whether the deployment of attention is
deliberate a n d controlled or an “automatic’’ consequence of a peripheral
Determinants of Attentional Control
visual onset. These spatiotemporal constraints on the deployment of
selective attention are very likely imposed by hard-wired properties of
the visual system such as the receptive field structure and the temporal
precision of early vision.
3.2 SELECTION BY FEATURE
According to Marr (1980, p. 3), the purpose of vision is to “know what is
where by looking.’’ This might imply simply opening one’s eyes to see
what is present, but often it entails searching for a particular object (e.g.,
red berries). While selection by location (either by moving the eyes or
through covert shifts of attention) is sometimes a reasonable strategy
because one knows where to look, one may also seek objects with known
visual properties (e.g., round and red) but u n k n o w n location, which sug-
gests that selection by feature is possible.
Among the first to investigate this issue, von Wright (1970) asked
whether selection in the partial report paradigm used by Sperling (1960)
could be based on simple attributes other than location, such as color, or
more abstract properties, such as meaning. von Wright found that atten-
tion could be guided efficiently by simple features (e.g., “Report the
names of the red letters’’), but not so efficiently by meaning (e.g., “Report
the names of the vowels’’). Corroborating evidence from studies of
visual search by Neisser (1967) and by Egeth, Jonides, and Wall (1972)
showed that simple shape differences (e.g., searching for a 4 among Cs)
could be used to direct attention efficiently.
In their seminal paper on search for features or conjunctions of fea-
tures, Treisman and Gelade (1980) found that “feature search,’’ in which
the target differs from nontargets in a single salient property (e.g., search
for a red target among green nontargets) was much more efficient (as
measured by visual search slopes) than was “conjunction search,’’ in
which the target w a s defined by the conjunction of two properties (e.g.,
search for a red vertical target—a conjunction of color and orientation—
among red horizontal a n d green vertical nontargets). They were able to
account for the efficiency of visual search in these tasks by proposing a
framework called “feature integration theory.’’ By offering a specific func-
tion for attention, the theory led to a surge in research on visual selection
during search.
The central claims of feature integration theory were, first, that the
visual system represents simple visual features like distinct colors a n d
orientations in separate feature m a p s (roughly consistent with the neuro-
physiological results of Hubel and Wiesel 1968, a n d many others since);
and, second, that the function of attention is to bind together the sepa-
rately represented features belonging to a given object via their common
spatial locations. According to the theory, feature search is efficient
because one need only monitor, say, a “red map’’ and if any activation
Yantis
occurs there, a positive response can be made; attention need not be
devoted to spatial locations one at a time. When, however, a target is
defined by a conjunction of features (e.g., red vertical target among red
horizontal and green vertical nontargets), search is inefficient because
attention must be directed to one location in the scene at a time, binding
the features at that location a n d allowing a decision about whether the
representation so created is the target.
Feature integration theory, in its original form, held that visual selec-
tion was essentially an unguided spatial search, at least in conjunction
search tasks. Egeth, Virzi, a n d Garbart (1984) showed, however, that even
conjunction search could be guided to some extent. They asked observers
to search for targets defined by a conjunction of features (e.g., a red O in
a field of black Os and red Ns) a n d found that search could be restricted
to the red target among the Os, or the O target among the red items.
Subsequent experiments verified and expanded on this finding (e.g.,
Driver, McLeod, a n d Dienes 1992; Nakayama a n d Silverman 1986; Wolfe,
Cave, and Franzel 1989; see Wolfe 1998 for a comprehensive review).
Wolfe a n d colleagues (Wolfe 1994; Cave a n d Wolfe 1990, Wolfe, Cave,
a n d Franzel 1989) proposed a theory of visual search called “guided
search.’’ Although strongly influenced by feature integration theory,
guided search takes into account the guidance by feature values revealed
in studies such as Egeth, Virzi, a n d Garbart 1984. An initial parallel stage
represents items in features maps (as in feature integration theory), a n d
then assigns priorities to items according to two criteria: items differing
significantly from their neighbors in any given dimension (e.g., color or
orientation) receive high bottom-up activation and items similar to the
target in any given dimension receive high top-down activation. These
two types of activation are combined in a priority m a p that determines
the order of search. The second stage (again as in feature integration
theory) involves selecting an item, binding its features into an object
representation, a n d making a decision about whether it is the target. The
order in which items are selected is determined by the priority m a p . Even
though search is strictly serial, the guidance provided by the priority m a p
yields efficient search under many circumstances where feature integra-
tion theory w o u l d have predicted inefficient search. For example, guided
search provides a straightforward account of the results of Egeth, Virzi,
a n d Garbart (1984).
Is Location Special?
The research reviewed thus far suggests that one can select an object by
directing attention to a location (either randomly or according to an atten-
tional priority schedule). In some sense, guided search holds that an
item’s features can guide search, but only indirectly through locations
that have been assigned high attentional priority in the activation m a p .
Determinants of Attentional Control
These studies do not reveal whether an item can be selected directly by
virtue of its having a particular feature value (e.g., the red object). Among
the few theories of attention that offer a mechanism for purely feature-
based selection is Bundesen’s “theory of visual attention’’ (1990), which
holds that only the discriminability of values within a feature dimension
affects the efficiency of selection; all dimensions, including location, are
assumed to be otherwise equivalent.
There is now substantial evidence, however, that feature-guided selec-
tion typically operates by directing attention to a spatial location con-
taining the target-defining feature value (e.g., Tsal and Lavie 1993). In
their investigation of this issue, Shih and Sperling (1996) asked whether
selection by feature was possible without spatial mediation. On each trial
of their experiment, several circular arrays of six letters were presented in
rapid succession in the same location, each replacing the previous one.
One array contained a single digit, a n d subjects were to report its iden-
tity, location, a n d color. In the alternating-feature condition, the letters in
each array were of the same color, but the color of the arrays alternated
(e.g., red, green, red, green); the target w a s known to be, say, red with
high probability. In this condition, if feature-based selection w a s possible,
an improvement in performance should be observed (relative to a base-
line in which the target’s color is unknown) because at least some of the
green items should have been rejected. In the feature-defined location
condition, an array consisted of five red items and one green item, alter-
nating with arrays containing five green items and one red item. Again,
the target w a s k n o w n to be, say, red with high probability. Shih a n d
Sperling found that w h e n spatial selection was impossible (in the
alternating-feature condition), knowledge of the target’s feature did not
improve performance at all. In contrast, when attention could be directed
to a location, as in the feature-defined location condition, performance
improved dramatically. They concluded that feature-based selection is
mediated by location. Several other reports corroborate this conclusion
(e.g., Cepeda et al. 1998; Johnston and Pashler 1990; Moore a n d Egeth
1998; Tsal and Lavie 1993, 1988; but see van der Heijden et al. 1996). These
studies suggest that location should not be viewed as just another feature
dimension; instead, it is the m e d i u m in which all features are expressed
a n d therefore enjoys a privileged status in visual selection (as Kubovy
1981 p u t it, location is an “indispensable attribute’’ in vision).
A potential exception to this claim is worth noting. O’Craven et al.
(1997) reported evidence using functional magnetic resonance imaging
that observers can selectively attend to stimuli exhibiting motion or sta-
tionarity, respectively. Their display consisted of a field of black a n d
white dots on a gray background; the white dots were moving as a con-
verging flow field toward fixation, while the black dots remained sta-
tionary. The observer was to shift attention every 20 seconds from the
white dots to the black dots a n d vice versa. Activation in the cortical
Yantis
Figure 3.2 Displays a n d data from Theeuwes 1992. To p . Solid contours are green, dashed
contours are red. In this example, subjects were to press a button corresponding to the ori-
entation (vertical or horizontal) of the line segment contained within the shape singleton
(i.e, the circle). In the “no distractor’’ condition, all the shapes were the same color (in this
case, green). In the “color distractor’’ condition, one nontarget shape w a s a different color
(here, red). Display size 7 is illustrated. Bottom. Response time w a s significantly slowed in
the color distractor condition, suggesting that the color distractor captured attention despite
its irrelevance to the task.
motion area MT was strongly modulated by the observer’s attentive
state. The authors concluded that selective attention to motion per se, not
just to a particular spatial location, w a s possible.
Salience and Attentional Capture by Feature Singletons
An item unique in some feature dimension (e.g., a red item in a scene con-
taining only blue items) is subjectively salient, a n d is sometimes said to
“ p o p out’’ of the display. The possibility that such feature singletons may
capture attention in a purely stimulus-driven fashion has proven to be a
contentious issue. A series of studies by Theeuwes a n d colleagues start-
ing in the early 1990s suggested that salient feature singletons indeed
capture attention despite strategic efforts to the contrary. In particular,
Theeuwes showed that when an observer searches for a singleton ele-
ment, then singletons in a different feature dimension capture attention
even though they are known to be irrelevant (see Theeuwes, Atchley, a n d
Kramer, chap. 4, this volume). In Theeuwes 1992, subjects were to report
Determinants of Attentional Control
the orientation of a target line segment (horizontal or vertical) that
appeared within a green circle, presented together with 4, 6, or 8 green
diamonds (figure 3.2). On half the trials, one of the green diamond dis-
tractors w a s replaced by a red diamond, and observers were told that this
color singleton distractor could never contain the target and should be
ignored. Theeuwes found that the presence of this distracting singleton
slowed responses (figure 3.2, bottom), suggesting that even a singleton in
an irrelevant visual dimension may capture attention. He concluded that,
when searching for a target singleton—an important proviso—there is
virtually no top-down control over attention: stimuli will be attended in
order of their salience.
A similar result was reported earlier by Pashler (1988): the time
required to find a target differing in orientation from the nontargets (i.e.,
an orientation singleton) w a s slowed by the presence of an irrelevant
color singleton. These findings were interpreted by Bacon a n d Egeth
(1994) as specifically reflecting the observer’s intent to select items that
are distinct from their neighbors in some feature dimension (see
Nothdurft 1993). Calling this state of attentional readiness “singleton
detection mode,’’ they argued that, in such a state, an observer relies on
a mechanism that computes the magnitude of local feature difference, but
that does not supply the identity of the singleton’s feature dimension or
value (e.g., shape or circle). Search is somewhat unselective in this case:
if one must rely on an item’s status as a feature singleton, one cannot
restrict search to the circle singleton or to the green singleton; instead, one
selects the item that differs most from its neighbors, and this may not be
the target of search (as in figure 3.2). Singleton detection m o d e is thus
only effective u n d e r circumstances in which the target happens to be the
most salient element in the display.
Bacon and Egeth (1994) supported this idea by showing that when sub-
jects are prevented from using singleton detection m o d e in tasks such as
those in Theeuwes 1992—for example, by using displays in which there
were multiple instances of the target so that the target was no longer a
singleton—irrelevant singletons no longer produce the distraction effect
observed by Theeuwes. Thus the apparent capture of attention by feature
singletons appears to be the result of a deliberate search strategy adopted
by observers and that sometimes yields inefficient search. This is a clear
example of h o w a top-down selective strategy is modulated by an early,
autonomous visual process such as the computation of local feature
contrast.
In a refinement of this conclusion, Folk a n d Remington (1998) sug-
gest that there are at least two possible causes for the sort of slowing
observed by Theeuwes (1992, 1994, 1996). The first is the account offered
by Theeuwes himself: w h e n searching for a shape singleton, the local
salience of each element in the display is computed, a n d attention is
directed spatially to the most salient element without regard to its featu-
ral identity. If this happens to be a color singleton, then responses are
82 Yantis
slowed because additional time is required to redirect attention to the
next most salient element, a n d so forth. Folk a n d Remington offer a sec-
on d possible mechanism for the interference observed by Theeuwes: the
presence of a distracting singleton may slow the deployment of attention
to the target item by requiring an effortful a n d time-consuming operation
to filter the distractor, but this may not entail a shift of spatial attention to
the distractor’s location.
In order to determine which of these two possible sources of the
interference effect is operative in Theeuwes’s experiments, Folk a n d
Remington (1998) employed a paradigm used successfully by Folk a n d
colleagues (e.g., Folk, Remington, and Johnston 1992) to study the inter-
action between stimulus-driven and goal-directed attentional control
(discussed in greater detail below). Their approach was to ask observers
to search for a target that differed from nontargets in a single dimension
(e.g., a red target among white nontargets, thus inducing an attentional
set for red elements). A distracting display was briefly presented before
the target display; this display could contain a red singleton or a single-
ton in some other dimension, a n d that singleton could appear at the same
location as the target or not. Folk and colleagues h a d already demon-
strated that a distracting singleton that matched the target-defining fea-
ture (in this case, a red distractor) slowed performance when it did not
appear in the target location and speeded performance when it appeared
in the upcoming target location (relative to a nondistractor baseline
condition); this pattern is taken as showing that the distractor captured
visual attention.
In the present case, Folk a n d Remington (1998) observed that a non-
matching distractor (e.g., green distractor when searching for a red tar-
get) failed to show position effects (that is, response time w a s the same
whether the distractor appeared in the target location or elsewhere) a n d
yet it did produce an overall slowing relative to a no-distractor baseline.
Thus distracting feature singletons were shown to have two dissociable
effects: if they matched the target’s defining feature, then they showed
location-specific effects, suggesting that they influenced the deployment
of spatial attention; if they did not match the target’s defining feature,
then they produced a filtering cost that w a s not spatially specific.
Theeuwes, Atchley, and Kramer (chap. 4, this volume) have argued that
the paradigm used by Folk and Remington (1998) probed the deployment
of attention too late to reveal an early spatial capture of attention that
could be overridden by top-down attentional control. Nevertheless,
attentional capture by an irrelevant but salient feature singleton in the
experiments described thus far has been observed only when the target
of search is itself a feature singleton. This sort of attentional capture must
therefore be viewed as a stimulus-driven modulation of a top-down
selection strategy (i.e., singleton detection mode).
In a study that does not appear to involve a strategic singleton detec-
tion mode, Joseph a n d Optican (1996) used a paradigm similar to the
83 Determinants of Attentional Control
one invented by Folk, Remington a n d Johnston (1992). Observers were
required to search for a target L shape in a dense array of T shapes—a
difficult search task that would not be expected to evoke singleton detec-
tion m o d e because the target consists of a particular arrangement of ori-
ented line segments (Beck a n d Ambler 1972). The target array w a s flashed
briefly a n d then masked, and the task w a s to report in which quadrant of
the display the L appeared. A distractor array preceding the target display
consisted of a single vertical (or horizontal) bar embedded in an array of
horizontal (or vertical) bars. This orientation singleton appeared at one of
the possible target locations, but was unpredictive of the target’s location.
Subjects were told to ignore the distractor because it contained no rele-
vant information. Localization accuracy was substantially greater when
the target appeared in the location previously occupied by the distractor
singleton than otherwise; Joseph and Optican concluded that attention
was involuntarily d r a w n to the location of the singleton, which suggests
that, at least under some conditions, irrelevant feature singletons may
capture attention. On the other hand, Hendel and Egeth (1998) found that
even the difficult search for an L in an array of Ts may cause observers to
adopt a search strategy in which oriented bars are task relevant: when the
target was a color singleton, an orientation singleton distractor failed to
capture attention. Thus here, too, feature singletons apparently capture
attention only when they are part of the subject’s search strategy.
Folk a n d colleagues (Folk, Remington, a n d Johnston 1992; Folk a n d
Remington 1998; Folk, Remington, a n d Wright 1994) have suggested that
all deployments of attention, including those that may appear to be
purely stimulus driven, are necessarily implementations of a top-down
attentional control setting. The idea is that all organisms are at all times
perceptually set for some input, and this perceptual set biases the visual
system to give higher priority to sensory representations satisfying the
contents of the current attentional set. As suggested by Hendel a n d Egeth
(1998), the findings of Joseph a n d Optican (1996) can be viewed as aris-
ing from an intent to attend to a particular combination of vertical a n d
horizontal bars, and because the singleton location alone contained, say,
a vertical bar, the attentional set caused a seemingly “involuntary’’ shift
of attention to it.
In all of their experiments on this topic, Folk a n d colleagues adopted
the following experimental approach (see figure 3.3). Subjects were
explicitly instructed to search a multielement array for an object defined
by one or more features (e.g., a red element among white elements), a n d
to report some other property of that object (e.g., its shape). Instructions
are presumed to establish a known a n d well-defined attentional control
setting in the observer, a current top-down state of attentional readiness
that will influence the observers’ perception in the upcoming events.
Each trial then consisted of two parts: a to-be-ignored distractor display
a n d a closely following target display. In Folk, Remington, a n d Johnston
Yantis
Figure 3.3 Procedure used by Folk, Remington, and Johnston (1992). On each trial, sub-
jects were to press one key if the target was an “=’’ and another key if it was an “X.’’ The
target was defined as the only stimulus in the display (onset target) or as the uniquely col-
ored element in the display (color target). Each target display was preceded by either an
onset or a color cue.
1992, exp. 1, the target display consisted of two “X’’s and two “ = ”s; the
primary task was to report whether the element with the target-defining
feature (say, red) was an “X’’ or an “ = .’’ The distractor display consisted
of elements clearly distinguishable from the targets: a cluster of four
small dots surrounding one or more of the potential target locations. One
of the distractor clusters could either match the target-defining feature
(e.g., it, too, could be a red element) or have another irrelevant feature
(e.g., abrupt onset). The distractor display appeared briefly (e.g., for 50
msec), followed after 100 msec by the target display. Subjects were told to
ignore the distractor because typically it would not occur in the target
location and was therefore irrelevant to the task. Folk and colleagues
found that response time was longer when the distractor and target loca-
tions were different than when they were the same; this indicated that the
distractor drew attention even though subjects were instructed to ignore
it. However, this pattern only obtained when the distractor matched the
target-defining feature; that is, a color distractor failed to draw attention
when the target was defined by abrupt onset. Subsequent experiments
verified and extended this observation. Folk and colleagues concluded
that attentional capture is often, perhaps always, a manifestation of some
top-down attentional set.
Although the claims of Folk and colleagues seem to directly contradict
those of Theeuwes (e.g., 1994), w h o has asserted that there is no top-down
control of attention when subjects are engaged in “preattentive visual
search,’’ the conflict may be more apparent than real. Theeuwes has
shown quite clearly that if one is to search for a target that differs from its
85 Determinants of Attentional Control
neighbors in some dimension, the feature difference computation only
represents the magnitude of the difference, a n d not the identity of the
dimension exhibiting that difference. In this sense, there is limited top-
d o w n control. The adoption of singleton detection m o d e is a strategic
choice, however, and therefore represents a clear case of an interaction
between top-down strategic control a n d modulating stimulus factors.
Several studies have shown that w h e n a feature singleton is com-
pletely task irrelevant (both in its identity and, critically, in that subjects
need not enter singleton detection m o d e to find the target), then the pres-
ence of a salient singleton distractor has virtually no effect on perfor-
mance, suggesting that feature singletons do not autonomously capture
attention in a purely stimulus-driven fashion. For example, Yantis a n d
Egeth 1999 asked subjects to search for a target that was difficult to dis-
criminate from nontargets (in this case, a vertical bar among bars tilted
slightly to the left a n d right), so that singleton detection m o d e was not a
viable strategy. In a control condition, the tilted target was always colored
red a n d the nontargets blue, a n d this yielded highly efficient search,
verifying that the color difference w a s sufficient to be labeled “salient.’’ In
the experimental condition, one item w a s always red, but because it was
only rarely the target, there was no incentive for subjects to use it to guide
attention. Response times to color singleton targets were no faster than to
nonsingleton targets, suggesting that the singleton failed to d r a w atten-
tion. Other examples of this result have been reported (Folk and Annett
1994; Gibson a n d Jiang 1998; Hillstrom a n d Yantis 1994; Jonides a n d
Yantis 1988; a n d Theeuwes 1990; for counterexamples, see Todd a n d
Kramer 1994; Theeuwes a n d Burger 1998). In other words, the salience of
feature singletons apparently does not control the deployment of atten-
tion unless it is licensed to do so by the adoption of singleton detection
m o d e .
3.3 SELECTION BY SEGMENTED OBJECT
Kahneman and Henik (1981, 183) asked the following prescient question:
“If attention selects a stimulus, what is the stimulus that it selects?’’ The
standard answer at the time would have been that attention selects the
contents of a spatial location; an attentional “spotlight’’ illuminates a con-
vex region of space. Our everyday commerce with the world, however,
involves interactions with segmented objects, not with empty locations or
with free-floating features. Although objects occupy spatial locations, we
are sometimes faced with a scene in which two objects spatially overlap
one another (e.g., a cat that is partly occluded by foliage); in such cases,
visual selection of an object via its location is not straightforward. It
seems possible to select one object a n d ignore another even w h e n the two
objects occupy a common two-dimensional spatial location. This is the
advantage offered by early scene segmentation.
Yantis
Object-based selection also imposes a stimulus-driven constraint on
the implementation of a goal-directed selection strategy. As I stated at the
beginning of this chapter, among the earliest hard-wired computations
carried out by the visual system is the segmentation of a scene into its
constituent objects and the separation of figure from background: the per-
ceptual organization of the spatiotemporally fragmented retinal image
(e.g., parts of a cat intermingled with parts of occluding foliage) into a
collection of coherent object representations (a single cat whose head,
legs, a n d tail are linked by their common motion, color, texture, depth,
a n d collinear contour). The principles of perceptual organization articu-
lated by the Gestalt psychologists in the early part of this century (e.g.,
proximity, similarity, common fate) describe h o w image features guide
grouping a n d segmentation (see Nakayama, He, and Shimojo 1996 for a
recent review) . For example, edges that are collinear will tend to be per-
ceived as bounding a common object even if they are partly occluded by
an intervening surface; image regions with the same color, texture, a n d
motion will tend to be perceived as part of a common surface; and so
forth. These grouping mechanisms are autonomous and indeed may
require cognitive effort to override, as suggested by Rensink and Enns
1998, discussed earlier (figure 3.1).
Among the first to demonstrate the constraints on selection imposed
by scene segmentation, Duncan 1984 clearly articulated the distinction
between space-based a n d object-based theories of visual selection. In
Duncan’s experiments, a display containing two superimposed objects
(an outline square and a tilted line) was flashed briefly a n d followed by
a mask. Each object h a d two attributes (e.g., line tilt and texture) with two
possible values per attribute (e.g., tilt left or right). Subjects were to report
one or two attributes, a n d in the latter case, the two attributes could
belong to the same object or come from two different objects. Duncan
found that whereas there w a s little cost in accuracy for reporting two
attributes from the same object, compared to reporting just a single attrib-
ute, there was a larger cost when the two attributes came from different
objects. He suggested that when an object is selected, all of its attributes
automatically become available for report. When attributes from two dif-
ferent objects are to be reported, there is a time cost associated with select-
ing the second object.
The effects of object segmentation appear to occur even when the task
does not require it. For example, in Egly, Driver, and Rafal 1994, a display
containing two parallel outline rectangles was presented vertically to the
right a n d left of fixation or horizontally above and below fixation (figure
3.4). Attention was cued to one end of one rectangle by brightening
the contours of the rectangle in that region. Shortly thereafter, one end of
one rectangle was filled in, a n d this target event h a d to be detected as
rapidly as possible. The cue w a s valid on most trials, and response time
(RT) w a s shortest when the target appeared in the cued location. On
Determinants of Attentional Control
Figure 3.4 Sample displays from Egly, Driver, a n d Rafal 1994. Each trial began with a 100
msec cue, indicating one end of one of the rectangles, followed after 200 msec by a target,
the filling in of one end of a rectangle. As illustrated in the t o p row, the cue was valid on
75% of the trials. On the 25% of the trials where the cue was invalid, the target appeared
within the cued object on half the trials (same-object condition) a n d within the uncued
object on the other half (different-object condition). The distances between the cued location
a n d each of the t w o uncued locations were the same.
trials where the cue was invalid, however, RT was shorter when the tar-
get w a s in the uncued end of the cued object (the same-object condition)
than when it w a s in the uncued object (the different-object condition).
This object-specific advantage occurred even though both locations were
equidistant from the cued location a n d equally likely to contain a target,
a n d even though there w a s no need to respect object boundaries in this
task. Moore, Yantis, a n d Vaughan (1998) observed a similar object-specific
benefit for targets appearing in uncued regions of a cued object even
w h e n the object was partly occluded. Behrmann, Zemel, a n d Mozer
(1998), Lavie a n d Driver (1996), a n d Vecera a n d Farah (1994) have
reported related corroborating evidence. Thus the process of perceptual
organization operating on visual scenes proceeds automatically a n d
influences the attentional priorities within the scene even when segmen-
tation into distinct objects is not part of the current attentional set.
Several other studies (e.g., Bacon a n d Egeth 1991; Baylis a n d Driver
1993; Grossberg, Mingolla, a n d Ross 1994; H u m p h r e y s a n d Müller 1993;
Treisman 1982), have shown that the Gestalt principles governing the
construction of perceptual object representations systematically influence
visual selection. Such influences are manifestly object-based a n d not
space-based ones, a n d they provide a further instance of stimulus-driven
constraints modulating top-down control settings. Duncan a n d H u m -
88 Yantis
phreys’s attentional engagement theory (1989, 1992) emphasizes the role
of perceptual grouping in visual search. According to the theory, the
similarities among targets and nontargets determine the efficiency of
selection in two ways. First, when target items are similar to nontarget
items, search will be inefficient because the detectability of the targets will
be low in a signal detection–theoretic sense. In other words, the targets
will tend to be grouped with the nontargets, making selection difficult.
Second, when nontarget items are similar to one another, search will be
relatively efficient because similar items are grouped into structures
whose constituents are treated similarly. If one is to reject (or suppress the
representation of) an item because it contains features known to be task
irrelevant, then all other items grouped with that item are also going to
be suppressed (what Duncan and Humphreys call “spreading suppres-
sion’’). This promotes efficient search when the nontargets are all similar
because they can be grouped a n d rejected all at once.
Much empirical support exists for the role of perceptual grouping in
visual selection. Baylis a n d Driver 1992 showed that the identification of
a central red target letter w a s influenced more by the (conflicting or con-
gruent) identity of distant red distractor letters than by adjacent green
distractor letters. Here grouping by color similarity caused the red letters
to be perceived “together,’’ even though color similarity was not relevant
to the task. Other examples of this sort include Driver a n d Baylis 1989
a n d Kramer a n d Jacobson 1991.
Thus we see that although perceptual objects can be selected according
to a top-down selection criterion, object-based selection seems to require
that the object or perceptual group be selected or rejected as a whole,
bottom-up, even when only a single part or attribute is desired.
3.4 STIMULUS-DRIVEN ATTENTIONAL CAPTURE
As we have shown, certain highly efficient forms of visual search (e.g.,
search for a red object in an array of green objects) sometimes produce
the subjective impression that the target item effortlessly “pops out’’ of
the display. In these cases, however, the feature singleton is the target of
search, or the subject has entered singleton detection mode, which
amounts to much the same thing. The observer is thus deliberately
searching for that stimulus, a n d there is almost certainly a goal-directed
component to the search strategy. In other words, such searches cannot be
characterized as purely stimulus driven. The question then remains
whether any search is purely stimulus driven.
My colleagues and I (Remington, Johnston, and Yantis 1992; Yantis a n d
Hillstrom 1994; Yantis and Johnson 1990; Yantis a n d Jonides 1984, 1990;
see also Oonk a n d Abrams 1998) have argued that an abrupt visual onset
enjoys high priority in vision and often captures attention in the absence
of a specific attentional set for abrupt onset. Our studies were designed
Determinants of Attentional Control
Figure 3.5 Displays a n d data from Jonides a n d Yantis 1988. To p . Each trial began with the
presentation of a target letter for that trial (not shown), followed by a set of six figure-eight
placeholders presented for 1 second. At the end of this interval, a subset of the line segments
in some of the figure eights disappeared to reveal letters. In conditions with display size 3
a n d 5, some of the figure-eight placeholders disappeared altogether. The test display con-
tained one abrupt onset letter and 2, 4, or 6 no-onset letters (display size 5 is illustrated).
Subjects were to press one of t w o buttons to indicate whether the specified target was pres-
ent or absent. The target was the onset item on 1 / n of the trails, where n is display size.
Bottom. Response time for trials in which the target was the onset item did not increase with
display size, suggesting that the onset item captured attention despite its irrelevance to the
task.
specifically to ask whether a visual event would capture attention w h e n
it was explicitly not part of the observers’ state of attentional readiness.
In the experiments of Yantis a n d Jonides (1984, 1990; Jonides a n d Yantis,
1988; see figure 3.5), for example, the task was to search for a prespecified
target letter in a multielement array containing one element that
appeared abruptly in a previously blank location (the onset element) a n d
several other elements that were present but camouflaged before the
appearance of the search array (the no-onset elements). The display items
were easily confusable, which typically would require an inefficient
90 Yantis
serial search for the specified letter. Because the target happened to
have an abrupt onset only rarely, there was no incentive for observers to
adopt an attentional set that conferred high priority on such elements.
Nevertheless, we found that when the target was the onset element, RT
was short a n d did not depend on the number of elements in the display,
whereas when one of the no-onset elements w a s the target, RT increased
almost linearly with the number of elements in the display (figure 3.5,
bottom). This pattern strongly suggests that the abrupt onset element
captured attention in a purely stimulus-driven fashion. This distin-
guishes abrupt onset from other salient features, such as color or bright-
ness singletons, that do not capture attention (e.g., Yantis a n d Egeth 1999;
see section 3.2).
The capture produced by abrupt onsets is not absolute, however. Yantis
a n d Jonides (1990) presented a central arrow at various moments in time
before a search display was to appear. The target of search was likely (in
some experiments, certain) to appear in the location indicated by the
arrow (eye position w a s monitored to ensure that fixation w a s main-
tained). An abrupt onset always appeared at the same time as the target,
sometimes in the expected (cued) location, and sometimes elsewhere. If
an abrupt visual onset captures attention regardless of the observers’
attentive state (in this case, their spatial focus of attention), then we
would expect performance to be disrupted (in this case, slowed) when
the onset appeared at an uncued location, reflecting the involuntary cap-
ture of attention by the onset, followed by the effortful redeployment of
attention to the target location. Instead, Yantis a n d Jonides (1990) found
that w h e n sufficient time was provided to shift attention in advance to the
cued location, a n d when the predictive validity of the cue was high
enough, capture by an abrupt onset was averted. That the cue could over-
ride attentional capture by the abrupt onset only w h e n the cue was pre-
dictive is crucial evidence of goal-directed attentional control, rather than
of competition between two abrupt onsets (i.e., the cue a n d the onset
letter). Several other studies (e.g., Juola, Koshino, a n d Warner 1995;
Koshino, Warner, and Juola 1992; Müller a n d Rabbitt 1989; and Theeuwes
1991) have corroborated the conclusion that deliberate deployments of
attention can prevent capture by abrupt onset. It remains an open ques-
tion whether the low-level, reflexive neural responses to abrupt onsets
such as those discussed by Rafal et al. (chap. 6, this volume) still occur
but are dominated by the top-down attentional set, or are suppressed
entirely by top-down deployments of attention.
Jonides a n d Yantis 1988, Hillstrom a n d Yantis 1994, and Yantis a n d
Egeth 1999 have demonstrated that the uniqueness of the onset element
per se cannot be the crucial factor in the observed attentional capture;
they showed that highly salient but uninformative feature singletons in
dimensions other than onset (e.g., color) do not d r a w attention (see sec-
tion 3.2), whereas an abrupt onset does. In an effort to determine the
Determinants of Attentional Control
mechanism for attentional capture by abrupt onsets, Yantis a n d Hillstrom
(1994) considered two possibilities. First, attentional capture might be
mediated by the abrupt luminance change associated with the onset
letter, which would implicate a low-level visual mechanism sensitive to
the spatiotemporal profile of the stimulus. Alternatively, the appearance
of a new perceptual object, independent of the luminance change, might
d r a w attention automatically.
Yantis and Hillstrom found that the appearance of a target letter
defined by equiluminant discontinuities in texture, motion, or depth nev-
ertheless captures attention, showing that a new object is sufficient to cap-
ture attention without a luminance increment (see also Oonk a n d Abrams
1998; Gellatly, Cole, and Blurton 1999). According to recent studies in our
lab, new objects defined by equiluminant discontinuities in color, using
the flicker photometry method, also capture attention.1 These studies
show that luminance change is not necessary to produce attentional cap-
ture by new perceptual objects, but rather that the appearance of new
objects alone can capture attention.
Moreover, Enns, Yantis, and Di Lollo (1998) have shown that a lumi-
nance change is not sufficient to capture attention. Subjects were asked to
search for a target letter (E or H) in an array of black and white letters on
a gray background. In an initial control experiment, we verified that
when one of the letters appeared in a previously blank location among
no-onset letters, it captured attention, even though the new object was
not predictive of the target location. In other words, the heterogeneity of
the letter colors (some black a n d some white) did not affect the standard
result that new objects capture attention. A second experiment then
examined search performance when all of the stimuli were no-onset
letters. At the moment the camouflage w a s removed from the figure-eight
placeholders to reveal the letter forms, one of the objects exhibited a
polarity reversal (e.g., black to white or white to black). Although the
luminance change in this case was at least as much as that exhibited by
the onset letter in the control condition, the element undergoing the lumi-
nance change nevertheless failed to capture attention. Thus luminance
change is not sufficient to produce attentional capture by abrupt onsets.
Together, these t w o lines of evidence suggest that the appearance of a
new object captures attention, not by virtue of the luminance change that
typically accompanies it, but because the visual system is predisposed to
attend to the creation of a new perceptual representation in a purely
stimulus-driven fashion.
This claim has not gone unchallenged. Miller (1989) a n d Martin-
Emerson and Kramer (1997) have reported that contour offsets can com-
pete to some extent for attention with abrupt onsets. Folk, Remington,
a n d Johnston (1992) found that w h e n searching for a color singleton
target, a preceding peripheral onset cue failed to capture attention; this
may well be an instance of top-down control over attentional capture by
Yantis
abrupt onset analogous to the findings of Yantis a n d Jonides (1990).
Recent experiments by Gellatly, Cole, a n d Blurton (1999) have shown
that a new object defined by equiluminant discontinuities in motion
failed to capture attention, suggesting that new objects do not always
capture attention, although there is a question about the strength of
the object representation in the equiluminant case. Generally speaking,
items defined by equiluminant discontinuities in dimensions other than
luminance are difficult to see, as evidenced by the relatively slow re-
sponse times observed in these tasks. It would not be surprising if a near-
threshold object failed to capture attention w h e n it appeared.
Folk, Remington, a n d Johnston (1992, 1993; see also Yantis 1993) have
argued that attentional capture by new objects is a result of an implicit
attentional control setting for abrupt onset, based on the assumption that
there is a subtle contingency in the experimental procedure that encour-
ages subjects to selectively attend to luminance change (each trial begins
with a luminance change, for example), and that capture by abrupt onset
is therefore a side effect of top-down attentional control. This assumption
is undermined, however, by the experiments of Enns, Yantis, and Di Lollo
(1998), which showed no attentional advantage for items reversing their
polarity at the beginning of each trial, even though the procedures were
precisely analogous to the onset case.
Folk, Remington, a n d Johnston (1992) have also argued that there may
be a “default’’ attentional control setting for new perceptual objects that
operates in the absence of any specific feature-based attentional set. This
suggests, of course, that the visual system is predisposed, perhaps even
hard-wired, to treat new perceptual objects with higher priority than
other attributes. That the visual system should have this bias for new
objects is hardly surprising: new perceptual objects are of obvious behav-
ioral significance a n d such an “early warning system’’ (Breitmeyer a n d
Ganz 1976, 31) for new objects w o u l d be expected to increase reproduc-
tive fitness. As we have seen, however, even this form of stimulus-driven
attentional capture is subject to some degree of top-down modulation
(Yantis and Jonides 1990).
3.5 INTERACTIVE CONTROL OF VISUAL ATTENTION
The experiments reviewed in this chapter reveal that most instances of
visual selection involve an interaction between top-down attentional
control a n d autonomous neural responses to visual stimuli. For example,
when directing attention to locations in space, we can readily observe
constraints on the spatial a n d temporal precision of selection imposed by
neuroanatomical and neurophysiological properties of the brain. In the
case of search for a feature singleton, the adoption of singleton detection
m o d e is a deliberate strategy that has implications for the efficiency
with which a target can be found, and for whether items to be ignored
Determinants of Attentional Control
will d r a w attention. Autonomous scene segmentation a n d perceptual
grouping mechanisms cause all the features of an object to be selected
as a unit, whether that is part of the attentional goal or not. And while
the appearance of new perceptual objects can capture attention in the
absence of a specific intent to attend to such changes, top-down control
can be exerted to avert such capture. Overall, the evidence suggests that
purely stimulus-driven attentional capture is rare; instead, interactions
between top-down attentional control settings and stimulus-driven fac-
tors that modulate deliberate control are the rule.
H o w might current behavioral goals interact with early visual modules
to yield the observed influences on selection? The goals that drive top-
d o w n attentional deployment are presumably contained in working
memory representations of the observer’s current task. In most cases
reviewed here, the current task is stipulated by the instructions conveyed
to the participant in an experimental psychology laboratory. These mem-
ory representations generally contain the target-defining features (e.g.,
the red item), the reported attribute of the target (e.g., its presence or its
name), the manual or vocal responses associated with the various possi-
ble outcomes of search, the contingencies in the experimental design (e.g.,
the relative probabilities with which various objects will appear), along
with expectations about the properties of the nontargets, the display in
general, a n d other aspects of the testing session. All of these are aspects
of the observer’s explicit state of attentional readiness. Other factors
that may influence the implementation of attentional goals may include
long-term or implicit memory representations, such as the participant’s
memory of previous similar experiences, together with autonomous early
perceptual mechanisms, such as perceptual organization and object seg-
mentation or pure attentional capture by new perceptual objects.
The stored memory representation of the task at hand comprises an
attentional set, which gates the neural representation of the sensory
input. For example, if there is a positional expectancy (e.g., “Name the
letter appearing four degrees to the right of the present point of fixa-
tion’’), then the neurons with receptive fields in that location may be
primed to receive input, those with receptive fields elsewhere may be
suppressed if a target appears in the expected location, or both (Moran
a n d Desimone 1985). Duncan and colleagues (Desimone and Duncan
1995; Duncan, Humphreys, a n d Ward 1997) have articulated an approach
to this problem in which multiple brain systems exhibit competitive
responses to inputs (i.e., a given brain region will tend to represent one
object at a time and suppress representations of other objects), whereas
cooperative integration across brain regions will tend to yield concurrent
activation of the same object. According to this idea, there is no one place
where attention originates; instead, it is an emergent property of the com-
petition and cooperation among multiple brain regions. Recent neuro-
biological and computational models of attention (e.g., Mozer a n d Sitton
1998; Niebur, Koch, and Rosin 1993; Olshausen, Anderson, and Van Essen
Yantis
1993; Tsotsos 1995; Usher and Niebur 1996) suggest how this neural gat-
ing might be implemented.
Although it is well known that there are massive feedback pathways in
the brain from higher centers to early visual areas (e.g., Van Essen a n d
DeYoe 1995), details of the mechanism by which memory representations
of the current behavioral goal modulate sensory responses are not well
understood. Rather than discuss any specific proposal, I will simply out-
line the properties any candidate mechanism must have. First, there must
be a working memory representation that specifies task requirements a n d
relevant stimulus attributes (much recent evidence points to prefrontal
regions as being particularly involved in such representation; e.g.,
Courtney et al. 1998; Goldman-Rakic 1995). Miller (chap. 22, this volume)
provides neurophysiological evidence that prefrontal neural representa-
tions are modulated by task d e m a n d s . Second, there must be direct or
indirect feedback connections between the neural representation for the
current attentional set (including the properties of the desired object, its
probable location, or both) and the early visual areas whose responses are
subject to attentional modulation, including V1 and V2 (Gandhi, Heeger,
a n d Boynton 1999; Motter 1993), extrastriate areas including V4 (e.g.,
Connor et al. 1997; Hopfinger et al., chap. 5, this volume; Motter 1994;
Moran and Desimone 1985), IT (e.g., Miller, Li, and Desimone 1991), MT
(e.g., Beauchamp, Cox, and DeYoe 1997; O’Craven et al. 1997; Treue a n d
Maunsell 1996), a n d LIP (e.g., Gottlieb, Kusuoki, and Goldberg 1998).
Candidate areas that appear to have the requisite feedback connections
a n d that are active during attentive tasks include the posterior parietal
cortex (e.g., Bushnell, Goldberg, a n d Robinson 1981; Corbetta et al. 1993;
Mountcastle, Anderson, a n d Motter 1981), parts of the thalamus, includ-
ing the pulvinar a n d the reticular formation (Crick 1984; LaBerge 1995,
Olshausen, Andersen, a n d Van Essen 1993), and some prefrontal areas
(Miller, chap. 22, this volume).
Visual pathways specialized to represent rapidly changing input (the
M-pathway) may well mediate stimulus-driven attentional capture by
new perceptual objects, which can in turn produce efficient control over
eye movements (Rafal et al., chap. 6, this volume). Early scene segmen-
tation mechanisms, presumably operating primarily in occipital a n d
occipital-temporal areas, further constrain the implementation of selec-
tion strategies (Driver 1995).
Despite recent efforts to characterize attentional control in terms of
neural systems, the problem of translating task requirements into specific
attentional goals has not yet been solved. The working memory repre-
sentations that specify attentional set remain as givens, outside the scope
of most models. We are usually left with a “central executive’’ to sort out
multiple competing goals. Although tackling this problem is among the
most compelling challenges we face in this area, any attempt to explain
the efficiency of visual selection must confront the constraints imposed by
bottom-up factors on the successful implementation of behavioral goals.
95 Determinants of Attentional Control
NOTES
Preparation of this chapter was supported by National Institute of Mental Health grant R01-
MH43924. I thank Howard Egeth for many valuable discussions, a n d Jon Driver, Nilli Lavie,
Stephen Monsell, and Jan Theeuwes for valuable comments on an earlier version.
1. Theeuwes (1995) has reported that the appearance of a new object equiluminant with its
background failed to capture attention. For technical reasons, however, the duration of the
“old’’ items in the display was only 50 msec (as compared to 1,000 msec in many previous
studies). Thus the new object may not have been perceived as “new’’ relative to these
fairly new “old’’ objects, thereby preventing it from capturing attention.
REFERENCES
Atchley, P., Kramer, A. F., Andersen, G. J., and Theeuwes, J. (1997). Spatial cuing in a stereo-
scopic display: Evidence for a “depth-aware’’ attentional spotlight. Psychonomic Bulletin and
Review, 4, 524–529.
Bacon, W. F., and Egeth, H. E. (1991). Local processes in preattentive feature detection.
Journal of Experimental Psychology: Human Perception and Performance, 17, 77–90.
Bacon, W. F., a n d Egeth, H. E. (1994). Overriding stimulus-driven attentional capture.
Perception and Psychophysics, 55, 485–496.
Bahcall, D. O., and Kowler, E. (1998). Attentional interference at small spatial separations.
Vision Research, 39, 71–86.
Baylis, G. C., and Driver, J. S. (1992). Visual parsing a n d response competition: The effect of
grouping factors. Perception and Psychophysics, 51, 145–162.
Baylis, G. C., a n d Driver, J. S. (1993). Visual attention and objects: evidence for hierarchical
coding of locations. Journal of Experimental Psychology: Human Perception and Performance, 19,
451–470.
Beauchamp, M. S., Cox, R. W., a n d DeYoe, E. A. (1997). Graded effects of spatial a n d
featural attention on h u m a n area MT and associated motion processing areas. Journal of
Neurophysiology, 78, 516–520.
Beck, J., and Ambler, B. (1972). Discriminability of differences in line slope and in line
arrangement as a function of mask delay. Perception and Psychophysics, 12, 201–204.
Behrmann, M., Zemel, R., and Mozer, M. (1998). Object-based attention and occlusion:
Evidence from normal subjects and a computation model. Journal of Experimental Psychology:
Human Perception and Performance, 24, 1011–1036.
Breitmeyer, B. G., a n d Ganz, L. (1976). Implications of sustained and transient channels for
theories of visual pattern masking, saccadic suppression, and information processing.
Psychological Review, 83, 1–36.
Bundesen, C. (1990). A theory of visual attention. Psychological Review, 97, 523–547.
Bushnell, M. C., Goldberg, M. E., a n d Robinson, D. L. (1981). Behavioral enhancement of
visual responses in monkey cerebral cortex: 1. Modulation in posterior parietal cortex
related to selective visual attention. Journal of Neurophysiology, 46, 755–772.
Cave, K. R., and Wolfe, J. M. (1990). Modeling the role of parallel processing in visual search.
Cognitive Psychology, 22, 225–271.
Cepeda, N. J., Cave, K. R., Bichot, N. P., a n d Kim, M. S. (1998). Spatial selection via feature-
driven inhibition of distractor locations. Perception and Psychophysics, 60, 727–746.
Yantis
Cheal, M. L., a n d Lyon, D. R. (1991). Central and peripheral precuing of forced-choice dis-
crimination. Quarterly Journal of Experimental Psychology, 43A, 859–880.
Colegate, R. L., Hoffman, J. E., and Eriksen, C. W. (1973). Selective encoding from multi-
element visual displays. Perception and Psychophysics, 14, 217–224.
Connor, C. E., Preddie, D. C., Gallant, J. L., a n d Van Essen, D. C. (1997). Spatial attention
effects in macaque area V4. Journal of Neuroscience, 17, 3201–3214.
Corbetta, M., Miezin, F. M., Shulman, G. L., and Petersen, S. E. (1993). A PET study of visuo-
spatial attention. Journal of Neuroscience, 13, 1202–1226.
Courtney, S. M., Petit, L., Maisog, J. M., Ungerleider, L. G., and Haxby, J. V. (1998). An area
specialized for spatial working memory in h u m a n frontal cortex. Science, 279, 1347–1351.
Crick, F. (1984). The function of the thalamic reticular complex: The search-light hypothesis.
Proceedings of the National Academy of Sciences, U.S.A., 81, 4586–4590.
Desimone, R., a n d Duncan, J. (1995). Neural mechanisms of selective visual attention.
Annual Review of Neuroscience, 18, 193–222.
Downing, C. J., a n d Pinker, S. (1985). The spatial structure of visual attention. In M. Posner
a n d O. Martin (Eds.), Attention and Performance XI, p p . 171–187. Hillsdale, NJ: Erlbaum.
Driver, J. (1995). Object segmentation a n d visual neglect. Behavioural Brain Research 71,
135–146.
Driver, J. S., and Baylis, G. C. (1989). Movement and visual attention: The spotlight meta-
phor breaks d o w n . Journal of Experimental Psychology: Human Perception and Performance,
15, 448–456.
Driver, J., McLeod, P. , a n d Dienes, Z. (1992). Motion coherence and conjunction search:
Implications for guided search theory. Perception and Psychophysics, 51, 79–85.
Duncan, J. (1984). Selective attention and the organization of visual information. Journal of
Experimental Psychology: General, 113, 501–517.
Duncan, J., a n d H u m p h r e y s , G. W. (1989). Visual search a n d stimulus similarity.
Psychological Review, 96, 433–458.
Duncan, J., a n d Humphreys, G. W. (1992). Beyond the search surface: Visual search a n d
attentional engagement. Journal of Experimental Psychology: Human Perception and Per-
formance, 18, 578–588.
Duncan, J., Humphreys, G. W., a n d Ward, R. (1997). Competitive brain activity in visual
attention. Current Opinion in Neurobiology, 7, 255–261.
Egeth, H., Jonides, J., a n d Wall, S. (1972). Parallel processing of multielement displays.
Cognitive Psychology, 3, 674–698.
Egeth, H. E., Virzi, R. A., and Garbart, H. (1984). Searching for conjunctively defined targets.
Journal of Experimental Psychology: Human Perception and Performance, 10, 32–39.
Egly, R., Driver, J., a n d Rafal, R. D. (1994). Shifting visual attention between objects and loca-
tions: Evidence from normal and parietal lesion subjects. Journal of Experimental Psychology:
General, 123, 161–177.
Egly, R., a n d Homa, D. (1984). Sensitization of the visual field. Journal of Experimental
Psychology: Human Perception and Performance, 10, 778–793.
Engel, F. L. (1971). Visual conspicuity, directed attention and retinal locus. Vision Research,
11, 563–576.
Determinants of Attentional Control
Enns, J. T., Yantis, S., a n d Di Lollo, V. (1998). Luminance transients capture attention, but
only in “new’’ display locations. Paper presented at the Thirty-eighth Annual Meeting of
the Association for Research in Vision a n d Ophthalmology, Fort Lauderdale, FL, May.
Eriksen, C. W., and Hoffman, J. E. (1972). Temporal a n d spatial characteristics of selective
encoding from visual displays. Perception and Psychophysics, 12, 201–204.
Eriksen, C. W., and Hoffman, J. E. (1973). The extent of processing noise elements during
selective encoding from visual displays. Perception and Psychophysics, 14, 155–160.
Eriksen, C. W., and Yeh, Y. (1985). Allocation of attention in the visual field. Journal of
Experimental Psychology: Human Perception and Performance, 11, 583–597.
Folk, C. L., and Annett, S. (1994). Do locally defined feature discontinuities capture atten-
tion? Perception and Psychophysics, 56, 277–287.
Folk, C. L., a n d Remington, R. W. (1998). Selectivity in distraction by irrelevant featural sin-
gletons: Evidence for two forms of attentional capture. Journal of Experimental Psychology:
Human Perception and Performance, 24, 847–858.
Folk, C. L., Remington, R. W., a n d Johnston, J. C. (1992). Involuntary covert orienting is con-
tingent on attentional control settings. Journal of Experimental Psychology: Human Perception
and Performance, 18, 1030–1044.
Folk, C. L., Remington, R. W., and Johnston, J. C. (1993). Contingent attentional capture: A
reply to Yantis (1993). Journal of Experimental Psychology: Human Perception and Performance,
19, 682–685.
Folk, C. L., Remington, R. W., and Wright, J. H. (1994). The structure of attentional control:
Contingent attentional capture by apparent motion, abrupt onset, a n d color. Journal of
Experimental Psychology: Human Perception and Performance, 20, 317–329.
Gandhi, S. P. , Heeger, D. J., and Boynton, G. M. (1999). Spatial attention affects brain activ-
ity in h u m a n primary visual cortex. Proceedings of the National Academy of Sciences USA, 96,
3314–3319.
Gellatly, A., Cole, G., a n d Blurton, A. (1999). Do equiluminant object onsets capture visual
attention? Journal of Experimental Psychology: Human Perception and Performance, 25,
1609–1624.
Gibson, B. S., a n d Jiang, Y. (1998). Surprise! An unexpected color singleton does not capture
attention in visual search. Psychological Science, 9, 176–182.
Ghirardelli, T. G., a n d Folk, C. L. (1996). Spatial cuing in a stereoscopic display: Evidence for
a “depth-blind’’ attentional spotlight. Psychonomic Bulletin and Review, 3, 81–86.
Goldman-Rakic, P. S. (1995). Architecture of the prefrontal cortex and the central executive.
In J. Grafman, K. J. Holyoak, a n d F. Boller (Eds.), Structure and functions of the human pre-
frontal cortex, p p . 71–83. N e w York: New York Academy of Sciences.
Gottlieb, J. P. , Kusunoki, M., a n d Goldberg, M. E. (1998). The representation of visual
salience in monkey parietal cortex. Nature, 391, 481–484.
Grossberg, S., Mingolla, E., a n d Ross, W. D. (1994). A neural theory of attentive visual
search: Interactions of boundary, surface, spatial, and object representations. Psychological
Review, 101, 470–489.
Helmholtz, H. von (1866). Treatise on physiological optics. 3d ed. Vol. 3, ed. a n d trans. J. P. C.
Southhall. Reprint, Washington, DC: Optical Society of America, 1925.
Hendel, S. K., a n d Egeth, H. E. (1998). Orientation singletons do not automatically capture
attention. Poster presented at the thirty-ninth Annual Meeting of the Psychonomic Society,
Dallas, TX.
Yantis
Henderson, J. M., a n d Macquistan, A. D. (1993). The spatial distribution of attention fol-
lowing an exogenous cue. Perception and Psychophysics, 53, 221–230.
Hillstrom, A. P. , and Yantis, S. (1994). Visual motion a n d attentional capture. Perception and
Psychophysics, 55, 399–411.
Hoffman, J. E. (1979). A two-stage model of visual search. Perception and Psychophysics, 25,
319–327.
Hoffman, J. E., and Nelson, B. (1981). Spatial selectivity in visual search. Perception and
Psychophysics, 30, 283–290.
Hubel, D. H., and Wiesel, T. N. (1968). Receptive fields and functional architecture of mon-
key striate cortex. Journal of Physiology (London), 195, 215–243.
Humphreys, G. W., and Müller, H. J. (1993). Search via recursive rejection (SERR): A con-
nectionist model of visual search. Cognitive Psychology, 25, 43–110.
Iavecchia, H. P., a n d Folk, C. L. (1995). Shifting visual attention in stereographic displays: A
timecourse analysis. Human Factors, 36, 606–618.
Johnson, D. N., a n d Yantis, S. (1995). Allocating visual attention: Tests of a two-process
model. Journal of Experimental Psychology: Human Perception and Performance, 21, 1376–1390.
Johnston, J. C., and Pashler, H. (1990). Close binding of identity and location in visual fea-
ture perception. Journal of Experimental Psychology: Human Perception and Performance, 16,
843–856.
Jonides, J. (1980). Towards a model of the mind’s eye’s movement. Canadian Journal of
Psychology, 34, 103–112.
Jonides, J. (1981). Voluntary versus automatic control over the mind’s eye’s movement. In
J. B. Long a n d A. D. Baddeley (Eds.), Attention and Performance IX, p p . 187–203. Hillsdale,
NJ: Erlbaum.
Jonides, J. (1983). Further toward a model of the mind’s eye’s movement. Canadian Journal
of Psychology, 34, 103–112.
Jonides, J., a n d Yantis, S. (1988). Uniqueness of abrupt visual onset in capturing attention.
Perception and Psychophysics, 43, 346–354.
Joseph, J. S., and Optican, L. M. (1996). Involuntary attentional shifts d u e to orientation dif-
ferences. Perception and Psychophysics, 58, 651–665.
Juola, J. F., Koshino, H., a n d Warner, C. B. (1995). Tradeoffs between attentional effects of
spatial cues a n d abrupt onsets. Perception and Psychophysics, 57, 333–342.
Kahneman, D., Henik, A. (1981). Perceptual organization a n d attention. In M. Kubovy a n d
J. R. Pomerantz (Eds.), Perceptual organization, p p . 181–211. Hillsdale, NJ: Erlbaum.
Kim, M. S., and Cave, K. R. (1995). Spatial attention in visual search for features and feature
conjunctions. Psychological Science, 6, 376–380.
Koshino, H., Warner, C. B., and Juola, J. F. (1992). Relative effectiveness of central, periph-
eral, a n d abrupt-onset cues in visual search. Quarterly Journal of Experimental Psychology,
45A, 609–631.
Kramer, A. F., a n d Jacobson, A. (1991). Perceptual organization a n d focused attention: The
role of objects and proximity in visual processing. Perception and Psychophysics, 50, 267–284.
Kubovy, M. (1981). Concurrent-pitch segregation and the theory of indispensable attributes.
In M. Kubovy and J. R. Pomerantz (Eds.), Perceptual organization, p p . 55–98. Hillsdale, NJ:
Erlbaum.
Determinants of Attentional Control
LaBerge, D. (1983). The spatial extent of attention to letters and words. Journal of Experi-
mental Psychology: Human Perception and Performance, 9, 371–379.
LaBerge, D. (1995). Attentional processing. Cambridge, MA: Harvard University Press.
LaBerge, D., a n d Brown, V. (1986). Variations in size of the visual field in which targets are
presented: An attentional range effect. Perception and Psychophysics, 40, 188–200.
LaBerge, D., Brown, V. Carter, M., Bash, D., and Hartley, A. (1991). Reducing the effects of
adjacent distrators by narrowing attention. Journal of Experimental Psychology: Human
Perception and Performance, 17, 90–95.
LaBerge, D., Carlson, R. L., Williams, J. K., a n d Bunney, B. G. (1997). Shifting attention in
visual space: Tests of moving-spotlight models versus an activity distribution model.
Journal of Experimental Psychology: Human Perception and Performance, 23, 1380–1392.
Lavie, N., and Driver, J. (1996). On the spatial extent of attention in object-based visual selec-
tion. Perception and Psychophysics, 58, 1238–1251.
Marr, D. (1980). Vision. San Francisco: W. H. Freeman and Co.
Martin-Emerson, R., a n d Kramer, A. F. (1997). Offset transients modulate attentional capture
by s u d d e n onsets. Perception and Psychophysics, 59, 739–751.
Miller, E. K., Li, L., and Desimone, R. (1991). A neural mechanism for working and recogni-
tion memory in inferior temporal cortex. Science, 254, 1377–1379.
Miller, J. (1989). The control of attention by abrupt visual onsets and offsets. Perception and
Psychophysics, 45, 567–571.
Moore, C. M., and Egeth, H. (1998). How does feature-based attention affect visual process-
ing? Journal of Experimental Psychology: Human Perception and Performance, 24, 1296–1310.
Moore, C. M., Yantis, S., and Vaughan, B. (1998). Object-based visual selection: Evidence
from perceptual completion. Psychological Science, 9, 104–110.
Moran, J., and Desimone, R. (1985). Selective attention gates visual processing in the extra-
striate cortex. Science, 229, 782–784.
Motter, B. C. (1993). Focal attention produces spatially selective processing in visual cortical
areas V1, V2, and V4 in the presence of competing stimuli. Journal of Neurophysiology, 70,
909–919.
Motter, B. C. (1994). Neural correlates of attentive selection for color or luminance in extra-
striate area V4. Journal of Neuroscience, 14, 2178–2189.
Mountcastle, V., Anderson, R., and Motter, B. (1981). The influence of attentive fixation u p o n
the excitability of the light-sensitive neurons of the posterior parietal cortex. Journal of
Neuroscience, 1, 1218–1232.
Mozer, M. C., and Sitton, M. (1998). Computational modeling of spatial attention. In H.
Pashler (Ed.), Attention, p p . 341–393. London: Psychology Press.
Müller, H. J., and Rabbitt, P. M. A. (1989). Reflexive and voluntary orienting of visual atten-
tion: Time course of activation and resistance to interruption. Journal of Experimental
Psychology: Human Perception and Performance, 15, 315–330.
Nakayama, K., He, Z. J., and Shimojo (1996). Visual surface representation: A critical link
between lower-level and higher-level vision. In S. M. Kosslyn and D. N. Osherson (Eds.), An
invitation to cognitive science: Visual cognition, p p . 1–70. Cambridge, MA: MIT Press.
Nakayama, K., and Silverman, G. H. (1986). Serial and parallel processing of visual feature
conjunctions. Nature, 320, 264–265.
Yantis
Nakayama, K., and Mackeben, M. (1989). Sustained a n d transient components of focal
visual attention. Vision Research, 29, 1631–1647.
Neisser, U. (1967). Cognitive psychology. N e w York: Appleton-Century-Crofts.
Niebur, E., Koch, C., a n d Rosin, C. (1993). An oscillation-based model for the neuronal basis
of attention. Vision Research, 33, 2789–2802.
Nothdurft, H. C. (1993). Saliency effects across dimensions in visual search. Vision Research,
33, 839–844.
O’Craven, K. M., Rosen, B. R., Kwong, K. K., Treisman, A., and Savoy, R. L. (1997). Voluntary
attention modulates fMRI activity in h u m a n MT-MST. Neuron, 18, 591–598.
Olshausen, B. A., Anderson, C. H., a n d Van Essen, D. C. (1993). A neurobiological model of
visual attention and invariant pattern recognition based on dynamic routing of information.
Journal of Neuroscience, 13, 4700–4719.
Oonk, H. M., and Abrams, R. A. (1998). New perceptual objects that capture attention pro-
duce inhibition of return. Psychonomic Bulletin and Review, 5, 510–515.
Pashler, H. (1988). Cross-dimensional interaction and texture segregation. Perception and
Psychophysics, 43, 307–318.
Posner, M. I. (1978). Chronometric explorations of mind. Englewood Cliffs, NJ: Erlbaum.
Posner, M. I. (1980). Orienting of attention. Quarterly Journal of Experimental Psychology, 32,
3–25.
Posner, M. I., Snyder, C. R. R., and Davidson, B. J. (1980). Attention a n d the detection of sig-
nals. Journal of Experimental Psychology: General, 109, 160–174.
Remington, R. W., Johnston, J. C., a n d Yantis, S. (1992). Involuntary attentional capture by
abrupt onsets. Perception and Psychophysics, 51, 279–290.
Rensink, R. A., and Enns, J. T. (1998). Early completion of occluded objects. Vision Research,
38, 2489–2505.
Shaw, M. L. (1978). A capacity allocation model for reaction time. Journal of Experimental
Psychology: Human Perception and Performance, 4, 586–598.
Shih, S. I., and Sperling, G. (1996). Is there feature-based attentional selection in visual
search? Journal of Experimental Psychology: Human Perception and Performance, 22, 758–779.
Sperling, G., (1960). The information available in brief visual presentations. Psychological
Monographs, 74, 1–29.
Theeuwes, J. (1990). Perceptual selectivity is task-dependent: Evidence from selective
search. Acta Psychologica, 74, 81–99.
Theeuwes, J. (1991). Exogenous and endogenous control of attention: The effect of visual
onsets and offsets. Perception and Psychophysics, 49, 83–90.
Theeuwes, J. (1992). Perceptual selectivity for color a n d form. Perception and Psychophysics,
51, 599–606.
Theeuwes, J. (1994). Endogenous and exogenous control of visual selection. Perception, 23,
429–440.
Theeuwes, J. (1995). Abrupt luminance change p o p s out; abrupt color change does not.
Perception and Psychophysics, 57, 637–644.
Theeuwes, J. (1996). Perceptual selectivity for color and form: On the nature of the inter-
ference effect. In A. F. Kramer, M. Coles, a n d G. Logan (Eds.), Converging operations in the
Determinants of Attentional Control
study of visual selective attention, p p . 297–314. Washington, DC: American Psychological
Association.
Theeuwes, J., and Burger, R. (1998). Attentional control during visual search: The effect of
irrelevant singletons. Journal of Experimental Psychology: Human Perception and Performance,
24, 1342–1353.
Todd, S., a n d Kramer, A. F. (1994). Attentional misguidance in visual search. Perception and
Psychophysics, 56, 198–210.
Treisman, A. (1982). Perceptual grouping a n d attention in visual search for features a n d for
objects. Journal of Experimental Psychology: Human Perception and Performance, 8, 194–214.
Treisman, A., a n d Gelade, G. (1980). A feature-integration theory of attention. Cognitive
Psychology, 12, 97–136.
Treue, S., and Maunsell, J. H. R. (1996). Attentional modulation of visual motion processing
in cortical areas MT and MST. Nature, 382, 539–541.
Tsal, Y., a n d Lavie, N. (1988). Attending to color a n d shape: The special role of location in
selective visual processing. Perception and Psychophysics, 44, 15–21.
Tsal, Y., and Lavie, N. (1993). Location dominance in attending to color and shape. Journal of
Experimental Psychology: Human Perception and Performance, 19, 131–139.
Tsotsos, J. K. (1995). Toward a computational model of visual attention. In T. V. Papathomas,
C. Chubb, A. Gorea, a n d E. Kowler (Eds.), Early vision and beyond, p p . 207–218. Cambridge,
MA: MIT Press.
Usai, M. C., Umiltà, C., and Nicoletti, R. (1995). Limits in controlling the focus of attention.
European Journal of Cognitive Psychology, 7, 411–439.
Usher, M., and Niebur, E. (1996). Modeling the temporal dynamics of IT neurons in visual
search: A mechanism for top-down selective attention. Journal of Cognitive Neuroscience, 8,
311–327.
van der Heijden, A. H. C., Kurvink, A. G., de Lange, L., de Leeuw, F., a n d van der Geest,
J. N. (1996). Attending to color with proper fixation. Perception and Psychophysics, 58,
1224–1237.
Van Essen, D. C., a n d DeYoe, E. A. (1995). Concurrent processing in the primate visual cor-
tex. In M. S. Gazzaniga (Ed.), The cognitive neurosciences, p p . 383–400. Cambridge, MA: MIT
Press.
Vecera, S. P., a n d Farah, M. J. (1994). Does visual attention select objects or locations? Journal
of Experimental Psychology: General, 123, 146–160.
von Wright, J. M. (1970). On selection in immediate visual memory. In A. F. Sanders (Ed.),
Attention and Performance III, p p . 280–292. Amsterdam: North-Holland.
Warner, C. B., Juola, J. F., and Koshino, H. (1990). Voluntary allocation versus automatic cap-
ture of visual attention. Perception and Psychophysics, 48, 243–251.
Wolfe, J. M. (1994). Guided search 2.0: A revised model of visual search. Psychonomic Bulletin
and Review, 1, 202–238.
Wolfe, J. M. (1998). Visual search. In H. Pashler (Ed.), Attention, p p . 13–73. London:
Psychology Press.
Wolfe, J. M., Cave, K. R., a n d Franzel, S. L. (1989). Guided search: An alternative to the fea-
ture integration model for visual search. Journal of Experimental Psychology: Human Perception
and Performance, 15, 419–433.
Yantis
Yantis, S. (1992). Multielement visual tracking: Attention and perceptual organization.
Cognitive Psychology, 24, 295–340.
Yantis, S. (1993). Stimulus-driven attentional capture a n d attentional control settings. Journal
of Experimental Psychology: Human Perception and Performance, 19, 676–681.
Yantis, S., a n d Egeth, H. E. (1999). On the distinction between visual salience a n d stimulus-
driven attentional capture. Journal of Experimental Psychology: Human Perception and Per-
formance, 25, 661–676.
Yantis, S., a n d Hillstrom, A. P. (1994). Stimulus-driven attentional capture: Evidence from
equiluminant visual objects. Journal of Experimental Psychology: Human Perception and
Performance, 20, 95–107.
Yantis, S., and Johnson, D. N. (1990). Mechanisms of attentional priority. Journal of
Experimental Psychology: Human Perception and Performance, 16, 812–825.
Yantis, S., a n d Jonides, J. (1984). Abrupt visual onsets and selective attention: Evidence from
visual search. Journal of Experimental Psychology: Human Perception and Performance, 10,
601–621.
Yantis, S., a n d Jonides, J. (1990). Abrupt visual onsets a n d selective attention: Voluntary
versus automatic allocation. Journal of Experimental Psychology: Human Perception and
Performance, 16, 121–134.
103 Determinants of Attentional Control
4 On the Time Course of Top-Down and Bottom-Up Control of Visual Attention
Jan Theeuwes, Paul Atchley, and Arthur F. Kramer
ABSTRACT Previous research showed that a salient feature singleton captured attention
bottom-up (Theeuwes 1991a, 1992, 1994a). A salient color singleton interfered with search
for a less salient shape singleton, which suggested that early processing was driven by
bottom-up saliency factors. The present experiments examined h o w bottom-up a n d top-
d o w n processing develops over time. Subjects searched for a shape singleton target and h a d
to ignore a color singleton distractor presented at different stimulus onset asynchronies
prior to the search display. The results indicate that when the target a n d distractor were pre-
sented simultaneously, the salient singleton distractor captured attention, whereas when
the distractor singleton was presented about 150 msec before the target singleton, the dis-
tractor did not disrupt performance. The findings suggest a stimulus-driven model of selec-
tion in which early processing is solely driven by bottom-up saliency factors. In later pro-
cessing, the early bottom-up activation of the distractor can be overridden by top-down
attentional control.
One of the most basic questions in the study of attention is the extent to
which top-down attentional control can prevent distraction from irrel-
evant stimuli. Visual selective attention is thought to be (1) goal directed
when attentional priority is given to only those objects and events that
are in line with the current goals of the observer: a n d (2) stimulus driven
when, irrespective of the intentions or goals of the observer, objects
a n d events involuntarily receive attentional priority—a phenomenon re-
ferred to as “attentional capture’’ (for recent reviews, see Egeth and Yantis
1997; Theeuwes 1993, 1994b; Yantis, 1993, 1996). These two mechanisms
of selection have been referred to as “top-down’’ and “bottom-up” atten-
tional control, respectively, (e.g., Eriksen and Hoffman 1972; Posner 1980;
Theeuwes 1991b; Yantis a n d Jonides 1984).
Many models of visual search assume that visual selection is the result
of an interaction between goal-directed a n d stimulus-driven factors (e.g.,
Cave a n d Wolfe 1990; Treisman a n d Sato 1990). Typically, it is assumed
that bottom-up activation occurs during early preattentive processing in
which the visual field is segmented into functional perceptual units.
Bottom-up activation is a measure of h o w salient an item is in its context.
An item that is locally unique in some basic visual dimension—usually
referred to as a “feature singleton’’ or simply a “singleton’’—will gener-
ate a large bottom-up activation (e.g., a red p o p p y in a field of green
grass). Top-down activation may also operate during attentional process-
ing. Various studies have demonstrated that in more complex search
tasks, knowledge of the specific task d e m a n d s may guide attention to
only those locations that match the target-relevant feature. For example,
Kaptein, Theeuwes, and van der Heijden (1995) showed that when
searching for a red vertical line segment between red tilted a n d green ver-
tical line segments, subjects searched serially among the red items while
ignoring the green line segments (see also Egeth, Virzi, and Garbart 1984).
Top-down guidance is typically assumed to proceed either by activation
of features that match those of the target (e.g., Wolfe 1994) or by inhibition
of features that do not (Treisman a n d Sato 1990).
In a series of experiments, Theeuwes (1991a, 1992, 1996) showed that a
salient feature singleton captured attention bottom-up. Even though sub-
jects h a d a clear top-down attentional set to search for a particular sin-
gleton, performance w a s disrupted by a distractor with a salient, unique
feature in a task-irrelevant dimension. Top-down control of attention
could not entirely override bottom-up interference from a singleton dis-
tractor known to be irrelevant. For example, Theeuwes (1992) presented
subjects with displays consisting of colored circles or diamonds appear-
ing on the circumference of an imaginary circle. Line segments of differ-
ent orientations appeared in the circles a n d diamonds. Subjects h a d to
determine the orientation of the line segment appearing in the target
shape. Subjects searched for a shape singleton, a single green diamond
among green circles. Time to find the shape singleton increased when an
irrelevant color singleton was present (i.e., one of the circles was red).
Even though subjects h a d a clear top-down set to search for the shape
singleton (i.e., the single green diamond), the presence of an irrelevant
singleton (i.e., the single red circle) caused interference. It was shown that
selectivity d e p e n d e d on the relative salience of the stimulus attributes:
when the color singleton was m a d e less salient than the shape singleton
(by reducing the color difference between the target a n d the nontarget
elements), the shape singleton interfered with search for the color single-
ton, whereas the color singleton no longer interfered with the search for
the shape singleton.
Based on these experiments, Theeuwes (1991a, 1992, 1994a, 1996) con-
cluded that early preattentive processing is driven by bottom-up factors
such as salience. Attention is captured by the most salient singleton in
the display, regardless of whether the property defining that singleton is
relevant for the task or not (for more recent evidence, see Bacon a n d
Egeth 1994, exp. 1; Caputo a n d Guerra 1998; Joseph a n d Optican 1996;
Kawahara and Toshima 1996; Kim a n d Cave 1999; Kumada 1999; Todd
a n d Kramer 1994). When engaged in parallel search for a particular fea-
ture singleton (e.g., a diamond among circles), the extent to which sin-
gletons capture attention is determined by the relative salience of the
singletons present in the visual field. It was suggested that, irrespective
Theeuwes, Atchley, a n d Kramer
of any top-down control, spatial attention is automatically and involun-
tarily captured by the most salient singleton. The shift of spatial attention
to the location of the singleton implies that the singleton is selected for
further processing. If this singleton is the target, a response is made. If it
is not the target, attention is directed to the next most salient singleton.
The initial shift of attention to the most salient singleton is thought to
be the result of relatively inflexible, “hard-wired’’ mechanisms, triggered
by the presence of these difference signal interrupts. Consistent with
proposals by Sagi and Julesz (1985) and Koch and Ullman (1985), it is
assumed that the parallel process can only perform local mismatch detec-
tion (i.e., indicating the presence of a discontinuity, but not its nature) fol-
lowed by a serial stage directed to areas of the visual field with the largest
magnitude mismatches.
Contrary to these findings, a group of other researchers have claimed
that the ability of a singleton to capture attention is contingent on
whether an attention-capturing stimulus is consistent with top-down set-
tings established “off-line’’ on the basis of current attentional goals (Folk,
Remington, and Johnston 1992; Folk and Remington 1998). According to
this “contingent capture’’ model, only stimuli that match the top-down
control settings will capture attention; stimuli that do not match the top-
d o w n settings will be ignored. Top-down control is thus possible even
when target and distractor are both singletons. Along these lines, it was
argued that in Theeuwes’s experiments the irrelevant singleton captured
attention because subjects were set to find a singleton (e.g., a local mis-
match) rather than a particular feature, such as a red circle (see Bacon and
Egeth 1994). It was claimed that irrespective of the bottom-up saliency
the singleton that matched the top-down setting would capture attention.
These claims are based on evidence from experiments in which subjects
had to ignore a cue that appeared 150 msec before the presentation of the
target display (Folk, Remington, and Johnston 1992). Subjects responded
to a character shape (X versus =) that, in different conditions, had either
a unique color or a unique abrupt onset. When the search display was
preceded by a to be ignored featural singleton (the cue) that matched the
singleton for which they were searching, the cue captured attention as
evidenced by a prolonged reaction time to identify the target (i.e., when
the cue and target appeared in different spatial locations). On the other
hand, if the to be ignored featural singleton cue did not match the single-
ton for which they were searching, its appearance apparently did not cap-
ture attention. This “contingent’’ capture of attention occurred for both
color and onset conditions, and is considered evidence that involuntary
capture is contingent on the adoption of some attentional set.
The critical finding in these studies is that a cue that does not match the
top-down search goal (i.e., the defining property of the target) does not
affect response time (RT), whereas a cue that matches the search goal
does. In other words, if subjects were searching for a red plus sign, they
Top-Down a n d Bottom-Up Control of Attention
were more likely to be distracted by a red cue than by an abrupt onset
cue, and vice versa. Folk, Remington, and Johnston (1992) have sug-
gested that the absence of an effect on RT for a cue that does not match
the target indicates that the cue did not capture attention. On the other
hand, the irrelevant cue may indeed have captured attention, but because
the cue display came on 150 msec before the search display, subjects may
have been able to overcome the attentional capture by the time the search
display was presented (see also Theeuwes 1994a,b). Disengagement of
attention from the cue may have been relatively fast when the cue a n d
target did not share the same defining properties (e.g., the cue is red a n d
the target is an onset), whereas disengagement from the cue may have
been relatively slow in the case where the cue a n d target share the same
defining properties (e.g., both were red). Such a mechanism could explain
w h y there are RT costs when the cue a n d target have the same defining
characteristics—and no costs when cue a n d target are different. This does
not imply, however, that there is no capture of attention by the irrelevant
cue singleton; it simply indicates that, after a certain time, subjects are
able to exert top-down control over the erroneous capture of attention by
the irrelevant singleton, to overcome its effects.
This account holds that early preattentive processing is driven by
solely bottom-up feature salience factors, generating an activation pat-
tern on which later attentive processing may then exert control to give
priority to elements that match the top-down attentional set. It thus
remains consistent with the claim of Theeuwes (1991a, 1992, 1994a, 1996)
that during early preattentive processing, top-down control is not possi-
ble. It is also in line with models of visual search suggesting that during
attentive processing either top-down inhibition may be applied to features
that match the distractors (Treisman a n d Sato 1990) or top-down activa-
tion, to features that match the target (Wolfe 1994).
If the model presented above is correct, it should be possible to reveal
h o w bottom-up a n d top-down processing develop over time. As in pre-
vious studies (e.g., Theeuwes 1992), subjects searched multielement dis-
plays for a shape singleton a n d reported the letter located inside the
shape singleton. On some trials, an irrelevant salient color singleton was
presented along with a premask display at different stimulus onset asyn-
chronies (SOAs) before the onset of the search display. When the target
a n d distractor singleton are presented close in time, a n d attention is cap-
tured by the distractor, search for the target singleton should be slowed.
If, however, the singleton distractor is presented well in advance of the
search display, subjects may be able to exert top-down control over the
irrelevant singleton, ensuring that, by the time of the arrival of the search
display, attention is directed to the target singleton. In these latter con-
ditions, there should be no effect of the singleton distractor on search
time.
Theeuwes, Atchley, a n d Kramer
4.1 EXPERIMENT 1
A visual search task similar to that in Theeuwes 1992 was employed,
where subjects h a d to search for a feature singleton, and where this
singleton is typically detected by means of preattentive parallel search.
Subjects searched for a shape singleton (a single gray diamond among
eight gray circles) a n d h a d to determine the orientation of the letter C (C
or reversed C) appearing in the diamond. Determining the orientation of
the letter C requires the allocation of focal attention to the location of the
shape singleton. In the distractor condition, one of the circles w a s red.
Because previous studies (see Theeuwes 1991a, 1992) have shown that
such a color singleton is more salient than a shape singleton, it was
expected that, in line with previous studies, the color singleton (i.e., the
distractor) w o u l d interfere with the search for the shape singleton (i.e.,
the target). To determine the time course of bottom-up and top-down
activation, the singleton distractor (the red circle) appeared at different
SOAs prior to the presentation of the target display.
Subjects
Twelve subjects, ranging in age from 18 to 30, participated as paid vol-
unteers. All had self-reported normal or corrected-to-normal vision a n d
reported having no color vision defects.
Apparatus
A 486 computer with an SVGA color monitor controlled the timing of the
events, generated stimuli a n d recorded reaction times. The “/’’-key a n d
the “z’’-key of the computer keyboard were used as response buttons. All
subjects were tested in a sound-attenuated, dimly lit room, with their
head resting on a chinrest. The monitor w a s located at eye level, 60 cm
from the chinrest.
Stimuli
Subjects performed a visual search task in which they searched for a
uniquely shaped element (a diamond located between circles) a n d
responded to the letter located inside this uniquely shaped singleton. The
display consisted of nine elements equally spaced around the fixation
point on an imaginary circle whose radius was 3.4 degrees. In the control
condition each display contained one gray outline diamond (1.4 degrees
on a side) surrounded by eight gray outline circles (1.4 degrees in
diameter). In the distractor condition, one of the gray outline circles
was replaced by a red circle producing a condition identical to that of
Theeuwes 1992, in which the target had a unique shape (shape singleton)
109 Top-Down a n d Bottom-Up Control of Attention
Figure 4.1 Trial events in experiment 1. The premask display (left panel) was presented for
700 msec. At stimulus onset asynchronies of 50, 100, 150, 200, 250, or 300 msec before the
presentation of the search display, the color of one of the elements of the premask changed
from gray into equiluminant red (middle panel). The search display (right side) contained
both the color singleton distractor (the red element) and a shape target singleton (the
diamond).
while the distractor had a unique color (color singleton). To ensure that
distractor effects were not due to attention encompassing both the target
singleton and the neighboring color singleton, the color singleton dis-
tractor was never placed adjacent to the target (i.e, there was always one
gray element between the target and color distractor).
Each display element contained a letter (0.4X0.8 degrees). The uniquely
shaped outline diamond (i.e., the target) contained either a C or a reversed
C, the orientation of the letter determining the response (subjects pressed
the “z’’-key for a C and the “/’’-key for a reversed C). The letters inside
the other eight circles were randomly sampled from the set E, P, F, U and
S. The letters were presented in white (11.0 c d / m 2 ) and the circle and dia-
mond were presented in gray (6.4 c d / m 2 ) . The color singleton distractor
was presented in red (6.3 c d / m 2 ) .
Design and Procedure
The sequence of events was as follows: Initially, a fixation dot was pre-
sented for 1,000 msec. Then the premask display came on consisting of
nine premask elements, each composed of a single outline circle and dia-
mond, and each containing a figure-eight premask letter (see figure 4.1).
The premask display was presented for 700 msec. The color of one of the
elements of the premasks changed from gray to equiluminant red with
SOAs of 50, 100, 150, 200, 250, or 300 msec before the presentation of the
search display. The search display was revealed by removing particular
diamonds or circles of the premask display resulting in a search display
consisting of eight circles and one diamond. Simultaneously with the
removal of the premask, the letters inside the outline elements were
110 Theeuwes, Atchley, a n d Kramer
Figure 4.2 Experiment 1: Mean RTs a n d error percentages as a function of stimulus onset
asynchrony (SOA) for the distractor a n d no-distractor conditions.
displayed by removing line elements from the figure eights. The search
display remained present for a maximum of 2 sec until a response was
emitted.
Each subject performed 240 trials, 120 no-distractor and 120 distractor
trials which were presented randomly within blocks of trials. SOA be-
tween premask and search display was varied randomly between trials
as well. Subjects were told to keep their eyes fixated at the fixation dot.
Subjects received 240 practice trials prior to the experimental trials, as
well as feedback about their performance in terms of RT and error rates
after each block of 60 trials. Prior to the start of the experiment, subjects
were instructed to search for the diamond and respond to the orientation
of the letter inside the diamond by pressing the appropriate response key.
They were told to ignore the uniquely colored red singleton.
Results
Response times longer than 1,200 msec were counted as errors, which led
to a loss of less than 1% of the trials. A one-way analysis of variance
(ANOVA) with no distractor or SOAs of 50, 100, 150, 200, 250, and 300
msec as levels showed a significant main effect: F(6, 66) = 2.3; p < 0.05.
Additional planned comparisons showed that the RT at SOAs of 50 msec
(731 msec) and 100 msec (742 msec) were significantly slower (p < 0.05)
than the RT in the no-distractor control condition (711 msec), indicating
that at the early SOA the singleton distractor interfered with search for
the target singleton. However, the RTs of the later SOAs (150, 200, 250,
and 300 msec) were not significantly different from the no-distractor con-
Top-Down a n d Bottom-Up Control of Attention
dition suggesting that in these conditions search for the target singleton
was not affected by the presence of the singleton distractor (see figure
4.2). Note that a distractor, when presented close in time to the target,
slows d o w n search by about 25 msec, an effect size very similar to that
reported in Theeuwes 1992. The error rates were low (about 4.9%) a n d
did not vary systematically with any of the conditions.
Discussion
The present results confirm earlier findings (e.g., Theeuwes 1991a, 1992)
that the presence of a irrelevant salient distractor interferes with search
for a relevant target singleton. The analysis of SOA suggests that there is
a reliable effect of the distractor at the early SOAs (50 and 100 msec) but
not at the later SOAs (150, 200, 250, 300 msec).
The results regarding SOA are in line with our predictions: at the early
SOAs w h e n distractor a n d target are presented in close succession, there
is a clear interference effect of the distractor. It was argued that in these
conditions, when target and distractor were presented in close temporal
proximity, there was not enough time to exert top-down control that
could have overcome attentional capture by the salient distractor. When,
however, the singleton distractor w a s presented a considerable time
(SOAs of 150 to 300 msec) before the presentation of the target singleton,
sufficient top-down control could be exerted that there was no sign of
attentional capture by the distractor. Indeed, response times at SOAs of
150 to 300 msec did not differ significantly from that in the no-distractor
condition.
4.2 EXPERIMENT 2
Experiment 1 suggests that early in processing attention is captured by
the salient distractor and that, later, attentional capture is overcome by
top-down attentional control. To determine whether spatial attention was
indeed captured by the distractor, we used the response congruency par-
adigm (Eriksen and Eriksen 1974; Eriksen and Hoffman 1972), in which
subjects have to ignore a stimulus that is either congruent or incongruent
with the response to the target. In previous studies (Theeuwes 1996;
Theeuwes and Burger 1998; Theeuwes et al. 1999) investigating whether
subjects could intentionally ignore salient but irrelevant singleton ele-
ments, the element to ignore was either identical to or different from the
target element they were looking for. The results showed that the identity
of the element to be ignored h a d an effect on response time suggesting
that indeed spatial attention was directed at the location of the distractor
element. Subjects were faster when the distractor element was identical
to the target (congruent with the response) than w h e n the distractor ele-
ment was different from the target (incongruent with the response).
Theeuwes, Atchley, a n d Kramer
To determine whether spatial attention was shifted to the location of
the color singleton distractor, we also presented a C or reversed C inside
the color singleton distractor at the various SOAs used here. This letter
was either identical with the letter inside the target shape singleton (and
therefore congruent with the response) or different from the letter inside
the target shape singleton (and therefore incongruent with the response).
If attention is indeed captured by the color singleton distractor, then the
identity of the letter inside the colored singleton distractor should have
an effect on responding, that is, a letter congruent with the response
should produce faster RTs than a letter incongruent with the response. If
attention is not captured by the colored singleton, then there should be no
congruency effect on RT.
Subjects
Fifteen subjects, ranging in age from 18 to 30, participated as paid
volunteers.
Stimuli
The stimuli were identical to those in experiment 1. The letter located
inside the irrelevant color singleton distractor was either a C or a reversed
C, and this could be congruent or incongruent with the target letter inside
the relevant shape singleton.
Design and Procedure
Only SOAs of 50, 100, 200, and 400 msec were used. In the current exper-
iment, there was always a red singleton distractor present in each display.
A congruent or incongruent letter was presented inside the distractor sin-
gleton.1 Note that the letter inside the singleton distractor was revealed
simultaneously with the red singleton distractor element. In other words,
the letter (which could be congruent or incongruent with the response)
was presented simultaneously with the singleton distractor and therefore
this letter was presented 50, 100, 200, or 400 msec before the presentation
of the other letters of the search display (including the target letter). SOA
was varied randomly within blocks of trials. Subjects received 240 prac-
tice trials and 240 experimental trials.
Results
Response times longer than 1,300 msec were counted as errors, which led
to a loss of 0.9% of the trials. An ANOVA with SOA (50, 100, 200, 400
msec) and congruency (congruent versus incongruent) as orthogonal
within subject factors showed an effect of SOA: F(3,42) = 7.8; p < 0.001;
Top-Down a n d Bottom-Up Control of Attention
Figure 4.3 Experiment 2: Mean RTs a n d error percentages as a function of stimulus onset
asynchrony (SOA) for the congruent a n d incongruent conditions.
and of congruency: F(1, 14) = 10.3; p < 0.001. The interaction between
SOA and congruency was also reliable: F(3, 42) = 3.37; p < 0.05. As is clear
from figure 4.3, response times become faster with increasing SOA, sug-
gesting that (in line with experiment 1) the effect of the singleton distrac-
tor diminishes with increasing SOA. Additional planned comparisons
show that a reliable congruency effect at SOA, 50 and 400 msec (p < 0.05)
and a marginally significant congruency effect at SOA 100 msec
(p = 0.07). At SOA 200 msec, congruency failed to reach significance
(p = 0.29; 678 msec versus 670 msec). Also, as is clear from figure 4.3, when
the letter inside the singleton distractor was congruent with the response
to the letter inside the target singleton response times were faster than
when it was incongruent. The finding that the letter inside the singleton
distractor did affect responding to the target singleton can only be
explained by assuming that at some point attention resided at the loca-
tion of the singleton distractor (but see Folk and Remington 1998). The
error rates were low (4.4%) and did not vary systematically with any of
the conditions.
Discussion
Experiment 2 shows that response latencies become shorter with increas-
ing SOA, suggesting again that presenting the distractor in advance of
the target overcomes attentional capture by the distractor. As in experi-
ment 1, the distractor seems to slow search by about 25 msec at the two
short SOAs (50 and 100 msec).
Theeuwes, Atchley, a n d Kramer
The overall congruency effect indicates that the identity of the letter
inside the singleton distractor had an effect on the response to the letter
appearing inside the target singleton. When the letter inside the distrac-
tor was identical to the letter inside the target singleton, and therefore
congruent with the response, response times were faster than when the
letter inside the distractor was incongruent with the response to the let-
ter inside the target singleton, a result identical to that in Theeuwes 1996.
These findings are consistent with attention being captured by the irrel-
evant singleton. Because capturing attention implies that focal attention
was directed to the irrelevant singleton, the identity of the letter became
available, thereby affecting the speed of responding to the target.
Folk and Remington (1998) have suggested an alternative explanation
for such findings. Instead of assuming that attention was captured by
the irrelevant singleton, they suggested that the congruency effect as
observed in Theeuwes 1996 and in Theeuwes and Burger 1998 was the
result of processing the target and distractor letter in parallel. Such an
explanation, though possible, is unlikely: at the eccentricities used in the
current experiments, letters cannot be processed efficiently in parallel for
discriminations such as C versus reversed C (see, for example, Theeuwes
1991c; Wolfe 1994). Usually, when subjects search for a target letter
among nontarget letters, search time increases linearly with the number
of nontarget letters in the display, a result typically seen as evidence for
spatially serial search. Given these considerations, the most likely expla-
nation is that the identity of the letter in the irrelevant singleton affected
responding because attention was directed at the location of the singleton
distractor before a response was made. In addition, the control experi-
ment (see note 1), in which a congruent or incongruent letter was placed
in a nonsingleton item, showed no effect of congruency (F = 1), provid-
ing evidence that the congruency effect only shows when attention is at-
tracted to the location of the colored singleton. This finding suggests that
parallel processing of all letters (including the congruent or incongruent
letter placed in the nonsingleton) is highly unlikely.
It is important to note that there is a clear congruency effect at SOA 400
msec (p = 0.0065). This finding is important because it implies that even
when the singleton distractor (with the congruent or incongruent letter
inside) is presented 400 msec before the presentation of the search dis-
play, attention was captured by the singleton. In other words (as demon-
strated in experiment 1), at SOAs of 200 msec, subjects had enough time
between the presentation of distractor and target, not to prevent atten-
tional capture, but to gain attentional control after their attention had
been erroneously captured by the salient distractor.
Another interesting finding is that at SOA 200 msec, the congruency
effect is absent (if anything, the effect is reversed). This suggests that to
gain attentional control subjects may have inhibited the singleton dis-
Top-Down a n d Bottom-Up Control of Attention
tractor location a n d thereby reduced the influence of the letter inside the
singleton distractor. Because of this inhibition, the letter inside the dis-
tractor no longer influences responding to the target letter. The fact that
the congruency effect is absent at SOA 200 msec but not at SOA 400 msec
suggests that the inhibition may be transient.
4.3 EXPERIMENT 3
The goal of experiment 3 was to investigate the possible role of inhibition
of the distractor color over trials. Experiment 3 w a s identical to experi-
ment 2 except that the color of the singleton distractor could be either red
or green a n d changed randomly from trial to trial. The results of experi-
ment 2 suggesting inhibition of the distractor at SOA 200 msec and not at
SOA 400 msec implies that the inhibition may be relatively short-lived. If
inhibition is relatively brief, then changing the color of the distractor from
trial to trial should produce the same pattern of effects as that observed
in experiment 2.
If, however, attentional set (e.g., in the sense of inhibiting a specific
color) is carried over from one trial to the next, then response latencies
should be faster when the singleton distractor has the same color as on
the previous trial than when it does not. Such a result would be consis-
tent with Maljkovic and Nakayama 1994, which showed that visual
search responses were faster when the color of the target singleton was
repeated from the previous trial than when it was changed. Subjects were
considered to be relatively fast on same-color trials because they could
retrieve an attentional set identical to the one used in the previous trial.
Although it is not clear whether such a repetition effect also occurs when
distractor rather than target colors are changed, if repeating the same
attentional set produced a general effect, then a repetition effect should
also be observed for the distractors in the present studies. Note that, as in
Maljkovic a n d Nakayama 1994, any repetition effect in the current exper-
iment cannot be a response-based effect because subjects did not respond
to the color, but to the letters inside the elements.
Subjects
Twelve subjects, ranging in age from 18 to 30, participated as paid
volunteers.
Stimuli
The stimuli were identical to those in experiment 2. The singleton dis-
tractor was either red or green and changed color randomly from trial to
trial.
Theeuwes, Atchley, a n d Kramer
Figure 4.4 Experiment 3: Mean RTs a n d error percentages as a function of SOA for the
congruent a n d incongruent conditions.
Design and Procedure
Subjects received 512 practice and 512 experimental trials.
Results
Response times longer than 1,300 msec were counted as errors, which led
to a loss of 1.1% of the trials. An ANOVA with SOA (50, 100, 200, 400
msec) and congruency (congruent versus incongruent) as factors
showed an effect of congruency: F(1, 11) =4.90; p<0.05; and of cong-
ruency X SOA: F(3, 33) = 6.54; p < 0.001. Planned comparisons indicate
that, for all SOAs, the difference between the congruent and incongruent
conditions is reliable (all p < 0.05). Note, however, that at SOA 200 msec
this effect is reversed (p = 0.02), that is, incongruent responses are faster
than congruent responses (see figure 4.4).
An additional analysis was carried out to determine whether changing
the color of the singleton distractor over trials had an effect on response
latencies. An ANOVA showed no effect of color change: F(1, 11) = 3.0;
p = 0.11; nor did distractor color change interact with any of the other
variables—color change X congruency: F(1, 11) = 0.07; color change X
SOA: F(3, 33) = 0.86. This suggests subjects were not able to carry over
the attentional set (including the color to inhibit) from the previous trial
in order to speed up responding. The error rates were low (5.5%) and did
not vary systematically with any of the variables.
Top-Down a n d Bottom-Up Control of Attention
Discussion
The finding that RTs in trials in which the distractor color switched were
the same as when the color remained the same suggests that attentional
set in the sense of which color to inhibit does not carry over from one trial
to the next. Unlike the findings in Maljkovic and Nakayama 1994, which
showed a repetition effect for the target color, the current findings indicate
that this does not hold for the distractor color. The results suggest that the
color to inhibit may not be part of the attentional set that transfers from
one trial to the next. Note, however, that in experiment 3 the target
remained fixed over trials. If specifying the target is the most important
feature of the attentional set, then one may argue that repetition effects of
the distractor color were not observed in experiment 3 because the target
remained the same. Future studies may address whether switching the
color of the distractor produces a repetition effect when the color of the
target also changes from trial to trial. Overall, consistent with the findings
of experiment 2 that overcoming of the distractor effect was relatively
short-lived, the current findings suggest that rejection of the relevant
color singleton does not transfer from one trial to the next.
The congruency effects are similar to those of experiment 2. For SOAs
50, 100, and 400 msec, there is a clear congruency effect in the sense that
congruent responses are faster than incongruent responses. Yet, consis-
tent with a trend in experiment 2, at an SOA of 200 msec, the congruency
effect is reversed, that is, congruent responses are faster than incongruent.
The findings suggest that in order to redirect attention away from the sin-
gleton distractor location, subjects may have inhibited the location of the
distractor, and thereby inhibited the letter inside the singleton distractor.
When the inhibited letter is identical to the target letter (i.e., the congru-
ent condition), subjects are relatively slow. On the other hand, when the
inhibited letter is different from the target letter (the incongruent condi-
tion), the letter is not inhibited and subjects are relatively fast.
Distractor inhibition also appears in many experiments demonstrating
negative priming, in which the response to a stimulus is slowed when the
previously inhibited stimulus becomes relevant for responding (e.g.,
Neill and Valdes 1996). For example, Tipper and Cranston (1985) showed
that when subjects ignored a letter on trial n, the response to a letter with
the same identity on trial n + 1 was impaired, a condition comparable to
the congruency manipulation in experiments 2 and 3. It is hypothesized
that actively inhibiting the potentially competing response from the letter
in the singleton to be ignored, may cause a reversal of the congruency
effect; that is, a response congruent with the letter inside the distractor is
slower than a response that is incongruent. Note that this reversal only
occurs when the distractor is presented 200 msec before the presentation
of the target, suggesting it takes time for inhibition to accrue. A similar
pattern of facilitation and inhibition appears in experiments addressing
Theeuwes, Atchley, a n d Kramer
“inhibition of return’’: targets appearing on the cued side show an RT
advantage for the first 150 msec, which is replaced by an inhibition after
250 to 300 msec (Posner and Cohen 1984).
The observation in experiment 3 that the congruency effect returns
after a time interval of 400 msec is not in line with findings from either
the “negative priming’’ nor the “inhibition of return’’ literature. At the
early SOAs, in which distractor and target are presented within 100 msec,
active top-down inhibition at the location of the distractor may start to
build u p , yet, before it is complete, the appearance of the target singleton
causes attention to be automatically captured by the location of the target
singleton. In other words, there may not be enough time to allow active
top-down inhibition at the early SOA, resulting in a “typical’’ congruency
effect, as observed in previous studies, where target a n d singleton dis-
tractor were presented simultaneously (see Theeuwes 1996).
At the later SOA of 200 msec, as evidenced by the absence of an inter-
ference effect of the distractor, subjects may have enough time to exert
top-down control. Top-down control results in active inhibition of the
singleton distractor, including the letter located inside the distractor.
Inhibition is important because the distractor a n d target are presented in
relatively close succession (i.e., within a 200 msec time frame), a n d will
compete for attention. At this point, it is not clear w h y the congruency
effect returns at SOA 400 msec. Perhaps it is impossible to maintain this
type of inhibition over a longer time period.
4.4 GENERAL DISCUSSION
The current experiments were designed to examine the time course of
bottom-up and top-down processing in visual search. The results indicate
that a salient singleton distractor presented close in time to the target sin-
gleton causes interference, as demonstrated by response times that are
significantly longer than those in the no-distractor condition. The finding
that the letter inside the singleton distractor had an effect on responding
to the target (i.e., the congruency effect) also suggests that spatial atten-
tion was d r a w n to the location of the distractor providing evidence that
the increase in RT is indeed d u e to attentional capture.
When a singleton distractor is presented at least 150 msec in advance
of the target, the interference effect is no longer observed, although the
finding that the letter inside the singleton distractor has an effect on RT at
still longer SOAs indicates that attention w a s captured by the singleton
distractor. Yet, with an interval of 150–200 msec between the presentation
of distractor a n d target, there was sufficient time to reorient spatial atten-
tion from the location of the distractor. When, at that point, the target
singleton is presented, attention is immediately directed to the target sin-
gleton resulting in response times equivalent to those in the no-distractor
condition.
Top-Down a n d Bottom-Up Control of Attention
When a singleton distractor is presented 150–200 msec in advance of
the target, it is assumed that top-down control can reduce or eliminate the
effect of the distractor. Note, however, that the presence of a congruency
effect at the longer SOAs indicates that top-down attentional control can-
not prevent attention from being captured by the singleton distractor, but
rather it allows a fast a n d efficiently redirection of attention from the dis-
tractor to the target location.
The present findings are consistent with those in Kim a n d Cave 1999,
which investigated the temporal interaction between top-down a n d
bottom-up control of attention by means of probe RTs. Kim a n d Cave also
used a task similar to that in Theeuwes 1992, where subjects searched for
a shape singleton (a circle among diamonds) while an irrelevant color sin-
gleton distractor (a red element among green elements) was present.
Either 60 or 150 msec after the presentation of the search display contain-
ing the target and singleton distractors, probes could appear at any of the
locations. It was hypothesized that if the early preattentive processing is
solely driven by bottom-up salience, as suggested by Theeuwes (1991,
1992), then the location of the salient singleton distractor should be
attended first, a n d thus the probe RT at the distractor location should
be faster than at any of the other locations in the short-SOA condition
regardless of whether the unique feature is relevant. On the other hand,
if top-down control is possible somewhat later in time, as the current
experiments suggest, then in the late-SOA condition, attention should no
longer be at the distractor location but instead at the location of the tar-
get singleton. For conditions in which target a n d distractor were locally
unique (and therefore salient enough) Kim a n d Cave (1999) did indeed
find these results. At an SOA of 60 msec, the probe RT at the location
of the singleton distractor w a s about 20 msec faster than at the target
singleton location. At an SOA of 150 msec, however, this pattern was
reversed: the probe RT at the target location w a s about 15 msec faster
than at the distractor location.
The current findings fit very well with those reported in Kim a n d Cave
1999, namely, that after 150 msec, attention is no longer at the location of
the distractor but instead at the location of the target. In our experiment
1, we show that, at an SOA of at least 150 msec, the singleton distractor
no longer interferes with search for the target singleten: there is no differ-
ence in RTs between the long-SOA conditions and the no-distractor con-
dition. These findings both suggest that it takes somewhere between 100
a n d 150 msec to disengage attention from the location of the distractor
a n d redirect it to the location of the target singleton.
The current results shed some new light on the findings obtained with
the spatial cuing paradigm of Folk and colleagues (Folk, Remington, a n d
Johnston 1992; Folk and Remington 1998) in which subjects have to
ignore a cue that appears 150 msec before the search display. The critical
finding is that a cue that does not match the top-down search goal (e.g.,
Theeuwes, Atchley, a n d Kramer
as in our experiments, the search goal is a shape singleton; the cue is a
color singleton) does not affect RT, whereas a cue that does match the
search goal slows search. The current findings and those in Kim a n d Cave
1999 show w h y with an SOA of 150 msec, a cue that does not match the
search goal has no effect on RT: by the time the search display is pre-
sented, subjects are able to exert enough top-down control to allow a re-
direction of attention from the location of the distractor to the location of
the target. The finding that there is an effect on RT in Folk and colleagues’
experiments when the cue a n d target share the same defining property
(e.g., the cue is red a n d the target is red) is not surprising because it is
likely that disengagement and redirection of attention from the distractor
location will take much longer when the distractor a n d target have the
same defining property. It will be clear that this explanation of Folk a n d
colleagues’ data does not suggest anything like a “contingent capture’’
hypothesis, but merely confirms Theeuwes’s stimulus-driven model
of selection (1992), in which early processing is driven by bottom-up
saliency factors. Note that our current findings a n d those of Kim a n d
Cave (1999) also disconfirm a more recent notion p u t forward by Folk
a n d Remington (1998), which suggests that irrelevant singletons do not
capture spatial attention but merely cause a “filtering’’ cost. Both the
effect of congruency of the letter inside the distractor, as found in our
experiments, a n d spatial RT probe effect, as found in Kim a n d Cave 1999,
clearly indicate that spatial attention w a s in fact captured by the location
of the distractor.
Even though we suggested that the effect of the distractor at the later
SOAs was reduced because of top-down control, the time course of the
distractor effect could also be explained in a purely bottom-up fashion.
Along these lines, it is assumed that attention is captured bottom-up by
the most salient singleton and, after being disengaged from the most
salient singleton, automatically reoriented to the next most salient single-
ton. If it takes about 150 msec to disengage a n d reorient attention, then it
is not surprising that, at an SOA of 150 msec the interference effect was
reduced. Note, however, that we assume that top-down control (i.e.,
knowing that one is looking for a diamond shape) does facilitate the dis-
engagement of attention from the colored distractor singleton. After
selecting the colored distractor, knowing that one is looking for a dia-
m o n d and not for a red circle will most likely speed up the disengage-
ment of attention and facilitate reorienting (see Theeuwes 1994b for a
similar account).
We interpreted the current results in a strictly serial fashion, assuming
that attention is first shifted to the most salient singleton and then to the
next. Parallel processing models could also explain the current findings,
assuming that on some trials the distractor finishes processing first, while
on others, the target singleton finishes first. A purely parallel model, in
which not only the two singletons are processed in parallel but all items
Top-Down a n d Bottom-Up Control of Attention
are processed in parallel is somewhat less likely, given the findings of
the control experiment (see note 1), which showed no congruency effect
when a congruent or incongruent letter was placed in a nonsingleton. If
all letters were processed in parallel, there should have been a clear con-
gruency effect in the control study because the response-related letter
inside a nonsingleton would have been processed at least as fast as any
of the other letters in the display (possibly faster because of a top-down
setting to look for this letter).
The current study indicates that during early preattentive processing,
selection is driven bottom-up, that is, attention is captured by the most
salient singleton present in the visual field. After attention is captured
by the location of the singleton distractor, “attentive’’ processing exerts
top-down activation that allows attention to be shifted elsewhere. The
current model assumes that visual selection is the result of an interac-
tion between goal-directed and stimulus-driven factors, consistent with
models of visual search (e.g., Cave and Wolfe 1990; Wolfe 1994; Treisman
and Sato 1990). Yet, unlike other models, the current model assumes that
early preattentive parallel processing (assumed to calculate differences
among stimulus features) is not accessible to top-down control. Only
after an item has been selected does top-down processing help to speed
up the disengagement of attention, allowing attention to be shifted to the
next location.
NOTES
This research was supported by cooperative research agreement DAAAL01-96-2-0003 with
the U.S. Army Research Laboratory. We thank Angela Glass and Meredith Minear for their
assistance in conducting the studies.
1. To ensure that any congruency effect does indeed depend on attention being attracted to
the colored singleton distractor, we ran a control study in which twelve subjects searched
for a shape singleton while a congruent or incongruent letter was placed in one of the non-
singletons, instead of being placed in the colored distractor. There were no reliable effect of
congruency on RT: F(1,11) = 1.06; p = 0.32; nor on error rate: F(1,11) = 1.32; p = 0.27.
REFERENCES
Bacon, W. F., and Egeth, H. E. (1994). Overriding stimulus-driven attention capture.
Perception and Psychophysics, 55,485-496.
Cave, K. R., and Wolfe, J. M. (1990). Modeling the role of parallel processing in visual search.
Cognitive Psychology, 22,225-271.
Caputo, G., and Guerra, S. (1998). Attentional selection by distractor suppression. Vision
Research, 38, 669-689.
Egeth, H. E., Virzi, R. A., and Garbart, H. (1984). Searching for conjunctively defined targets.
Journal of Experimental Psychology: Human Perception and Performance, 10,32-39.
Egeth, H. E., and Yantis, S. (1997). Visual attention: control, representation and time course.
Annual Review of Psychology, 48,269-297.
Theeuwes, Atchley, a n d Kramer
Eriksen, B. A., and Eriksen, C. W. (1974). Effects of noise letters upon the identification of a
target letter nonsearch task. Perception and Psychophysics, 16, 143–149
Eriksen, C. W., and Hoffman, J. E. (1972). Temporal a n d spatial characteristics of selective
encoding from visual displays. Perception and Psychophysics, 12, 201–204.
Folk, C. M., a n d Remington, R. W. (1998). Selectivity in distraction by irrelevant featural sin-
gletons: Evidence for two forms of attentional capture. Journal of Experimental Psychology:
Human Perception and Performance, 24, 847–858.
Folk, C. L., Remington, R. W., a n d Johnston, J. C. (1992). Involuntary covert orienting is con-
tingent on attentional control settings. Journal of Experimental Psychology: Human Perception
and Performance, 18, 1030–1044.
Joseph, J. S., and Optican, L. M. (1996). Involuntary attentional shifts d u e to orientation dif-
ferences. Perception and Psychophysics, 12, 201–204.
Kaptein, N. A., Theeuwes, J., a n d van der Heijden, A. H. C. (1995). Search for a conjunctively
defined target can be selectively limited to a color-defined subset of elements. Journal of
Experimental Psychology: Human Perception and Performance, 21, 1053–1069.
Kawahara, J., and Toshima, T. (1996). Stimulus-driven control of attention: Evidence from
visual search for moving target among static nontargets. Japanese Journal of Psychonomic
Science, 15, 77–87.
Kim, M. S., and Cave, K. R. (1999). Top-down a n d bottom-up attentional control: On the
nature of interference from a salient distractor. Perception and Psychophysics, 61, 1009–1023.
Koch, C., and Ullman, S. (1985). Shifts in selective visual attention: Towards the underlying
neural circuitry. Human Neurobiology, 4, 219–227.
Kumada, T. (1999). Limitations in attending to a feature value for overriding stimulus-
driven interference. Perception and Psychophysics, 61, 61–79.
Maljkovic, V., a n d Nakayama, K. (1994). Priming of pop-out: 1. Role of features. Memory and
Cognition, 22, 657–672.
Neil, W. T., a n d Valdes, L. A. (1996). Facilitatory and inhibitory aspects of attention. In A. F.
Kramer, M. G. H. Coles and G. D. Logan (Eds.), Converging operations in the study of visual
attention, p p . 77–106. Washington DC: American Psychological Association.
Posner, M. I. (1980). Orienting of attention. Quarterly Journal of Experimental Psychology, 32,
3–25.
Posner, M. I., and Cohen, Y. (1984). Components of visual orienting. In H. Bouma and D.
Bouwhuis (Eds.), Attention and Performance X, p p . 531–556. Hillsdale, NJ: Erlbaum.
Sagi, D., a n d Julesz, B. (1985). Detection versus discrimination of visual orientation.
Perception, 14, 619–628.
Theeuwes, J. (1991a). Cross-dimensional perceptual selectivity. Perception and Psychophysics,
50, 184–193.
Theeuwes, J. (1991b). Exogenous a n d endogenous control of attention: The effect of visual
onsets and offsets. Perception and Psychophysics, 49, 83–90.
Theeuwes, J. (1991c). Categorization and identification of simultaneous targets. Acta
Psychologica, 76, 73–86.
Theeuwes, J. (1992). Perceptual selectivity for color a n d form. Perception and Psychophysics,
51, 599–606.
Theeuwes, J. (1993). Visual selective attention: A theoretical analysis. Acta Psychologica, 83,
93–154.
Top-Down a n d Bottom-Up Control of Attention
Theeuwes, J. (1994a). Stimulus-driven capture a n d attentional set: Selective search for color
a n d visual abrupt onsets. Journal of Experimental Psychology: Human Perception and Per-
formance, 20, 799–806.
Theeuwes, J. (1994b). Endogenous and exogenous control of visual selection. Perception, 23,
429–440.
Theeuwes, J. (1996). Perceptual selectivity for color and form: On the nature of the inter-
ference effect. In A. F. Kramer, M. G. H. Coles and G. D. Logan (Eds.), Converging operations
in the study of visual attention, p p . 297–314. Washington DC: American Psychological
Association.
Theeuwes, J., and Burger, R. (1998). Attentional control during visual search: The effect of
irrelevant singletons. Journal of Experimental Psychology: Human Perception and Performance,
24, 1342–1353.
Theeuwes, J., Kramer, A. F., Hahn, S., Irwin, D. E., a n d Zelinsky, G. J. (1999). Influence of
attentional capture on oculomotor control. Journal of Experimental Psychology: Human
Perception and Performance, 25, 1595–1608.
Todd, S., a n d Kramer, A. F. (1994). Attentional misguidance in visual search. Perception and
Psychophysics, 56, 198–210.
Tipper, S. P. , a n d Cranston, M. (1985). Selective attention and priming: Inhibitory a n d facil-
itory effects of ignored primes. Quarterly Journal of Experimental Psychology, 37A, 591–611.
Treisman, A. M., a n d Sato, S. (1990). Conjunction search revisited. Journal of Experimental
Psychology: Human Perception and Performance, 16, 451–478.
Wolfe, J. M. (1994). Guided search 2.0: A revised model of visual search. Psychonomic Bulletin
and Review, 1, 202–238.
Yantis, S. (1993). Stimulus-driven attentional capture. Current Directions in Psychological
Science, 2, 156–161.
Yantis, S. (1996). Attentional capture in vision. In A. F. Kramer, M. G. H. Coles, and G. D.
Logan (Eds.), Converging operations in the study of visual attention, p p . 45–76. Washington DC:
American Psychological Association.
Yantis, S., a n d Jonides, J. (1984). Abrupt visual onsets and selective attention: Evidence from
selective search. Journal of Experimental Psychology: Human Perception and Performance, 10,
601–621.
124 Theeuwes, Atchley, a n d Kramer
5 Electrophysiological and Neuroimaging Studies of Voluntary and Reflexive
Attention
Joseph B. Hopfinger, Amishi P. Jha, Jens-Max Hopf,
Massimo Girelli, a n d George R. Mangun
ABSTRACT Powerful brain systems specialized for voluntary and reflexive attentional
control influence visual information processing. Studies of voluntary selective attention
have shown that the amplitudes of visual event-related potentials (ERPs) are greater for
events occurring at attended locations. Using ERPs, we recently investigated the neural cor-
relates of reflexive attention a n d found that early visual processing in the cortex is also mod-
ulated by reflexive orienting. By integrating functional imaging with ERP recording, we
related ERP signs of voluntary attention to underlying neural mechanisms. We used event-
related functional magnetic resonance imaging (fMRI) and trial-by-trial spatial cuing to
investigate the time course a n d functional anatomy of these attentional control systems.
Attentional mechanisms in frontal, parietal, and temporal cortex were found to produce
changes in visual cortical processing at multiple loci in the visual hierarchy, facilitating or
attenuating information from competing loci to reduce interference from irrelevant events
during perception a n d performance.
Over the past three decades, psychophysical studies of the effects of
selective attention on perception a n d performance in h u m a n s have estab-
lished that voluntarily directing covert attention to selected locations or
events facilitates perception a n d performance. For example, observers are
typically faster a n d more accurate in responding to stimuli at attended
than at unattended locations (see Yantis, chap. 3, this volume). In order to
elucidate the neurobiological underpinnings of these attentional phenom-
ena, physiological approaches have been employed in both h u m a n s a n d
animals. Much of this work initially investigated where in the ascending
processing stream top-down attentional control could influence stimulus
analysis. Studies in both h u m a n s a n d animals have now clearly demon-
strated that sensory-perceptual processes are modulated by top-down
spatial attention (see Mangun, Hillyard, a n d Luck 1993; Desimone a n d
Duncan 1995).
Other research has aimed to understand the control systems them-
selves (e.g., Corbetta et al. 1993; Harter et al. 1989; Posner et al. 1984).
Widespread regions of the brain, including frontal a n d parietal cortex,
subcortical structures such as the basal ganglia, portions of the thalamus,
a n d brain stem structures such as the superior colliculus, have been
implicated in attentional control (see Posner and Petersen 1990).
After contrasting the effects of top-down a n d bottom-up control over
sensory processing in visual cortex, we review our recent efforts to inte-
grate electrophysiological recordings with functional neuroimaging
measures to provide detailed anatomical information about where in the
h u m a n visual cortex top-down control acts to influence sensory process-
ing. We conclude by examining the control circuitry itself.
The emerging picture is that visual analyses in multiple areas of h u m a n
extrastriate cortex are modulated under the control of both top-down a n d
bottom-up mechanisms during spatial attention. These modulations
include changes in the gain of sensory inputs for attended and ignored
locations. As a result, the signal-to-noise ratio is improved for relevant
versus irrelevant inputs across visual space. At the earliest levels of
visual cortical analysis subject to attentional control, these effects of atten-
tion are distributed in a gradient across the visual field, and are specific
for selection based on location (see, for example, Mangun 1995).
5.1 VOLUNTARY ATTENTIONAL CONTROL OF VISUAL
PROCESSING VIA SPATIAL SELECTION
In the late 1960s, Eason, Harter, a n d White (1969) used scalp-recorded
ERPs in studies of spatially selective visual attention. They instructed
their subjects to attend and respond manually to stimuli flashed to one
visual field, a n d to ignore those flashed to the opposite field. Comparing
attended versus passively viewed ERPs to the same physical stimuli, the
authors observed changes in occipital sensory-evoked responses at
latencies between 150 and 200 msec after stimulus onset. Subsequently,
Van Voorhis and Hillyard (1977) replicated a n d extended these findings
in experiments designed to control for nonselective effects such as behav-
ioral arousal. They did so by comparing one attention condition directly
to another roughly equivalent condition (e.g., attend right versus attend
left location), rather than to passive conditions. Amplitude modulations
of sensory-evoked components as a function of spatial selective attention
began as early as 70–80 msec after the onset of the visual stimulus (see
also Eason 1981). The earliest effect of spatial attention was on the ampli-
tude of the so-called P1 component, a positive polarity wave recorded
over the lateral occipital scalp at 70–130 msec after the onset of the stim-
ulus a n d believed to be a reflection of activity in striate cortex, or even in
subcortical pathways. We n o w know, however, that the P1 attention effect
arises at later stages of visual cortical processing. Numerous studies have
demonstrated the reliability of these spatial attention effects in cortical
sensory-evoked ERPs and have significantly clarified their properties
(e.g., Eimer 1994; Harter, Aine, and Schroeder 1982; Hillyard and Münte
1984; Mangun and Hillyard 1991; Mangun, Hillyard, and Luck 1993). One
key finding is that the P1 component is affected only by spatial attention,
a n d not by attention selectively directed to nonspatial stimulus features
Hopfinger, Jha, Hopf, Girelli, and Mangun
Brain Structures Sensitive To Visual Attention
Attentional Control Structures
Parietal Cortex
Frontal Cortex
MT/MST 1 IVI I /IV
D—|.
Perceptual Processing Structures
V2
V3/VP
V4 Inferotemporal cortex
Figure 5.1 Schematic diagram of brain structures sensitive to visual attention. Various
brain regions have been implicated in attentional processing based on single-neuron record-
ings a n d pharmacological manipulations in monkeys, plus neuropsychological, event-
related potential (ERP), and functional imaging studies in h u m a n s . These structures can be
segregated into those hypothesized to be involved in attentional control, such as the frontal
a n d parietal cortex a n d subcortical structures like the pulvinar nuclei of the thalamus and
superior colliculus (SC), and those involved in perceptual analyses influenced by atten-
tional control systems.
such as color (e.g., Anllo-Vento, Luck, and Hillyard 1998; Harter and Aine
1984; Hillyard a n d Münte 1984).
Stages of Information Processing Influenced by Top-Down Control
Although much evidence indicated that sensory ERPs were affected by
attention, until very recently, there was little direct evidence about which
anatomically defined brain areas were being affected. Precisely where in
the complex visual system of the primate brain are top-down attentional
control processes able to influence incoming information?
Recent evidence indicates that spatial selective attention exerts its
greatest control over visual input processing at the cortical rather than
subcortical levels of the ascending pathways (figure 5.1), a n d it is n o w
well established that the P1 attention effect reflects modulation in the
extrastriate cortex (e.g., Heinze et al. 1994; Mangun et al. 1997; Woldorff
et al. 1997). Consistent with evidence from single-cell studies of spatial
127 Electrophysiology and Neuroimaging of Attention
attention in n o n h u m a n primates (e.g., Luck et al. 1997; Moran a n d Desi-
mone 1985), ERP activity in the P1 latency range is thought to arise from
visual areas V2, V3/VP, a n d V4, although it remains unclear whether the
P1 attention effect is generated in one or several of these areas. Some evi-
dence suggests that, under certain stimulus and task conditions, incom-
ing sensory signals can be weakly modulated earlier in primary visual
cortex (V1). We will return to this issue later. First, however, we turn to
our studies of reflexive attention, in which we ask whether visual cortical
processing is influenced by bottom-up control as well as by the more
well-established top-down mechanisms discussed so far.
5.2 REFLEXIVE ATTENTIONAL CONTROL OF VISUAL PROCESSING
Visual attention can be oriented reflexively (automatically) as well as vol-
untarily w h e n sensory events trigger (cue) attention to their locations in
the visual field (for a review, see Yantis, chap. 3, this volume). Both
reflexive and voluntary attention produce facilitation in reaction times
(RTs) to target stimuli occurring at attended or cued locations, but the
time courses of these effects differ. Reflexive attention is more rapidly
engaged and more transient than voluntary attention (Posner, Snyder,
a n d Davidson 1980; Jonides 1981). In addition, reflexive attention
includes inhibitory processes that lead to slowed RTs for cued-location
events as time between the reflexive cue a n d target increases (e.g., Posner
a n d Cohen 1984). Known as “inhibition of return’’ (IOR), this may lead
the reflexive system to favor novel locations, promoting effective search
of the scene. The neural correlates of reflexive attention are less well
understood than those for voluntary attention. Does, for example, the
fast RT facilitation observed with reflexively attended stimuli involve
changes in visual input processing, or does it reflect later changes in deci-
sion criteria or motor activation for cued-location events?
Event-Related Potential Evidence for Reflexive Attentional Control
over Visual Cortical Processes
To test the effects of reflexive attention on cortical processing, we (Hop-
finger a n d Mangun 1998) presented spatially nonpredictive cues (a brief
offset-onset of white dots) in left or right hemifields, and followed these
with task-relevant targets in either the same or opposite field. The targets
were either tall or short bars (0.5 probability) and required a discrimina-
tive button press. The interstimulus interval (ISI) between reflexive cues
a n d targets w a s either short (34–234 msec) or long (566–766 msec).
Targets that followed on the same side as the reflexive cues at the short
ISI elicited occipital ERPs of enhanced amplitudes, in addition to faster
RTs. Furthermore, these ERP enhancements appeared to be at the same
neural locus as the earliest enhancements typically produced by vol-
Hopfinger, Jha, Hopf, Girelli, and Mangun
untary spatial attention, in the occipital P1 component (90–140 msec
latency). In contrast, at the longer ISI, the facilitation in ERPs w a s replaced
by a reversal in the ERP pattern. That is, the P1 tended to be of smaller
amplitude to cued location targets, a pattern reminiscent of IOR. We
interpreted these data as evidence that reflexive attention produces a
short-lived, spatially restricted facilitation in visual processing in extra-
striate visual cortex. Interestingly, this appears to occur at the same stage
of visual processing influenced by voluntary attention.
A limitation in the foregoing ERP study of reflexive attention was that
the subjects’ task involved a discrimination of target features (tall versus
short vertical bars). Although the subjects were informed that the cues
were completely uninformative about where the target would occur,
because the analysis of target features required focal allocation of atten-
tion for task performance, the discrimination task might have introduced
a voluntary component (e.g., Egly et al. forthcoming; Treisman 1988);
under these stimulus and task conditions, enhancements of the P1 com-
ponent might not reflect activity of a purely reflexive mechanism. This
seems unlikely given that the reflexive P1 attention effect w a s rapidly
engaged a n d transient, as is typically seen with RT facilitation for
reflexive cues at short ISIs. To eliminate this possibility, however, a n d to
further investigate the relationship between IOR a n d processing in
visual cortex, we conducted a study that utilized the same stimulus
parameters as Hopfinger a n d Mangun (1998) but required only a simple,
speeded RT response to the suprathreshold target bars.
Methods Four small white dots were continuously displayed in the left
a n d right visual hemifields, forming an imaginary rectangle 1.0 by 1.4
degrees in size (the center of the imaginary rectangle w a s 1.5 degrees
above fixation a n d 6.4 degrees lateral to fixation). Each trial began when
the four dots on one side of fixation (equally probable on the left or right
of fixation) blinked off for 34 msec before reappearing (reflexive cue).
Subjects were explicitly told that the blinking of the dots w a s nonpredic-
tive of the location of the subsequent target bar, a n d were told not to
attend to the dots because this would be an unproductive strategy in
responding rapidly to the targets.
As in our prior study, the targets followed the reflexive cues by either
34–234 or 566–766 msec, in a r a n d o m fashion. The intertrial interval
varied randomly between 1,500 and 2,000 msec. The target remained on
the screen for 50 msec and was randomly either 1.8 or 2.3 degrees in
height by 0.60 degree in width, but the height w a s irrelevant for the pres-
ent study. Subjects pressed a button with their index finger as soon as the
bar was detected (response h a n d w a s counterbalanced). Trials were pre-
sented in 80 total blocks over t w o separate days of testing for each sub-
ject; 20% were catch trials, in which no target followed the reflexive cue—
a n d to which subjects virtually never responded.
Electrophysiology and Neuroimaging of Attention
-2pV
LVF Target
A OR |f
-IjlV^
OL ^
RVF Target
500
msec
500
r/l msec
Cued
L-o.77
(iVolts
-0.72
pVolts
LVF Target RVF Target
-2fiV
C PZ fr
- 2 ^ - ^
500
msec
P300—i
Component
Cued-location
Uncued-location
500
msec
P300
Component
Figure 5.2 A. Event-related potential (ERP) waveforms and topographic m a p s in reflexive
cuing study. ERPs at lateral occipital scalp sites to left (LVF) and right (RVF) target bars are
shown at the top. Tick marks are 50 msec, target onset is indicated by the arrow and upright
calibration bar. Positive voltages are plotted d o w n w a r d . Overlaid are the responses to tar-
gets when preceded by a reflexive cue in the same location (cued location) and when pre-
ceded by a cue in the opposite visual hemifield (uncued location) after correction with Adjar
filtering. Differences in the amplitude of the occipital P1 component are shaded. B.
Topographic voltage maps of the responses to left and right targets during the time range
of the P1 effect (100–150 msec). Each line on the head represents an isovoltage contour. The
scalp topographic maxima of the P1 for cued a n d uncued targets is shaded and labeled. C.
ERPs from midline parietal electrode site Pz are shown for left a n d right visual field target
bars. Cued-location targets (solid line) elicited larger P300 components than did uncued-
location targets (dashed line), and the difference is shaded in the figure.
The ERPs were collected from 64 tin electrodes placed on the scalp, but
only selected sites are shown in the figures. Eye position was monitored
with an infrared video camera system and by recording the electrooculo-
gram from electrodes placed around each eye. Trials with eye movements
or blinks were rejected. The adjacent response filter (Adjar) method
(Woldorff 1993) was employed to separate the brain responses to the cues
from those to the targets, something that is critical at short ISIs. The
details of the recording and analysis were identical to those in Hopfinger
and Mangun 1998.
Results and Conclusions The subjects were significantly faster in
responding to targets at the cued location than at the uncued location
(cued = 282 msec versus uncued = 290 msec; p < 0.05) at short cue-to-
target ISIs, although this pattern changed at the longer ISI, where a typi-
cal IOR pattern was observed (cued = 290 msec versus uncued = 277
msec; p < 0.05). The data presented in figure 5.2A (top) are the mean (i.e.,
group-averaged) ERP responses over 8 right-handed subjects. In line
with our prior report (Hopfinger and Mangun 1998), cued location tar-
gets in the short-ISI range elicited significantly enhanced P1 components
in comparison to targets at the uncued location (cued = 0.79 f£V versus
uncuedd = 0.31 fV; p < 0.05). At the longer ISIs, this pattern was no longer
present, and the P1 tended to be smaller at the cued location, although
this difference was not statistically reliable (cued = 0.79 //V versus un-
cued = 0.92 //V, p>0.05; not shown in figure 5.2). This reflexive effect
appears at the same latency (P1 latency range) as has been observed for
the effects of voluntary attention, suggesting that a similar processing
stage is being modulated by reflexive and voluntary attentional control,
although clearly the control mechanisms may not be identical.
The topographic maps of figure 5.2B (middle) show the scalp maxima
of the P1 components for cued and uncued targets. The scalp distribution
of these effects is quite similar to that observed for the P1 in studies of
voluntary attention, being maximal over contralateral occipital scalp
locations. These topographic distributions are consistent with activity in
the ventral extrastriate cortex (e.g., Heinze et al. 1994).
To assess whether the targets at cued and uncued locations are treated
differently at later stages of analysis, we also examined activity in the
P300 latency range (200-400 msec) elicited to the targets. The P300 is a
cognitive ERP elicited by stimuli that have higher perceived relevance or
require contextual updating (e.g., Donchin and Coles 1988). At short
cue-to-target ISIs, when the P1 to the target was enhanced by reflexive
attention, the P300 to that target was also larger (cued = 2.41 //V versus
uncued = 1.69 //V; p< 0.001). At longer cue-to-target ISIs, however, there
was no difference in the amplitude of the P300 component (3.02 //V
versus 2.76 |JiV; p>0.05; figure 5.2C, bottom).
Electrophysiology and Neuroimaging of Attention
The data from the present study demonstrate that reflexive attention
triggered by sensory events leads to a brief facilitation of target process-
ing for subsequent stimuli. Because they were obtained for simple as well
as more difficult detection tasks (Hopfinger and Mangun 1998), these
data strengthen our proposal that the effects of reflexive attention on the
P1 component are automatic. An unexpected result of this work has been
to demonstrate that both voluntary and reflexive attention manifest their
effects on sensory signals at similar stages of cortical analysis, the stages
reflected in the P1 attention effect.
5.3 THE FUNCTIONAL ANATOMY OF EARLY SPATIAL ATTENTION
An important next step is to determine where in the visual system the
modulation of the P1 component of the ERP is generated. Studies of the
intracranial generators of scalp-recorded ERPs all suffer from the same
general limitation—the recordings are m a d e relatively far from the site of
generation, making accurate localization difficult. Neuroelectric model-
ing can be used to infer the intracranial locus of scalp-recorded activity,
but the well-known “inverse problem’’ limits this approach. Although a
given distribution of charges inside the head will specify a unique pattern
on the scalp (the so-called forward solution), the inverse is not true (e.g.,
Dale a n d Sereno 1993). Thus no unique solution can be obtained when
going in the inverse direction from scalp recordings to neural generators.
Many studies have used inverse modeling to investigate the neural gen-
erators of scalp-recorded activity, but for the reasons noted above, it is
difficult to accept or reject any particular model.
Nonetheless, inverse modeling with computer algorithms can be em-
ployed to test possible models, especially when combined with addi-
tional information. For example, Dale a n d Sereno (1993) outlined the use
of anatomical information obtained from anatomical MRI scans to con-
strain the locations of possible neuroelectric sources to regions of the cor-
tex, thereby eliminating many areas of the head from consideration as
possible sites of generation of scalp-recorded ERPs. Similarly, we used
functional neuroimaging to identify active brain regions during a spatial
selective attention task that could serve to constrain source localization
models of ERPs (Heinze et al. 1994; see Mangun, Hopfinger, and Heinze
1998 for a review).
Integrating Event-Related Potentials and Neuroimaging in Studies of
Attention
In our first study integrating electrophysiology and functional imaging
methods, we combined ERP recording and positron-emission tomogra-
p h y (PET; Heinze et al. 1994). The design was similar to those in several
of our ERP experiments (e.g., Heinze and Mangun 1995). Subjects were
Hopfinger, Jha, Hopf, Girelli, and Mangun
presented with bilateral stimulus arrays containing two nonsense sym-
bols within each lateral hemifield at a rate of about 3 per second. The task
was to fixate a central point and, by attending covertly to the symbol pair
in one hemifield, to determine whether the two symbols on that side were
identical. Matching symbol targets required a rapid button press. The
symbols in the opposite field were ignored during that block. In different
blocks, subjects were instructed to attend to the left or right field stimuli.
PET activations showed that spatial selective attention activated
extrastriate visual cortex (posterior fusiform gyrus) in the hemisphere
contralateral to the attended stimuli. This PET information w a s used to
constrain modeling of ERP sources. We modeled neuroelectric sources at
the anatomical loci identified using PET, a n d calculated the patterns of
electrical activity that sources at these sites w o u l d produce on the scalp
model. Because we placed (or seeded) these model neural sources within
the PET-defined brain loci in the computer simulation, we referred to
them as “seeded forward solutions.’’ We found that dipoles located with-
in the PET-defined brain loci yielded highly accurate accounts of the
scalp-recorded ERP attention data, but only in the time range corre-
sponding to the P1 component (80–130 msec latency). This suggests that
changes in input processing in extrastriate visual cortex, in the region of
the posterior fusiform gyrus, were generating the P1 attention effect in
the ERPs. We were able to localize the site of top-down attentional con-
trol over ascending visual sensory processing in both time (80–130 msec
poststimulus) a n d space (posterior fusiform gyrus). An important
methodological feature of this experiment was that we compared identi-
cal experimental conditions, in the same volunteers, to isolate the same
attention effects in the functional imaging a n d ERP data.
Covariations in Event-Related Potential and Functional Imaging
Measures If the P1 attention effect in the ERPs really is related to the
attentional modulation revealed by changes in regional cerebral blood
flow (rCBF) in the posterior fusiform gyrus, then these measures should
covary with one another as a function of experimental manipulations. In
Mangun et al. 1997, we tested this directly by manipulating the percep-
tual load of the task (see Lavie, chap. 7, this volume) to determine
whether the P1 attention effect and the posterior fusiform activations
would be similarly affected. As before, subjects viewed bilateral arrays,
a n d in separate blocks attended to either the right or left of the arrays.
Two different tasks were n o w compared. One task was identical to that in
Heinze et al. 1994, with subjects having to respond to matching symbols
at the attended location (high-load task). In the other, only a simple lumi-
nance detection was required (low-load task); subjects were required to
respond to a small dot appearing on one side within the confines of the
bilaterally flashed symbol arrays. ERPs a n d PET measures were obtained
in separate sessions for each subject.
Electrophysiology and Neuroimaging of Attention
Attend Left Attend Left-Right Attend Right
Figure 5.3 Positron emission tomography (PET) activations and event-related potential
(ERP) topographic maps during voluntary spatial attention. Top. Main effects of attending
left versus right are shown when the subjects performed the symbol discrimination task.
The PET activations (outlined in black lines) are overlaid onto a horizontal section from MRI
scans. The Z-value scale next to each MRI scan refers only to the activated regions outlined
with black lines. The topographic voltage m a p shown at the top is the attend-left minus
attend-right difference m a p in the P1 latency range. The contour lines on the topographic
maps indicate polarity and voltage (thick solid = positive; dashed = negative). Because,
however, the polarity is an artifact of the direction of subtraction (left minus right), the P1
over the left hemisphere has a negative polarity in the maps, but is actually a positive
enhancement. Bottom. Plots of PET activation are statistical interaction maps of regions
where the amplitude of the attention effect was different for symbol discrimination versus
luminance detection (high versus low perceptual load). The topographic difference m a p
was derived by subtracting the attend-left minus attend-right attention m a p for the lumi-
nance detection task from that in the symbol discrimination task. A = anterior, P = posterior,
L = left, and R = right.
134 Hopfinger, Jha, Hopf, Girelli, and Mangun
There was a complete replication of our earlier study with respect to
the ERP a n d PET effects in the fusiform gyrus during discrimination.
When subjects attended to one visual hemifield, there was a significant
increase in the P1 component over contralateral scalp sites, and a corre-
sponding increase in rCBF in the contralateral posterior fusiform gyrus
(figure 5.3, top). Additional activations were also found in the contralat-
eral middle occipital gyrus, probably d u e to the use of more sensitive PET
methods (i.e., 3-D imaging).
Importantly, the amplitude of the attention effects (attend left versus
attend right) in both the ERP (P1 component) a n d PET (posterior fusiform
activation) measures were found to covary with perceptual load. This
was observed as significant interactions between attention (attend left
versus attend right) a n d task (symbol discrimination versus dot detec-
tion) for both the P1 component of the ERP a n d the activations in the
posterior fusiform gyrus (figure 5.3, bottom). The attention effects were
larger when perceptual load was higher. This covariation between the P1
effect a n d the fusiform gyrus PET effect supports the idea that the stage
of visual processing indexed by the P1 component occurs in extrastriate
cortex in the posterior fusiform gyrus. Although the PET activity in the
medial occipital gyrus showed a tendency in the same direction as the
fusiform activity, this was not reliable (no statistical interaction).
The increased attention effects with higher perceptual load can be
interpreted as the result of more attentional resources being dedicated to
the attended location, so that the differences between attended versus
unattended locations are enlarged. These data provide physiological
support for the proposal of Lavie a n d colleagues (Lavie, chap. 7, this
volume; Lavie a n d Tsal 1994) that perceptual load of target discrimi-
nation influences early selection processes (see also H a n d y a n d Mangun
2000).
Attentional Modulations in Functionally Defined Visual Areas
Having demonstrated that modulations of incoming sensory signals
occur as a function of spatial attention within visual cortex, we must n o w
identify whether these mechanisms are occurring within a single visual
cortical area or in multiple visual areas. The presence of multiple areas
in visual cortex is n o w well established in n o n h u m a n primates based
on single-cell studies (e.g., Van Essen and DeYoe 1995). Homologous
visual cortical areas can now be m a p p e d in h u m a n s using functional
neuroimaging (e.g., Engel et al. 1994; Sereno et al. 1995). Such mapping
allows one to refine the localization of visuo–spatial attention effects by
relating them to visual areas (e.g., V1, V2, V3/VP, and V4), not merely to
anatomical structures (e.g., lingual, fusiform, and middle occipital gyri),
as we (Jha et al. 1997) have done.
Electrophysiology and Neuroimaging of Attention
Visual Areas Attentional
Activations
Left –<— —>– Right
• VI D V P • Attend Right
• V2 D V4v D Attend Left
Figure 5.4 Derivations of visual areas a n d activations during spatial selective attention
from fMRI. Data from one representative subject (the first of six to be analyzed). Based on
the activations to meridia stimuli, the extent of visual areas V1 through V4v is shown for the
upper visual hemifield field representation on the ventral surface of the brain. Traced sec-
tions are sequential coronal slices beginning near the occipital pole (top) and continuing
anteriorly. The attentional activations from the same subject (right) are shown for the attend-
right (dark) and attend-left (lighter) conditions. By comparing these to the derived bound-
aries of the visual areas (left), one can observe that attention effects in the posterior fusiform
gyrus/lingual gyrus include activity in visual areas V2, VP, a n d V4v, as well as in other
regions that may be homologous to area TEO in monkeys (see Kastner et al. 1998).
Methods and Results We used fMRI to functionally define the borders
of the early visual areas in each of six subjects. The methods, though
similar to those of Engel et al. (1994) and Sereno et al. (1995), stimulated
only the meridia of the visual field (Kastner et al. 1998; Tootell et al. 1995).
Under passive viewing conditions, the upper and lower vertical meridia,
and left and right horizontal meridia were separately stimulated by
pattern-reversing checkerboard stimuli. Because the visual borders
between V1 and V2, between V2 and V P / V 3 , and between V P / V 3 and
V4 occur at the meridia of the visual field, we were able to determine the
Hopfinger, Jha, Hopf, Girelli, and Mangun
extent of the first few visual areas, whose derivation from the fMRI data
for one subject is shown in the left column of figure 5.4.
The subjects also performed a visual attention task that required
matching symbols at the attended location, every 16 sec a central arrow
cue instructed the subjects where to attend (see Mangun et al. 1998). It
w a s then possible to determine which early visual areas were modulated
during the spatial attention task by comparing the attention-related acti-
vations (attend left versus attend right) to the functionally defined visual
areas for each subject (compare left versus right columns of figure 5.4).
Attention-related activations were found in multiple visual areas, includ-
ing V2, VP, and V4.
Conclusions and Discussion Prior studies in h u m a n s using ERPs or
functional imaging have been unable to identify the precise areas of
visual cortex displaying attentional modulations. In this study, we used
fMRI to define the borders of cortical visual areas V1–V4, a n d were thus
able to demonstrate that spatial attention modulates neuronal processing
in multiple visual areas (V2–V4), but not in V1. Knowing that activations
previously viewed as singular sources of activity in extrastriate cortex
(as in our earlier PET studies) actually reflect activities in adjacent visual
cortical maps should allow more complex neuroelectric models to be
developed a n d tested. These will prove crucial in helping to define the
role that attention plays within different regions of visual cortex.
For example, modeling of ERP activity constrained by functional acti-
vations in adjacent, functionally defined visual areas might help resolve
h o w the primary visual cortex (V1 or striate cortex) is involved in visual
spatial selective attention. Many studies have failed to find any evidence
that the striate cortex could be modulated by spatial selective attention
either in animals (Luck et al. 1997; Moran and Desimone 1985) or in
humans, using ERPs (e.g., Clark et al. 1996; Mangun, Hillyard, and Luck
1993) or functional neuroimaging (e.g., Heinze et al. 1994; Kastner et al.
1998; Mangun et al. 1997, 1998). On the other hand, single-neuron record-
ing in monkeys (Motter 1993; Vidyasagar 1998) a n d fMRI in h u m a n s
(Somers et al. 1999; Worden, Schneider, a n d Wellington 1996) have occa-
sionally detected modulations of striate cortex during spatial selective
attention as well as during nonselective attention, where the nonspecific
effects of arousal are not well controlled (e.g., Watanabe et al. 1997). These
findings raise the possibility that, under certain conditions, incoming sen-
sory signals can be influenced by top-down attentional processes as early
as striate cortex (see Posner a n d Gilbert 1999 for review).
With the exception of Motter 1993, most studies showing attention
effects in V1 have measured regional cerebral blood flow in h u m a n s
in ways that could not specify the time course of the effects, or have
measured effects at very long latencies not consistent with input gating
(Roelfsema, Lamme a n d Spekreijse 1998). To interpret fMRI attention
Electrophysiology and Neuroimaging of Attention
effects in V1, it is essential that the time course of the activations be estab-
lished. Martinez a n d colleagues 1999 combined ERPs and fMRI to do pre-
cisely this. Their subjects selectively attended to stimuli in the left or right
visual field (ignoring the opposite hemifield). In separate sessions, fMRI
a n d ERP measures of attention were obtained (attend left versus attend
right). Mapping their effects onto functionally defined visual cortical
areas, the authors found that attention-related fMRI activations occurred
in visual areas V1–V4, but that short-latency ERPs generated in V1 were
not affected by attention. Rather, attentional modulations in the ERPs,
occurred later, at latencies consistent with activity in extrastriate cortex.
The Martinez et al. study suggests that increased rCBF in V1 during spa-
tial selective attention does not reflect an early gain control process over
incoming signals in striate cortex. Instead, V1 modulation may be a
reflection of reafferent activation of V1 from later stages in the visual
hierarchy, a view consistent with observations of long-latency attention
effects in V1 from single-neuron recordings in monkeys (Roelfsema,
Lamme, and Spekreijse 1998).
5.4 ATTENTIONAL CONTROL CIRCUITRY
Thus far we have considered the effect of attentional control on incoming
sensory signals. In the remainder of this chapter, we turn to consideration
of the control systems responsible for top-down effects of attention. The
issue of which brain systems participate in attentional control is some-
what less well understood than where attention influences sensory
inputs. Research in neurological patients, animals, and also in healthy
observers using neuroimaging suggests that the control of visuospatial
attention involves a complex network of widely distributed neuronal
populations, including those in dorsolateral-prefrontal, anterior cingu-
late, posterior parietal cortex, and thalamic and midbrain structures (e.g.,
Bushnell, Goldberg, a n d Robinson 1981; Corbetta 1998; Goldberg a n d
Bruce 1985; Heilman, Watson, a n d Valenstein 1994; Mesulam 1981;
LaBerge 1997; Posner a n d Petersen 1990; Posner and Driver 1992). The
specific functions of these structures in attentional control are only par-
tially understood, however, perhaps in part because the time course of
their relative activations during attentional orienting has not yet been
clarified. ERP a n d functional imaging studies incorporating new analyti-
cal approaches can be used to address the time course and functional
anatomy of attentional control systems, just as they have been used to
investigate their modulatory effects on perceptual processes.
Electrophysiological Studies of Attentional Orienting and Control
Although most ERP studies of attention focused on how attention affects
sensory processing, some have also investigated neuroelectric correlates
Hopfinger, Jha, Hopf, Girelli, and Mangun
of attentional preparation prior to the arrival of the target stimulus. For
example, in voluntary, trial-by-trial spatial cuing paradigms, ERPs can be
recorded in response to an attention-directing cue, and brain activity can
be monitored in the period after the instruction about where to attend,
but before the target is delivered (Harter et al. 1989; Mangun 1994;
Yamaguchi, Tsuchiya, and Kobayashi 1994).
Harter and coworkers (e.g., 1989) first studied the ERP correlates of
shifts of visuospatial attention. In their studies, a small (—0.5 degree)
central arrow cue (located at fixation) pointed either to the right or left
visual field. The cues defined the relevant side for that trial. Targets
appearing on the cued side were responded to as fast as possible, while
targets appearing on the uncued side were ignored. The subtraction of
ERP responses triggered by left-pointing cues from that of right-pointing
cues, revealed two principal attention shift-related ERP effects. The first,
denoted “early directing attention negativity’’ (EDAN), was a negative
polarity deflection over the parietal scalp contralateral to the direction
indicated by the attention-directing cue, starting 200 msec after cue onset
and lasting until 400 msec past cue onset. Presumably the EDAN is
related to attentional control processes that establish selective spatial
attention. Later (500-700 msec after the cue), at occipital electrode sites in
the hemisphere contralateral to the arrow direction, the ERP was more
positive (in comparison to the ipsilateral hemisphere). Referred to as “late
directing attention positivity’’ (LDAP), this second effect was proposed to
reflect the modulation of cortical excitability in regions of the brain cor-
responding to where attention has been directed in space. The LDAP has
been difficult to observe in adults, however, and has been identified only
in studies of children. Because no differences were observed in the early
(< 200 msec) sensory-evoked responses to the left versus right cues, it is
unlikely that any of the foregoing effects resulted from simple physical
differences between the left and right cue stimuli.
The attention-orienting ERP effects described above have been repli-
cated and extended in several reports. In Mangun 1994, we reported an
EDAN-like effect in adult subjects over parietal-temporal scalp sites
contralateral to the direction of the cue between 250 and 350 msec after
an endogenous central arrow cue. We also reported a longer-latency (300-
500 msec) right-hemisphere negative wave over frontal scalp sites.
Replicating the EDAN effect described by Harter et al. (1989), Yamaguchi,
Tsuchiya, and Kobayashi (1994) reported that it first occurred at poste-
rior temporal and parietal sites, but then appeared to spread over central
and frontal sites after about 380 msec. Independent of cue direction, a
right posterior temporal negativity was found starting 500 msec after cue
onset and lasting until the target onset. None of these studies observed
the LDAP of Harter and colleagues when recording in adults.
Although the foregoing studies correlated ERPs with an instruction to
shift attention from one location to another, we still do not know how
Electrophysiology and Neuroimaging of Attention
these potentials relate to underlying brain structures implicated in atten-
tional processes. Evidence from single-neuron recordings in monkeys
(e.g., Bushnell, Goldberg, a n d Robinson 1981; Colby, Duhamel, a n d
Goldberg 1993; Steinmetz et al. 1994), from studies of patients with focal
cortical lesions (e.g., Posner et al. 1984), and from h u m a n functional
neuroimaging (e.g., Corbetta et al. 1993; Corbetta 1998) indicates that
the parietal cortex is involved in visuospatial attention. This has led to
various models for the role of parietal cortex in attention, most of which
emphasize attentional control processes such as shifting attention, dis-
engaging attention from a current locus to enable shifting, or mapping
locations to be attended so that visual processing can be influenced in a
spatially defined manner (e.g., Posner a n d Petersen 1990; LaBerge 1997).
If the EDAN component of Harter et al. 1989 reflects neural processes
involved in the control of visual attention by parietal cortex, then one
might expect it to have a narrow scalp maximum over parietal cortical
regions. Moreover, this topography should be distinct from that of the
later LDAP, which, if related to effects in visual cortex, should have a
distribution similar to that for attention-related enhancements of target
processing (e.g., P1 attention effect).
In a recent study, we sought to clarify the scalp distributions of cue-
related ERPs using detailed topographical analysis of the ERPs in the cue-
target interval in a voluntary, trial-by-trial spatial cuing paradigm (Hopf
a n d Mangun, forthcoming). High-resolution mappings of the ERP com-
ponents were obtained.
Methods ERPs from 92 scalp sites were recorded from 14 healthy, right-
handed student volunteers. Subjects fixated a point (0.19 degree diame-
ter) in the center of a computer monitor. Two white outline boxes (3.3
degrees wide by 5.5 degrees tall) were continuously present (10.7 degrees
lateral to fixation, measured to center) in the upper visual field to demar-
cate possible target locations. Each trial began with an arrow cue (100
msec duration) flashed to fixation. It randomly pointed to the left or right,
a n d was followed 900 msec later by a pair of symbols flashed (35 msec
duration) to one of the two lateral locations.
While maintaining fixation, subjects had to attend covertly to the cued
box a n d discriminate whether the symbols presented there were identi-
cal. Approximately 16% of cued-location stimuli were matching targets,
a n d required a button press response (response hand was counterbal-
anced between experimental blocks within and across subjects). The cue
was not predictive of target location, but instructive, indicating to the
subjects that the cued box was relevant for that trial. They were to ignore
the uncued location, and no response was required on trials where the
targets appeared in the uncued location.
The perceptual load of the target discrimination was manipulated in
separate experimental blocks. In the low-load condition, the symbols dif-
Hopfinger, Jha, Hopf, Girelli, and Mangun
c s c
» LH * »
Parietal - ^ *
+5 [MV]
200 400 600 800 1000
msec
Left cue
— Right cue
Figure 5.5 Event-related potentials (ERPs) to attention-directing cues. Grand average
ERPs over 14 subjects are shown in response to an attention-directing arrow cue located
near fixation. ERPs to left cues are shown in solid lines; those to right cues in dashed lines,
with differences shaded. The onsets of the cues (C) and subsequent target stimuli (S) are
indicated above the ERPs. Left-hemisphere (LH, left column) and right-hemisphere (RH,
right column) electrode sites were from parietal (top row), frontal (middle row), and occip-
ital (bottom row) scalp regions.
fered in more than one detail (mean of 2-3). In the high-load condition,
the symbols differed in only one detail, a small horizontal bar of about 1.1
degrees.
Analysis of variance (ANOVA) was used to assess significant differ-
ences between conditions for the target-evoked ERPs (e.g., P1 and N1)
where a priori predictions guided statistical tests at specific electrode
sites. For the cue-related ERPs whose topographies were under investi-
gation, however, statistical evaluation was performed within the context
of topographic mapping using a procedure developed to assess the scalp
maps. An F-test was performed for each electrode location separately,
and a modified Bonferroni correction applied. The corrected critical
F-value in the present experiment is F(1, 13) = 10.58. The F-value for the
single-electrode test where each effect discussed below was maximal will
be reported in the text.
Results and Discussion Figure 5.5 shows superimposed grand-averaged
waveforms (over the 14 subjects) for left- and right-pointing cues in the
time interval between the onset of the cue and the onset of the target sym-
bols. The ERPs are from parietal, frontal, and occipital electrodes. Starting
approximately 200 msec after cue onset, a statistically significant differ-
141 Electrophysiology and Neuroimaging of Attention
LEFT CUE - RIGHT CUE
Figure 5.6 Topographic voltage maps of the differences obtained by subtracting right from
left cue-related activity (grand averages over the 14 subjects). The topographic maps were
computed over the indicated time ranges, and thus correspond to the EDAN, frontal effect,
and LDAP components. Darker shades indicate negative voltages. Because, however, polar-
ities are dependent on the direction of subtraction, the EDAN has a positive polarity in
these maps, and the LDAP over the two hemispheres appear to be of opposite polarity, but
each is actually a contralateral positive-going deflection in the ERPs.
ence for right- versus left-pointing cues can be observed over the poste-
rior parietal scalp of the left hemisphere (figure 5.5, top, shaded areas):
F(1, 13) = 15.90. The scalp voltage topographies of this posterior left
hemisphere effect are shown in the difference maps (left-right cue, low-
load condition only) in figure 5.6 between 200 and 400 msec from the cue
onset. (Note that as a result of the direction of the subtraction, this effect
is seen as a positive focus in the topographic maps of figure 5.6, but can
be interpreted as greater left hemisphere negativity for right-pointing
versus left-pointing cues.) This effect may be related to the EDAN com-
ponent of Harter and colleagues (1989; Harter and Anllo-Vento 1991).
By about 300 msec after cue onset, there was a statistically significant
effect over right frontal scalp regions. The shaded area in figure 5.5
(middle) illustrates this effect in the average waveforms for lateral frontal
Hopfinger, Jha, Hopf, Girelli, and Mangun
Valid-Invalid
Figure 5.7 Scalp topographic maps of the attention difference effects (attended minus
unattended) for left (LVF) and right (RVF) target stimuli in the P1 time range. Attentional
modulations of the P1 evoked by left-field stimuli showed a scalp maximum over right lat-
eral occipital regions, whereas attention effects on for right-field stimuli it showed a scalp
maximum over left lateral occipital regions.
electrodes. This cue-related effect lasted until 500 msec after cue onset
and was reliably larger at right than at left frontal sites (figure 5.6, middle
row, left map). Statistically, the right frontal effect was most robust
between 300 and 400 msec latency: F(1, 13) = 40.49, consistent with the
right frontal component described in Mangun 1994.
A longer-latency effect of cue direction began at about 400 msec after
arrow onset (figure 5.5, bottom) at occipital sites over both hemispheres.
This took the form of a statistically significant focal scalp positivity con-
tralateral to the direction of the attention-directing arrow cue: F(1, 13) =
14.6 at electrode T5 in the left hemisphere; F(1, 13) = 25.73 at electrode T6
in the right hemisphere. The topographic maps of figure 5.6 show these
effects as a posterior right-hemisphere positive focus, and a mirror image
left-hemisphere negative focus. (Note again that the opposite polarities
over the two hemispheres were caused by the direction of the subtraction
of the ERPs in the topographic maps.) These contralateral positivities
lasted until 850 msec after cue onset, b u t terminated approximately 150
msec before the onset of the symbols. This occipital effect may be related
to the LDAP effect previously observed in children (Harter and Anllo-
Vento 1991). Finally, at around 600 msec after the cue onset, a left anterior
scalp difference occurred as a function of the direction of the attention-
directing cues (figure 5.5, middle, and figure 5.6, bottom left map), and
this frontal effect lasted until the end of the cue-target interval: F(1, 13) =
15.82 from 700-900 msec latency.
Bearing close similarity to prior studies, a sequence of ERPs related
to the direction of attention-directing cues were obtained in the period
following the cue, but prior to the onset of the target stimuli. Did percep-
tual load influence these cue-related ERPs? Although there were
significant main effects of perceptual load in the ERPs to the cues, these
effects did not interact with the responses to the attention-directing cues,
Electrophysiology and Neuroimaging of Attention
with the possible exception of a marginally significant effect between 300
and 400 msec latency over the left frontal scalp: F(1, 13) = 5.8. Thus, to our
surprise, the load manipulation did not influence the ERP signs of atten-
tional control to the cues. Likewise, the effects of the load manipulation
only weakly influenced the attention effects on the subsequent target-
evoked ERPs. Although cued symbols showed a significantly larger
occipital positivity between 80 and 120 msec (the P1 attention effect; see
topographic maps in figure 5.7) and an enhanced negativity between 130
and 200 msec (the N1 attention effect—not shown in figures), the magni-
tude of the attention effects were not different for low versus high load for
the P1 or N1 (c.f., H a n d y and Mangun 2000).
Conclusions The present results provide some insights into the nature
of attentional control processes during visual spatial attention. First,
when attention is directed in space by an endogenous cue, a series of
ERP components is generated, providing additional tools for investig-
ating attentional controls processes in the absence of overt behavioral
responses. Second, these cue-related effects have distinct scalp topog-
raphies and time courses.
The EDAN had a focus over parietal scalp, b u t contrary to prediction,
it was only significant over left parietal scalp regions. Although the left
lateralization of the EDAN effect might signal a special role of the left
hemisphere in spatial attention, such an interpretation would be incon-
sistent with other evidence. For example, neuropsychological evidence
has demonstrated that hemispatial neglect is worse following right
versus left parietal lesions (e.g., Heilman, Watson, and Valenstein 1994).
Neuroimaging suggests that both left and right parietal lobes are involved
in spatial attention (Corbetta et al. 1993), but that the right parietal lobe
may play a greater role.
In light of the foregoing evidence for a right-hemisphere role in spatial
processing and attentional orienting in space, an alternative inter-
pretation for the present left-lateralized effects is that they may be an arti-
fact of our comparison, which subtracted right- from left-pointing cues.
That is, if the right hemisphere were equally active for both left- and
right-pointing cues, then the subtraction would yield no difference over
the right parietal scalp. If the left hemisphere were differentially active for
leftward versus rightward orienting, then the subtraction would yield
a left-lateralized effect such as we have observed here. Such a model
closely fits our behavioral studies in split-brain patients (Mangun et al.
1994), where we found evidence that the right hemisphere was affected
by both right and left attention-directing cues, whereas the left hemi-
sphere was not. We return to this question later in this chapter.
Interestingly, the earliest ERP signs of cue-related activity were over
the parietal scalp, with the frontal effects occurring later. Although many
models of attention posit the frontal cortex as the seat of initiation of
Hopfinger, Jha, Hopf, Girelli, and Mangun
attentional control (e.g., LaBerge 1997; Posner and Petersen 1990), the
time course information of the present study is not consistent with such
a proposal, at least not for activity differential for right- a n d left-pointing
cues.
The LDAP showed topographic foci over the lateral occipital scalp. The
maxima of the LDAP on the scalp closely matched those for the atten-
tional enhancements of the P1 component to subsequent target stimuli.
Thus the occipital activity in response to the cues and that in response to
the subsequent early attention effects h a d similar topographies, as pre-
dicted by the hypothesis that the LDAP is a sign of increased neural
excitability in the neurons coding the regions of space to be attended. A
key finding, however, w a s that the LDAP effect terminated prior to the
onset of the target stimuli, even though the targets showed significant
modulation by attention. This raises significant questions about the func-
tion of the neural processes involved in the LDAP effect. For example,
perhaps the LDAP is a sign of control signals to neurons coding the
region of visual space to be attended, rather than evidence of the resul-
tant enhanced excitability of those visual neurons.
ERP data as used here can provide only indirect clues about the under-
lying neural structures involved in top-down control of spatial attention.
We must turn to functional imaging to help fill in the details about func-
tional anatomy.
Event-Related Functional Magnetic Resonance Imaging Studies of the
Control of Spatial Attention
Until about 1995, neuroimaging studies of attentional control systems
suffered from two serious methodological limitations: (1) the inability to
provide temporal information; and (2) the use of “blocked’’ designs, pre-
cluding separation of individual trials within blocks, as commonly done
with behavioral and electrophysiological data (see Rugg 1998). By con-
trast, event-related fMRI now permits the brain activity related to differ-
ent intermingled trials to be decomposed (e.g., Buckner 1998; McCarthy
et al. 1997), an approach conceptually similar to that used in ERP re-
search. For example, in the cuing paradigms described in the last section,
we separately derived ERPs to the cues and those to the subsequent tar-
gets, and a similar approach can n o w be used with fMRI. We employed
this method to investigate control of spatial attention in a trial-by-trial
cuing task (Hopfinger, Buonocore, and Mangun 2000).
Methods Three healthy adult subjects participated in the data reported
here. All were right-handed and had normal vision. Each trial began with
a voluntary cue at fixation (500 msec duration) that randomly pointed
either to the left or right visual hemifield. The direction of the arrow was
an instructive cue, telling the observer to attend selectively to the cued
Electrophysiology and Neuroimaging of Attention
hemifield. To eliminate any differences in sensory activations to the cue
between conditions, the cue consisted of two overlapping arrows, one
blue a n d one yellow, pointing in opposite directions. Some subjects were
told to use the blue cue to direct attention, while the others were told to
use the yellow.
Following the cue by a random interval of 1,000 msec (17% of trials) or
8,160 msec (83% of trials), bilateral black-and-white checkerboard targets
were presented to the upper visual field (4 Hz reversal rate; 750 msec
duration). This permitted brain responses to be modeled when the cues
a n d targets were separated by several seconds (long-ISI trials), but
required the subjects to prepare for the possibility that a target would
appear at shorter ISIs. The task was to maintain fixation on central cross-
hairs while covertly attending to the cued side to discriminate whether
elements of the checkerboard on that side were missing (on 50% of trials,
some checks were gray). From 3 to 9 checks were missing for infrequent
target checkerboards. Subjects pressed one button for targets and another
for nontargets. Approximately 8 sec separated trials (from target off to
onset of next cue).
The hemodynamic responses to the cues a n d targets were separately
modeled with event-related fMRI methods using SPM97 for the trials
where the cue and target were separated by 8,160 msec. The hemody-
namic response was modeled as the sum of two gamma functions and its
temporal derivative. A statistical significance level of p < 0.001 was set.
The activated regions were overlaid on the canonical T1-weighted MRI
scans of SPM97 for the images shown here.
Results Figure 5.8 shows the average activations over three subjects
to the onset of the arrow-directing cues (left panels) a n d the subsequent
target stimuli (right panels). For the present discussion, we will consider
only a single posterior parieto-occipital slice in which cue-related activa-
tions and target-related activations were both visible, and we will not
consider activations present in other brain regions (e.g., frontal, temporal,
or subcortical regions).
Let us t u r n first to the h e m o d y n a m i c responses to the cues a n d tar-
gets collapsed over the direction of cuing (figure 5.8, top row). The cue
resulted in bilateral inferior parietal cortex activations, whereas the tar-
gets produced no such inferior parietal activity, being restricted to the
visual cortex. Thus parietal and occipital cortical regions were differen-
tially activated by attention directing cues a n d subsequent targets.
Were these activations different as a function of the direction in which
attention was cued? The activations in the inferior parietal region to the
cues were not significantly different for left versus right cues (figure 5.8,
left panels, middle and bottom rows). In contrast to the findings for the
parietal cortex, the visual cortex activations in both the right and left
hemispheres were significantly modulated by the direction of attention in
Hopfinger, Jha, Hopf, Girelli, and Mangun
Cues Targets
Cues Left>Right Targets Left>Right
Cues Right>Left Targets Right>Left
Figure 5.8 Event-related fMRI during spatial cuing. Activations in response to cues (left)
a n d targets (right) in the spatial cuing paradigm described in the text. Bilateral inferior pari-
etal (IP) activations were obtained in responses to cues (left, collapsed over cue direction),
a n d visual cortical (VC) activation in response to the bilateral target stimuli (right, also col-
lapsed over cue direction). There were contralateral activations in the visual cortex to cues.
For the targets, attention to the left and right (right, middle, a n d bottom) halves of the bilat-
eral target stimulus showed contralateral medioventral occipital activations, consistent with
activity in extrastriate cortex (ES).
response to both cues and targets. In responses to the cues, there were dif-
ferential activations of the contralateral medioventral occipital cortex.
These responses to the cues in occipital cortex may represent either
attentional control signals acting in visual cortex to alter neuronal
excitability or regional cerebral blood flow related to the increased neu-
ronal excitability itself. They do not, however, represent simple sensory
activations by the cues, which were localized more posterior in the brain,
147 Electrophysiology and Neuroimaging of Attention
near the foveal representation on the occipital pole as determined by con-
trol sessions (not shown in figures). The ERP data described in the last
section suggest that, in response to cues, the occipital effects followed
the parietal activations in time. That is, the occipital activations may be
related to the LDAP component in the ERPs. This must remain spec-
ulative until combined ERP and fMRI studies are performed with this
paradigm.
In response to the targets, activations contralateral to the direction of
attention were observed in visual cortex (figure 5.8, right panels, middle
a n d bottom rows). These effects on target processing are in line with our
prior PET (Heinze et al. 1994; Mangun et al. 1997) and fMRI (Mangun et
al. 1998) studies. Increased rCBF was observed in lingual and fusiform
gyri in the hemisphere contralateral to the attended visual hemifield.
Conclusions These findings while preliminary, demonstrate differential
activation of parietal and occipital cortical areas during distinct time
periods of a spatial cuing task. They support the thesis that the parietal
cortex is engaged during attentional orienting, presumably as part of a
cortical network for top-down attentional control (e.g., LaBerge 1997).
The result of these orienting processes is the selective activation of extra-
striate cortex to filter inputs from relevant versus irrelevant locations.
Although the complete circuit remains to be elucidated, these and our
ERP findings for related tasks paint similar pictures.
5.5 GENERAL CONCLUSIONS
While the physiological mechanisms of visuospatial selective attention
are certainly not completely understood, there has been significant prog-
ress in that direction. The findings we have presented demonstrate
that not only is extrastriate cortex modulated by top-down attentional
mechanisms, as we showed previously (Heinze et al. 1994; Mangun et al.
1997), but this occurs in multiple visual areas, including V2, V3/VP, a n d
V4. The related ERP recordings indicate that these effects represent mod-
ulations of initial input processing, as opposed to reafferent activations
of these areas by later stages in the visual hierarchy. Our ERP studies
also suggest that visual cortical processing is significantly modulated
by reflexive attention, at sites similar to those for voluntary attention,
although fine-grained functional anatomical studies remain to be done.
Finally, the control systems involved in top-down voluntary spatial atten-
tion can be studied by monitoring brain activity in response to attention-
directing cues. Although spatially distinct brain regions, including
frontal, parietal, and occipital brain areas, are engaged by instructions
to shift attention, these activities have very different time courses, as
indexed by ERP recordings, with activity over parietal scalp appearing
earliest in response to an informative cue. Event-related fMRI studies
Hopfinger, Jha, Hopf, Girelli, and Mangun
show that cue-related hemodynamic responses in inferior parietal cortex
occur, a n d that this activity can be modeled separately from that in the
visual cortex in response to an attention-directing cue or subsequent tar-
get stimulus. Together, these results support a model in which parietal
brain regions are involved in the initial control of attention to affect
changes in stimulus input processing by visual cortex.
NOTE
This research was supported by grants to Joseph B. Hopfinger from the National Science
Foundation, to Amishi P. Jha from the National Institute of Mental Health, to Jens-Max Hopf
from the German Academic Exchange Program (DAAD) a n d to George R. Mangun from the
National Institute of Mental Health, the National Institute of Neurological Disorders a n d
Stroke, and the H u m a n Frontier Science Program. We are grateful to Maryam Soltani a n d
Michael H. Buonocore for their contributions, and to Karl Friston a n d Christian Buchel for
their assistance.
REFERENCES
Anllo-Vento, L., Luck, S. J., a n d Hillyard, S. A. (1998). Spatio-temporal dynamics of atten-
tion to color: Evidence from h u m a n electrophysiology. Human Brain Mapping, 6, 216–238.
Buckner, R. (1998). Event-related fMRI and the hemodynamic responses. Human Brain
Mapping, 6, 394–398.
Bushnell, M. C., Goldberg, M. E., and Robinson, D. L. (1981). Behavioral enhancement
of visual responses in monkey cerebral cortex: 1. Modulation in posterior parietal cortex
related to selective visual attention. Journal of Neurophysiology, 46, 755–772.
Clark, V. P., and Hillyard, S. A. (1996). Spatial selective attention affects extrastriate but not
striate components of the visual evoked potential. Journal of Cognitive Neuroscience, 8,
387–402.
Colby C., Duhamel, J., and Goldberg, M. (1993). Ventral intraparietal area of the macaque:
Anatomic location and visual response properties. Journal of Neurophysiology, 69, 902–914.
Corbetta, M. (1998). Frontoparietal cortical networks for directing attention and the eye to
visual locations: Identical, independent, or overlapping neural systems? Proceedings of the
National Academy of Sciences, U.S.A., 95, 831–838.
Corbetta, M., Miezin, F. Shulman, G., and Petersen, S. (1993). A PET study of visuospatial
attention. Journal of Neuroscience, 13, 1202–1226.
Dale, A. M., and Sereno, M. I. (1993). Improved localization of cortical activity by combin-
ing EEG and MEG with MRI cortical surface reconstruction: A linear approach. Journal of
Cognitive Neuroscience, 5, 162–176.
Desimone, R., a n d Duncan, J. (1995). Neural mechanisms of selective visual attention.
Annual Review of Neuroscience, 18, 193–222.
Donchin, E., and Coles, M. G. H. (1988). Is the P300 component a manifestation of context
updating? Behavioral and Brain Sciences, 11, 357–427.
Eason, R. G. (1981). Visual evoked potential correlates of early neural filtering during selec-
tive attention. Bulletin of the Psychonomic Society, 18, 203–206.
Electrophysiology and Neuroimaging of Attention
Eason, R., Harter, M., and White, C. (1969). Effects of attention and arousal on visually
evoked cortical potentials and reaction time in man. Physiology and Behavior, 4, 283–289.
Egly, R., Rafal, R., Henik, A., and Berger, A. (Forthcoming). Reflexive a n d voluntary covert
orienting in detection a n d discrimination tasks. Journal of Experimental Psychology: Human
Perception and Performance.
Eimer, M. (1994). Sensory gating as a mechanism for visuospatial orienting: Electro-
physiological evidence from trial-by-trial cuing experiments. Perception and Psychophysics,
55, 667–675.
Engel, S. A., Rumelhart, D., Wandell, B., Lee, A., Glover, G., Chichilinsky, E., and Shadlen,
M. (1994). FMRI of h u m a n visual cortex. Nature, 370, 106.
Goldberg, M. E., and Bruce, C. J. (1985). Cerebral cortical activity associated with the orien-
tation of visual attention in the rhesus monkey. Vision Research, 25, 471–481.
Handy, T. C., a n d Mangun, G. R. (2000). Attention and spatial selection: Electrophysiological
evidence for modulation by perceptual load. Perception and Psychophysics, 62, 175–186.
Harter, M. R., and Aine, C. J. (1984). Brain mechanisms of visual selective attention. In
R. Parasuraman and D. R. Davies (Eds.), Varieties of attention, p p . 293–321. N e w York:
Academic Press.
Harter, M. R., Aine, C., a n d Schroeder, C. (1982). Hemispheric differences in the neural pro-
cessing of stimulus location and type: Effects of selective attention on visual evoked poten-
tials. Neuropsychologia, 20, 421–438.
Harter, M. R., a n d Anllo-Vento, L. (1991). Visual-spatial attention: preparation a n d selection
in children a n d adults. In C. H. M. Brunia and M. N. Verbaten (Eds.), Event-related brain
research, vol. EEG suppl. 42, p p . 183–194. Amsterdam: Elsevier.
Harter, M. R., Miller, S. L., Price, N. J., LaLonde, M. E., and Keyes, A. L. (1989). Neural
processes involved in directing attention. Journal of Cognitive Neuroscience, 1, 221–237.
Heilman, K. H., Watson, R. T., and Valenstein, E. (1994). Localization of lesions in neglect
a n d related disorders. In A. Kertesz (Ed.), Localization and neuroimaging in neuropsychology,
p p . 495–524. San Diego, CA: Academic Press.
Heinze, H. J., and Mangun, G. R. (1995). Electrophysiological signs of sustained and tran-
sient attention to spatial locations. Neuropsychologia, 33, 889–908.
Heinze, H. J., Mangun, G. R., Burchert, W., Hinrichs, H., Scholz, M., Münte, T. F., Gös, A.,
Johannes, S., Scherg, M., Hundeshagen, H., Gazzaniga, M. S., a n d Hillyard, S. A. (1994).
Combined spatial and temporal imaging of spatial selective attention in h u m a n s . Nature,
392, 543–546.
Hillyard, S. A., a n d Münte, T. F. (1984). Selective attention to color and location: An analy-
sis with event-related brain potentials. Perception and Psychophysics, 36, 185–198.
Hopf, J.-M., and Mangun, G. R. (Forthcoming). Shifting visual attention in space: An elec-
trophysiological analysis using high spatial resolution mapping. Clinical neurophysiology.
Hopfinger, J., a n d Mangun, G. R. (1998). Reflexive attention modulates processing of visual
stimuli in h u m a n extrastriate cortex. Psychological Science, 9, 441–447.
Hopfinger, J. B., Buonocore, M. H., and Mangun, G. R. (2000). The neural mechanisms of
top-down attentional control. Nature Neuroscience, 3, 284–291.
Jha, A. P., Buonocore, M., Girelli, M., a n d Mangun, G. R. (1997). FMRI a n d ERP studies of
the organization of spatial selective attention in h u m a n extrastriate visual cortex. Society for
Neuroscience Abstracts, 23, 301.
Hopfinger, Jha, Hopf, Girelli, and Mangun
Jonides, J. (1981). Voluntary versus automatic control over the mind’s eye movement. In
J. B. Long a n d A. D. Baddeley (Eds.), Attention and Performance IX, p p . 187–203. Hillsdale,
NJ: Erlbaum.
Kastner, S., DeWeerd, P., Desimone, R., and Ungerleider, L. C. (1998). Mechanisms of
directed attention in the h u m a n extrastriate cortex as revealed by functional MRI. Science,
282, 108 – 111.
LaBerge, D. (1997). Attention, awareness, and the triangular circuit. Consciousness and
Cognition, 6, 149–181.
Lavie, N., and Tsal, Y. (1994). Perceptual load as a major determinant of the locus of selec-
tion in visual attention. Perception and Psychophysics, 56, 183–197.
Luck, S. J., Chelazzi, L. Hillyard, S. A. and Desimone, R. (1997). Neural mechanisms of
spatial selective attention in areas V1, V2 and V4 of macaque visual cortex. Journal of
Neurophysiology, 77, 24–42.
Mangun, G. R. (1994). Orienting attention in the visual fields: An electrophysiological analy-
sis. In H.-J. Heinze, T. F. Münte, and G. R. Mangun (Eds.), Cognitive electrophysiology, p p .
81–101. Boston: Birkhäuser.
Mangun, G. R. (1995). Neural mechanisms of visual selective attention in h u m a n s .
Psychophysiology, 32, 4–18.
Mangun, G. R., Buonocore, M., Girelli, M., and Jha, A. (1998). ERP and fMRI measures of
visual spatial selective attention. Human Brain Mapping, 6, 383–389.
Mangun, G. R., a n d Hillyard, S. A. (1991). Modulation of sensory-evoked brain potentials
provide evidence for changes in perceptual processing during visual-spatial priming.
Journal of Experimental Psychology: Human Perception and Performance, 17, 1057–1074.
Mangun, G. R., Hillyard, S. A., and Luck, S. J. (1993). Electrocortical substrates of visual
selective attention. In D. Meyer a n d S. Kornblum (Eds.), Attention and Performance XIV, p p .
219–243. Cambridge, MA: MIT Press.
Mangun, G. R., Hopfinger, J., Kussmaul, C., Fletcher, E., a n d Heinze, H. J. (1997). Covaria-
tions in ERP and PET measures of spatial selective attention in h u m a n extrastriate cortex.
Human Brain Mapping, 5, 273–279.
Mangun, G. R., Hopfinger, J., and Heinze, H. J. (1998). Integrating electrophysiology a n d
neuroimaging in the study of h u m a n cognition. Behavior Research Methods, Instruments and
Computers, 30, 118–130.
Mangun, G. R., Luck, S. J., Plager, R., Loftus, W., Hillyard, S. A., Clark, V., Handy, T., a n d
Gazzaniga, M. S. (1994). Monitoring the visual world: Hemispheric asymmetries a n d sub-
cortical processes in attention. Journal of Cognitive Neuroscience, 6, 265–273.
Martinez, A., Anllo-Vento, L., Sereno, M. I., Frank, L. R., Buxton, R. B., Dubowitz, D. J.,
Wong, E. C., Heinze, H. J., a n d Hillyard, S. A. (1999). Involvement of striate and extrastriate
visual cortical areas in spatial selective attention. Nature Neuroscience, 2, 364–369.
McCarthy, G., Luby, M., Gore, J., and Goldman-Rakic, P. (1997). Infrequent events tran-
siently activate h u m a n prefrontal and parietal cortex as measured by functional MRI.
Journal of Neurophysiology, 77, 1630–1634.
Mesulam, M.-M. (1981). A cortical network for directed attention a n d unilateral neglect.
Annals of Neurology, 10, 309–325.
Moran, J., and Desimone, R. (1985). Selective attention gates visual processing in the extra-
striate cortex. Science, 229, 782–784.
Electrophysiology and Neuroimaging of Attention
Motter, B. C. (1993). Focal attention produces spatially selective processing in visual cortical
areas V1, V2 a n d V4 in the presence of competing stimuli. Journal of Neurophysiology, 70,
909–919.
Posner, M. I., a n d Cohen, Y. (1984). Components of visual attention. In H. Bouma and D.
Bowhuis (Eds.), Attention and Performance X, p p . 531–556. Hillsdale, NJ: Erlbaum.
Posner, M. I., and Driver, J. (1992). The neurobiology of selective attention. Current Opinion
in Neurobiology, 2, 165–169.
Posner, M. I., and Gilbert, C. D. (1999). Attention and primary visual cortex. Proceedings of
the National Academy of Science, U.S.A., 96, 2585–2587.
Posner, M. I., and Petersen, S. E. (1990). The attention system of the h u m a n brain. Annual
Review of Neuroscience, 13, 25–42.
Posner, M. I., Snyder, C. R. R., and Davidson, B. J. (1980). Attention a n d the detection of sig-
nals. Journal of Experimental Psychology: General, 109, 160–174.
Posner, M. I., Walker, J. A., Friedrich, F. A., a n d Rafal, R. D. (1984). Effects of parietal injury
on covert orienting of attention. Journal of Neuroscience, 4, 1863–1874.
Roelfsema, P. R., Lamme, V. A. F., a n d Spekreijse, H. (1998). Object-based attention in the
primary visual cortex of the macaque monkey. Nature, 395, 376–381.
Rugg, M. D. (1998). Convergent approaches to electrophysiological a n d hemodynamic
investigations of memory. Human Brain Mapping, 6, 394–398.
Sereno, M., Dale, A., Reppas, J., Kwong, K., Belliveau, J., Brady, J., Rosen, B., and Tootell, R.
(1995). Borders of multiple visual areas in h u m a n s revealed by functional magnetic reso-
nance imaging. Science, 268, 889–893.
Somers, D. C., Dale, A. M., Seiffert, A. E., and Tootell, R. B. H. (1999). Functional MRI reveals
spatially specific attentional modulation in h u m a n primary visual cortex. Proceedings of the
National Academy of Science, U.S.A., 96, 1663–1668.
Steinmetz, M. A., Connor, C. E., Constantinidis, C., and McLaughlin, J. R. (1994). Covert
attention suppresses neuronal responses in area 7a of the posterior parietal cortex. Journal of
Neurophysiology, 72, 1020–1023.
Treisman, A. M. (1988). Features a n d objects: The fourteenth Bartlett memorial lecture.
Quarterly Journal of Experimental Psychology, 40A, 201–237.
Tootell, R., Reppas, J., Kwong, K., Malach, R., Born, R., Brady, T., Rosen, B., and Belliveau,
J. W. (1995). Functional analysis of h u m a n MT and related visual cortical areas using mag-
netic resonance imaging. Journal of Neuroscience, 15, 3215–3230.
Van Essen, D., and DeYoe, E. (1995). Concurrent processing in the primate visual cortex. In
M. S. Gazzaniga (Ed.), The cognitive neurosciences, p p . 383–400. Cambridge, MA: MIT Press.
Van Voorhis, S. T., and Hillyard, S. A. (1977). Visual evoked potentials and selective atten-
tion to points in space. Perception and Psychophysics, 22, 54–62.
Vidyasagar, T. R. (1998). Gating of neuronal responses in macaque primary visual cortex by
an attentional spotlight. NeuroReport, 9, 1947–1952.
Watanabe, T., Sasaki, Y., Miyauchi, S., Putz, B., Fujimaki, N., Nielsen, M., Takino, R., a n d
Miyakawa, S. (1997). Attention-regulated activity in h u m a n primary visual cortex. Journal of
Neurophysiology, 79, 2218–2221.
Woldorff, M. G. (1993). Distortion of ERP averages d u e to overlap from temporally adjacent
ERPs: Analysis and correction. Psychophysiology, 30, 98–119.
Hopfinger, Jha, Hopf, Girelli, and Mangun
Woldorff, M. G., Fox, P., Matzke, M., Lancaster, J., Veeraswamy, S., Zamarripa, F., Seabolt,
M., Glass, T., Gao, J., Martin, C., and Jerabeck, P. (1997). Retinotopic organization of the early
visual-spatial attention effects as revealed by PET and ERPs. Human Brain Mapping, 5,
280–286.
Worden, M., Schneider, W., a n d Wellington, R. (1996). Determining the locus of attentional
selectivity with functional magnetic resonance imaging. Cognitive Neuroscience Society
Abstracts, 3, 101.
Yamaguchi, S., Tsuchiya, H., and Kobayashi, S. (1994). Electroencephalographic activity
associated with shifts of visuospatial attention. Brain, 117, 553–562.
153 Electrophysiology and Neuroimaging of Attention
6 Looking Forward to Looking: Saccade Preparation and Control of the Visual
Grasp Reflex
Robert Rafal, Liana Machado, Tony Ro,
a n d Harris Ingle
ABSTRACT Studying eye movements is a useful avenue to understanding the neural
basis of automaticity and control from an evolutionary perspective. Two experiments inves-
tigated control of the midbrain circuits responsible for the primitive visual grasp reflex.
Experiment 1 showed that saccade preparation by normal participants reduced the fixation
offset effect—the benefit in saccade latency afforded by elimination of a fixation stimulus.
This implies strategic control over fixation neurons in the rostral pole of the superior col-
liculus, suggesting some autonomy from reflexive activation by external signals at fixation.
Experiment 2 showed that in patients with chronic, unilateral lesions involving the frontal
eye fields the visual grasp reflex is selectively disinhibited to signals in the contralesional
field. These findings are consistent with the view of the frontal cortex as a tool maker
that manipulates phylogenetically older neural circuits, evolved to provide for reflexive
responses to the environment, putting them to new uses in the service of coherent, goal-
directed behavior a n d creative problem solving.
Our neural machinery for visual orienting, like the rest of us, is the
product of a long evolutionary history (Ingle 1973). All vertebrates
have midbrain circuits for reflexively orienting the eyes toward salient
events occurring in the visual periphery—the visual grasp reflex (VGR).
In foveate mammals, including humans, these archetypal pathways func-
tion to align high-acuity regions of the retina to the location of a s u d d e n
change in the visual periphery; they must also be integrated with cortical
mechanisms involved in strategic search under endogenous control. In
everyday life, the outside world and internally generated goals place
constantly competing d e m a n d s on visual orienting systems. Coherent
a n d adaptive behavior requires control mechanisms to arbitrate between
these competing d e m a n d s a n d to coordinate responding. Much is known
about the neural basis of visual orienting; the coordination of its reflexive
a n d voluntary deployment provides a useful framework for understand-
ing the psychobiology of automaticity a n d control from an evolutionary
perspective.
In this chapter, we consider strategic control over the midbrain circuits
responsible for the primitive VGR. We proceed from a perspective that
the nervous system routinely goes about its business through an orches-
tration of reflexes by endogenous processes that can activate or inhibit
them (Easton 1973). The first experiment examines the effects of volun-
tary saccade preparation on the fixation offset effect (FOE; Klein a n d
Kingstone 1993). We use the FOE—the reduction in saccade latency
afforded by the offset of a fixation point—as a marker for the status of the
collicular circuits with respect to the generation and inhibition of the
VGR. This experiment shows that normal individuals are endogenously
able to control the midbrain fixation reflex that inhibits the VGR. The
second experiment examines the effects of chronic unilateral lesions,
restricted to either dorsolateral prefrontal cortex (DLPFC) or parietal cor-
tex, on errors in an antisaccade task. The antisaccade task requires both
the inhibition of the VGR toward the peripheral stimulus a n d the endoge-
nous generation of a voluntary saccade in the other direction. It allows
examination of direct competition between reflexive a n d voluntary neu-
ral systems for visual orienting.
6.1 EXPERIMENT 1: THE EFFECTS OF VOLUNTARY SACCADE
PREPARATION ON THE FIXATION OFFSET EFFECT
The visual grasp reflex is controlled by an opponent process mechanism
within the superior colliculus (SC) of the midbrain. Its activation—or
inhibition—is determined by competition between collicular neurons for
maintaining fixation and others for generating a saccadic eye movement
to a new location. Both in the superficial sensory layers a n d in the
deeper layers in which movement cells generate saccadic eye move-
ments, the SC has a topographic m a p of the visual field (Wurtz a n d
Albano 1980). The colliculus uses a place code for eye movements such
that the vector of a saccade is contingent on the location activated within
the colliculus. Cells in the rostral pole help to hold the eyes anchored at
fixation (Munoz and Wurtz 1992). As one moves caudally in the SC, neu-
rons code saccades of increasing amplitude into the contralateral field.
Like all other collicular neurons, fixation neurons in the rostral pole have
mutually inhibitory connections with movement neurons throughout
the colliculus (Munoz and Istvan 1998), a n d pharmacological inactivation
of these neurons is associated with disinhibition of reflexive saccades
(Munoz and Wurtz 1993a,b). Thus the potential for an eccentric visual
stimulus to trigger a VGR—pulling the eyes to the stimulus—is deter-
mined by the relative activity of neurons in the caudal colliculus, with a
movement field toward the stimulus, and of rostral pole neurons that
inhibit movement cells and maintain fixation.
Fixation neurons are active during fixation, even in darkness, a n d
increase their activity when a fixation signal is present. Hence the pres-
ence of a fixation point inhibits the VGR, increasing saccade latencies.
The reduction in saccade latency w h e n a fixation point offsets—the
fixation offset effect—thus provides a measure of the degree to which
Rafal, Machado, Ro, a n d Ingle
fixation neurons are being driven by a fixation stimulus. Saslow (1967)
first observed that the offset of a fixation point decreases the latency of
saccades to visual targets in an experiment where there was a temporal
gap between fixation offset and target onset. This facilitation has been
called the “gap effect.’’ A gap of approximately 200 msec is optimal for
reducing saccade latencies. Under some circumstances, a robust gap
effect may generate a bimodal distribution of saccade latencies, with a
separate peak of very fast “express saccades’’ having latencies of less than
100 msec (Fischer and Ramsperger 1984). On the other hand, Kingstone
a n d Klein (1993a) and Reuter-Lorenz, Hughes, and Fendrich (1991)
emphasize that a temporal gap between fixation offset and the target pro-
vides a component of general alerting, in addition to the specific effects
on saccades afforded by fixation offset. This component, specifically d u e
to oculomotor disengagement and not to general alerting, represents the
FOE a n d can be obtained even when fixation offset is simultaneous with
target onset (Fendrich, Demirel, and Danziger 1999). The decrease in
activity of the fixation neurons with fixation offset represents a neural
correlate of express saccades (Dorris a n d Munoz 1995).
The fixation reflex, in which a visual signal at the point of fixation
reflexively activates fixation neurons that inhibit the VGR to eccentric
events, is especially strong in early infancy (Johnson 1990). At about 2
months of age, the colliculi come under the unopposed inhibitory
influence of the basal ganglia (substantia nigra pars reticulata). Infants
may become distressed because they are unable to break the lock of a
visual stimulus in order to move their eyes. The FOE decreases during
infant development (Hood, Atkinson, and Braddick 1997; Johnson a n d
Gilmore 1997), marking maturation of frontobasal ganglia-colliculus cir-
cuits that brings the fixation reflex under voluntary control, permitting
efficient visual search with alternating saccades and fixations.
Moreover, normal adults can learn to make express saccades with prac-
tice, even while the fixation point remains visible (Fischer and Breitmeyer
1987). The implication here is that normal participants may be able to vol-
untarily inhibit collicular pole fixation neurons even in the presence of a
visual fixation stimulus. With fixation cell activity being more under
strategic control than under the exogenous influence of a fixation point,
the FOE should be reduced because fixation cell activity is less influenced
by the presence or absence of a fixation point. In experiment 1, we exam-
ined whether normal adults can modulate the FOE when they voluntarily
prepare an eye movement.
Participants
Twenty-six undergraduates participated for course credit: 13 in experi-
ment 1a a n d 13 in experiment 1b.
Control of the Visual Grasp Reflex
Apparatus, Stimuli, and Procedure
The apparatus for stimulus display a n d eye movement recording is
detailed in Ro, et al. 1997. Saccade latency was defined as the time when
eye velocity exceeded 60 degrees/second. Participants were tested sitting
in a quiet, dimly lit room facing a video display monitor 54 cm in front of
them. The display consisted of white stimuli on a black background. After
an intertrial interval of 1 sec, each trial began with presentation of a 0.35-
degree fixation dot flanked by 2-degree unfilled squares, 8-degrees to the
left a n d right. After 500 msec, the fixation dot was replaced by a 1-degree
central precue. On one half of the trials, the precue was an arrowhead that
pointed (with 100% probability, except on catch trials—(16.7% of total
trials) to the location of the forthcoming saccade target, thus permitting
saccade preparation prior to target appearance. On the other random half
of trials, the precue w a s a double-headed arrow that w a s uninformative
about the location of the forthcoming target, so that participants could
not fully program a saccade until the target appeared. Fixation stimulus
(the precue) offset versus overlap with target onset was also manipulated
independent of precue validity. On half the trials, the precue offset simul-
taneously with target onset (fixation offset condition); on the other half,
the precue remained visible until response to the target (fixation overlap
condition). The target was a 1.5-degree asterisk that appeared with
equal frequency in the center of either of the two peripheral boxes a n d
remained visible until either a response was recorded or 3 sec lapsed. In
experiment 1a, the target appeared either 200 or 700 msec after precue
onset; in experiment 1b, the cue-target intervals were 75 and 700 msec.
Participants were instructed to prepare a saccade if a single arrowhead
was presented, but to maintain fixation until a target appeared, at which
time they were to make an eye movement to the target as quickly as pos-
sible. Catch trials were used to test for compliance with these instruc-
tions, and participants practiced until it w a s clear that they understood
the task. Eye movements initiated during the interval between fixation
onset a n d target onset terminated the trial and triggered the computer to
buzz. In both experiment 1a and 1b, participants completed a single ses-
sion of test trials (384 and 192 trials, respectively).
Results
In both experiments, trials were excluded from analysis if saccade
latencies were less than 75 msec (2.6%) or more than 1,000 msec (4.5%).
Responses on catch trials (8.9%, not including responses that appeared to
be blinks) exceeded 10% for 6 individuals in experiment 1a a n d for 4 in
experiment 1b. The analyses presented below included all participants,
although analyses that excluded individuals with more than 10% catch
trial responses gave comparable results. Saccade latencies for each par-
Rafal, Machado, Ro, a n d Ingle
Table 6.1 Mean Saccade Latencies in Milliseconds (Standard Deviation in Parentheses) for
Experiments 1a and 1b
Condition
Uninformative
Precue
Informative
Precue
Condition
Uninformative
Precue
Informative
Precue
Experiment 1a
200 msec precue-target interval
Fixation Fixation
overlap offset
330 271
(34) (40)
292 253
(33) (36)
Fixation
offset
effect
59
39
Experiment 1b
75 msec precue-target interval
Fixation Fixation
overlap offset
347 285
(40) (36)
298 268
(40) (49)
Fixation
offset
effect
62
30
700 msec precue-target interval
Fixation
overlap
322
(44)
323
(32)
Fixation
offset
255
(31)
274
(36)
Fixation
offset
effect
67
54
700 msec precue-target interval
Fixation
overlap
334
(43)
324
(44)
Fixation
offset
280
(39)
284
(36)
Fixation
offset
effect
54
40
ticipant were subjected to an analysis of variance (ANOVA). Within-
subject factors included precue (informative or uninformative), fixation
(offset or overlap), and cue-target interval (stimulus onset asynchrony of
200 or 700 msec) in experiment 1a, and of 75 or 700 msec in experiment
1b).
Both experiment 1a and experiment 1b showed effects of precue:
F(1, 12) =4.6, p =0.05; F(1, 12) =7.4, p<0.02, respectively; and fixation
offset: F(1, 12) = 96.0, p < 0.001; F(1, 12) = 95.7, p < 0.001, respectively. Both
showed an interaction between precue and cue-target interval: F(1, 12) =
31.6, p< 0.001; F(1, 12) = 14.3, p< 0.005, respectively. As shown in table
6.1, there was a benefit of an informative precue on saccade latency at the
short intervals. Because saccade latency increased in the informative pre-
cue condition between the short and long intervals, however, no benefit
of an informative precue was present at the 700 msec interval in either
experiment 1a or 1b. Both showed a reduction of the FOE in the saccade
preparation (informative precue) condition: (F(1, 12) = 14.4, p< 0.005;
F(1, 12) = 10.9, p < 0.01), respectively. The effect of precue on the FOE was
present at both cue-target intervals and did not interact with their SOA.
Discussion
The major result of this experiment, with regard to strategic control of the
VGR, is that saccade preparation reduced the FOE. This finding suggests
159 Control of the Visual Grasp Reflex
that the midbrain fixation reflex—the otherwise automatic tendency of
a fixation stimulus to hold the eyes—can be voluntarily controlled. It
appears that collicular fixation neurons can be inhibited voluntarily when
a saccade is prepared, even when a fixation point is visible. Because the
frontal eye fields are considered responsible for initiating voluntary sac-
cades, one possibility is that their projections can inhibit the rostral pole
fixation neurons in the ipsilateral colliculus, either directly or through the
basal ganglia, even in the presence of a fixation stimulus that would
otherwise activate those neurons.
It seems likely that the reduction of the FOE afforded by an informative
precue is attributable to voluntary preparation of a saccade, not simply
from covert orienting to the cued field. Walker, Kentridge, a n d Findlay
(1995) showed that precues that enabled covert orienting of attention, but
not saccade preparation, did not influence the gap effect. In their experi-
ment, saccade targets could appear at either of two possible locations
(near or far) in either the left or right field. Informative precues enabled
participants to orient their attention covertly toward the cued field.
Because, however, the cue d i d not indicate which location the target
would appear at in the cued field, near or far, participants could not fully
program a saccade until the target appeared.
An intriguing dissociation in this experiment was observed between
the effects of an informative precue over the fore period on mean saccade
latency and on the FOE. The benefit of an informative precue on saccade
latency at the short precue-target interval was not sustained at the long
interval, whereas the reduction in the FOE afforded by informative pre-
cues w a s maintained throughout the fore period. Our previous work
(Rafal et al. 1989) has shown that preparation, and then cancellation, of
an endogenous saccade activates an inhibitory tag called “inhibition of
return’’ (IOR) at the location toward which the saccade had been pre-
pared. One possibility suggested by the current results is that IOR is
generated even when an endogenously prepared eye movement is not
canceled, resulting in the loss of informative precue benefit over time.
However, while the sustained effect of an informative precue on the FOE
indicates that some preparatory state was sustained, it is not clear what
type of preparation was being maintained: the orienting of covert atten-
tion, the preparation of a saccade program, or some other aspect of prepa-
ration. We are conducting further experiments to test whether the loss of
the informative precue benefit is d u e to generation of IOR by saccade
preparation; and hence whether IOR can influence target detection even
at actively attended locations.
6.2 EXPERIMENT 2: EFFECTS OF CORTICAL LESIONS ON THE
VISUAL GRASP REFLEX IN AN ANTISACCADE TASK
The antisaccade task, in which a saccade must be m a d e away from a
peripheral target, d e m a n d s both that the visual grasp reflex be inhibited
160 Rafal, Machado, Ro, a n d Ingle
a n d that a voluntary saccade be generated toward the opposite field. One
mechanism for preventing reflexive glances toward a peripheral visual
stimulus could be to increase the level of activity of rostral pole fixation
neurons that inhibit the VGR. The fixation offset effect has been shown to
be much reduced in an antisaccade situation (Forbes a n d Klein 1996;
Reuter-Lorenz, Hughes, a n d Fendrich 1991; Reuter-Lorenz et al. 1995),
possibly because frontal eye field (FEF) projections increase activity in
fixation neurons. In this situation, strategic control over fixation neurons
could allow individuals to maintain a high level of fixation cell activity
even in the absence of an external fixation point, reducing reflexive eye
movement errors.
Thus, both in antisaccade tasks a n d w h e n prosaccades are prepared,
the FOE is reduced because fixation neurons are less influenced by the
presence or absence of the external fixation point. In the case of antisac-
cades, a high rate of fixation cell activity is maintained even in the
absence of a fixation point, whereas in experiment 1, saccade preparation
may have decreased the rate of fixation cell activity, even in the presence
of a fixation point. Thus the FOE is reduced in both cases, but for differ-
ent reasons in response to opposite strategic requirements.
Neuropsychological evidence for a role of dorsolateral prefrontal cor-
tex in the control of the VGR in the antisaccade task w a s first reported by
Guitton, Buchtel, and Douglas (1985), w h o showed that unilateral dam-
age caused an increase in reflexive glances, that is, prosaccade errors, in
the antisaccade task. One possible mechanism for this deficit postulates
that the frontal lobes have a critical role in inhibition, a n d that lesions of
oculomotor cortex result in collicular disinhibition. On the other hand, it
has also been shown that the d e m a n d s of working memory are critical
determinants of errors in the antisaccade task (Roberts, Hager, and Heron
1994). Because lesions of DLPFC cause impairments of working memory
(e.g., Funahashi, Bruce, and Goldman-Rakic 1991), the reflexive glances
m a d e by patients with frontal lobe damage may not necessarily reflect
loss of inhibitory control, but instead may result from reduced working-
memory capacity (Kimberg and Farah 1993; see also Kimberg a n d Farah,
chap. 32, this volume). A reduction in working memory that prevents
patients from maintaining task instructions should cause patients to
make errors to both the ipsilesional and contralesional field. In contrast,
if regions of oculomotor cortex are normally involved in inhibiting the
ipsilateral colliculus, then a unilateral lesion in that cortex might result in
an asymmetry where more reflexive glances are m a d e to targets appear-
ing in the contralesional visual field. Thus lesions involving the FEF
might be expected to result in disinhibition of the ipsilesional colliculus,
a n d an asymmetric pattern of reflexive glances. Some patients with
extensive DLPFC lesions may have bilateral increases in reflexive glances
d u e to impaired working memory, but if the lesion also involves the
FEF, the deficit may be asymmetric, with more contralesional than ipsi-
lesional errors.
161 Control of the Visual Grasp Reflex
Table 6.2 Clinical Information for Patients with Frontal Lesions
Patient
A.A.
W.A.
O.A.
J.C.
M.K.
K.K.
A.L.
R.M.
J.M.
L.S.
R.T.
E.B.
W.T.
Lesion
side
Left
Left
Left
Left
Right
Left
Left
Left
Left
Left
Left
Right
Right
FEF
damage
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
N o
N o
A g e /
Sex
30F
74F
64M
71M
64M
65M
68F
71M
71M
67F
80M
79F
52M
Volume
(cc)
59
26
18
106
200
14
56
14
15
28
46
17
9
Etiology
Stroke
Stroke
Stroke
Stroke
Aneurysm
Stroke
Stroke
Stroke
Shrapnel
Resection*
Stroke
Stroke
Resection*
Chronicity
(years)
3
10
12
9
17
13
16
8
3
15
11
14
8
* L.S. and W.T.’s brain lesions resulted from surgical resection of a meningioma and colloid
cyst, respectively.
Experiment 2 studied antisaccade performance in patients with
chronic, unilateral lesions of the FEF. Its goal was to determine whether
chronic lesions of frontal cortex cause an asymmetric impairment in in-
hibiting the VGR attributable to frontocollicular disinhibition, or bilateral
deficits attributable to reductions in working memory. One further goal
of this research was to examine the effect of cortical lesions on the ability
to use precues to modulate the FOE strategically in both prosaccade a n d
antisaccade tasks. Here, however, we report only the effect of these cor-
tical lesions on the incidence of reflexive glances m a d e toward the ipsi-
lesional and contralesional field, that is, errors d u e to disinhibition of
the VGR.
Participants
Eleven patients with chronic (tested at least t w o years after brain injury),
unilateral lesions of oculomotor cortex in the superior dorsolateral pre-
frontal region were studied (FEF group). Control subjects included 24
normal elderly individuals (mean age: 72 years; standard deviations: 6),
a n d neurological patients with unilateral lesions sparing the FEF: 2
with DLPFC lesions and 9 with lesions of parietal cortex (PAC; clinical
details of individual patients are provided in tables 6.2 a n d 6.3). The
region of lesion overlap in the FEF group is depicted in figure 6.1. All of
the patients with FEF damage except one had left-hemisphere lesions.
Both patients with DLPFC lesions sparing the FEF had right-hemisphere
lesions. Of the 9 PAC-lesioned patients, 4 h a d left-hemisphere lesions a n d
five h a d right-hemisphere lesions.
162 Rafal, Machado, Ro, a n d Ingle
Table 6.3 Clinical Information for Patients with Parietal Lesions
Patient
R.A.
L.L.
J.S.
R.S.
M.K.
T.E.
K.T.
L.P.
R.R.
Lesion
side
Left
Left
Right
Right
Left
Left
Right
Right
Right
TPJ
damage?
Yes
Yes
Yes
Yes
N o
N o
N o
N o
N o
A g e /
Sex
65M
73M
38M
51M
51M
46F
48F
72M
69M
Volume
(cc)
71
40
73
80
33
27
46
6
34
Etiology
Stroke
Stroke
Shrapnel
Stroke
Shrapnel
Stroke
Resection*
Stroke
Stroke
Chronicity
(years)
7.5
2.5
10
6.5
29
2
22
> 6
11
* K.T.’s brain lesion resulted from surgical resection of a glioma.
Figure 6.1 Composite reconstruction of neuroimages showing the common area of lesion
at the intersection of the superior frontal sulcus and the precentral sulcus in all patients in
the FEF group.
All of the patients h a d normal oculomotor function on standard clini-
cal testing of saccades, pursuit eye movements, and optokinetic nystag-
m u s . None h a d any clinical signs of hemispatial neglect, a scotoma in
the region of the stimuli, or a coexisting neurological disorder (e.g.,
Parkinson’s disease). Patients were not included in the study if review of
their neuroimages showed the lesion to approach or to undercut the mar-
gins of the FEF such that they could not be clearly classified as having or
not having FEF involvement. PET activation studies (reviewed in Paus
1996) a n d a study in our laboratory using transcranial magnetic stimu-
lation (Ro et al. 1999) have localized the brain region responsible for
generating voluntary, contraversive saccades to cortex centered at the
intersection of the superior frontal sulcus and the precentral sulcus
(Brodmann’s area 6), approximately 2 cm rostral to the motor h a n d area.
Our selection criterion for patients assigned to the FEF group was guided
by these findings, a n d also by stimulation studies in h u m a n s that have
163 Control of the Visual Grasp Reflex
Table 6.4 Mean Percentage of Reflexive Eye Movement Errors (Standard Deviation in
Parentheses) Made by Each Group of Subjects
Uninformative precue
Overlap
Field: Normal elderly (n = 24)
Left 14
(11)
Right 17
(15)
PAC-lesioned patients (n = 9)
Ipsilesional 16
(12)
Contralesional 11
(7)
Offset
14
(10)
19
(16)
17
(12)
9
(9)
DLPFC-lesioned patients with FEF involvement (n = 11)
Ipsilesional 11
(10)
Contralesional 22
(10)
13
(11)
27
(18)
DLPFC-lesioned patients without FEF involvement (n = 2)
Ipsilesional 16
(5)
Contralesional 13
(18)
20
(4)
11
(9)
Informative precue
Overlap
8
(7)
11
(12)
12
(10)
6
(6)
9
(7)
23
(10)
11
(12)
9
(9)
Offset
10
(9)
11
(10)
12
(10)
10
(13)
13
(9)
21
(16)
18
(25)
15
(21)
Total
12
14
14
9
11
23
16
12
demonstrated that saccades may be elicited from regions somewhat more
rostral, including Brodmann’s area 8 (Penfield a n d Rasmussen 1950).
Apparatus, Stimuli, and Procedure
The same apparatus was used as in experiment 1. We measured the n u m –
ber of reflexive glances toward peripheral targets during an antisaccade
task using the same stimuli a n d procedure as in experiment 1 except for
the following: (1) participants were instructed to make an eye movement
to the box opposite the target as soon as the target appeared; (2) the
unfilled squares that marked the two possible target locations, a n d hence
the targets, appeared 6-degrees to the left and right of fixation; (3) only
the 200 msec precue-target interval was used; (4) the target was a 0.35-
degree white dot; (5) the intertrial interval was 1,500 msec; and (6) there
was a total of 384 test trials, 320 target-present trials plus 64 catch trials.
Results
All reflexive glances (i.e., errors in which saccades were m a d e toward,
rather than away from, the peripheral target) with saccadic latencies less
164 Rafal, Machado, Ro, a n d Ingle
Table 6.5 Percentage of Reflexive Glances to Ipsilesional and Contralesional Fields for
Each Patient
Patient
O.A.
J.C.
J.M.
R.M.
M.K.
A.A.
K.K.
L.S.
A.L.
W.A.
R.T.
E.B.
W. T.
Frontal lesion
Ipsilesional Contralesional
FEF lesion
2
19
4
4
13
10
18
8
19
25
3
12
28
11
12
19
11
28
43
40
36
17
No FEF lesion
24
8
22
2
Patient
R.S.
L.L.
R.A.
J.S.
L.P.
K.T.
M.K.
R.R.
T.E.
Parietal lesion
Ipsilesional Contralesional
TPJ lesion
14 1
22 9
14 9
11 5
No TPJ lesion
28 21
5 3
1 4
25 23
9 3
than 80 msec or greater than 500 msec were excluded from analysis. We
conducted an ANOVA of the percentage of remaining reflexive eye move-
ments made by the FEF-lesioned, PAC-lesioned, and normal elderly
groups, collapsed across field of target, with precue (informative or uninfor-
mative) and fixation (offset or overlap) as within-subject factors. There
was a main effect of precue condition, showing that informative precues
reduced the number of reflexive glances to targets: F(1, 41) = 16.896,
p < 0.001. No other main effects or interactions were statistically reliable
(p > 0.1). The FEF and PAC groups were each separately compared to the
normal elderly group, confirming that neither of these patient groups dif-
fered from the normal elderly group in its overall percentage of reflexive
eye movements (p > 0.1 for each comparison).
Table 6.4 shows the percentage of erroneous reflexive eye movements
made toward each visual field for each group. Individual ANOVAs of
the normal elderly, FEF, and PAC group data, with precue type, fixation
condition, and target side as within-subject variables, were conducted to
test for any visual field asymmetries. The results showed that the normal
elderly group did not show a significant asymmetry (p > 0.2) between the
left and right fields. FEF patients made more reflexive glances toward
contralesional targets: F(1, 10) = 18.018, p= 0.002; however, the two
patients with DLPFC lesions that spared the FEF did not make more
reflexive glances toward the contralesional field than the ipsilesional
field. PAC patients made significantly fewer reflexive glances toward con-
tralesional targets: F(1, 8) = 10.840, p = 0.011. (See table 6.5 for individual
patient data.)
165 Control of the Visual Grasp Reflex
Because difficulties in disengaging attention in PAC patients have been
associated with lesions of the temporal-parietal junction (TPJ; Friedrich et
al. 1998), five of w h o m also participated in the current study, we did a
lesion subregion analysis of the PAC group, comparing the patients
whose lesions involved the TPJ to those whose lesions did not, based on
whether neuroimaging showed the lesion to involve the posterior part of
the superior temporal gyrus (area 22). The results showed an interaction
between lesion site and target side: F(1, 7) = 5.555, p < 0.05. Separate
analysis of the PAC patients with TPJ involvement (n = 4) and those with-
out TPJ involvement (n = 5) revealed that only the TPJ group made
significantly fewer reflexive glances toward the contralesional field:
F(1,3) = 19.667, p = 0.02. The group whose lesions spared the TPJ did not
show a significant effect of target side (p > 0.2). Because the average lesion
volume was greater in TPJ than in non-TPJ-lesioned patients (66.0 versus
29.4 cc), the data were examined to determine whether the apparent dif-
ferences between the two groups could be attributed to differences in
lesion volumes. Inspection of tables 6.3 and 6.5 shows no correlation
between the degree of field asymmetry and lesion volume.
Discussion
The current observations extend our understanding of corticocollicular
interactions as a model system for the control of automaticity. FEF lesions
resulted in a disinhibition of the VGR specifically to contralesional sig-
nals. This is consistent with evidence that chronic FEF lesions result in
hyperactivity of the ipsilesional colliculus (Henik, Rafal, and Rhodes
1994). Note, however, that in our study there were only two DLPFC
control patients without FEF involvement, and both of them had right-
hemisphere lesions. By contrast, 10 of the 11 patients with FEF lesions had
left-hemisphere involvement. Unlike the FEF patients, patients with
lesions of the TPJ showed a decrease in reflexive glances toward the con-
tralesional field. This is consistent with evidence that parietal lesions
result in hypoactivity of the ipsilesional colliculus (Sprague 1966).
The field-specific effect shown by the FEF patients in our study sug-
gests that the FEF may normally suppress unwanted reflexive eye move-
ments by exciting fixation cells in the ipsilateral SC. Fixation cells in turn
project to the brain stem omnipause neurons (Paré & Guitton, 1994) which,
like fixation cells, show a high firing rate when the eyes are stationary, but
which pause during saccades and inhibit brain stem premotor neurons
that innervate the oculomotor muscles (reviewed in Büttner-Ennever and
Horn 1997; Everling et al. 1998). Alternatively, the FEFs may inhibit
reflexive glances through their direct projection to the omnipause neu-
rons (reviewed in Moschovakis, Scudder, and Highstein 1996).
To date, reports of antisaccade performance in groups of patients
with unilateral frontal damage have been inconsistent. Several investiga-
Rafal, Machado, Ro, a n d Ingle
tors reported bilateral disinhibition of the VGR (Fukushima et al. 1994;
Guitton, Buchtel, and Douglas 1985; Pierrot-Deseilligny et al. 1991), while
some found either ipsilateral disinhibition (Fukushima et al. 1994) or no
disinhibition (Rivaud et al. 1994).
Discrepancies in whether unilateral disruption of FEF activity results in
contralesional or ipsilesional deficits are also present in the monkey liter-
ature. Burman and Bruce (1997) found that electrically stimulating some
cells in the monkey FEF, but not anterior or posterior to the FEF, inhibited
both memory a n d visually guided saccades, especially those directed
toward the contralesional field. Accordingly, permanent lesions of the
monkey FEF led to frequent premature saccades toward contralesional
targets during a memory-guided saccade task (Deng et al. 1986). On the
other hand, acute chemical inactivation of the monkey FEF led to prema-
ture saccades primarily toward ipsilateral targets during the delay of a
memory-guided saccade task (Dias, Kiesau, and Segraves 1995; Dias a n d
Segraves 1997; Sommer a n d Tehovnik 1997).
The apparent inconsistencies in the effects of FEF damage may relate to
the heterogeneity of the lesions, both in terms of (1) anatomical extent—
not all of the frontal patients in previous investigations h a d FEF damage;
a n d (2) chronicity—most previous investigations examined patients in
the acute stage of illness, w h e n diaschesis (the remote effects of an acute
lesion on neural structures with which damaged tissue has been inter-
connected) may have contributed to collicular dysfunction on the side of
the frontal lesion. After acute lesions of the FEF in monkeys, there is
hypometabolism of the SC on the side of the lesion (Deuel a n d Collins
1984), a n d clinical hemispatial neglect during this acute stage is common
because both cortical and subcortical orienting systems on the side of the
lesion are dysfunctional. After the acute phase of diaschesis resolves,
however, neglect recovers, and monkeys are able, within weeks, to make
saccades to contralesional targets with normal latencies (Schiller, Sandell,
a n d Maunsell 1994).
H u m a n s with chronic lesions of the FEF show evidence that the ipsi-
lesional colliculus becomes hyperactive and the contralesional colliculus
becomes hypoactive. Henik, Rafal, a n d Rhodes (1994) demonstrated that
patients with chronic, unilateral FEF lesions (four of w h o m also partici-
pated in the current study) have shorter latencies to initiate saccades to
contralesional targets. The patients in the current study were also tested
in a prosaccade task using the same display as used for the antisaccade
task. The results, to be reported in more detail elsewhere, replicated
Henik et al.: latencies of saccades toward contralesional targets were
shorter.
Two single-case studies of patients with unilateral FEF damage demon-
strated the effects of the chronicity of the lesion on changes in collicular
activity for behavior in an antisaccade task. In a series of sessions begin-
ning 5 days after a stroke and ending 170 days after the stroke, Butter et
Control of the Visual Grasp Reflex
al. (1988) reported that a patient with a right frontal infarct including the
FEF initially exhibited contralesional sensory neglect. Although the per-
centage of saccades erroneously m a d e toward ipsilesional targets did not
change over the testing sessions, the percentage of reflexive saccades
m a d e toward contralesional targets increased dramatically as the patient
recovered from neglect. Kwon a n d Heilman (1991) later replicated this
effect of neglect on disinhibition of the VGR, a n d extended it to the limb
system using a line bisection task, in a patient with right frontal damage,
including Brodman’s areas 6 a n d 8.
In addition to the evolving effects of FEF lesions on collicular circuitry,
some patients with frontal lobe lesions also have reduced working mem-
ory that may disrupt antisaccade performance bilaterally (Walker et al.
1998). Some models providing a unified account of the deficits associated
with frontal lobe damage suggest that the ability to inhibit automatic
responses depends on working memory (e.g., Kimberg a n d Farah 1993;
see also Kimberg a n d Farah, chap. 32, this volume). Asking college stu-
dents to perform an antisaccade task simultaneously with a variety of
secondary tasks that placed different d e m a n d s on working memory,
Roberts, Hager, and Heron (1994) found that performing a concurrent
task with high working memory d e m a n d s caused normal participants
to make twice as many erroneous reflexive glances. These authors sug-
gested that the difficulty in inhibiting reflexive glances shown by frontal
patients, a n d by normal participants when working memory is taxed,
stems from not “maintaining a high enough level of activation of the rel-
evant self-instructions.’’ Guitton, Buchtel, a n d Douglas (1985) reported
that simplifying their antisaccade task in patients with frontal lesions, by
eliminating an identification response that was required after each eye
movement, diminished the frequency of reflexive glances. This reduction
in reflexive errors supports the possibility that working memory plays a
role in inhibiting reflexive glances and that this effect could be indepen-
dent of the unilateral effect that results from disruption of a cortico-
collicular pathway.
The unique contribution of the current investigation among both animal
a n d patient studies of antisaccade performance is the specific examina-
tion of the effects of chronic cortical lesions. The results help to reconcile
apparent inconsistencies in the neurological literature. The cortico-
subcortical circuits for controlling eye movements are a dynamic system,
a n d the effects of FEF lesions on collicular function may evolve over time.
Initially, frontal lesions cause hypometabolism in the ipsilesional col-
liculus (Deuel and Collins 1984). In the acute stage, patients often have
transient hemispatial neglect because both the cortical a n d subcortical
components of the orienting system are dysfunctional on the side of the
lesion. This causes hyperorienting toward the side of the lesion, and a
disinhibited ipsilesional VGR. By contrast, the current findings a n d our
earlier findings in a prosaccade task (Henik, Rafal, a n d Rhodes 1994)
Rafal, Machado, Ro, a n d Ingle
suggest that, in the chronic stage, the ipsilesional colliculus is hyperac-
tive, causing a disinhibited contralesional VGR, whereas the colliculus
contralateral to the FEF lesion is relatively hypoactive. The result is an
asymmetric deficit, with a disinhibited VGR only toward contralesional
targets. The effects of frontal lesions on antisaccade performance are thus
determined by (1) the chronicity of the lesion; (2) the state of dynamic
interaction between cortex a n d subcortex; (3) the specific regions of
DLPFC involved; and (4) the degree of working-memory impairment.
6.3 CONCLUSIONS
This research w a s motivated by the hypothesis that fixation neurons in
the rostral pole of the superior colliculus, which are activated by stimuli
at fixation to hold the eyes in place, may also be under endogenous con-
trol via frontocollicular pathways. These pathways enable us to free the
oculomotor system from reflexive responses to the environment, and per-
mit coherent behavior based on strategic goals. The results of the current
experiments are consistent with this hypothesis. Saccade preparation in
response to informative precues reduced the fixation offset effect in nor-
mal individuals, a n d damage to the frontal eye field released the visual
grasp reflex toward signals appearing in the contralesional field.
The current evidence for endogenous control over fixation converges
with other evidence for cortical control of collicular circuitry. Express sac-
cades are classically associated with conditions in which there is a tem-
poral gap between fixation offset a n d target onset. Nevertheless, with
practice, some individuals can be trained to make express saccades even
when a fixation point is present (Fischer a n d Breitmeyer 1987). Machado
a n d Rafal (in press) have recently examined another manipulation of
strategic set on the FOE—the proportion of catch trials (in which no sac-
cade target appeared a n d fixation h a d to be maintained). In these exper-
iments, prosaccades were made, and there were no informative precues;
rather, the proportion of catch trials was manipulated systematically.
When catch trials were less frequent, the FOE was smaller. Kingstone a n d
Klein (1993b) have shown that decreasing the proportion of catch trials
decreases saccade latency in a gap condition. A reduction in the FOE d u e
to fewer catch trials, as found by Machado a n d Rafal, would require a
greater decrease in saccade latencies in the fixation overlap condition
than in the offset condition. We can infer, then, that the decrease in the
FOE with fewer catch trials may result from cortical inhibition of fixation
neurons even w h e n a fixation point is present—the same mechanism we
are proposing for the reduction of the FOE in experiment 1.
The reduction of the FOE in an antisaccade task may also be d u e to
strategic control of the VGR (Forbes and Klein 1996). In this circumstance,
however, the FOE is presumably decreased because cortical control can
maintain a high level of activity of fixation neurons, even when there is
Control of the Visual Grasp Reflex
no external stimulus at fixation. In both cases, prosaccade a n d anti-
saccade, the FOE is reduced because the oculomotor set makes fixation
neurons more autonomous from external stimuli a n d more u n d e r
endogenous control.
Experiment 2 showed that FEF lesions caused disinhibition of the VGR
specifically to contralesional targets. It demonstrated that the same region
of the superior dorsolateral prefrontal cortex, the FEF, that is responsible
for generating contralateral voluntary saccades is also responsible for
inhibiting the VGR. The critical role of the FEF in voluntary saccade gen-
eration in h u m a n s has been established by converging evidence. It is acti-
vated during voluntary saccades (Paus 1996); a n d lesions in it (Henik,
Rafal, and Rhodes 1994) or its transient inactivation by transcranial
magnetic stimulation (TMS; Ro et al. 1999, 1997) increases the latency of
contralateral, voluntary saccades. An important mission of the FEF in
controlling voluntary saccades is the ability to inhibit reflexive eye move-
ments, when necessary. We suggest that this mission is accomplished by
the same kind of voluntary regulation of the opponent neural circuitry of
the colliculus that normal participants exhibited in experiment 1.
Although the antisaccade task is an artificial situation contrived in the
laboratory, h u m a n s (as well as prey and other social animals) frequently
inhibit reflexive eye movements in natural conditions. The ability to
inhibit eye movements may have evolved in conjunction with the ability
to covertly orient visual attention to meet specific adaptive requirements,
for example, the need of a prey animal to attentively track a predator
without establishing eye contact that could attract attention to itself; the
need of a juvenile primate to keep track of the doings of the alpha male
while avoiding confrontation; or the need of a h u m a n to pay attention to
the h a n d s of an approaching stranger while maintaining eye contact.
One way of thinking about the frontal cortex is as the brain’s tool user
a n d tool maker. Its great expansion in h u m a n s is paralleled by increasing
flexibility in goal-directed behavior, and creativity in problem solving.
The frontal cortex orchestrates the novel use of phylogenetically older
neural circuits, which may have evolved to meet the needs of entirely dif-
ferent environmental pressures. In this sense, the circuitry of reflexes may
be thought of as tools used by frontal cortex to make new mental tools to
solve new problems. In the mission of tool maker, the frontal lobes
require competence in a number of functions, to include holding the com-
putations of several operations of a complex task on line in working
memory; sequencing their implementation in time with frontostriatal
switching circuitry; and, in circumstances like the antisaccade task,
inhibiting the primitive functions for which the component circuits ini-
tially evolved.
In summary, we have focused on a simple primitive midbrain visuo-
motor reflex as a model system for understanding the neural basis of
automaticity a n d control. The strategic control of the VGR provides for an
Rafal, Machado, Ro, a n d Ingle
efficient coordination of two opponent oculomotor systems for moving
a n d fixing the eyes. Visual search involves a sequence of saccades a n d
intervening fixations during which attention dwells on the attended item
to extract visual information. The oculomotor system implements this
through anatomically distinct pathways with mutually inhibitory inter-
actions for fixation and saccades. This implementation occurs at the level
of an opponent physiological process within the SC itself, and is under
control of the FEF.
NOTE
This research was supported by U.S. Public Health Service grants MH41544, MH54100, a n d
MH19930.
REFERENCES
Burman, D. D., a n d Bruce, C. J. (1997). Suppression of task-related saccades by electrical
stimulation in the primate’s frontal eye field. Journal of Neurophysiology, 77, 2252–2267.
Butter, C. M., Rapcsak, S. Z., Watson, R. T., a n d Heilman, K. M. (1988). Changes in sensory
inattention, directional akinesia, a n d release of fixation reflex following a unilateral frontal
lesion: A case report. Neuropsychologia, 26, 533–545.
Büttner-Ennever, J. A., a n d Horn, A. K. E. (1997). Anatomical substrates of oculomotor con-
trol. Current Opinion in Neurobiology, 7, 872–879.
Deng, S.-Y., Goldberg, M. E., Segraves, M. A., Ungerleider, L. G., and Mishkin, M. (1986).
The effect of unilateral ablation of the frontal eye fields on saccadic performance in the
monkey. In E. L. Keller and D. S. Zee (Ed.), Adaptive processes in visual and oculomotor systems,
p p . 201–208. Oxford: Pergamon Press.
Deuel, R. K., and Collins, R. C. (1984). The functional anatomy of frontal lobe neglect in
monkeys: Behavioral and 2-deoxyglucose studies. Annals of Neurology, 15, 521–529.
Dias, E. C., Kiesau, M., and Segraves, M. A. (1995). Acute activation a n d inactivation
of macaque frontal eye field with GABA-related drugs. Journal of Neurophysiology, 74,
2744–2748.
Dias, E. C., and Segraves, M. A. (1997). The primate frontal eye field and the generation of
saccadic eye movements: Comparison of lesion a n d acute inactivation/activation studies.
Revista Brasileria de Biologia, 56, 239–255.
Dorris, M. C., and Munoz, D. P. (1995). A neural correlate for the g a p effect on saccadic reac-
tion times in monkey. Journal of Neurophysiology, 73, 2558–2562.
Easton, T. A. (1973). On the normal use of reflexes. American Scientist, 60, 591–599.
Everling, S., Paré, M., Dorris, M. C., and Munoz, D. P. (1998). Comparison of the discharge
characteristics of brainstem omnipause neurons a n d superior colliculus fixation neurons in
monkey: Implications for control of fixation and saccade behavior. Journal of Neuro-
physiology, 79, 511–528.
Fendrich, R., Demirel, S., a n d Danziger, S. (1999). The oculomotor gap effect without a
foveal fixation point. Vision Research, 39, 833–841.
Fischer, B., a n d Breitmeyer, B. (1987). Mechanisms of visual attention revealed by saccadic
eye movements. Neuropsychologia, 25, 73–84.
Control of the Visual Grasp Reflex
Fischer, B., and Ramsperger, E. (1984). H u m a n express saccades: Extremely short reaction
times of goal-directed eye movements. Experimental Brain Research, 57, 191–195.
Forbes, K., and Klein, R. M. (1996). The magnitude of the fixation offset effect with endoge-
nously a n d exogenously controlled saccades. Journal of Cognitive Neuroscience, 8, 344–352.
Friedrich, F. J., Egly, R., Rafal, R., and Beck, D. (1998). Spatial attention deficits in h u m a n s :
A comparison of superior parietal and temporal parietal junction lesions. Neuropsychology,
12, 193–207.
Fukushima, J., Fukushima, K., Miyasaka, K., a n d Yamashita, I. (1994). Voluntary control of
saccadic eye movement in patients with frontal cortical lesions and parkinsonian patients in
comparison with that in schizophrenics. Biological Psychiatry, 36, 21–30.
Funahashi, S., Bruce, C. J., and Goldman-Rakic, R. P. (1991). Neuronal activity related to sac-
cadic eye movements in the monkey’s dorsolateral prefrontal cortex. Journal of Neuro-
physiology, 65, 1464–1483.
Guitton, D., Buchtel, H. A., and Douglas, R. M. (1985). Frontal lobe lesions in man cause
difficulties in suppressing reflexive glances a n d in generating goal-directed saccades.
Experimental Brain Research, 58, 455–472.
Henik, A., Rafal, R., and Rhodes, D. (1994). Endogenously generated and visually guided
saccades after lesions of the h u m a n frontal eye fields. Journal of Cognitive Neuroscience, 6,
400–411.
Hood, B. M., Atkinson, J., and Braddick, O. J. (1997). Selection-for-action and the develop-
ment of visual selective attention. In J. E. Richards (Ed.), Cognitive neuroscience of attention: A
developmental perspective, p p . 219–250. Hillsdale, NJ: Erlbaum.
Ingle, D. (1973). Two visual systems in the frog. Science, 181, 1053–1055.
Johnson, M. H. (1990). Cortical maturation and the development of visual attention in early
infancy. Journal of Cognitive Neuroscience, 2, 81–95.
Johnson, M. H., Gilmore, R. O., a n d Lsibra, G. (1997). Toward a computational model of the
development of saccade planning. In J. E. Richards (Ed.), Cognitive neuroscience of attention:
A developmental perspective, p p . 103–130. Hillsdale, NJ: Erlbaum.
Kimberg, D. Y., a n d Farah, M. J. (1993). A unified account of cognitive impairment follow-
ing frontal lobe damage: The role of working memory in complex, organized behavior.
Journal of Experimental Psychology: General, 122, 411–428.
Kingstone, A., and Klein, R. M. (1993a). Visual offsets facilitate saccadic latency: Does pre-
disengagement of visuospatial attention mediate this gap effect? Journal of Experimental
Psychology: Human Perception and Performance, 19, 1251–1265.
Kingstone, A., and Klein, R. M. (1993b). What are express saccades? Perception and Psycho-
physics, 54, 260–273.
Klein, R., a n d Kingstone, A. (1993). Why do visual offsets reduce saccadic latencies?
Behavioral and Brain Sciences, 16, 583–584.
Kwon, S. E., a n d Heilman, K. M. (1991). Ipsilateral neglect in a patient following a uni-
lateral frontal lesion. Neurology, 41, 2001–2004.
Machado, L., and Rafal, R. (Forthcoming). Strategic control over the visual grasp reflex:
Studies of the fixation offset effect. Perception and Psychophysics.
Moschovakis, A. K., Scudder, C. A., a n d Highstein, S. M. (1996). The microscopic anatomy
a n d physiology of the mammalian saccadic system. Progress in Neurobiology, 50, 133–254.
Munoz, D. P., and Istvan, P. J. (1998). Lateral inhibitory interactions in the intermediate
layers of the monkey superior colliculus. Journal of Neurophysiology, 79, 1193–1209.
172 Rafal, Machado, Ro, a n d Ingle
Munoz, D. P., a n d Wurtz, R. H. (1992). Role of the rostral superior colliculus in active
visual fixation and execution of express saccades. Visual Neuroscience, 9, 409–414.
Munoz, D. P., and Wurtz, R. H. (1993a). Fixation cells in monkey superior colliculus: 1.
Characteristics of cell discharge. Journal of Neurophysiology, 70, 559–575.
Munoz, D. P. , and Wurtz, R. H. (1993b). Fixation cells in monkey superior colliculus: 2.
Reversible activation a n d deactivation. Journal of Neurophysiology, 70, 576–589.
Paré, M., and Guitton, D. (1994). The fixation area of the cat superior colliculus: Effects
of electrical stimulation a n d direct connection with brainstem omnipause neurons.
Experimental Brain Research, 101, 109–122.
Paus, T. (1996). Location and function of the h u m a n frontal eye-field: A selective review.
Neuropsychologia, 34, 475–484.
Penfield, W., a n d Rasmussen, T. (1950). The cerebral cortex of man: A clinical study of localiza-
tion of function. N e w York: Macmillan.
Pierrot-Deseilligny, C., Rivaud, S., Gaymard, B., and Agid, Y. (1991). Cortical control of
reflexive visually-guided saccades. Brain, 114, 1473–1485.
Rafal, R., Calabresi, P., Brennan, C., and Sciolto, T. (1989). Saccade preparation inhibits reori-
enting to recently attended locations. Journal of Experimental Psychology: Human Perception
and Performance, 15, 673–685.
Reuter-Lorenz, P. A., Hughes, H. C., and Fendrich, R. (1991). The reduction of saccadic
latency by prior offset of the fixation point: An analysis of the gap effect. Perception and
Psychophysics, 49, 167–175.
Reuter-Lorenz, P. A., Oonk, H. M., Barnes, L. L., a n d Hughes, H. C. (1995). Effects of warn-
ing signals a n d fixation point offsets on the latencies of pro- versus antisaccades: Implica-
tions for an interpretation of the gap effect. Experimental Brain Research, 103, 287–293.
Rivaud, S., Muri, R. M., Gaymard, B., Vermersch, A. I., and Pierrot-Deseilligny, D. C. (1994).
Eye movement disorders after frontal eye field lesions in h u m a n s . Experimental Brain
Research, 102, 110–120.
Ro, T., Cheifet, S., Ingle, H., Shoup, R., and Rafal, R. (1999). Localization of the h u m a n
frontal eye fields a n d motor h a n d area with transcranial magnetic stimulation a n d mag-
netic resonance imaging. Neuropsychologia, 37, 225–231.
Ro, T., Henik, A., Machado, L., and Rafal, R. D. (1997). Transcranial magnetic stimulation of
the prefrontal cortex delays contralateral endogenous saccades. Journal of Cognitive Neuro-
science, 9, 433–440.
Roberts, R. J., Hager, L. D., a n d Heron, C. (1994). Prefrontal cognitive processes: Working
memory and inhibition in the antisaccade task. Journal of Experimental Psychology: General,
123, 374–393.
Saslow, M. G. (1967). Effects of components of displacement-step stimuli u p o n latency for
saccadic eye movements. Journal of the Optical Society of America, 57, 1024–1029.
Schiller, P. H., Sandell, J. H., a n d Maunsell, J. H. (1994). The effect of frontal eye field a n d
superior colliculus lesions on saccadic latencies in the rhesus monkey. Experimental Brain
Research, 98, 179–190.
Sommer, M. A., a n d Tehovnik, E. J. (1997). Reversible inactivation of macaque frontal eye
field. Experimental Brain Research, 116, 229–249.
Sprague, J. M. (1966). Interaction of cortex and superior colliculus in mediation of periph-
erally summoned behavior in the cat. Science, 153, 1544–1547.
Control of the Visual Grasp Reflex
Walker, R., Husain, M., Hodgson, T. L., Harrison, J., and Kennard, C. (1998). Saccadic eye
movements a n d working memory deficits following damage to h u m a n prefrontal cortex.
Neuropsychologia, 36, 1141–1159.
Walker, R., Kentridge, R. W., a n d Findlay, J. M. (1995). Independent contributions of the ori-
enting of attention, fixation offset a n d bilateral stimulation on h u m a n saccadic latencies.
Experimental Brain Research, 103, 294–310.
Wurtz, R. H., and Albano, J. (1980). Visual-motor function of the primate superior colliculi.
Annual Review of Neuroscience, 3, 189–226.
174 Rafal, Machado, Ro, a n d Ingle
7 Selective Attention and Cognitive Control: Dissociating Attentional Functions through
Different Types of Load
Nilli Lavie
ABSTRACT Selective attention acts to ensure that behavior is controlled by goal-relevant
rather than goal-irrelevant stimuli. Here I propose two very different mechanisms for atten-
tional control. The first, a passive mechanism, excludes irrelevant stimuli from perception,
but can only operate in situations of high perceptual load which exhaust perceptual capac-
ity with processing of relevant material. In situations of low perceptual load, a second active
mechanism comes into play to suppress responses to irrelevant distractors that cannot be
excluded from perception. Such suppression requires higher-level control functions (e.g.,
working memory). High cognitive load (e.g., in working memory) engages these functions,
a n d therefore leads to inefficient rejection of distractors. Evidence for the distinction
between passive a n d active control mechanisms is obtained from the contrasting effects of
different types of load on distractor processing in behavioral studies. The implications of
this distinction for neural activity, as well as for aging of attentional functions, are also
considered.
Selective attention usually allows efficient and focused processing of
goal-relevant stimuli, with minimal intrusions from goal-irrelevant stim-
uli. In this chapter, I argue for two forms of such attentional selection. The
first is a relatively passive form of control, whereby interference from
irrelevant distractors is prevented simply because they are not perceived.
Efficient exclusion of irrelevant stimuli from perception is not always
possible, however.1 A second, more active control function therefore
comes into play in situations where irrelevant as well as relevant stimuli
are perceived. In such cases, active control is crucial for suppressing
response tendencies toward the irrelevant, yet processed distractors.
Specifically, I argue that high load in the perception of relevant stimuli
results in reduced perception of distractor stimuli because there is insuf-
ficient capacity to process them all. Low perceptual load in the relevant
task, on the other hand, results in the processing of irrelevant as well
as relevant information, a n d therefore requires some active means of
rejecting distractors for maintaining appropriate control of behavior.
These active control processes rely on higher mental functions, such as
working memory (WM), which are required to maintain current priori-
ties and thus ensure that low-priority items can be suppressed. Contrary
to the predicted effect for perceptual load, however, increasing the load
on these higher mental functions will drain the capacity available for
active control a n d result in more, rather than fewer, intrusions from irrel-
evant distractors.
The two proposed mechanisms for selective attention, active a n d pas-
sive, should thus be dissociable from one another through the opposite
effects that different types of load are predicted to have on the efficiency
of selective attention. In particular, the extent to which intrusions from
distractors can be avoided should depend on the level and type of load
in relevant processing. In the following sections, the two mechanisms are
described in greater detail a n d evidence is discussed for the role of dif-
ferent processing loads in determining whether the passive or active
mechanism of control comes into play. Some implications of this distinc-
tion for the normal aging of selective attention are also discussed.
7.1 PASSIVE CONTROL OF SELECTIVE ATTENTION
In this section, I consider the role of perceptual load in selective attention,
placing the perceptual load model in the context of previous research on
selective attention, then describing the model in greater detail and report-
ing the empirical evidence available thus far to support it.
The Perceptual Load Theory: Background and Model
The extent to which selective attention can prevent perception of irrele-
vant distractors has been debated for the last four decades, within the tra-
dition of debate on whether attentional selection has an early or a late
locus in processing (e.g., Kahneman a n d Treisman 1984). On the one
hand, numerous reports of distractors excluded from perception (e.g.,
Treisman 1969; Yantis and Johnston 1990) seemed to support the early-
selection view of attention. On the other, reports of failures to exclude dis-
tractors from perception (e.g., Shiffrin and Schneider 1977; Driver a n d
Tipper 1989) seemed to support the rival, late-selection view, according to
which attention can affect only processes of response selection that occur
after distractors are perceived. These conflicting reports led to a theoreti-
cal impasse for many years.
A resolution to the early- versus late-selection debate may be possible,
however, if we consider a hybrid model for attention, which combines
aspects of both views (Lavie 1995). According to this model, perceptual
processing has capacity limits (as in the early-selection approach) but
operates automatically on all stimuli (as in the late-selection approach)
accommodated within those limits. Thus voluntary control is restricted to
determining priorities between relevant a n d irrelevant stimuli, although
any remaining capacity from processing relevant stimuli will “spill over’’
to the processing of irrelevant distractors.
The extent to which the perception of irrelevant distractors can be pre-
vented should thus depend on the perceptual load imposed by the pro-
Lavie
cessing of relevant information. Situations of low perceptual load will
inevitably result in perception of irrelevant stimuli, despite the assign-
ment of low priority to their processing. By contrast, when relevant per-
ceptual processing imposes a high load, this will exhaust perceptual
capacity leaving none for distractors, so that perception of distractors is
prevented by passive exclusion: there is simply insufficient capacity to
process them.2 The more active process of maintaining current priorities
in WM is important for ensuring the correct distinction between relevant
a n d irrelevant information, in situations of both high and low perceptual
load. But this is in itself insufficient to cause selective perception, as long
as perceptual load in relevant processing is low, a n d thus leaves spare
perceptual capacity. Irrelevant information that is involuntarily perceived
must therefore be actively suppressed at a later stage.
A review of past studies in the early- versus late-selection debate (Lavie
a n d Tsal 1994) has lent support to this hybrid model. Results indicating
selective perception have typically been obtained under high perceptual
load in the relevant task; those indicating unselective perception, under
low perceptual load.
The Role of Perceptual Load in Distractor Interference: Behavioral
Studies
A series of new experiments using various manipulations of perceptual
load (Lavie 1995; Lavie a n d Cox 1997) has provided further evidence for
the model, showing that perceptual processing of distractors is reduced
only by a high perceptual load in the relevant task. A new method was
used to manipulate the perceptual load imposed by processing of rele-
vant stimuli a n d to assess its effect on processing irrelevant stimuli. Stim-
uli for the relevant task were presented in the display center, while an
irrelevant distractor stimulus w a s presented in a peripheral position.
Subjects were told to ignore the peripheral distractor a n d to focus on
the central task, a n d the extent to which distractors were nonetheless
processed was assessed by measuring response competition effects
(cf. Eriksen and Eriksen 1974) from distractor stimuli that were either
response incongruent, response congruent, or neutral with respect to the
target response. Perceptual load was manipulated in the central task by
varying either the number of stimuli relevant for processing (i.e., set size
of the relevant items in the display center) or the processing requirements
for constant items (e.g., requiring feature versus conjunction processing
for the same displays). For example, Lavie a n d Cox (1997) presented sub-
jects with a visual search task at the center of the display, asking them to
ignore an irrelevant peripheral distractor (presented outside the relevant
search area) while searching for one of t w o target letters (e.g., X or N)
among other nontarget letters in the central array.3 Note that, unlike the
peripheral distractor, these nontarget stimuli were always neutral with
Attentional Control and Processing Load
respect to target response, a n d only served to load target perception by
forcing subjects to search for the target among them. Search load was
manipulated in one experiment by varying the similarity between targets
a n d nontargets, (e.g., the X or N targets were presented either among Os
in the low-load condition, or among angular letters in the high-load con-
dition). In another experiment, search load w a s manipulated by varying
the set size of similar targets and nontarget letters.
We found that efficient searches, involving target pop-out, led to
inefficient rejection of the peripheral distractor because the search load of
the relevant task was low. By contrast, inefficient searches, with a steep
search slope indicating that each potentially relevant item imposed an
additional d e m a n d on attention, led to efficient rejection of irrelevant
peripheral distractors, as long as more than four items were involved in
the relevant search to exhaust capacity (see also Fisher 1982; Yantis a n d
Jones 1991; Kahneman, Treisman, a n d Gibbs 1993; Pylyshyn et al. 1994 for
similar reports of capacity limits).
Because these manipulations of search load involved either varying the
nontarget letters (e.g., curved versus angular letters) that appeared in the
relevant central area or varying their number, the appearance of the dis-
play in this study differed between the load conditions. Another study
(Lavie 1995) demonstrated that the load imposed by the processing of
relevant items determines irrelevant distractor processing even for
cases that did not involve any variation in the stimulus displays with
load, which was now manipulated via different processing requirements
for the same displays. In one experiment for example, subjects m a d e
speeded choices discriminating the identity of a central target letter,
while attempting to ignore an irrelevant peripheral distractor. Whether
subjects should respond to the central target, however, w a s conditional
on another shape adjacent to the target. In the low-load condition, the
mere presence of this additional shape was sufficient to license a response
(which was to be withheld if that shape w a s absent). In the high-load con-
dition, subjects had to identify the combination of shape, exact size, a n d
position of the adjacent shape to decide whether to respond to the target
letter. Distractor interference w a s observed when mere detection of the
shape w a s required (low load), but w a s significantly reduced when it h a d
to be identified (high load) for the very same displays.
The consistent decrease in interference from incongruent distractors
with higher perceptual loads in all these previous experiments was taken
as supporting our hypothesis that perceptual load imposed by relevant
stimuli reduces the perceptual processing of irrelevant distractors, thus
also supporting early selection.
Recent late-selection views (e.g., Tipper a n d Milliken 1996) offer an
alternative account for our results, however, one that stresses the role of
inhibitory mechanisms in distractor exclusion, as revealed by negative
priming effects (i.e., the slowing d o w n of subsequent responses to items
that served as the irrelevant distractor on the preceding trial). If inhibition
178 Lavie
Figure 7.1 Example displays from the low-load (panel A) a n d high-load (panel B) condi-
tions in Experiment 1 of Lavie and Fox 2000. A prime display a n d the immediately follow-
ing probe display are shown for each condition. IR = ignored repetition; C = control; AR =
attended repetition condition.
is the primary means for selective processing (e.g., Driver a n d Tipper
1989), then the reduced interference we found in situations of high load
may not necessarily reflect reduced distractor processing, but rather
increased inhibition of processed distractors. On the other hand, if per-
ceptual load determines distractor processing, as we claim, inhibition
will only be required w h e n distractors are perceived, that is, only u n d e r
low-load conditions.
A study of negative priming (NP) by the author a n d Elaine Fox (Lavie
a n d Fox 2000) provided support for the perceptual load hypothesis in this
179 Attentional Control a n d Processing Load
respect. Presenting subjects with pairs of prime a n d probe displays, we
assessed NP effects from prime distractors as a function of perceptual
load in the processing of prime targets. Figure 7.1 displays the task we
used in this study. Subjects searched for a target letter among a varying
number of nontarget letters in the center of the prime display a n d ignored
an irrelevant peripheral distractor, which was always incongruent with
the prime target. NP from this distractor was found to depend on the rel-
evant search set size, decreasing as this set size was increased.
Several experiments allowed us to rule out alternative accounts for this
effect of load on N P. For example, the effect of load in our first experiment
might be attributed to the greater similarity between prime and probe
displays in the low-load versus high-load conditions (as they both
involved the same number of items, with a relevant set size 1; see figure
7.1). In experiment 2, however, prime and probe similarity was greater in
the high-load than in the low-load prime conditions: all the probes in
experiment 2 involved a relevant set size of 6. The same result was found
in both experiments, namely, less negative priming for conditions with a
high perceptual load in the prime display. The two experiments taken
together thus rule out any account of the results in terms of retrieval of
episodic memory for the distractor, which can depend on the similarity
between the prime a n d probe displays (see Fox a n d DeFockert 1998; Neil,
1997).
In a d d i t i o n , these experiments d e m o n s t r a t e d that NP crucially
d e p e n d s on the level of perceptual load in the relevant processing for the
prime displays, rather than on general task difficulty, as the same effect of
prime load on NP w a s obtained regardless of the level of load in the
probes. Finally, NP did not depend on reaction times overall (RTs; a n d
their associated variability): it was obtained in all the conditions of low
prime load even when their overall probe RTs were just as slow as those
for our high prime loads (as was the case for experiment 2, which h a d
high probe loads).
We conclude that high perceptual load in the relevant task reduces
perceptual processing of distractors, hence protects the postperceptual
processing of relevant stimuli from distractor intrusions. Moreover, be-
cause high perceptual load reduces response competition effects from
distractors on concurrent targets (Lavie and Fox 2000, exp. 4; Lavie a n d
Cox 1997)—as well as any NP in responses to subsequent targets—we
think that distractor interference is reduced by high perceptual load in a
rather passive manner, without requiring any active inhibition mecha-
nisms such as those indicated by NP.
We conclude that with high perceptual loads, the reason distractors do
not interfere is simply that they are not identified (i.e., early selection). On
the other hand, a more active means of suppressing distractor responses
(as indexed by NP effects; see Tipper a n d Milliken 1996) may become cru-
cial in situations of low perceptual load, when distractors are processed
Lavie
Figure 7.2 Example displays of the moving (panel A) or static (panel B) dot distractors
from the functional imaging study by Rees et al. (1997). The stimuli were the same for low-
a n d high-load conditions; only the task performed on the central stream of w o r d s differed.
Comparing the fMRI response for moving versus static dot distractors allowed a measure
of processing for irrelevant background motion.
a n d may thus compete to control behavior (i.e., late selection). The nature
of these active control mechanisms will be discussed in section 7.2.
The Role of Perceptual Load in Determining Neural Activity for
Distractors
Our perceptual load hypothesis raises some interesting predictions for
the brain activity that should be produced by distractors. If a high load in
relevant processing actually reduces irrelevant distractor perception, as we
claim, then neural responses in sensory cortices associated with distrac-
tor perception should d e p e n d on the load imposed by the relevant task,
even if that task is quite unrelated to the distractors in question. Speci-
fically, we claim that brain activity for entirely irrelevant distractors
should be found despite subjects’ attempts to ignore them, provided the
relevant task load is low. Activity to irrelevant distractors should only be
reduced by higher load in the relevant task.
We recently tested these predictions using fMRI to assess the neural
responses to moving distractors (Rees, Frith, a n d Lavie 1997; see figure
7.2). A stream of w o rd s was presented at fixation at a rate of one word per
second. A full field of dots was presented in the periphery. These dots
were either static (figure 7.1B) or moving (figure 7.1A), a n d subjects were
requested to focus on the words a n d ignore the dots under t w o task con-
ditions. In the low-load condition, subjects discriminated between lower-
a n d uppercase letters in the w o r d stream, a n d in the high-load condition,
they discriminated between bisyllabic a n d mono- or trisyllabic words for
the same streams.
The results confirmed our predictions exactly: motion related activity
in cortical area V 5 / M T varied as a function of the word task. Activity in
V 5 / M T for moving versus static dots w a s apparent in the case discrimi-
nation conditions (low load), but w a s eliminated in the syllable discrimi-
Attentional Control a n d Processing Load
nation conditions (high load). This interaction between load a n d the
neural responses to background motion was also found in other areas
likely to be involved in motion perception, such as the V1/V2 border,
a n d the superior colliculus (SC; Shipp and Zeki 1985; Ungerleider et al.
1984).4 In sum, we found that a whole network of sensorimotor areas
that are likely to be involved in motion perception were active in the
presence of irrelevant motion distractors, provided that the relevant task
involved only low load; but that this distractor-induced activity was then
significantly reduced as load in relevant processing was increased.
In a further psychophysical experiment, we used the same task a n d
displays as those used in our scanning experiment, while assessing the
processing of irrelevant motion via the duration of the motion aftereffect
it induced (see Chaudhuri 1991). We found that the duration of the
motion aftereffect induced by the irrelevant moving distractors w a s
significantly reduced in the syllable discrimination condition. These two
experiments thus provide evidence for our claim that the relevant task
load can decrease perception of irrelevant moving distractors. Additional
evidence that attentional modulation of neural activity in early visual
cortices is most apparent under high perceptual load comes also from
some recent fMRI a n d single-cell studies (see Motter 1994; Kastner et al.
1998). It is important to note, however, that our current conclusion about
the role of load in distractor processing, as determined by fMRI, is con-
fined to the particular manipulation of load we used, and the specific
type of moving distractors presented. As with our behavioral studies,
additional experiments with different load manipulations as well as
various types of distractors need to be run to allow us to reach a more
definitive conclusion
The Role of Perceptual Load in the Normal Aging of Selective
Attention
Elizabeth Maylor and I tested some implications of the perceptual load
theory for the normal aging of attention (Maylor a n d Lavie 1998). It is
often claimed (e.g., Ball et al. 1988) that aging can lead to a greater restric-
tion in perceptual processing capacity. Because, according to our model,
distractor processing d e p e n d s on the amount of available processing
capacity, we predicted that older adults should benefit more than
younger adults from smaller increases in relevant perceptual load with
respect to susceptibility to distractors. Because smaller increases in load
should be needed to exhaust capacity for the elderly g r o u p .
To test this prediction, we compared the effect of a graded increase in
perceptual load on distractor processing for 16 younger (aged 19–30) a n d
16 older (aged 65–79) adults. Subjects were presented with a relevant set
of letters in the center of the display, a n d had to make speeded choices
indicating which of two target letters was present among these relevant
Lavie
Figure 7.3 Distractor effects as a function of relevant set size a n d age group in experiment
1 of Maylor a n d Lavie 1998. A. Mean differences in reaction time between incongruent a n d
neutral conditions, with standard error bars. B. Proportional differences in reaction time for
these conditions.
letters, while attempting to ignore an irrelevant distractor in the periph-
ery. This irrelevant distractor w a s either response incongruent or neutral
with respect to the current target letter, to provide a response competition
measure for distractor processing. Perceptual load in the relevant target
processing was manipulated by varying the set size of the central array
(i.e., by adding neutral nontarget letters).
The results (presented in figure 7.3) support our prediction. Although
elderly subjects suffered from greater distraction in situations of very low
load (i.e., with just 1 target a n d 1 distractor), very small increases in load
(e.g., to just 2 relevant items, rather than only 1) were indeed sufficient for
reducing distractor effects in the old but not in the young subjects. Thus
older adults seem capable of benefiting from their greater restriction in
the available capacity for perception to reduce irrelevant distractor pro-
cessing at intermediate perceptual loads.
The finding of greater distraction for older versus younger adults at
very low levels of load (i.e., relevant set size 1; see figure 7.3) cannot,
however, be explained by such capacity limits in perception for the elderly
because response compatibility effects indicate identification of the dis-
tractor a n d its associated response for both groups (as we might expect
u n d e r situations of low perceptual load). Also, this larger distractor effect
in the elderly cannot be explained by general slowing with age because
the distractor effect at low load in the elderly was significantly larger than
that for the young even when the effect was calculated as a proportion of
the overall RTs for each population (figure 7.3).5 Finally differential eye
movements toward the distractor (e.g., Olincy et al. 1997) cannot account
for this aging effect because the display durations used (100 msec) were
too brief to allow eye movements. This effect seems therefore to reflect
183 Attentional Control a n d Processing Load
an additional age-related deficit in the ability to suppress irrelevant
response tendencies to distractors when these do get processed (as at
very low perceptual loads). The hypothesis that aging involves a specific
decline in inhibitory control mechanisms (see Hasher and Zacks 1988)
has received support from a number of previous studies. For example, it
is often found that NP is reduced with age (e.g., Hasher et al. 1991; Kane
1994; McDowd a n d Oseas-Kreger 1991; but see Kramer et al. 1994;
Sullivan a n d Faust 1993 for evidence of some age-related equivalence in
negative priming). Thus we conclude that the normal aging of attention
seems to involve (at least) two components. First, there is a decreased
capacity for perception, which can actually lead to some improvement in
passive selectivity: reduced processing of distractors as a natural conse-
quence of perceptual capacity being more readily exhausted by relevant
processing. Second, there is an additional age-related decline in the abil-
ity to actively reject distractors that do get processed, in situations of very
low load. Thus aging also seems to involve a deficit in the mechanisms of
active control.
7.2 ACTIVE CONTROL OF SELECTIVE ATTENTION
Our previous perceptual load studies support a simple account of an
early selective attention mechanism that can prevent distractors from
being perceived. We have provided substantial evidence for our claim
that distractors are excluded from perception as a matter of course in sit-
uations of high perceptual load, which exhaust perceptual capacity in the
relevant processing. This is a somewhat passive form of early selection:
distractors do not interfere simply because they are not processed. A com-
plete account of selective attention, however, also requires consideration
of a more active form of selection, one that allows appropriate selective
behavior even in situations of low perceptual load. In such situations, our
results show that irrelevant as well as relevant stimuli are perceived, a n d
thus can compete to guide behavior. Some late-selection mechanism is
then needed to actively suppress responses to processed distractors, a n d
thus ensure that behavior is appropriately controlled by relevant rather
than irrelevant stimuli.
The importance of such active mechanisms of attentional control can
be seen from the various “slips of action’’ that can occur if irrelevant
response tendencies are not suppressed. While such failures of attention
are relatively infrequent in young healthy adults, they become more pro-
nounced in older adults (see Maylor a n d Lavie 1998; see also Hasher a n d
Zacks 1988). Moreover, they can arise in extreme form for patients suffer-
ing from frontal lobe damage (see, for example, Shallice a n d Burgess
1991). Indeed, the greater distraction we found at low perceptual loads in
older versus younger subjects might be explained by deterioration of
the frontal lobes. Although aging involves a loss of cells in both posterior
Lavie
a n d anterior cortices, the greatest proportion of cell loss is frontal (e.g.,
Kramer et al. 1994). Moreover, frontal areas are known to be involved
in various high-level cognitive processes, such as working memory
(Baddeley 1986; D’Esposito, a n d Postle, chap. 26, this volume; Goldman-
Rakic and Friedman 1991; Petrides, chap. 23, this volume), multiple task
coordination (e.g., Burgess, chap. 20, this volume; Della Sala et al. 1995;
Shallice and Burgess 1996), and inhibition of irrelevant responses (e.g.,
Foster, Eskes a n d Stuss 1994; Posner and DiGirolamo 1998; Tipper,
Howard, a n d Houghton, chap. 10, this volume; but see Kimberg a n d
Farah, chap. 32, this volume), all of which seem crucial for maintaining
priorities between relevant and irrelevant stimuli, to guide behavior in
accordance with current goals. Thus our functional distinction between
early-selection a n d late-selection mechanisms of attentional control
seems likely to m a p onto an anatomical distinction that has been m a d e
between posterior and anterior attention systems in the brain (e.g.,
Posner and Petersen 1990).
In drawing an analogy with Posner’s general distinction between ante-
rior a n d posterior mechanisms, however, I do not wish to imply that
there is only one mechanism of frontal control. Evidence from imaging
a n d lesion studies in monkeys a n d h u m a n s suggests several distinct con-
trol functions, with particular frontal areas being differentially involved
(see D’Esposito a n d Postle, chap. 26, Keele a n d Rafal, chap. 28, Petrides,
chap. 23, a n d Robbins and Rogers, chap. 21, this volume). My point is
that selective attention, and in particular the ability to reject irrelevant
distractors, might d e p e n d on all of these various control functions being
intact (i.e., not lesioned, or not loaded).
Crucially, I propose that these two major psychological functions of
attention can be distinguished by contrasting the effects of different types
of load on selective processing. As described above, the exclusion of
distractors improves with high perceptual load in the relevant task.
However, a high load on the “frontal’’ processes important for cognitive
control (e.g., working memory, task coordination) should lead to a dete-
rioration of selective attention—an effect functionally similar to the effect
of a frontal lobe lesion. This is because increased load on those anterior
processes involved in cognitive control should leave less capacity for the
active suppression of intrusions from irrelevant but perceived distractors
into behavior. Thus the two major control functions of attention, namely,
selective perception and control of response selection, should be distin-
guishable from one another by contrasting the effects of different types of
load on distractibility. Increases in perceptual load should decrease distrac-
tion, by engaging perceptual capacity in the relevant processing. By con-
trast, increases in higher-level cognitive control load (at low perceptual
load) should increase distraction, by engaging cognitive control mecha-
nisms so that they become less able to block responses to perceived irrel-
evant distractors.
Attentional Control and Processing Load
Figure 7.4 Example display sequences from single trials for the low-working-memory-
load (panel A) a n d high-working-memory-load (panel B) conditions of experiment 1 in
Hirst a n d Lavie 1998.
I n o w review evidence for these contrasting effects of different types of
load on selective attention from a series of n e w studies conducted with
my graduate student Sandra Hirst (Lavie et al. in preparation).
The Role of Working-Memory Load in Distractor Rejection for
Selective Attention Tasks
Directing attention appropriately requires the active maintenance of
goals a n d task priorities in working memory (WM), specifying which
stimulus types are currently relevant, a n d which irrelevant. Accordingly,
we reasoned that loading WM in a situation of low perceptual load
should lead to reduced differentiation between high- a n d low-priority
items (i.e., between targets versus distractors), a n d hence result in more
intrusions from items that should have been given low priority. To
manipulate WM load during a selective attention task, we developed the
following n e w paradigm (figure 7.4). A selective attention task was inter-
leaved with a WM task. Each trial began with a memory set (e.g., several
visual digits) that subjects h a d to maintain in WM. The identity of the
characters in the memory set differed on each trial, to ensure that active
memorizing of items was required, so that any process of recency detec-
tion (e.g., Monsell 1978) that might be involved in recognition would still
require active maintenance through rehearsal.
During the retention interval (which typically lasted for about 1.6 sec)
a display for the selective attention task appeared (e.g., a central target
186 Lavie
Figure 7.5 Mean correct reaction time and percentage error (in parentheses) for perfor-
mance in the selective attention task, plotted as a function of the compatibility between
concurrent target a n d distractor in that task, a n d also as a function of the working mem-
ory load in the interleaved task
letter for speeded discrimination, together with a flanking distractor let-
ter). After a speeded-choice response was m a d e in this selective attention
task, a single memory probe then appeared, a n d subjects h a d to indicate
whether it h a d been present in the memory set that began the trial. WM
load was manipulated by the size of the memory set. In the low-WM-load
condition, only one digit was present in the memory set for each trial. In
the high-WM-load condition, six digits were presented in this set (see
figure 7.4). Our prediction w a s that increasing WM load in this way
should lead to greater distractibility in the unrelated selective attention
task, by drawing on resources otherwise used to control selection in situ-
ations of low perceptual loads. Recall that, according to our model, dis-
tractors are always perceived in such situations, so that active control is
required to prevent a response to them.
The results supported our prediction. As can be seen in figure 7.5, a
greater distractor effect was found in the selective attention task with a
high WM load (mean interference of 193 msec), than with a low WM
load (mean of 140 msec).6 Hence an increase in WM load can lead to
increased distractibility, supporting our hypothesis that loading WM
engages active mechanisms of attentional control, a n d therefore leads to
a reduced ability to reject perceived distractors u n d e r low perceptual
load. In further experiments, we replicated this effect of WM load for
additional memory tasks (e.g., implementing recall procedures within
our interleaved paradigm).
Note that, as predicted, WM load led to a result opposite to that typically
found for perceptual load. As shown repeatedly earlier (see section 7.1),
higher perceptual load reduces distractor interference, while here we
187 Attentional Control a n d Processing Load
found that higher WM load increases distractor interference. This contrast
seems to confirm our distinction between the two control mechanisms of
selective attention. To corroborate this, we checked that the usual per-
ceptual load effect could still be found when the selective attention task
was interleaved with a WM task, as in the new paradigm described
above. Perceptual load was manipulated by varying the set size or the rel-
evant central letters presented in each display for the selective attention
task, while again interleaving this task with the WM task. WM load was
n o w held constant (and low), with a memory set of just one item on each
trial. As predicted, higher perceptual load again resulted in a decreased
distractor effect (a distractor effect of 128 msec was obtained with low
perceptual load, but one of only 11 msec with high perceptual load).
These two experiments, with interleaved tasks, thus confirm that effects
of perceptual and WM load on selective attention can indeed be dissoci-
ated within the same paradigm. Whereas perceptual load decreases the
effects from irrelevant distractors, WM load increases these effects.
Note also that these experiments provide an entirely new form of evi-
dence for the importance of active control mechanisms in attention. Many
previous studies have shown that increasing the load on cognitive control
functions can lead to a performance cost (e.g., to a greater dual-task
decrement with a greater load in WM; Baddeley 1986). However, such an
overall d r o p in performance is the typical result of any increase in load;
for instance, higher perceptual load in a relevant task will also produce
an overall decline in performance. Our approach differs from previous
work on the loading of control processes because we specifically measure
processing of irrelevant distractors, rather than merely overall perfor-
mance in the relevant task. Thus our approach allows us to tie control
processes more closely to specific functions of selective attention (i.e., to
the rejection of perceived distractors in particular).7
While these experiments clearly showed greater distractor effects
under high WM load, even under low WM load, levels of distractor
effects were fairly high (provided that perceptual load was low). For
example, compare the distractor effects produced in the conditions of low
perceptual and low WM load of our experiment 1 (mean interference of
140 msec) a n d of our second study (128 msec) in this series, against the
typical range of distractor effects found in all the studies from section 7.1,
or in traditional studies of response competition effects from flanking dis-
tractor letters (where distractor interference effects typically range from
20 to 50 msec; see Lavie and Tsal 1994 for a comprehensive review). Why
should the overall level of distractor interference be so much increased in
our WM studies, even when WM was low? One likely reason is that our
new paradigm required subjects to switch back and forth between the
WM task a n d the selective attention task. As noted earlier, the coordina-
tion of multiple tasks has long been associated with frontal control
Lavie
processes (e.g., Della Sala et al. 1995; Shallice and Burgess 1996; see also
Keele a n d Rafal, chap. 28, this volume; Robbins and Rogers, chap. 21, this
volume). Accordingly, we hypothesized that the requirement to coordi-
nate task switching between our interleaved WM and selective attention
tasks may have loaded another anterior component of cognitive control,
which w o u l d again impair the ability to reject perceived distractors in
both low and high memory loads. The following subsection directly con-
siders whether a task-switching requirement can impair the active rejec-
tion of perceived distractors.
Effects of Task Coordination on Distractor Rejection in Selective
Attention Tasks
Much previous work has established the importance of anterior control
functions in the coordination of multiple tasks. Patients suffering from
frontal lesions are impaired at such coordination (e.g., Baddeley 1986;
Shallice and Burgess 1996). Recent functional imaging studies also
demonstrate the involvement of frontal areas in dual-task coordination
within the normal brain (see Keele a n d Rafal, chap. 28, this volume).
Finally, behavioral studies of normals have also highlighted the special
d e m a n d that is posed by the requirement to coordinate two tasks rather
than one. For example, Della Sala et al. (1995) a n d others have reported
that the cost involved in coordinating two WM tasks versus performing
one of them far exceeds the performance decrement caused by increas-
ing the load in either one of the tasks alone. In this subsection, we test
whether imposing a greater d e m a n d on task coordination can lead to
greater failures of selective attention, by exhausting subjects’ control
capacity, and thus leaving them less able to reject perceived distractors.
Although the procedure described in the previous subsection involved
an aspect of dual-task coordination (because a selective attention task
was interspersed with a WM task), this was held constant across the
experimental conditions, with only WM load or only perceptual load
being varied within an otherwise constant setting of two interleaved
tasks. We (Lavie et al. in preparation) n o w manipulated the requirement
for dual-task coordination directly, while keeping memory set and per-
ceptual load constant. Distractor processing in a selective attention task
was measured as before, but was n o w compared between single- a n d
dual-task situations.
We compared distractor effects between single- a n d dual-task con-
ditions, in a similar paradigm to the one used in our previous WM
experiments, except for two major changes. First, we n o w presented the
memory probe before the display for the selective attention task, so that
the entire WM task a n d the entire selective attention task now alternated,
rather than the WM task spanning the interval during which the selective
Attentional Control and Processing Load
Figure 7.6 Example display sequences for individual trials from the single- a n d dual-task
conditions of the experiment on task coordination. Note that task conditions were blocked
a n d the display sequences were the same in all respects between the blocks except that the
“memory probe’’ was always an asterisk in the single-task condition, a n d thus required no
response.
attention task w a s performed. Second, we kept the memory set constant
(with a set size of one item), a n d varied only whether subjects h a d to per-
form the attentional or both tasks.
Figure 7.6 presents the sequence of events a n d the experimental condi-
tions used. In the dual-task condition, subjects were presented with a
memory set, followed by a retention interval, a n d then a memory probe
to which they h a d to respond. After the memory response h a d been
m a d e , a n d a further 1 sec h a d elapsed (blank except for a fixation dot), the
display for the selective attention task appeared. This again required
subjects to make a speeded-choice response to a central letter, while any
compatibility effects were measured from an irrelevant flanking letter to
provide a measure of distractor interference. In the single-task condition,
a similar sequence of events w a s used, except subjects did not have to
make any response to the “memory’’ probe (which was n o w simply an
asterisk on every trial). As the two conditions were presented in separate
blocks, the subjects presumably m a d e no attempt to memorize the digit
in the single-task blocks.
We predicted more distractor interference for the selective attention
task in the dual-task condition. Even though the memory set no longer
h a d to be maintained in WM while performing the selective attention
task, we expected that the requirement to alternate between the WM a n d
selective attention tasks would load the anterior control processes that
190 Lavie
coordinate task switching, and thus disrupt the ability to actively sup-
press perceived distractors. This prediction was confirmed. The distrac-
tor effect of 63 msec (M = 704 for incongruent RTs; M = 641 for congruent
RTs) in the single-task condition was significantly increased to 90 msec in
the dual-task condition (M = 781 for incongruent RTs; M = 691 for con-
gruent RTs); and error rates were increased from 3% to 7% in the single-
versus dual-task conditions. This experiment confirms that task coordi-
nation is another important component in the active control of selective
attention. Note that, once again, the loading of anterior control processes
is found to have the opposite effect to increases in perceptual load, lead-
ing to greater rather than less distractibility.
7.3 CONCLUSIONS
In summary, the work presented here establishes a distinction between
active late-selection mechanisms, and passive early-selection mecha-
nisms for the control of selective attention, and demonstrates the im-
portant role of relevant processing load in determining the extent of
distraction by irrelevant information. In our work on late-selection mech-
anisms of active attentional control, we have started to lay out in greater
detail the involvement of specific anterior control functions in deter-
mining distractibility for selective attention tasks. This work already
indicates the importance of control functions loaded by working memory
in selective attention, and also provides some preliminary evidence that
control functions involved in coordinating multiple tasks may also play
a crucial role. Future work should further specify the nature of these
control functions, and determine whether other components of anterior
control are similarly involved in distractor rejection for selective attent-
ion tasks. Finally, working out whether the distinction between passive
and active mechanisms of selective attention can be related, respectively,
to posterior and anterior attentional networks in the brain, should pro-
vide further insights into the influences of control processes on selective
attention.
NOTES
This work was supported by Medical Research Council (U.K.) grant G9805400. I thank Jon
Driver, Art Kramer, Stephen Monsell, and Steven Yantis for their helpful comments.
1. My usage of the term perception throughout this chapter follows the conventional usage
in the early- versus late-selection debate, namely, referring to processes that lead to stimu-
lus identification. From this perspective, elaborative semantic activation, memory, response
selection, and response execution are conceived as postperceptual processes. See Pashler
1989 and Pashler and Johnston 1989 for discussion of distinctions between perception, in
this sense, a n d later processes.
2. Situations of high perceptual load will result in selective perception even if the correct set
cannot be actively maintained. Selection in such cases may not be the correct one, however,
Attentional Control and Processing Load
which is to say, it may not follow the appropriate attentional set (e.g., some irrelevant stim-
uli may be perceived instead of some relevant stimuli).
3. The term nontargets is used to refer to stimuli presented for the relevant task in central
positions that could contain the target.
4. Although SC has been implicated in oculomotor control, eye movements do not provide
a plausible explanation for these findings: no significant eye movements were found during
the performance in the experimental conditions (when measured outside the scanner). For
a full discussion of this, see Rees, Frith, and Lavie (1997).
5. It should be noted that our analysis of proportional RTs can only discount linear general
slowing models. For a more detailed treatment of general slowing accounts, see Maylor a n d
Lavie (1998).
6. Although it might appear from figure 7.4 that the increased distractor effect with high
memory load was d u e to reduced RTs in the congruent condition, this simple effect was not
statistically significant. Moreover, our additional WM and selective attention experiments
typically showed a WM load effect on both incongruent and congruent RTs.
7. The opposite effects that different types of load have on distractibility allow us to rule out
alternative accounts of the effect on distractibility from each type of load alone. For exam-
ple, at the meeting Daniel Gopher suggested that the result of better selectivity obtained
with high perceptual load might be d u e to subjects increasing their motivation for selective
processing when anticipating a difficult trial. If this were the case, we should presumably
have also found better selectivity with higher WM load because this also led to a substan-
tial increase in task diffi culty. Any account of load effects in terms of general task difficulty
thus seems insufficient.
REFERENCES
Baddeley, A. D. (1986). Working memory, Oxford: Oxford University Press.
Baddeley, A. D., Bressi, S. Della Sala, S., Logie, R., and Spinnler, H. (1991). The decline of
working memory in Alzheimer’s disease: A longitudinal study. Brain, 114, 2521–2542.
Ball, K. K., Beard, B. L., Roenker, D. L., Miller, R. L., and Griggs, D. S. (1988). Age a n d
visual search: Expanding the useful field of view. Journal of the Optical Society of America A5,
2210–2219.
Chaudhuri, A. (1991). Modulation of the motion aftereffect by selective attention. Nature,
344, 60–62.
Della Sala, S., Baddeley, A., Papagno, C., and Spinnler, H. (1995). Dual-task paradigm: A
means to examine the central executive, structure a n d functions of the h u m a n prefrontal
cortex. Annals of the New York Academy of Sciences, 769, 161–171.
Driver, J., a n d Tipper, S. P. (1989). On the nonselectivity of “selective’’ seeing: Contrasts
between interference a n d priming in selective attention. Journal of Experimental Psychology:
Human Perception and Performance, 15, 304–314.
Eriksen, B. A., and Eriksen, C. W. (1974). Effects of noise letters upon the identification of a
target letter in a non search task. Perception and Psychophysics, 16, 143–149.
Fisher, D. L. (1982). Limited-channel models of automatic detection: Capacity and scanning
in visual search. Psychological Review, 89, 662–692.
Foster, J. K., Eskes, G. A., a n d Stuss, D. T. (1994). The cognitive neuropsychology of atten-
tion: A frontal lobe perspective. Cognitive Neuropsychology, 11, 133–147.
Lavie
Fox, E., and De Fockert, J. W. (1998). Negative priming d e p e n d s on prime-probe similarity:
Evidence for episodic retrieval. Psychonomic Bulletin and Review, 5, 107–113.
Goldman-Rakic, P. S., and Friedman, H. R. (1991). The circuitry of working memory
revealed by anatomy and metabolic imaging. In H. S. Levin, H. M. Eisenberg, and A. L.
Benton, (Eds.), Frontal lobe function and dysfunction, p p . 72 – 9 1 . N e w York: Oxford University
Press.
Hasher, L., Stoltzfus, E. R., Zacks, R. T., and Rypma, B. (1991). Age and inhibition. Journal of
Experimental Psychology: Learning, Memory and Cognition, 17, 163–169.
Hasher, L., and Zacks, R. T. (1988). Working memory, comprehension, a n d aging: A review
a n d a new view. In G. H. Bower (Ed.), The psychology of learning and motivation, vol. 22, p p .
193–225. N e w York: Academic Press.
Kane, M. J., Hasher, L., Stoltzfus, E. R., Zacks, R. T., a n d Connelly, S. L. (1994). Inhibitory
attentional mechanisms and aging. Psychology and Aging, 9, 103–112.
Kahneman, D., a n d Treisman, A. (1984). Changing views of attention a n d automaticity. In
R. Parasuraman and D. R. Davies (Eds.), Varieties of attention, p p . 29–61. New York:
Academic Press.
Kahneman, D., Treisman, A., and Gibbs, B. (1992). The reviewing of object files: Object-
specific integration of information. Cognitive Psychology, 24, 175–219.
Kastner, S., De Weerd, P., Desimone. R., and Ungerleider, L. G. (1998). Mechanisms of
directed attention in the h u m a n extrastriate cortex as revealed by functional MRI. Science,
282, 108 – 111.
Kramer, A. F., Humphrey, D., Larish, J. F., Logan, G. D., and Strayer, D. L. (1994). Aging a n d
inhibition: Beyond a unitary view of inhibitory processing in attention. Psychology and
Aging, 9, 491–512.
Lavie, N. (1995). Perceptual load as a necessary condition for selective attention. Journal of
Experimental Psychology: Human Perception and Performance, 21, 451–468.
Lavie, N., a n d Cox, S. (1997). On the efficiency of attentional selection: Efficient visual search
results in inefficient rejection of distraction. Psychological Science, 8, 395–398.
Lavie, N., and Fox, E. (2000). The role of perceptual load in negative priming. Journal of
Experimental Psychology: Human Perception and Performance, 26.
Lavie, N., Hirst, S., and Colledge, E. (In preparation). The role of working memory load in
control of selective visual attention.
Lavie, N., and Tsal, Y. (1994). Perceptual load as a major determinant of the locus of selec-
tion in visual attention. Perception and Psychophysics, 56, 183–197.
Maylor, E., and Lavie N. (1998). The influence of perceptual load on age differences in selec-
tive attention. Psychology and Aging, 13, 563–573.
McDowd, J. M., a n d Oseas-Kreger, D. M. (1991). Aging, inhibitory processes, and negative
priming. Journal of Gerontology: Psychological Sciences, 46, 340–345.
Monsell, S. (1978). Recency, immediate recognition memory and reaction time. Cognitive
Psychology, 10, 465–501.
Motter, B. C. (1994). Neural correlates of attentive selection for color or luminance in extra-
striate area V4. Journal of Neuroscience, 14, 2178–2189.
Neill, W. T. (1997). Episodic retrieval in negative priming and repetition priming. Journal of
Experimental Psychology: Learning, Memory, and Cognition, 18, 565–576.
Attentional Control and Processing Load
Olincy, A., Ross, R. G., Youngd, D. A., and Freedman, R. (1997). Age diminishes perfor-
mance on an antisaccade eye movement task. Neurobiology of Aging, 18, 483–489.
Pashler, H. (1989). Dissociations a n d dependencies between speed a n d accuracy: Evidence
for a two-component theory of divided attention in simple tasks. Cognitive Psychology, 21,
469–514.
Pashler, H., a n d Johnston, J. C. (1989). Chronometric evidence for central postponement in
temporally overlapping tasks. Quarterly Journal of Experimental Psychology, 41A, 19–45.
Posner, M. I., and DiGirolamo, G. J. (1998). Conflict, target detection a n d cognitive control.
In R. Parasuraman (Ed.), The attentive brain. Cambridge, MA: MIT Press.
Posner, M. I., and Petersen, S. E. (1990). The attention system of the h u m a n brain. Annual
Review of Neuroscience, 13, 25–42.
Pylyshyn, Z., Burkell, J., Fisher, B., and Sears, C. (1994). Multiple parallel access in visual
attention. Canadian Journal of Experimental Psychology, 48, 260–283.
Rees, G., Frith, C., a n d Lavie, N. (1997). Modulating irrelevant motion perception by vary-
ing attentional load in an unrelated task. Science, 278, 1616–1619.
Rogers, R., and Monsell, S. (1995). Costs of a predictable switch between simple cognitive
tasks. Journal of Experimental Psychology: General, 124, 207–231.
Salthouse, T. A. (1992). Mechanisms of age-cognition relations in adulthood. Hillsdale, NJ:
Erlbaum.
Shallice, T., a n d Burgess, P. (1991). Deficits in strategy application following frontal lobe
damage in man. Brain, 114, 727–741.
Shallice, T., a n d Burgess, P. (1996). The domain of supervisory processes and temporal
organization of behaviour. Philosophical Transactions of the Royal Society of London, 351,
1405–1412.
Shiffrin, R. M., and Schneider, W. (1977). Controlled and automatic h u m a n information
processing: 2. Perceptual learning, automatic attending and a general learning theory.
Psychological Review, 84, 127–190.
Shipp, S., and S. Zeki, (1985). Segregation of pathways leading from area V2 to areas V4 a n d
V5 of macaque monkey visual cortex. Nature, 315, 322–324.
Sullivan, M. P., a n d Faust, M. E. (1993). Evidence for identity inhibition during selective
attention in old adults. Psychology and Aging, 8, 589–598.
Tipper, S. P., a n d Milliken, B. (1996). Distinguishing between inhibition-based a n d episodic
retrieval-based accounts of negative priming. In A. F. Kramer, M. G. H. Coles, and G. D.
Logan (Eds), Converging operations in the study of visual selective attention, p p . 77–106.
Washington, DC: American Psychological Association.
Treisman, A. M. (1969). Strategies a n d models of selective attention. Psychological Review, 76,
282–299.
Ungerleider, R., Desimone, T. W. Galkin, and M. Mishkin, (1984). Subcortical projections of
area MT in the macaque. Journal of Comparative Neurology, 223, 368.
Yantis, S., and Johnston, J. C. (1990). On the locus of visual selection: Evidence from focused
attention tasks. Journal of Experimental Psychology: Human Perception and Performance, 16,
135–149.
Yantis, S., and Jones, E. (1991). Mechanisms of attentional selection: Temporally modulated
priority tags. Perception and Psychophysics, 50, 166–178.
Lavie
8 Relations among Modes of Visual Orienting
Raymond M. Klein a n d David I. Shore
ABSTRACT Selective processing of the visual world is accomplished through overt shifts
of gaze direction and covert shifts of attention. Such visual orienting, whether overt or
covert, can be controlled exogenously by environmental stimulation, or endogenously by
the observer’s intentions. The modes of orienting implied by these distinctions may inter-
act cooperatively or competitively. We illustrate, with examples often drawn from the chap-
ters in this section, (1) the interactions between endogenous and exogenous control over
overt and covert orienting, and (2) the relations between overt and covert orienting and
between endogenous and exogenous control.
8.1 MODES OF VISUAL ORIENTING
Visual orienting is a set of processes used to give a region of space and
the objects in it preferential access to the visual and cognitive routines
that control behavior. The need for eye movements (overt orienting) is
apparent when one considers that primate vision is characterized by a
relatively small central area of fine resolution that must be aligned with
potentially important targets. In contrast, a shift of attention (covert ori-
enting) involves an internal selection, accomplished without an overt shift
of gaze, whereby some regions or objects in space are given processing
preference over others. Overt and covert orienting can be directed by
environmentally generated inputs (spatially asymmetric stimulation to
the visual, auditory or tactile modalities) or by observer-generated signals
(based on momentary intentions or enduring dispositions). Following
Posner (1980; see also Klein, Kingstone, and Pontefract 1994), we will
refer to these sources of control as “exogenous’’ (coming from outside the
organism, this source is also referred to as “bottom-up’’ and characterized
as “reflexive’’ or “automatic’’) and “endogenous’’ (coming from within
the organism, this source is also referred to as “top-down’’ and charac-
terized as “voluntary’’ or “strategic’’). Working within the 2 X 2 matrix
implied by these two distinctions (figure 8.1), this chapter will explore
several issues dealing with the control of orienting.
A high degree of coordination between these modes of orienting char-
acterizes everyday behavior. A compelling demonstration is provided by
Yarbus (1967) who, in one study, presented observers the same stimulus
Figure 8.1 Modes of orienting and the relations among them ( **—• ) and competi-
tive interactions between them ( -—) discussed in the chapter sections (indicated here by
number).
(the painting The Unexpected Visitor, by I. E. Repin) repeatedly and each
time asked them a different question. Their oculomotor scanning behav-
ior was dramatically affected by the question they were trying to answer.
Thus, while the exogenous input remained the same, knowledge-driven
endogenous control of overt orienting was used to select those regions
where the answers might be found (faces for emotional expression;
objects and clothes for material circumstances; etc.). Although Yarbus did
not use an independent method to verify which regions were being
attended, because the regions fixated were those containing the most
useful information for answering the questions, it seems reasonable to
assume that the path of attention and the path of fixations would have
been highly correlated. Whereas the modes of orienting appear coopera-
tive in Yarbus’s study, dissociations between and competitive interactions
among them have been well demonstrated, as will be illustrated in the
sections that follow.
8.2 ENDOGENOUS VERSUS EXOGENOUS OVERT ORIENTING
Within the oculomotor machinery there is a network of gating mecha-
nisms in the brain stem and superior colliculus whose activation can pre-
vent overt orienting (Everling et al. 1998). Removal of a fixated stimulus
provides an exogenous signal to deactivate this inhibitory gating mecha-
nism, and when such removal precedes the appearance of a target with a
delay sufficient to allow endogenous preparatory processes to become
active, extremely rapid (“express’’) saccades can be initiated (Fischer and
Ramsperger, 1984; Kingstone and Klein 1993). Because natural scene
components rarely disappear, to permit overt orienting in the presence of
a fixated stimulus, an internally generated disinhibitory signal must be
hypothesized. Evidence for such an “endogenous oculomotor disengage-
ment’’ from fixation has been reported by Taylor, Kingstone, and Klein
(1998). Similarly, Rafal et al. (chap. 6, this volume) found that when a
196 Klein and Shore
stimulus driven saccade is highly probable, the retarding effect on sac-
cadic latencies of the fixation stimulus is reduced (their experiment 1).
This suggests that endogenous disengagement from the fixation stimulus
is part of the preparation for overt orienting.
A compelling interaction between exogenous and endogenous control
of overt orienting was reported by Theeuwes et al. (1998). Initially, 6 gray
objects were placed around an imaginary circle; when five of the objects
changed to red, subjects were to foveate the remaining gray object in
order to identify a target contained within it. Although the singleton
nature of the target object m a d e selection unambiguous, the exogenous
system might not be optimally engaged because there was no change in
that item. On one-half of the trials, an irrelevant, red item was a d d e d to
the array at the same time the target location w a s revealed. Overall,
responses were slowed by the appearance of this new object; more impor-
tant, subjects executed a large proportion of eye movements toward this
irrelevant distractor despite their intention to move to the target. These
errors were often rapidly redirected toward the target, a n d the observers
were usually not aware of their overt orienting errors. The authors pro-
pose that two eye movements were programmed in parallel—one,
endogenously, to move to the location of the target; the other, exoge-
nously elicited by the abrupt onset—and that w h e n the incorrect
response toward the distractor was launched by the exogenous system
it w a s quickly inhibited a n d overwritten by endogenous control. Left
with the intended result of the final fixation, subjects were unaware of
the exogenously generated behavior. This reinforces prior evidence
(Kaufman a n d Richards 1969) that we are often not aware of where our
eyes have been.
Although these results are reminiscent of the antisaccade task (Everling
a n d Fischer 1998; Forbes and Klein 1996), where a stimulus presented in
one location instructs observers to move their eyes in the opposite
direction, one important difference is that in the antisaccade task the
exogenous signal is task relevant until its location has been encoded a n d
inverted. Although one might expect that this would give the onset stim-
ulus greater salience a n d that errors would thus be higher in the antisac-
cade task than in the distractor paradigm, this is not the case. We think
this is d u e in part to the timing of oculomotor disengagement. In the anti-
saccade task, endogenous release from fixation is likely delayed until the
exogenous signal has been converted to an endogenous command. In the
distractor paradigm, the color change in the displayed items, revealing
the singleton target, is likely used to initiate disengagement from fixation,
which would thus occur in close temporal proximity to the onset distrac-
tor. To the reflexive machinery in the superior colliculus, the distractor
is much more salient than the unchanging target, a n d therefore it fre-
quently captures control of overt orienting. Exploring the performance of
patients with damage to different cortical systems on the antisaccade
Modes of Visual Orienting
task, Rafal et al. (chap. 6, this volume) noted the importance of the frontal
eye fields and posterior parietal cortex (particularly the parietotemporal
junction) in successful performance in this task (their experiment 2). We
think that exciting new data will be generated by examining the per-
formance of patients, such as those used by Rafal et al., on the distractor
paradigm of Theeuwes et al. (1998).
8.3 ENDOGENOUS VERSUS EXOGENOUS COVERT ORIENTING
The interaction described above for overt orienting is paralleled in
studies of covert orienting a n d highlighted in the debate concerning the
extent to which salient exogenous signals (e.g., abrupt onsets or single-
tons) capture attention or can be ignored w h e n they are task irrelevant
(see Yantis, chap. 3, this volume, and Theeuwes, Atchley a n d Kramer,
chap. 4, this volume, for reviews). Whereas early studies (Jonides a n d
Yantis 1988; Yantis and Jonides 1984, 1990) suggested that abrupt onsets
capture attention, this proposal has been modified in three distinct ways.
First, Theeuwes a n d colleagues (see also Joseph and Optican 1996) have
maintained that attention is captured by any salient stimuli (at least ini-
tially). In support of this conclusion, they have repeatedly shown that
when searching for a singleton (color or shape) the presence of a second
unique singleton (defined on a different dimension) retards search per-
formance. Second, others (Folk, Remington, and Johnston 1992; Folk and
Remington 1998) have claimed that exogenous attention can be pre-
vented from orienting to an irrelevant distractor if an appropriate atten-
tional control setting (ACS) is instantiated. ACSs are conceptualized as
endogenously generated rules that determine which exogenous signals
will result in orienting. Hence, if the unique singleton is task irrelevant,
orienting can be avoided. The findings that led to the ACS proposal pro-
vide a powerful demonstration of the ability of endogenous control to
modulate, even countermand, exogenous orienting. Third, Yantis (chap.
3, this volume) has also considered attentional control settings to be a pri-
mary determinant of exogenous features that might capture attention,
with one proviso—new objects have a special status in their ability to
capture attention whether or not they appear as abrupt onsets (Yantis a n d
Hilstrom 1994).
Yantis responds to the finding that singleton distractors attract atten-
tion even when they are in a dimension (e.g., color) different from that of
the target (e.g., form) by noting that in these tasks the observers may be
adopting a “singleton detection mode’’ (cf. Bacon a n d Egeth 1994). While
this may be true, we feel that an important question remains unanswered:
What is the default control setting? Schmidt (1994; McColl and Schmidt
1995) approached this question by avoiding a search task altogether. He
asked whether a singleton in an otherwise homogeneous visual array
would support the phenomenon of illusory line motion (ILM). If a line,
Klein a n d Shore
presented all at once, is preceded by a cue adjacent to one end of the line,
the observer perceives the line d r a w n on the screen away from the cue
(Hikosaka, Mayauchi, and Shimojo 1993). When Schmidt presented an
array of stimuli around fixation, ILM was observed away from an orien-
tation singleton in the array. Because there was no task to perform with
the array, an interpretation in terms of a “singleton detection mode’’
seems unlikely. Instead, it seems that the singularity in the visual array
creates an “attractor’’ region. All other things being equal (the default set-
ting), in paradigms where attention is labile, this region is more likely
than any other region to attract attention.
The empirical conflict between studies supporting salience-driven ori-
enting (exogenous control) to irrelevant distractors a n d successful filter-
ing via ACS (endogenous control) might be resolved by considering
methodological differences. Theeuwes, Atchley, and Kramer (chap. 4, this
volume; Theeuwes et al. 1998) show that the identity contained within an
irrelevant singleton is processed automatically when it appears at the
same time as the relevant singleton containing the target; whereas the
irrelevant identity does not affect performance when it precedes the
target by 200 msec. Because 200 msec is close to the cue-target interval
(150 msec) used in the studies by Folk a n d colleagues, those studies do
not directly challenge the salience-driven attention hypothesis.1 Neuro-
physiological data (Chelazzi et al. 1993; see also Desimone a n d Duncan
1995) show that the initial response of neurons in inferotemporal cortex
to a stimulus is uninfluenced by task-relevance, whereas the cell’s sub-
sequent (100–200 msec after target onset) response rate is much higher
to the task-relevant stimulus. This converges with Theeuwes’s proposal
that some early exogenously driven processes can be immune to endoge-
nous control. Behaviorally speaking, there are two time course issues
that warrant investigation: H o w m u c h forewarning does a subject re-
quire to establish an ACS in order to avoid distraction from an irrelevant
singleton? And how soon after the target does the distractor have to be
presented to interfere with ongoing processing?
It is interesting to consider how the ACS concept might be related to
concepts developed within the task-switching literature (see Allport,
Styles, and Hsieh 1994; Rogers a n d Monsell 1995; see also Allport a n d
Wylie, chap. 2, this volume a n d Pashler, chap. 12, this volume). When an
observer is searching an array for a target item, a n d the array is preceded
by an uninformative cue, it seems reasonable to assume that the task will
be accomplished by instantiating rules for finding the target a n d for
ignoring the irrelevant cue. Nonconflicting rules (e.g., ignore color, attend
onset; ignore onset, attend color) can be maintained in parallel, hence one
should be able to avoid reflexive orienting. Because conflicting rules
(ignore onset, attend onset), cannot be maintained in parallel, however,
switching between them may take considerable time. Hence, with con-
flicting rules and short cue-target stimulus onset asynchronies (SOAs) the
Modes of Visual Orienting
observer may merely maintain the rule necessary to find the target, in
which case the irrelevant cue necessarily attracts attention.
The control exerted in order to implement ACS (whereby orienting
toward task-irrelevant singletons is inhibited) may be similar to the con-
trol hypothesized by guided search (Wolfe, Cave, a n d Franzel 1989;
Treisman and Sato 1990) to be exerted against distractors sharing a non-
target attribute in some conjunction search tasks. It w o u l d be interesting
to use a neuropsychological or individual differences approach to obtain
evidence for or against such an association. For example, the work of
Kingstone et al. (1995), showing that the left, but not the right, hemi-
sphere can implement guided search, might be extended by looking at
ACS in the left and right hemisphere of the split-brain subject. Or it might
be found that aging disrupts both ACS and guided search while leaving
other attention functions relatively unaffected.
8.4 RELATIONS BETWEEN COVERT AND OVERT ORIENTING
One question that follows from our ability to attend where we are not
looking was posed during the discussion by David Meyer: What is the
relationship between overt a n d covert orienting? With exogenous orient-
ing, there is a consensus that overt a n d covert orienting are strongly
linked, perhaps because the same kinds of stimuli that tend to attract
attention also activate the oculomotor system. In contrast, the literature
on overt a n d covert relations with endogenous orienting is characterized
by contradictory claims as to whether covert orienting is prepared, but
unexecuted, overt orienting.
Following a long tradition in psychology suggesting that motor plans
play an important role in perception, Klein (1980) proposed that endoge-
nous shifts of attention are accomplished by oculomotor preparation to
fixate the location to be attended—an idea captured in Rizzolatti et al.’s
“premotor’’ theory (1987). When a saccade is executed, whether under
endogenous or exogenous control, the gaze shift is preceded by a shift
of attention toward the location to be fixated (cf. Posner 1980; Hoffman
a n d Subramaniam 1995; Shepard, Findlay, and Hockey 1986). Although
often taken as evidence for the idea that oculomotor readiness medi-
ates endogenous covert orienting, this finding is actually not pertinent.
Covert orienting is a shift of attention without a shift in gaze. By defin-
ition, a mechanism that could only shift attention when the eyes moved
could not be responsible for covert orienting. If the oculomotor readiness
proposal were true, then when a saccade is prepared, but not executed,
there should be a corresponding attention shift, and conversely, when
there is an endogenously generated attention shift, a corresponding sac-
cade should be prepared. Direct tests of these predictions (Klein 1980;
Klein a n d Pontefract 1994; Ennis a n d Kingstone 1998) seemed to dis-
confirm the oculomotor readiness proposal for endogenous covert visual
orienting (and the similar premotor theory).2
200 Klein a n d Shore
Sheliga et al. (1994) found that endogenously generated probe sac-
cades are biased away from a covertly attended location, whereas Kustov
a n d Robinson (1996) seem to have found the opposite with electrically
elicited saccades (for a review, see Klein, in preparation). Both teams
claimed that their biased saccades provide evidence that covert endog-
enous orienting was accomplished by the endogenous preparation of
overt orienting. The Sheliga et al. (1994) pattern can be explained, how-
ever, by assuming that there is a natural tendency to look where one is
attending and that the instruction to attend covertly (without making an
eye movement) causes the tendency to be inhibited, thus deflecting probe
saccades away from the attended location. In this account, neither the
natural tendency nor the inhibition would be responsible for causing the
covert shift of attention. Evolution of the ability to endogenously attend
without overtly looking would have required inhibitory control over
the natural tendency to gaze at the attended object, as Rafal et al. (chap.
6, this volume) a n d Klein (in preparation) have argued. Kustov a n d
Robinson’s (1996) study, which comes the closest to providing evidence
in favor of Klein’s original proposal (1980), contains a serious confound:
cues to attend spatially also indicated whether a right- or a left-limb
response w a s likely to be required, which creates two ambiguities.
Because the cuing effect could be d u e to motor preparation rather than to
visual orienting, we cannot be sure that the endogenous cue elicited a
shift of attention. Even if there were a shift of attention, it could not be
confidently determined whether the effect on saccades was d u e to this
shift or to the preparation of the likely manual response. The confound
precludes firm conclusions about the oculomotor readiness hypothesis.
This clever experimental test should certainly be repeated with this con-
found removed. If it is found that electrically elicited saccades are biased
in the direction of attention, the oculomotor readiness proposal will be
strongly supported. 3
8.5 RELATIONS BETWEEN ENDOGENOUS AND EXOGENOUS
ORIENTING
With respect to overt orienting, endogenous a n d exogenous signals con-
trol the same thing: where the fovea is directed. With respect to covert ori-
enting, it is often assumed that they control the same attentional system,
however, paraphrasing a remark by Nancy Kanwisher during the discus-
sion, we might note that calling two processes “attention’’ does not make
them the same and ask, Is there evidence linking or dissociating the atten-
tion oriented via endogenous and exogenous means?
Using central and peripheral cues to direct attention, Jonides (1976,
1981) provided the earliest evidence of differences between endogenous
a n d exogenous control of covert orienting. He showed that covert orient-
ing w a s faster under exogenous control, a n d that endogenous, but not
Modes of Visual Orienting
Figure 8.2 Whether covert orienting adds (+) or interacts (X) with opportunities for illu-
sory conjunctions and with nonspatial expectancies depends on the type of control (exoge-
nous or endogenous).
exogenous, control was sensitive to cognitive load and to the relative
probabilities of the two types of cues. These important differences are
consistent with a reflexive versus voluntary distinction (Müller and
Rabbitt 1989). Most investigators have assumed that the attentional
mechanisms brought by endogenous or exogenous control to a region of
or object within space are the same; all that differs is how attention is
“transported’’ to its spatial destination. In contrast, a behavioral double
dissociation (see figure 8.2) we will briefly describe (see also Klein,
Kingstone, and Pontefract 1992) suggests that these two types of “atten-
tion’’ might be fundamentally different.
According to Treisman’s “feature integration theory’’ (FIT; Treisman
and Gelade 1980), attention is required to correctly “glue’’ together the
features present in a region that belong to an object. Briand and Klein
(1987) tested whether feature integration was among the functions per-
formed by the attention system recruited by an informative precue. They
combined the Posner cuing paradigm with tasks where the target (the let-
ter R) could be discriminated from the distractors (P, B; feature task) by a
single feature (slanted line) or where the correct conjoining of features
was required because of the possibility of an illusory conjunction from
the distractors (P, Q; conjunction task). With exogenous orienting in
response to an informative peripheral cue, there was a larger cuing effect
for the conjunction task than for the feature task (see also Prinzmetal,
Presti, and Posner 1986; and Treisman 1985), whereas with endogenous
orienting in response to a similarly informative central cue, both tasks
showed similar cuing effects. Briand (1998) recently replicated and
extended this pattern, notably by using uninformative peripheral cues,
features from different dimensions (form and color), and a range or cue-
target SOAs. Thus the answer to the question posed by Briand and
Klein’s title (1987) “Is Posner’s Beam the Same as Treisman’s ‘Glue’?’’ is
yes, for the exogenous beam, and no, for the endogenous beam. This
was the first dissociation between these modes of control (see figure 8.2)
that suggested something more fundamental than how attention gets to
its destination might differentiate exogenous from endogenous covert
orienting.
Suppose there are several possible targets that might appear, and that
one target type (form, color, orientation) is much more likely to occur
Klein and Shore
than the other, causing the observer to generate a nonspatial expectancy.
When a location is now cued, will covert orienting interact with the non-
spatial expectancy? The findings obtained by Klein (1994; Klein a n d
Hansen 1990), which have been replicated and extended by Kingstone
a n d Egly (in preparation), reveal additivity with exogenous orienting a n d
an interaction with endogenous orienting (see figure 8.2). This suggests
that nonspatial expectancies and endogenously controlled orienting (spa-
tial expectancies) involve overlapping mechanisms or stages of process-
ing. In contrast, the effects on processing that follow exogenous orienting
elicited by a peripheral cue are independent of those associated with the
nonspatial expectancy.
Although it is possible to infer from this double dissociation that dif-
ferent types of “attention’’ are being oriented by endogenous and exoge-
nous means (see Klein 1994; Briand 1998), one need not go so far. It is pos-
sible to assume that a common attention system is oriented in response to
endogenous a n d exogenous signals, so long as one assumes that in addi-
tion unique stages of processing are affected by exogenous and endoge-
nous orienting. Both systems operate relatively early on feature encoding
or extraction stages. The evidence reviewed by Hopfinger et al. (chap. 5,
this volume) showing amplification of the event-related potential within
about 100 msec of stimulus onset, and Hopfinger et al.’s isolation of this
modulation to sources in extrastriate cortex strongly suggest that endoge-
nous orienting can involve early amplification of the sensory signals that
might give rise to exogenous orienting. Interactions between endogenous
a n d exogenous control (as implied by ACS, and discussed above) may
also arise at this stage of operation. Exogenous control, perhaps because
it typically entails visual information for peripheral pattern-recognizing
routines to analyze, interacts with opportunities for illusory conjunction,
suggesting that it plays a role in feature binding.4 In contrast, endoge-
nous control involves pigeonholing operations at the decision stage (cf.
Broadbent 1971), which would interact with other, nonspatial expec-
tancies that may be similarly implemented.
There are several examples of covert orienting, two of which we will
mention here, that seem to have features associated with both exogenous
a n d endogenous control. First, as noted earlier, when an eye movement
is m a d e under endogenous control, attention is d r a w n to the location to
be foveated before the eyes get there. This shift of attention appears oblig-
atory because the uniform distribution of probe stimuli (which were used
to determine the locus of attention) would warrant a uniform distribution
of attention. Second, recent studies (Langton and Bruce 1999; Driver et al.
1999; Friesen a n d Kingstone 1998) have shown that attention is shifted
rapidly a n d automatically in the direction that a foveally presented rep-
resentation of a conspecific (whether person or cartoon drawing) is look-
ing. We believe that progress in understanding the nature of these hybrid
forms of covert orienting will be advanced by exploring h o w they be-
Modes of Visual Orienting
have in relation to nonspatial expectancies and opportunities for illusory
conjunctions.
8.6 SUMMARY
We have highlighted a subset of the relations among the different modes
of orienting shown in figure 8.1. Both overt a n d covert orienting involve
an interplay between control by endogenous signals a n d control by
exogenous signals. This interplay can be cooperative or competitive. The
concept of attentional control settings provides a powerful tool for under-
standing a wide range of phenomena, including the competitive interac-
tions for which it was generated. The utility of the overall framework
(figure 8.1) is accentuated when considering two questions raised during
the meeting: What is the relation between overt a n d covert orienting?
And is the same form of attention shifted by exogenous a n d endogenous
signals? We answered the first question by noting the strong linkage
when orienting is controlled exogenously and the implicit competitive
interaction when attention is endogenously shifted in space while gaze
direction is maintained. In answer to the second question, a double dis-
sociation w a s briefly described (under exogenous versus endogenous
control, attention behaves differently with respect to feature integration
a n d nonspatial expectancies; see figure 8.2), which strongly suggests that
when attention is endogenously or exogenously elicited, subtly different
selective mechanisms are engaged. Phenomena such as inhibition of
return, illusory line motion, meridian effects, and the disengage deficit
seen with neglect patients present dissociations consistent with this con-
clusion. Finally, we propose that double dissociation be used to explore
the nature of attention elicited in ambiguous cases.
NOTES
This chapter was m a d e possible by a Collaborative Projects Grant to Raymond M. Klein
from the Natural Sciences and Engineering Research Council of Canada and a Killam
Fellowship to David I. Shore. We are grateful for the comments of Stephen Monsell a n d Jon
Driver on an earlier version, and for the advice of Susan Boehnke, Jeff Hancock, Amelia
H u n t , Jason Ivanoff, Bill Matheson, and William Schmidt.
1. Other differences in the methodology used by Theeuwes, Atchley, a n d Kramer and by
Folk and colleagues may also be responsible for the conflicting results that have been
observed. For example, Theeuwes, Atchley, and Kramer typically use a searchlike display,
where the features that define the target and distractor are likely to be grouped with the ele-
ments of the array, whereas Folk and colleagues use a cuing-type display, where the dis-
tracting cue is unlikely to be grouped with the items that must be reported.
2. That inhibition of return (see Taylor and Klein 1998 for a review) is generated following
endogenous motor preparation but not following endogenous covert orienting provides
converging evidence for this disconfirmation.
3. In this case, the apparent conflict with Sheliga et al. (1994) may be resolved by consider-
ing whether the signal generating the probe saccade is imposed exogenously with no
204 Klein a n d Shore
opportunity to modify the state of the oculomotor system (Kustov and Robinson 1996), or
whether it is generated endogenously and therefore might be accompanied by a voluntary
cancellation of the prepared saccade (Sheliga et al. 1994).
4. Whether exogenous covert orienting interacts with feature integration because it entails
peripheral visual stimulation that is an input to pattern recognition routines could be
tested by exploring whether exogenous visual orienting in response to a localizable audi-
tory event yields the same interaction with illusory conjunctions as does a peripheral
visual stimulus.
REFERENCES
Allport, A., Styles, E., and Hseieh, S. (1994). Shifting intentional set: Exploring the dynamic
control of tasks. In C. Umiltà a n d M. Miscovitch (Eds.), Attention and Performance XV, p p .
421–452. Cambridge, MA: MIT press.
Bacon. W. F., and Egeth, H. E. (1994). Overriding stimulus-driven attention capture.
Perception and Psychophysics, 55, 485–496.
Briand, K. A. (1998). Feature integration a n d spatial attention: More evidence of a dissocia-
tion between endogenous and exogenous orienting. Journal of Experimental Psychology:
Human Perception and Performance, 24, 1243–1256.
Briand, K. A., a n d Klein, R. M. (1987). Is Posner’s “beam’’ the same as Treisman’s “glue’’?
On the relation between visual orienting a n d feature integration theory. Journal of
Experimental Psychology: Human Perception and Performance, 13, 228–241.
Broadbent, D. E. (1971). Decision and stress. London: Academic Press.
Chelazzi, L., Miller, E. K., Duncan, J., and Desimone, R. (1993). A neural basis for visual
search in inferior temporal cortex. Nature, 363, 345–347.
Desimone, R., and Duncan, J. (1995). Neural mechanisms of selective viual attention. Annual
Review of Neuroscience, 18, 193–222.
Driver, J., Davis, G., Ricciardelli, P., Kidd, P., Maxwell, E., a n d Baron-Cohen, S. (1999). Gaze
perception triggers reflexive visuospatial orienting. Visual Cognition, 6, 509–540.
Ennis, T., a n d Kingtone, A. (1998). The oculomotor readiness hypothesis: Revisited (Again!).
Paper presented at the Canadian Society for Brain, Behavior and Cognitive Science, Ottawa,
June.
Everling, S., and Fischer, B. (1998). The antisaccade: A review of basic research a n d clinical
studies. Neuropsychologia, 36, 885–899.
Everling, S., Paré, M., Dorris, M. C., and Munoz, D. P. (1998). Comparison of the discharge
characteristics of brain stem omnipause neurons and superior colliculus fixation neurons
in m o n k e y : Implications for control of fixation a n d saccade behavior. Journal of
Neurophysiology, 79, 511–528.
Fischer, B., and Ramsperger, E. (1984). H u m a n express saccades: Extremely short reaction
times of goal-directed eye movements. Experimental Brain Research, 57, 191–195.
Folk, C. L., a n d Remington, R. W. (1998). Selectivity in distraction by irrelevant featural sin-
gletons: Evidence for two forms of attentional capture. Journal of Experimental Psychology:
Human Perception and Performance, 24, 847–858.
Folk, C. L., Remington, R. W., a n d Johnston, J. C. (1992). Involuntary covert orienting is con-
tingent on attentional control settings. Journal of Experimental Psychology: Human Perception
and Performance, 18, 1030–1044.
Modes of Visual Orienting
Forbes, K., and Klein, R. M. (1996). The magnitude of the fixation offset effect with endoge-
nously a n d exogenously controlled saccades. Journal of Cognitive Neuroscience, 8, 344–352.
Friesen, C. K., a n d Kingstone, A. (1998). The eyes have it! Reflexive orienting is triggered by
nonpredictive gaze. Psychonomics Bulletin and Review, 5, 490–495.
Hikosaka, O., Miyauchi, S., a n d Shimojo, S. (1993). Focal visual attention produces illusory
temporal order and motion sensation. Vision Research, 33, 1219–1240.
Hoffman, J. E., and Subramaniam, B. (1995). The role of visual attention in saccadic eye
movements. Perception and Psychophysics, 57, 787–795.
Jonides, J. (1976). Voluntary versus reflexive control of the mind’s eye’s movement. Paper
presented at the annual meeting of the Psychonomics Society, St. Louis, November.
Jonides, J. (1981). Voluntary versus automatic control over the mind’s eye’s movement. In
J. B. Long a n d A. D. Baddeley (Eds.), Attention and Performance IX, p p . 187–203. Hillsdale,
NJ: Erlbaum.
Jonides, J., a n d Yantis, S. (1988). Uniqueness of abrupt visual onset in capturing attention.
Perception and Psychophysics, 43, 346–354.
Joseph, J. S., and Optican, L. M. (1996). Involuntary attentional shifts d u e to orientation dif-
ferences. Perception and Psychophysics, 58, 651–665.
Kaufman, L., a n d Richards, W. (1969). Spontaneous fixation tendencies for visual forms.
Perception and Psychophysics, 5, 85–88.
Kingstone, A., and Egly, R. (In preparation). Endogenous and exogenous orienting during
the maintenance of a form expectancy.
Kingstone, A., Grabowecky, M., Mangun, G. R., Valsangkar, M. A., and Gazzinaga, M. S.
(1995). Paying attentionn to the brain: The study of selective visual attention in cognitive
neuroscience. In J. Burak and J. Enns (Eds.), Attention, development and psychopathology, p p .
263–287. N e w York: Guilford.
Kingstone, A., and Klein, R. M. (1993). What are h u m a n express saccades? Perception and
Psychophysics, 54, 260–273.
Klein, R. M. (1980). Does oculomotor readiness mediate cognitive control of visual atten-
tion? In R. Nickerson (Ed.), Attention and Performance VIII, p p . 259–275. N e w York:
Academic Press.
Klein, R. M. (1994). Perceptual-motor expectancies interact with covert visual orienting
under endogenous but not exogenous control. Canadian Journal of Experimental Psychology,
48, 151–166.
Klein, R. M. (In preparation). Shared neural control of attention a n d eye movements:
Intuitively appealing but not quite established. Journal of Experimental Psychology: Human
Perception and Performance.
Klein, R. M., a n d Hansen, E. (1990). Chronometric analysis of spotlight failure in endoge-
nous visual orienting. Journal of Experimental Psychology: Human Perception and Performance,
16, 790–801.
Klein, R. M., Kingstone, A., a n d Pontefract, A. (1992). Orienting of visual attention. In K.
Rayner (Ed.), Eye movements and visual cognition: Scene perception and reading, p p . 46–65. N e w
York: Springer-Verlag.
Klein, R. M., and Pontefract, A. (1994). Does oculomotor readiness mediate cognitive con-
trol of visual attention? Revisited! In R. Nickerson (Ed.), Attention and Performance XV:
Conscious and nonconscious information processing, p p . 333–350. Hillsdale, NJ: Erlbaum.
Klein a n d Shore
Klein, R. M., Schmidt, W. C., a n d Muller, H. J. (1998). Disinhibition of return: Unnecessary
a n d unlikely. Perception and Psychophysics, 60, 862–872.
Klein, R. M., and Taylor, T. L. (1994). Categories of cognitive inhibition, with reference to
attention. In D. Dagenbach and T. H. Carr (Eds.), Inhibitory processes in attention, memory, and
language, p p . 113–150. San Diego, CA: Academic Press.
Kustov, A. A., a n d Robinson, D. L. (1996). Shared neural control of attention shifts a n d eye
movements. Nature, 384, 74–77.
Langton, S. R. H., and Bruce, V. (1999). Reflexive visual orienting in response to the social
attention of others. Visual Cognition, 6, 541–567.
Maylor, E. (1985). Facilitatory a n d inhibitory components of orienting in visual space. In M.
I. Posner and O. S. M. Marin (Eds.), Attention and Performance XI, p p . 189–203. Hillsdale, NJ:
Erlbaum.
McColl, S. L., a n d Schmidt, W. C. (1995). Orientation singletons evoke facilitation as mea-
sured using illusory line motion. Investigative Opthalmology and Visual Science, 36, S373.
McCormick, P. A., a n d Klein, R. (1990). The spatial distributon of attention during covert
visual orienting. Acta Psychologica, 75, 225–242.
Müller, H. J., and Rabbitt, P. M. A. (1989). Reflexive and voluntary orienting of visual atten-
tion: Time course of activation and resistance to interruption. Journal of Experimental
Psychology: Human Perception and Performance, 15, 315–330.
Posner, M. I. (1980). Orienting of attention. Quarterly Journal of Experimental Psychology, 32,
3–25.
Posner, M. I., Snyder, C. R., a n d Davidson, B. J. (1980). Attention and the detection of sig-
nals. Journal of Experimental Psychology: General, 109, 160–174.
Prinzmetal, W., Presti, D., a n d Posner, M. I. (1986). Does attention affect feature integration?
Journal of Experimental Psychology: Human Perception and Performance, 12, 361–369.
Rizzolatti, G., Riggio, L., Dascola, I., and Umiltà, C. (1987). Reorienting attention across the
horizontal and vertical meridians: Evidence in favor of a premotor theory of attention.
Neuropsychologia, 25, 31–40.
Rogers, R. D., and Monsell, S. (1995). Costs of a predictable switch between simple cogni-
tive tasks. Journal of Experimental Psychology: General, 124, 207–231.
Schmidt, W. C. (1994). Stimulus-driven attentional capture: Evidence from illusory line
motion. Master’s thesis, University of Western Ontario, London.
Sheliga, B. M., Riggio, L., a n d Rizzolatti, G. (1994). Orienting of attention and eye move-
ments. Experimental Brain Research, 98, 507–522.
Shepherd, M., Findlay, J. M., and Hockey, R. J. (1986). The relationship between eye move-
ments a n d spatial attention. Quarterly Journal of Experimental Psychology, 38A, 475–491.
Subramaniam, B., and Hoffman, J. E. (1991). Saccadic eye movement a n d visual selective
attention. Paper presented before the Psychonomics Society, San Francisco, November.
Taylor, T., Kingstone, A. F., and Klein, R. M. (1998). Visual offsets and oculomotor disinhibi-
tion: Endogenous and exogenous contributions to the gap effect. Canadian Journal of Experi-
mental Psychology, 52, 192–200.
Taylor, T. L., and Klein, R. M. (1998). On the causes and effects of inhibition of return.
Psychonomic Bulletin and Review, 5, 625–643.
Theeuwes, J., Kramer, A. F., Han, S., and Irwin, D. E. (1998). Our eyes do not always go
where want them to go: Capture of the eyes by new objects. Psychological Science, 9, 379–385.
Modes of Visual Orienting
Treisman, A. (1985). Preattentive processing in vision. Computer Vision, Graphics and Image
Processing, 31, 156–177.
Treisman, A., and Gelade, G. (1980). A feature integration theory of attention. Cognitive
Psychology, 12, 97–125.
Treisman, A., a n d Sato, S. (1990). Conjunction search revisited. Journal of Experimental
Psychology: Human Perception and Performance, 16, 459–478.
Wolfe, J. M., Cave, K. R., a n d Franzel, S. L. (1989). Guided search: An alternative to the fea-
ture integration model for visual search. Journal of Experimental Psychology: Human Perception
and Performance, 15, 419–433.
Yantis, S., a n d Hillstrom, A. P. (1994). Stimul us-driven attentional capture: Evidence from
equiluminant visual objects. Journal of Experimental Psychology: Human Perception and
Performance, 20, 95–107.
Yantis, S., a n d Jonides, J. (1984). Abrupt visual onsets and selective attention: Evidence from
visual search. Journal of Experimental Psychology: Human Perception and Performance, 10,
601–621.
Yantis, S., a n d Jonides, J. (1990). Abrupt visual onsets a n d selective attention: Voluntary
versus automatic allocation. Journal of Experimental Psychology: Human Perception and
Performance, 16, 121–134.
Yarbus, A. L. (1967). In L. A. Riggs (Ed.), Eye movements and vision. New York: Plenum Press.
208 Klein a n d Shore
9 The Control of Visuomotor Control
A. David Milner
ABSTRACT There is growing evidence for a number of parallel systems transforming
visual inputs into action coordinates, as well as several transforming inputs into percepts.
Moreover, the perceptual system is itself implicated in several aspects of visuomotor
guidance. This multiplicity of routes participating in visuomotor control raises various
questions of integration and coordination.
9.1 SEPARATION OF PERCEPTION FROM VISUOMOTOR
CONTROL
Until recently, there has been a near-universal tendency in psychology
a n d neuroscience to think of visual processing as a means of constructing
a single, all-purpose perceptual representation, one that can serve us in
all of our dealings with the world, whether motor, mnemonic, aesthetic,
or social. Although it seems introspectively obvious, a n d almost absurd
to deny, that “what we see is what we get,’’ this view of vision has
become increasingly untenable, a n d we are currently in the midst of a
radical change in the way that visual scientists conceptualize their field of
study (Georgeson 1997; Milner a n d Goodale 1995). The change in per-
spective toward seeing vision as having multiple endpoints has sprung
largely from research in visual neuroscience—human neuropsychology,
primate electrophysiology, a n d neurobehavioral studies of animals
(Milner and Goodale 1995). Moreover, there are many observations in
normal h u m a n subjects of dissociations between different visual process-
ing systems (brought about, for example, through perceptual illusions)
that fit well into the same framework (Goodale and Haffenden 1998).
“ H u m a n perception,’’ as Von Hofsten (1987, 34) observed prophetically,
may be specialized enough to make it appropriate to speak of a number
of perception-action systems instead of regarding perception as a uni-
tary process separate from action . . . . To believe that the study of, for
instance, arbitrary finger movements as response to displayed letters or
numbers will reveal anything essential about the coordination between
perception a n d action is doubtful. It may actually be as erroneous as the
belief that verbal memory could be studied through the use of nonsense
syllable lists.
Current approaches have n o w led to a general recognition that there
are quasi-independent visual subsystems in the primate brain, each serv-
ing a different motor domain—saccadic eye movements, ocular pursuit,
locomotion, reaching with the arm, grasping with the hand. The neuronal
machinery underlying such “pragmatic’’ coding of visual information
(Jeannerod a n d Rossetti 1993) is vested principally in the occipitoparietal
complex of visual areas k n o w n as the “dorsal stream’’ and its associated
structures in the brain stem, thalamus, a n d frontal lobes. A quite different
complex of visual areas, the occipitotemporal “ventral’’ stream, carries
out the processing that provides the furniture for our perceptual experi-
ence a n d the raw materials for storing visual memories. This perceptual
system, itself not unitary, appears in large part to operate indepen-
dently of the dedicated visuomotor systems. While the efficient operation
of both streams m u s t d e p e n d on selective spatial gating, the visuo-
motor control systems appear to do this without the benefit of visual
consciousness.
The separate functioning of these two broad systems of visual process-
ing can be seen most dramatically in neurological “experiments of
nature’’ in which one of the systems has been damaged, leaving the other
to work largely in isolation. That is, patients may suffer from parietal lobe
damage that impairs visuomotor control but largely spares visual per-
ception, a condition known as “optic ataxia’’ (Perenin a n d Vighetto 1988);
or more rarely, they may suffer damage that preserves visuomotor control
despite a severe loss of form perception, a condition known as “visual
form agnosia’’ (Milner et al. 1991).
The existence of multiple visual pathways in the brain raises important
a n d puzzling questions as to their interrelationships: H o w do they inter-
act in the guidance of behavior? Are they subject to overall orchestration
by other systems, a n d if so, how? These questions, though scarcely
addressed in empirical investigations to date, are central to the theme of
this volume. I restrict my present comments to two broad issues:
1. H o w concurrently activated visuomotor systems or processes are
coordinated;
2. When and how perceptual processing is co-opted in the service of
visuomotor control.
9.2 COORDINATION OF VISUOMOTOR CONTROL
The first a n d most experimentally tractable question is h o w a movement
through space, such as a manual reach or a saccadic eye movement, can
be influenced by competing stimuli in other parts of space. Pioneering
studies to address this question were carried out by Tipper, Lortie, a n d
Baylis (1992) and Sheliga, Riggio a n d Rizzolatti (1994), discussed, along
Milner
with more recent work, by Tipper, Howard, and Houghton (chap. 10, this
volume). Depending on the distractor’s proximity a n d salience, it has
been found that the eye or h a n d trajectory might veer toward or away
from the distractor, a n d Tipper a n d colleagues present a model that can
encompass both of these types of effect.
Of course, given that studies of this kind reveal the operation of com-
peting motor programs, it may be expected that such competition will
occur not only between competing transport tendencies such as reaching,
but also between nontransport movements such as the in-flight prefor-
mation of h a n d grip as a function of the size of target a n d nontarget
objects.
Initial studies of this possibility (Chieffi et al. 1993; Jackson, Jackson,
a n d Rosicky 1995) found little evidence for anticipatory grip size to be
influenced by the size of a distractor during normal grasping movements.
More recently, however, Gangitano, Daprati, a n d Gentilucci (1998) a n d
Castiello (1998) have reported subtle interference effects under certain
conditions. For example, Castiello (1998) has found changes in the
rate of h a n d opening when three-dimensional distractors, but not two-
dimensional ones, are used. This may be because a 2-D shape only par-
tially activates the visuomotor module that governs the grasp component
of a prehension movement.
A complementary question that arises is h o w we can make on-line
changes during a reaching movement in response to a change in the
visual array that actually is relevant to our overall goals and has to be
responded to. This may require a change in the nature of the action itself,
or a change in some parameter of the action, such as the direction of a
reach. A number of experiments have addressed these issues. In an inter-
esting example of the first type of study, Pisella, Arzi, a n d Rossetti (1998)
have used a task in which the subject is required to either redirect or
interrupt a reaching movement in response to particular changes in the
stimulus. More commonly, however, researchers have used tasks in
which a target change during the course of a movement requires only
that the metrics, rather than the nature, of the movement be modified to
deal with the change. Interestingly, if a visual target is abruptly moved by
only a few degrees during a saccade, not only are subjects unaware of any
shift (Bridgeman, Hendry, a n d Stark 1975), but they will make a perfectly
normal recalibrated hand movement toward it, unwittingly incorporat-
ing an appropriate terminal correction (Goodale, Pélisson, a n d Prablanc
1986).
If the target is shifted well away from the “ballpark’’ of the original
location, however, the kinematic characteristics of a reach do change.
Even though the subject becomes subjectively aware of such major per-
turbations, however, this awareness may occur much later than the motor
adjustments m a d e . For example, Castiello, Paulignan, a n d Jeannerod
Control of Visuomotor Control
(1991), asked subjects to indicate (using a vocal response) when they per-
ceived a s u d d e n displacement in the position of an object to which they
were directing a grasp. On trials in which the object w a s displaced at
the onset of the reach, the vocal response w a s emitted 420 msec after the
onset of the movement. In contrast, adjustments to the trajectory of the
grasping movement could be seen as early as 100 msec after the pertur-
bation, that is, more than 300 msec earlier than the vocal response.
Other studies have examined the effects of on-line perturbations of the
intrinsic stimulus properties of a target object, such as its size or orienta-
tion, on wrist a n d finger movements preparatory to grasping the object
(e.g., Paulignan et al. 1991). These investigations show that our visuomo-
tor apparatus is flexible enough to respond adaptively to target changes,
a n d indeed that some adjustments will be absorbed into the movement
almost as if nothing had happened, offering a speed advantage that
allows us, for example, to pursue a fly in midflight. But while “natural’’
changes (such as small displacements) are handled as a matter of course
by the system, w i t h o u t any need for external m o d u l a t i o n , other
changes—particularly ones that would rarely or never occur in everyday
life—may require a reprogramming dependent on a conscious percep-
tion of the change, resulting in a slowing of the action.
Another well-studied question is h o w we coordinate different visuo-
motor subsystems to work together smoothly during the execution of
everyday actions. The unfolding of a prehension movement, in which the
transport a n d grasp components of the arm and h a n d along with the
requisite eye movements, are integrated in exquisite temporal and spatial
harmony (Jeannerod 1988), provides a prime example. Jeannerod a n d his
colleagues make a good case that this coordination can be achieved by
m e a n s of direct interactions between the visuomotor s u b s y s t e m s
involved. Yet such interactions would have to be able to cope with the
fact that spatial location for action is visually coded in different ways in
different subsystems within the parietal cortex (Colby a n d Duhamel 1997;
Snyder et al. 1998).
If different spatial “languages’’ are used to perform these visuomotor
transformations, how are the subsystems able to talk to each other?
Current evidence in the monkey suggests a solution: in both the reaching
a n d saccadic control areas of the monkey’s posterior parietal cortex,
many visuomotor neurons retain a retinotopic coding of target location,
although their responses may be modulated by eye and sometimes head
or limb position signals (Andersen 1997; Snyder et al. 1998). While this
“gain field’’ modulation provides a mechanism for ensembles of neurons
to code location with respect to the head or body, it does so without dis-
carding the retinal information (Andersen 1997). This preserved retinal
information could provide the single common language needed for the
different systems to work together (Goodale 1998).
Milner
9.3 COORDINATION OF PERCEPTION AND ACTION
But if there is a binding problem between different visuomotor systems,
there must be a greater one between the perceptual a n d visuomotor sys-
tems (first noted by Peter Milner in 1974). We have argued (Milner a n d
Goodale 1995; Milner 1997) that the ventral stream, through its close con-
nections with areas such as the perirhinal cortex (Parker a n d Gaffan
1998), can both inform a n d be informed by systems for storing the endur-
ing visual characteristics of objects. Once the perceptual system has con-
sulted its “semantic’’ knowledge base a n d identified a visual target as
deserving of further action, it is presumed that suitable motor instruc-
tions can be issued and the “pragmatic’’ dorsal system be brought into
play to guide the animal’s actions.
This proposed relationship finds a nice analogy in the use of “tele-
assistance’’ in robotic control (Goodale 1998). In this case, an intelligent
system in the form of a h u m a n operator may be able to identify an item
of interest on, say, the surface of the moon by means of a video signal, a n d
can send an instruction to a semiautonomous robot to carry out co-
ordinated actions u p o n that item using its o w n sensing a n d output de-
vices. This metaphor underlines the important point that in the proposed
m o d e of communication between the two visual systems, the perceptual
processing is not providing the visuomotor control but preparing the
way for it.
Neuropsychological studies make this point graphically. For example,
D.F., a patient with visual form agnosia (Milner et al. 1991), is well able to
perform a number of reaching a n d grasping tasks with normal levels of
skill even though she is unable to process the objects of her actions per-
ceptually. Thus she is unable to judge the width or orientation of a rec-
tangular block, a n d yet reaches out to pick it up with the same degree of
visually based wrist a n d grip calibration as a normal subject. This means
that providing she has some way of “tagging’’ the target object (e.g.,
through its color), she does not need to perceive its contours in order to
grasp it successfully.
Indeed, visuomotor control can proceed with modest success without
the intervention of any visual perception at all, provided the subject has
some means of localizing the target stimulus. It has been k n o w n for many
years that some patients with complete hemianopia caused by damage to
the primary visual cortex may still be able to direct the eye or h a n d
toward stimuli in the “blind’’ field despite having no conscious visual
experience of those stimuli (Weiskrantz et al. 1974). More recently, it has
been found that similar patients may show significant visual calibration
of the wrist a n d fingers when reaching for objects in their blind field
(Perenin and Rossetti 1996; Rossetti 1998; Marcel 1998). Evidently, an
object need not be present in awareness for the brain to be able to “tag’’
it spatially a n d to process its characteristics to guide action.
Control of Visuomotor Control
The problem in understanding h o w the real brain can use “teleassis-
tance’’ arises because of a profound difference in the way visual space is
encoded in the perceptual and visuomotor systems (Bridgeman et al.
1979; Wong and Mack 1981; Goodale, Pélisson, a n d Prablanc 1986;
Paillard 1987). The perceptual (sometimes called “cognitive’’ or “repre-
sentational’’) system relates stimulus location to a contextual framework,
a n d is consequently subject to various visual illusions, whereas the visuo-
motor system relates stimulus location directly to the observer, a n d is
therefore much less prone to systematic error. How, then, can an item
localized within a relative visual coordinate system be tagged in a way
that can be accessed by a visuomotor system that operates in egocentric
coordinates?
The answer may be that the perceptual system is itself able to bring
about m o v e m e n t s , albeit less directly t h a n the dedicated system
(Bridgeman et al. 1979; Wong a n d Mack 1981). This use of perceptual rep-
resentations to drive action is exemplified by our ability to direct the eyes
or h a n d to a target no longer physically present, a feat the dedicated sys-
tem is not equipped to perform. This limitation is apparent in patients
w h o cannot use their perceptual system, whether through visual form
agnosia (Goodale, Jakobson, a n d Keillor 1994; Milner, Dijkerman, a n d
Carey 1999) or hemianopia (Rossetti 1998). They are unable to guide their
reaching and grasping on the basis of visual information presented just a
few seconds earlier.
We may assume then that a normally functioning perceptual system
must be able to guide our body and eyes towards the location of a rele-
vant stimulus with respect to other items in the world. Once the appro-
priate action has been selected (probably through frontal systems;
Riddoch, Humphreys, a n d Edwards, chap. 27, this volume), our dorsal
visuomotor systems would then provide the precise guidance of limbs
a n d other effectors needed for performing the action.
A converse form of mutual assistance will of course arise regularly
whenever the visuomotor system initiates an orienting movement
toward a novel or salient visual stimulus, bringing it onto the fovea for
perceptual analysis. The ventral processing stream, which is specialized
for analyzing the central region of the visual field, can then do its work.
But h o w could a stimulus located through the dorsal stream that does not
result in an overt orienting response still receive detailed processing by
the perceptual system? We have suggested previously that a selected
location might be “broadcast’’ from parietal areas to other visual areas
(Milner a n d Goodale 1995), presumably again in retinal code. Some such
mechanism could underlie instances of “cross-priming,’’ whereby stimuli
selected as the targets of saccadic eye movements (Deubel and Schneider
1996) or manual reaching movements (Deubel, Schneider, a n d Paprotta
1998) gain higher perceptual discriminability. Thus it may be that “the
(dorsally based) preparation of a goal-directed motor response . . . binds
Milner
the (perceptual) processing capacities of the ventral stream to the same
object’’ (Deubel, Schneider, and Paprotta 1998, 100).
So far we have considered relatively indirect forms of interaction
between the visual streams. But there is little hard evidence that the dor-
sal stream can offer a detailed visual analysis of objects, beyond their
axial orientation, size, and spatiotemporal disposition. Might not the
brain need to supplement this in order to guide everyday actions ade-
quately? If the perceptual system does need to be recruited in this rather
more direct way, then the lack of perceptual ability in an agnosic patient
such as D.F. would be expected to impose sharp limits on her visuomo-
tor skills. There are in fact several examples of such limits. Thus, although
she can post a flat object through an oriented slot, she makes many 90-
degree errors when asked to post a T-shape into a T-shaped aperture
(Goodale et al. 1994). Similarly, she can grasp an elongated block at any
orientation with normal accuracy, yet fails to vary her h a n d orientation
when reaching to grasp a cross-shaped object set at different orientations
(Carey, Harvey, and Milner 1996). And although she can point accurately
to single points in space, she cannot open her t h u m b a n d forefinger to
match the separation of two holes set in a disk she is asked to grasp
(Dijkerman, Milner, a n d Carey 1998).
This need for perceptual information also becomes apparent in grasp-
ing tasks that depend on 3-D information, whether for calibrating the
amplitude of a reach (Marotta, Behrmann, a n d Goodale 1997) or for
adjusting the wrist when grasping an object tilted in depth (Dijkerman et
al. 1996). Whenever the use of binocular vision is prevented, the normal
visual system can fall back on using “pictorial’’ cues provided by per-
spective a n d figural context. Because these are cues that D.F. cannot use,
however, her performance is impaired under stationary monocular view-
ing conditions, although her monocular accuracy is restored to near nor-
mal when she moves her head sideways to provide herself with motion
parallax cues (Dijkerman, Milner, a n d Carey 1999).
All of these limitations on D.F.’s visuomotor ability, then, illustrate the
intact brain’s ability to benefit from perceptual processing in its execution
of visuomotor acts. Our prehension a n d other motor skills in the real
world of complex objects cannot depend entirely on the basic, if quick
a n d reliable, guidance that the dedicated visuomotor system provides.
But the ventral stream can offer other benefits to motor guidance
beyond simply more elaborate bottom-up analysis. It can also, through its
links with memory stores, access top-down information about the nature
of the objects themselves, for example, their fragility and weight—infor-
mation not given directly to the retina. That is, the perceptual system can
modulate aspects of our actions beyond those that can be computed from
the geometry of the target array. A good example of this is the visual
calibration of grip force, which is measurable on initial contact with an
object, prior to any proprioceptive feedback (Johansson and Cole 1992).
Control of Visuomotor Control
This modulation must be based on stored size-weight correlations. The
assumption that this force calibration is achieved via the perceptual
rather than the visuomotor system is consistent with recent studies of
geometric visual illusions. Although such illusions fail to influence antic-
ipatory grip aperture during our reaches for an object whose size we mis-
perceive, they do influence the force of our grip in the grasping act itself
(Brenner and Smeets 1996; Jackson a n d Shaw 2000).
Jeannerod, Decety, and Michel (1994) found what may be a clue to
h o w ventral processing is able to influence action in these direct ways.
Although their optic ataxic patient A.T. showed severe visuomotor prob-
lems in her attempts to grasp rectangular blocks of different widths, she
became much better able to calibrate her grip when faced with familiar
objects (e.g., a lipstick). Presumably, outputs from her functioning recog-
nition system were able to bypass the badly damaged control networks
in her parietal lobes a n d gain direct access to motor systems. An impor-
tant question for the future will be to delineate which outputs of the per-
ceptual system can follow such an independent route, without needing to
implicate the dorsal stream.
9.4 CONCLUSIONS
In this commentary, I have d r a w n attention to a number of questions
that arise from current conceptualizations of perception a n d action, but
have offered only a few tentative answers. Perhaps the next Attention
a n d Performance symposium, “Common Processes in Perception a n d
Action,’’ will provide a clearer picture.
NOTE
I am grateful to Mel Goodale for his comments on the draft manuscript, to Stephen Monsell
a n d Jon Driver for their helpful remarks, a n d to the Wellcome Trust for their financial
support.
REFERENCES
Andersen, R. A. (1997). Multimodal integration for the representation of space in the poste-
rior parietal cortex. Philosophical Transactions of the Royal Society of London, B352, 1421–1428.
Brenner, E., and Smeets, J. B. J. (1996). Size illusion influences how we lift but not how we
grasp an object. Experimental Brain Research, 111, 473–476.
Bridgeman, B., Hendry, D., and Stark, L. (1975). Failure to detect displacement of the
visual world during saccadic eye movements. Vision Research, 15, 719–722.
Bridgeman, B., Lewis, S., Heit, G., and Nagle, M. (1979). Relation between cognitive a n d
motor-oriented systems of visual position perception. Journal of Experimental Psychology:
Human Perception and Performance, 5, 692–700.
Carey, D. P. , Harvey, M., and Milner, A. D. (1996). Visuomotor sensitivity for shape and ori-
entation in a patient with visual form agnosia. Neuropsychologia, 34, 329–338.
Milner
Castiello, U. (1998). Attentional coding for three-dimensional objects a n d two-dimensional
shapes: Differential interference effects. Experimental Brain Research, 123, 289–297.
Castiello, U., Paulignan, Y., and Jeannerod, M. (1991). Temporal dissociation of motor
responses and subjective awareness. Brain, 114, 2639–2655.
Chieffi, S., Gentilucci, M., Allport, A., Sasso, E., a n d Rizzolatti, G. (1993). Study of selective
reaching a n d grasping in a patient with unilateral parietal lesion. Brain, 116, 1119–1137.
Colby, C. L., a n d Duhamel, J. R. (1997). Spatial representations for action in parietal cortex.
Cognitive Brain Research, 5, 105–115.
Deubel, H., a n d Schneider, W. X. (1996). Saccade target selection a n d object recognition:
Evidence for a common attentional mechanism. Vision Research, 36, 1827–1837.
Deubel, H., Schneider, W. X., and Paprotta, I. (1998). Selective dorsal a n d ventral processing:
evidence for a common attentional mechanism in reaching and perception. Visual Cognition,
5, 81–107.
Dijkerman, H. C., Milner, A. D., and Carey, D. P. (1996). The perception a n d prehension of
objects oriented in the depth plane: 1. Effects of visual form agnosia. Experimental Brain
Research, 112, 442–451.
Dijkerman, H. C., Milner, A. D., a n d Carey, D. P. (1998). Grasping spatial relationships: fail-
ure to demonstrate allocentric visual coding in a patient with visual form agnosia. Con-
sciousness and Cognition, 7, 424–437.
Dijkerman, H. C., Milner, A. D., and Carey, D. P. (1999). Prehension of objects oriented in
depth: motion parallax restores performance of a visual form agnosic when binocular vision
is unavailable. Neuropsychologia, 37, 1505–1510.
Gangitano, M., Daprati, E., a n d Gentilucci, M. (1998). Visual distractors differentially inter-
fere with the reaching and the grasping components of prehension movements. Experi-
mental Brain Research, 122, 441–452.
Georgeson, M. (1997). Vision and action: You ain’t seen nothin’ yet. Perception, 26, 1–6.
Goodale, M. A. (1998). Visuomotor control: Where does vision end and action begin?
Current Biology, 8, R489–R491.
Goodale, M. A., a n d Haffenden, A. M. (1998). Frames of reference for perception a n d action
in the h u m a n visual system. Neuroscience and Biobehavioral Reviews, 22, 161–172.
Goodale, M. A., Pélisson, D., and Prablanc, C. (1986). Large adjustments in visually guided
reaching do not depend on vision of the h a n d or perception of target displacement. Nature,
320, 748–750.
Goodale, M. A., Jakobson, L. S., a n d Keillor, J. M. (1994). Differences in the visual control of
pantomimed a n d natural grasping movements. Neuropsychologia, 32, 1159–1178.
Goodale, M. A., Jakobson, L. S., Milner, A. D., Perrett, D. I., Benson, P. J., a n d Hietanen, J. K.
(1994). The nature and limits of orientation and pattern processing supporting visuomotor
control in a visual form agnosic. Journal of Cognitive Neuroscience, 6, 46–56.
Jackson, S. R., a n d Shaw, A. (2000). The Ponzo illusion affects grip force but not grip aper-
ture scaling during prehension movements. Journal of Experimental Psychology: Human
Perception and Performance, 26, 1–6.
Jackson, S. R., Jackson, G. M., a n d Rosicky, J. (1995). Are non-relevant objects represented
in working memory? The effects of non-target objects on reach and grasp kinematics.
Experimental Brain Research, 102, 519–530.
Jeannerod, M. (1988). The neural and behavioural organization of goal-directed movements.
Oxford: Oxford University Press.
219 Control of Visuomotor Control
Jeannerod, M., Decety, J., a n d Michel, F. (1994). Impairment of grasping movements follow-
ing bilateral posterior parietal lesion. Neuropsychologia, 32, 369–380.
Jeannerod, M., and Rossetti, Y. (1993). Visuomotor coordination as a dissociable visual func-
tion: experimental and clinical evidence. In C. Kennard (Ed.), Visual perceptual defects,
Bailliere’s clinical neurology, vol. 2, no. 2, p p . 439–460. London: Bailliere Tindall.
Johansson, R. S., and Cole, K. J. (1992). Sensory-motor coordination during grasping a n d
manipulative actions. Current Opinion in Neurobiology, 2, 815–823.
Marcel, A. J. (1998). Blindsight and shape perception: Deficit of visual consciousness or of
visual function? Brain, 121, 1565–1588.
Marotta, J. J., Behrmann, M., a n d Goodale, M. A. (1997). The removal of binocular cues dis-
rupts the calibration of grasping in patients with visual form agnosia. Experimental Brain
Research, 116, 113–121.
Milner, A. D. (1997). Vision without knowledge. Philosophical Transactions of the Royal Society
of London, B352, 1249–1256.
Milner, A. D., and Goodale, M. A. (1995). The visual brain in action. Oxford: Oxford
University Press.
Milner, A. D., Dijkerman, H. C., and Carey, D. P. (1999). Visuospatial processing in a pure
case of visual-form agnosia. In N. Burgess, K. Jeffery a n d J. O’Keefe (Eds.), The hippocampal
and parietal foundations of spatial cognition, p p . 443–466. Oxford: Oxford University Press.
Milner, A. D., Perrett, D. I., Johnston, R. S., Benson, P. J., Jordan, T. R., Heeley, D. W., Bettucci,
D., Mortara, F., Mutani, R., Terazzi, E., and Davidson, D. L. W. (1991). Perception and action
in “visual form agnosia.’’ Brain, 114, 405–428.
Milner, P. M. (1974). A model for visual shape recognition. Psychological Review, 81, 521–535.
Paillard, J. (1987). Cognitive versus sensorimotor encoding of spatial information. In P. Ellen
a n d C. Thinus-Blanc (Eds.), Cognitive processes and spatial orientation in animal and man. Vol.
2, Neurophysiology and developmental aspects, p p . 43–77. Dordrecht: Nijhoff.
Parker, A., a n d Gaffan, D. (1998). Memory systems in primates: episodic, semantic, a n d per-
ceptual learning. In A. D. Milner (Ed.), Comparative neuropsychology, p p . 109–126. Oxford:
Oxford University Press.
Paulignan, Y., Jeannerod, M., MacKenzie, C., and Marteniuk, R. (1991). Selective perturba-
tion of visual input during prehension movements: 2. The effects of changing object size.
Experimental Brain Research, 87, 407–420.
Perenin, M. T., a n d Rossetti, Y. (1996). Grasping without form discrimination in a hemi-
anopic field. NeuroReport, 7, 793–797.
Perenin, M.-T., and Vighetto, A. (1988). Optic ataxia: A specific disruption in visuomotor
mechanisms: 1. Different aspects of the deficit in reaching for objects. Brain, 111, 643–674.
Pisella, L., Arzi, M., a n d Rossetti, Y. (1998). The timing of color and location processing.
Experimental Brain Research, 121, 270–276.
Rossetti, Y. (1998). Implicit perception in action: Short-lived motor representations of space.
Consciousness and Cognition, 7, 520–558.
Sheliga, B. M., Riggio, L., a n d Rizzolatti, G. (1994). Orienting of attention and eye move-
ments. Experimental Brain Research, 98, 507–522.
Snyder, L. H., Grieve, K. L., Brotchie, P., a n d Andersen, R. A. (1998). Separate body- a n d
world-referenced representations of visual space in parietal cortex. Nature, 394, 887–891.
Milner
Tipper, S. P. , Lortie, C., a n d Baylis, G. C. (1992). Selective reaching: Evidence for action-
centered attention. Journal of Experimental Psychology: Human Perception and Performance, 18,
891–905.
Von Hofsten, C. (1987). Catching. In H. Heuer and A. F. Sanders (Eds.), Perspectives on per-
ception and action, p p . 33–46. Hillsdale, NJ: Erlbaum.
Weiskrantz, L., Warrington, E. K., Sanders, M. D., a n d Marshall, J. (1974). Visual capacity in
the hemianopic field following a restricted occipital ablation. Brain, 97, 709–728.
Wong, E., and Mack, A. (1981). Saccadic programming and perceived location. Acta
Psychologica, 48, 123–131.
221 Control of Visuomotor Control
10 Behavioral Consequences of Selection from Neural Population Codes
Steven P. Tipper, Louise A. Howard,
a n d George Houghton
ABSTRACT The perceptual elicitation of actions takes place even for visual inputs that are
not the intended target of subsequent overt behavior. According to our proposed model,
this automatic analysis can activate competing population codes that represent different
actions. Because these codes can overlap, inhibitory mechanisms are necessary to select one
population to guide overt behavior. Selection should result in changes to the population
vector that can produce deviations of the movement trajectory either toward or away from
the stimulus to be ignored. Such trajectory deviations are observed in both manual reaches
a n d saccades. The polarity and extent of the deviation are determined by the potency of the
distractor.
The process of perception-action coupling can be so fluent that inhibitory
mechanisms are sometimes required to overcome inappropriate but dom-
inant responses to achieve behavioral goals. An interesting example of
the costs of failure to inhibit a dominant response is provided by Stins
(1998) from the work of Boysen (1993). It is relatively simple to train a
chimpanzee to point to numerals to receive a reward. For example, it will
quickly learn to point to the numeral 4 rather than 2 when it receives four
candies in the former case a n d only two candies in the latter case. It is also
easy to train the chimpanzee to point to the numeral 2 rather than 4 when
another chimpanzee will receive the two candies while it receives the
four candies. When, however, the actual candies are presented rather
than numerals, performance changes dramatically. The chimpanzee can-
not point to two candies to receive four. Rather, it always reaches to the
location containing the four candies, even though these always go to the
other chimpanzee. It seems clear that the stimuli are evoking action auto-
matically, a n d the level of activation is determined by the reward value of
the stimulus. The inhibitory mechanisms necessary to overcome the dom-
inant response, and to maximize reward, are not available to the chim-
panzee a n d hence it continually loses out to its companion.
As this example indicates, a n d as Diamond’s analysis (1990) of the “A
not B’’ error in babies also makes clear, to achieve free choice a n d control,
it is essential that organisms develop the capacity to resist the strongest
response of the moment. The ability to selectively direct action to achieve
our goals is one of the most distinctive components of h u m a n behavior.
In sharp contrast to the chimpanzee, h u m a n s have evolved remarkably
efficient inhibitory control mechanisms. Only in the earliest stages of
development (e.g., Diamond 1990), or after brain damage, is action con-
sistently captured by irrelevant objects (e.g., Lhermitte 1983; Riddoch et
al. 1998; Riddoch, Humphreys, and Edwards, chap. 27, this volume).
Because visual information can automatically stimulate action, the crit-
ical mechanism for behavioral control is not necessarily some means for
evoking a desired action, but rather, mechanisms for preventing u n d e –
sired action (see Tipper, Howard, and Houghton 1998). The visuomotor
system might function like a car with an automatic transmission: when
the engine is running and the transmission is engaged in drive, the car is
always attempting to act; indeed, one needs to depress the brake pedal
(constant tonic inhibition) to prevent action from being automatically ini-
tiated. Hommel (chap. 11, this volume) reviews the evidence for the auto-
matic initiation of actions and the complex interplay between control
processes a n d intentions.
10.1 SELECTION-FOR-ACTION MECHANISMS
In this chapter, we focus on two aspects of the visual control of action:
mechanisms of inhibition and the use of multiple frames of reference. We
discuss each in turn.
There is evidence for parallel activation both of manual actions (e.g.,
Coles et al. 1985) and of saccades (e.g., Henderson a n d Ferreira 1990;
Reichle et al. 1998; Theeuwes et al. 1998). The model we propose assumes
that sometimes both objects relevant for action (targets) and objects irrel-
evant for action (distractors) are processed in parallel to the level of action
planning, and compete for the control of effectors. Parallel encoding of
actions requires selection mechanisms to be engaged. We suggest that the
representations of the response activated by a distractor are inhibited to
facilitate responding to the target (Houghton and Tipper 1994). Evidence
for inhibition of irrelevant information has been found in studies of a
number of cognitive processes, including working memory (Hasher a n d
Zacks 1988), episodic retrieval (Anderson a n d Bjork 1994), language pro-
duction (e.g., Dell and O’Seaghdha 1994), language comprehension (e.g.,
Gernsbacher a n d Faust 1995), serial order (Houghton and Tipper 1996),
a n d selective attention (e.g., Tipper 1985; Tipper, Brehaut, and Driver
1990). The most direct evidence for inhibition comes from single-cell
recordings in the monkey brain (e.g., Moran a n d Desimone 1985). For
example, Schall and Hanes (1993) found that w h e n monkeys were re-
quired to direct a saccade toward a target among distractors, a distractor
stimulus initially evoked a competing saccade, which w a s encoded
a n d subsequently inhibited. The role of inhibition in selective attention
has been described formally, a n d simulated in a computational model
(Houghton a n d Tipper 1994).
Tipper, Howard, and Houghton
Our model also assumes that visual inputs can be represented in dif-
ferent reference frames. There are multiple spatial frames of reference,
such that information can be encoded in retinotopic (e.g., Abrams a n d
Pratt forthcoming), environment-based (e.g., Hinton and Parsons 1988),
head-centered (e.g., Andersen and Zipser 1988), shoulder-centered (e.g.,
Soechting and Flanders 1989), or hand-centered (e.g., Graziano a n d Gross
1996; Tipper, Lortie, a n d Baylis 1992) frames.
In tasks demanding that the h a n d be moved to make direct contact
with a target, as in the experiments to be described here, the information
must be encoded in an oculomotor frame (to enable saccades) a n d a
hand-centered frame, in which the distance a n d direction of the reach is
represented by activation in motor networks (see also de Graaf, Sittig,
a n d Denier van der Gon 1994; Ghez, Hening, and Gordon 1991). This
hand-centered coding requires that proprioceptive information concern-
ing hand location is integrated with visual input. For example, a visual
receptive field may surround the hand, and move as the h a n d moves
(Graziano a n d Gross 1996).
In our o w n work, we have demonstrated that target and distractor are
encoded in parallel into such hand-centered frames of reference (e.g.,
Howard and Tipper 1997; Meegan and Tipper 1998; Tipper, Howard, a n d
Jackson 1997; Tipper, Lortie, a n d Baylis 1992). From the pattern of inter-
ference effects produced by the distractor, we can infer the frame of ref-
erence into which visual inputs are analyzed. The amount of interference
produced by a distractor could be explained only by assuming that the
stimulus w a s represented in terms of the reaching action it evoked, a n d
not by any other form of internal representation. As the h a n d started its
reach from different positions, the pattern of distractor interference
changed, even though visual information, and other body-centered
frames (e.g., head a n d shoulder) remained static. Thus distractors close to
the h a n d produced much more interference than those far from the hand.
The results are consistent with the view that multiple objects evoke action
in parallel, a n d that there is competition between these simultaneously
active responses that is resolved by inhibition mechanisms (see also
Meegan a n d Tipper 1998).
10.2 POPULATION CODING AND SELECTION
Most of the work studying selection for action has relied on temporal
measures such as total time from stimulus onset to response completion,
or reaction time (RT) to begin, a n d movement time to complete, the
action. Consideration of the physiology mediating such behaviors sug-
gests, however, that h a n d trajectory may yield further insights into these
visuomotor processes.
A number of studies suggest that action parameters can be encoded in
populations of neurons. Investigating the neural basis of primate reach-
Selection from Neural Population Codes
ing behavior in area 5 of the parietal cortex a n d in premotor cortex,
Georgopoulos (1990a,b), Kalaska (1988), and Kalaska, Caminiti, a n d
Georgopoulos (1983) observed distributed neural activity in which the
direction of a particular reach is represented by the activity of a popula-
tion of cells. Each individual neuron’s level of activity is broadly tuned
around a preferred direction, at which greatest activity is evoked.
Accordingly, a given cell will contribute, to varying degrees, to reaching
movements in different directions. The actual direction of the reach is
determined by the s u m of the single-cell contributions to the population
vector. Importantly for our current concerns, Georgopoulos (1990a) has
also argued that the specification of movement direction involves similar
codes in both arm and eye movement systems. Thus information con-
cerning oculomotor (e.g., Sparks, Holland, a n d Guthrie 1976) a n d manual
behavior is distributed within neural ensembles in which direction of
movement is uniquely specified only at the population level.
This form of coding has important implications for models of selection.
Because each neuron’s activity is broadly tuned, each cell will contribute
to a variety of reaches. Thus, when two objects are present that both
evoke reaches, the cell activities coding their directions can overlap, that
is, some cells will be activated by both reaches. Inhibitory selection of one
reach over the other may shift the population distribution in such a way
that it affects the final reach to a target. In the model we have been devel-
oping, we have found that the form of the inhibition acting to control dis-
tractor activation can have differing effects on the reach path to targets
(Houghton and Tipper forthcoming).
10.3 EXPERIMENTAL APPROACH AND MODEL PREDICTIONS
The experimental procedure used in these studies is developed from the
work of Sheliga and colleagues (Sheliga et al. 1995, 1997; Sheliga, Riggio,
a n d Rizzolatti 1994, 1995). Consider figure 10.1. While fixating the central
cross a n d depressing the start key at the bottom of the board with the
right hand, subjects were required to attend to one of the four light-
emitting diode (LED) cues placed at equal distances around fixation. If
the LED flashed green, subjects were required to reach to a n d depress the
target key at the top of the board as fast as possible. In contrast, if the LED
flashed red, no response was to be emitted. Subjects were precued before
each trial as to which LED w o u l d contain the color cue, and hence atten-
tion was endogenously oriented to the LED until color onset. At color
onset, attention was presumably withdrawn from the LED as action was
directed toward the target. At no time in the first experiment w a s action
ever directed toward the LED cue. Therefore we refer to the LED as the
“distractor.’’
Following the premotor theory of Rizzolatti et al. (e.g., 1987), we
assume that directing covert attention to a location will also activate
Tipper, Howard, and Houghton
Figure 10.1 Stimulus board used in the present study.
motor systems (see also Deubel a n d Schneider 1996; Morrison 1984; a n d
Hoffman 1998 for a recent review). Rizzolatti a n d colleagues have argued
that saccades are automatically activated when attention is directed to a
location in space, a n d that these activated saccades are suppressed u n d e r
task instructions not to look at the LED. We employ Sheliga a n d col-
leagues’ procedure to examine distractor effects in both oculomotor a n d
manual frames of reference, a n d to test predictions derived from com-
puter simulations of selection mechanisms acting on overlapping p o p u –
lation codes.
In simulation work, we have investigated two ways in which distractor
activation may be controlled, based on current neural network models of
selective attention (Houghton a n d Tipper forthcoming). The first mecha-
nism uses lateral inhibition between cells (units) coding direction. Units
are organized in topographic fashion so that units coding similar direc-
tions are side by side. Each unit has excitatory connections to nearby
units (those representing similar directions) a n d inhibitory connections
to more distant units. This on-center, off-surround (oCoS) organization
among directionally sensitive neurons has physiological support (Geor-
gopoulos 1995). If target objects a n d their associated direction achieve
enhanced activation d u e to attention (Houghton a n d Tipper 1994), then,
provided distractor activation is not too strong, the oCoS interactions
among the direction units can resolve the conflict. The neural activity
caused by the distractor is largely suppressed, with all activity clustering
around the target direction. A residual asymmetry in the distribution may
persist, however, resulting in a shift of the population vector slightly
toward the distractor.
Figure 10.2 illustrates this situation. Initially, subjects attend to a stim-
ulus (an LED), which simply provides a cue as to whether to respond to
a target; no action to this stimulus is required. Nevertheless, neural activ-
ity encoding the direction toward this stimulus is produced (figure
227 Selection from Neural Population Codes
Figure 10.2 Example of a weakly evoked movement, such as a reach to an LED cue. In
each panel the directional preference of each cell is represented by a line whose height rep-
resents the activity of the cell. The dotted line shows baseline activity. The small rectangle
is the target key, a n d the small circle is an LED cue on the right. The arrows in panels C a n d
D show the direction of movement resulting from the summation of the population. A.
Low-level activity produced by the LED cue light. B. Reduced activity to the LED cue fol-
lowing lateral inhibition. C. High-level activity for the reach to the target. D. Summation of
the activity in panels B a n d C. The resultant movement to the target deviates to the right
(toward the cue; compare with panel C).
10.2A). Shortly afterward, a reach to a target is produced, represented in
figure 10.2C. At this time, the activity associated with the LED distractor,
though small, is still present (figure 10.2B). The resulting population code
is shown in figure 10.2D, where panels B a n d C are s u m m e d . Because
there are cells in common to both the reach evoked by the LED a n d the
target, the population is shifted slightly toward the LED.
In other circumstances in which a stimulus to be ignored evokes a very
powerful response, such lateral inhibition mechanisms are not sufficient
to resolve response conflict. That is, action can be captured by the wrong
stimulus. To resolve this level of competition, a further reactive inhibition
mechanism is required, one that specifically acts on the activation caused
by the distractor (this mechanism is described in detail in Houghton a n d
Tipper 1994). In our model, inhibition feeds back onto the distractor, a n d
the level of inhibition is related to the activation state of the distractor:
m o r e p o t e n t distractors p r o d u c e greater levels of self-inhibition.
Importantly, this form of inhibition has effects on population distribu-
tions distinct from that caused by the oCoS mechanism. In particular, it
can lead to trajectories that veer away from distractors (see Houghton
a n d Tipper forthcoming).
This situation, a n d its effect on trajectories, is shown in figure 10.3.
Again, action (a saccade) is evoked by the LED stimulus. In contrast to
the previous example, much greater activity is represented by higher
neural activity in figure 10.3A than in figure 10.2A. To subsequently
select against this stimulus a n d respond to the target (figure 10.3C), self-
inhibition feeds back onto the population of cells encoding action toward
the LED, a n d the effect of this reactive inhibition is shown in figure 10.3B.
The summation of neural activity (combining figures 10.3B a n d 10.3C) is
shown in figure 10.3D, in which it can be seen that trajectories veer away
from the LED to be ignored.
228 Tipper, Howard, a n d Houghton
Figure 10.3 Example of a strongly evoked movement, such as a saccade to an LED cue.
Conventions are as for figure 10.2. A. Cells respond strongly to the LED cue. B. Reactive
inhibition reduces cell activity below baseline. C. Activity to the target. D. Summation of
activity in panels B a n d C results in the saccade deviating away from the cue (compare with
panel C).
To investigate the relative strength of the actions evoked by the LED,
we examined both eye a n d h a n d trajectories. Recall that subjects initially
attend to the LED to discriminate its color, while maintaining fixation at
the center of the display. Subjectively, the urge to fixate the LED is
extremely powerful. Clearly, orienting the cone-rich fovea w o u l d greatly
facilitate the color discrimination task. We argue that reactive inhibition
is necessary to prevent such saccades to the LED, a n d that saccades
should therefore deviate away from the attended LED (figure 10.3).
In sharp contrast, there is no conscious urge to reach to the LED, sug-
gesting only weak activation of reaches toward the LED. We argue that
reaching actions are nevertheless automatically activated while attending
to the LED (cf. Hommel, chap. 11, this volume; Simon 1969). As noted
above, we have shown computationally that this conflicting activation
can be largely resolved by oCoS interactions, resulting in a fairly straight
movement path toward the target, though with a residual tendency in the
direction of the distractor. Hence we predict that h a n d movements will
exhibit small deviations toward the LED, as shown in figure 10.2.
Our other concern is frames of reference. Recall that other studies of
selective reaching have provided evidence for hand-centered frames
(e.g., Meegan a n d Tipper 1998; Tipper, Lortie, a n d Baylis 1992; Tipper,
Howard, a n d Jackson 1997), because distractors close to the h a n d pro-
duce greater levels of interference than those far from the h a n d . In the
current model, we therefore predict that LEDs close to the reaching h a n d
will evoke more powerful reaches, a n d hence larger deviation effects,
than LEDs far from the h a n d .
In contrast, our model predicts the opposite result for saccades. That is,
saccades will deviate away from LEDs far from the h a n d more than they
will from LEDs near the h a n d . This emerges from the amount of neural
overlap between saccades evoked by the LED a n d subsequent target. For
example, in figure 10.1, the right-side LED, which is far from the hand,
has a saccade direction closer to that of the target saccade (i.e., they are
both in the u p p e r hemifield) than does the right-side LED, which is
Selection from Neural Population Codes
closer to the hand. Hence the populations of cells encoding the two sac-
cades activated will overlap substantially more in the former case, a n d
suppression of one group will have more of an effect on the other. Indeed,
precisely this result, in which saccade deviation is greater when attend-
ing to an LED in the same hemifield as the target saccade (LED far from
hand) than when attending an LED in the hemifield opposite the saccade
direction (LED near hand) has been observed by Sheliga, Riggio, a n d
Rizzolatti (1994).
In summary, we are attempting to demonstrate (1) that similar selec-
tion processes take place in both the saccade generating systems a n d in
the manual reaching systems; (2) that two mechanisms (on-center, off-
surround; a n d reactive feedback) enable selection between competing
populations of cells; and (3) whether the second mechanism is engaged
determines the direction of changed trajectory, a n d d e p e n d s on task
d e m a n d s . Actions with a very low level of activity (e.g., manual reaches
to the present color cues) will have little or no reactive inhibitory feed-
back, resulting in deviations toward the distractor. In contrast, actions
that are powerfully evoked (e.g., saccades to color cues) will require sub-
stantial reactive inhibitory feedback to suppress them, resulting in devia-
tions away from the distractor.
10.4 EXPERIMENT 1
Experiment 1A
Subjects In return for course credit, 11 right-handed subjects (all
females) were recruited from our student subject pool, ranging in age
from 18 to 45 (mean age: 23.3). One subject h a d poor stereopsis, but her
performance on the task did not differ from that of the other subjects.
Visual acuity was normal or corrected to normal in all subjects. All re-
ported normal hearing.
Apparatus The experiment was programmed in LabView (version
4.0.1), running on an Apple Macintosh PowerPC 8100/100. A National
Instruments NB-DIO-24 I / O card w a s used to send and receive digital
signals. H a n d movements were recorded using a MacReflex system with
2 infrared cameras recording at 50 Hz, plus video processors running on
an Apple Macintosh Quadra 630. Subjects wore a reflective marker,
approximately 9 mm diameter, on an elastic band on their wrist over the
ulnar notch of the radius. The stimulus board, with start a n d target keys
a n d 4 two-colored LEDs, arranged as in figure 10.1, was oriented with the
start key closest to the subject.
Design A within-subject design w a s used. Cues were presented on the
left or right by lighting up one of the LEDs. The near versus far cue loca-
Tipper, Howard, and Houghton
tions were run in separate testing sessions separated by a short interval,
with order counterbalanced between subjects. The experiment consisted
of 8 “go’’ (green LED) and 2 “no-go’’ (red LED) trials per block, with an
equal number of each color on the left a n d right. Each block of 10 trials
was presented in a new r a n d o m order, and there were 8 blocks per
session.
Procedure The experiment’s t w o sessions together lasted approxi-
mately 50 minutes, and took place in a dimly lit room. Subjects’ vision
was tested first, then the experimenter demonstrated the task, and then
subjects were given two practice blocks of trials before beginning the
experimental trials. When after eight blocks, the cue location changed
from front to back or vice versa, subjects were given a further practice
block. The start key lit up yellow w h e n a trial was ready to begin. Subjects
were told to fixate the blue dot in the center of the display a n d to press
the start key a n d hold it d o w n to begin a trial, at which point the light
went out. If they released the key at this point, an error beep would
sound a n d the start key w o u l d illuminate again. After a variable interval
(range 510–1485 msec) a tone would sound for 250 msec. If it was a high-
pitched tone (800 Hz), then subjects were to orient their attention to the
LED on the left; if it was a low-pitched tone (300 Hz), they were to orient
their attention to the right (the validity of the tone cue was 100%, a n d
only one LED flashed on each trial). Fixation remained at the blue dot,
a n d this w a s monitored by the experimenter (see Tipper, Brehaut, a n d
Driver 1990 for similar procedure and reliability). There was then a 1,500
msec interval before the LED cue was presented for 100 msec, at which
point the cameras were triggered to start recording. If the cue was green,
subjects were to release the start key a n d press the target key as fast as
they could. The start key would light up again to signal the start of the
next trial 1,500 msec after depression of the target. If, however, the LED
h a d flashed red, they were to keep holding the start key d o w n (otherwise
an error beep would sound) until the start key flashed to signal the start
of the next trial.
Data Collection and Analyses Reaction time (RT) for the manual reach
was the interval between onset of the cue a n d the time at which the wrist
velocity achieved 25 m m / s e c . Because no mean RT contrasts between
attending to left/right or near/far LEDs were significant, these will not
be discussed further (means were 312, 301, 309, a n d 304 msec for near-
right, near-left, far-right and far-left cues, respectively).
Wrist trajectory was constructed by standardizing each reach spatially
(see Tipper, Howard, a n d Jackson 1997). Location of reach onset was
defined as for the RT, and the end of the reach was the greatest extent in
the Y-dimension (sagittal plane) achieved by the wrist marker on each
trial. In previous research using more than one target location, we have
Selection from Neural Population Codes
Table 10.1 Mean and Standard Error (SE) Trajectory Deviation (in Millimeters in the X
Dimension) at 25, 50, and 75% Stages through the Path (Y Dimension) of H a n d a n d Eye
Movements to the Target Following Cues at the Near-right, Near-left, Far-right, and Far-left
Locations in Experiments 1 and 2
Experiment n
1A (hand) 11
1B (eye) 7
2 (hand) 21
2 (eye) 6
Stage
(%)
25
50
75
25
50
75
25
50
75
25
50
75
Near right
M
17.92
11.39
1.26
0.00
–1.75
1.16
9.72
–4.20
–20.40
–8.34
–12.90
–12.20
SE
3.15
3.37
3.43
2.13
6.01
10.28
2.50
3.49
4.04
3.30
5.43
6.98
Near left
M
14.54
6.72
–3.24
6.40
9.89
13.77
8.40
–6.89
–23.80
11.06
17.84
18.81
SE
3.10
3.34
3.49
3.69
9.12
14.16
2.77
3.82
4.28
5.04
9.12
11.06
Far righ
M
15.97
9.02
–0.50
–1.16
–1.75
3.10
13.21
–0.70
–18.00
–7.18
–12.20
–11.60
t
SE
2.54
2.39
2.26
1.75
6.21
10.47
2.47
3.19
3.70
4.46
5.82
6.79
Far left
M
15.11
7.87
–1.55
4.07
6.40
10.47
9.62
–6.07
–23.44
8.73
6.40
10.47
SE
2.70
2.70
2.55
2.13
4.65
7.95
2.78
3.90
4.35
3.30
4.65
7.95
excluded outlying reaches by the extent to which they deviated from the
norm. In the present series of experiments, which had only one target
location, but which also had no-go trials, trials were excluded if they did
not show a smooth trajectory, defined as any decrease in wrist velocity
after onset but before peak velocity was achieved. On this basis, 19% of
trials were excluded (but inclusion of these trials does not change the pat-
tern of data). A further 7% of trials were excluded because of recording or
subject errors.
The dependent variable was the amount of deviation of the reach path
to left or right of the origin a quarter, half, and three-quarters of the way
through the reach. Data were analyzed using analysis of variance
(ANOVA), with cue distance (near and far), cue side (left and right), and
stage through the reach (25%, 50%, and 75%, respectively) included as
repeating factors.1
Results and Discussion
Hand Trajectory The main effect of the cue side just missed signifi-
cance, with reaches deviating toward the cue: F(1, 10) = 4.03, p = 0.07.
However, the interaction between cue distance and side was highly
significant: F(1, 10) = 10.64, p < 0.01. Post hoc ANOVAs conducted on the
data obtained with near and far cues separately indicated that the side
effect was significant for the near cues: F(1, 10) = 7.60, p < 0.05; but not the
far cues: F(1, 10) = 0.68, n.s. These effects are illustrated in figure 10.4A;
mean scores are shown in table 10.1.
232 Tipper, Howard, and Houghton
Figure 10.4 Mean trajectories from experiment 1 (go/no-go task). H a n d trajectories
(experiment 1A, panel A) and eye trajectories (experiment 1B, panel B). The approximate
location of the near cues (left panels) a n d far cues (right panels) are shown.
Experiment 1A therefore provides some evidence that when subjects
covertly attend to a location to analyze the color of a briefly presented
cue, manual action to that location is evoked. Furthermore, in line with
the theory of hand-centered frames, larger effects were produced by the
LEDs closer to the hand’s starting location. The effect observed w a s a
small deviation of h a n d trajectory toward the attended LED cue. This
result is consistent with the notion that reaching response activation is
very weak in this procedure, so that little or no reactive inhibition feeds
Selection from Neural Population Codes
back onto this representation. Neural activity representing a reach to the
distractor therefore remains slightly above baseline when the population
encoding the reach to the target is activated. As a result, the latter popu-
lation is slightly skewed such that trajectories veer toward the attended
LED.
Experiment 1B
Task design and procedure for experiment 1B was as described in exper-
iment 1A in that subjects reached for the target when the LED cue went
green, except that we now recorded eye movements. Seven new subjects
with normal or corrected-to-normal vision and a mean age of 23.6 (age
range: 19 to 31) were recruited from our subject panel. Horizontal and
vertical eye movements were recorded as electrooculograms (EOGs) on
two separate amplifiers operating at 200 Hz, and a Biopac Systems
MP100 processor.
Data Collection and Analyses Trials were excluded from both fixation
and saccade analyses if fixation was not maintained during the fixation
period according to the following criteria. Maximum, minimum, and
mean voltages in the horizontal dimension for the 1 sec period before the
LED cue was illuminated were collated for each of the four LED locations.
Any trial in which the maximum or minimum value fell outside 2 stan-
dard deviations (SDs) of the mean of the maximum or minimum values
w a s excluded. Difference scores were then calculated (maximum-
minimum) of the remaining trials, and again, those in which the differ-
ence exceeded 2 SDs of the mean were excluded. To examine presaccadic
drift, analyses were conducted on the mean of the means of the remain-
ing trials. Trials in which fixation was maintained were also used to ana-
lyze subsequent eye movements.
Saccade RTs were defined as follows. The onset of saccadic eye move-
ments was defined as the time at which either the vertical (Y) voltage
stopped decreasing, or the velocity of the Y became less than the maxi-
m u m velocity of the saccade divided by 150 (both measured backward in
time from the end of the saccade). When saccade RTs were analyzed, the
main effect of cue distance was significant: F(1, 6) = 5.65, p = 0.05 (means
were 347 and 334 msec for the near-right and near-left cues, 370 and 367
msec for the far-right and far-left cues). The end of the saccade was the
maximum voltage in the Y-dimension. X- and Y-coordinates were stan-
dardized in the same manner as the hand movements, and means were
obtained for the different cue location conditions (25%, 50%, and 75%
through the eye movement). Repeated measures ANOVAs used the same
factors as those in experiment 1A. Voltages were converted into approxi-
mate mm values for figure 10.4B, and table 10.1.
Tipper, Howard, and Houghton
Table 10.2 Mean and Standard Error (SE) Values (in Volts in the X Dimension) while
Attending to Each of Four Locations (Near-right, Near-left, Far-right, and Far-left) during
the Fixation Period and Before Onset of the Cue in Experiments 1 a n d 2
Near right Near left Far right Far left
Experiment n M SE M SE M SE M SE
1B 7 0.15 0.02 0.12 0.06 0.25 0.03 0.24 0.02
2 6 0.27 0.05 0.24 0.04 0.28 0.05 0.23 0.03
Results and Discussion
There was a significant main effect of cue side, F(1, 6) = 28.85, p < 0.005,
indicating saccadic deviations away from the cued side; and a significant
interaction between stage and side, F(2,12) = 21.50, p < 0.0001, indicating
that the difference between right and left cues increased as the saccade
progressed (figure 10.4). No other effects reached significance.
We also analyzed mean fixation in the 1 sec period before LED cue
onset to ensure that presaccadic drift could not account for the effects
obtained, with distance and side as repeated factors (see table 10.2).
The effect of cue distance was significant: F(1,6) = 18.72, p < 0.005 (mean
far = 0.245; mean near = 0.131 volts), indicating that in testing sessions
with far cues, fixation was slightly to the right of that in sessions with
near cues. Because, however, there was no reliable effect of cue side, pre-
saccadic drift is very unlikely to account for the significant deviations in
saccades that were found.
Experiment 1B clearly replicates Sheliga and colleagues’ finding
(Sheliga, Riggio, and Rizzolatti 1994, 1995; Sheliga et al. 1995, 1997) of a
deviation of the saccade away from the location covertly attended.
Furthermore, as Sheliga and colleagues have confirmed in numerous
studies, this saccade deviation cannot be explained by eye drift to the
attended side of space.
In line with our predictions, the hand (experiment 1A) and eye move-
ments (experiment 1B) show quite different trajectory deviations (com-
pare figures 10.4A and 10.4B). The eye clearly deviates away from the
attended LED cue, whereas the hand deviates slightly toward the cue.
Essentially similar principles are involved in the mechanisms governing
representations of eye and hand movements. We propose that the only
difference between these action systems is that the powerful saccades
evoked when attending to the LED require reactive inhibition; this
mechanism is not required when reaching, selection being achieved by
lateral inhibition alone. On the other hand, although our model predicted
larger deviations when neural populations overlap substantially than
when they are more separate, and although Sheliga, Riggio and Rizzolatti
(1994) have in fact observed this result, our data show a trend for the
Selection from Neural Population Codes
opposite result. Saccade deviations were slightly larger w h e n attending
to LEDs in the hemifield opposite to that of the subsequent saccade (near
to hand) in which population overlap is relatively small, than when
attending to LEDs in the same hemifield. This will be discussed further at
the end of section 10.5.
10.5 EXPERIMENT 2
In experiment 1, the LED cue was never the target for overt behavior.
Thus, even though subjects h a d to orient covert endogenous attention to
the LED to analyze the color of the brief stimulus, they knew in advance
that eye and h a n d movement to the keypress target was the only re-
sponse required; we assumed that the oculomotor and manual responses
to the target were prepared in advance. It is therefore surprising that
reaching a n d saccade actions to the LED cue were nevertheless still
covertly evoked merely by attending to the cue. Such data provide s u p -
port for Rizzolatti’s premotor theory of attention.
In experiment 2, we examined the effect on performance of actually
making the LED relevant to behavior. That is, rather than the cue being a
go (green) or no-go (red) signal, such that action w a s only ever directed
to the keypress target, here action toward the LED cue w a s required on
some trials. The green color cue n o w signaled a rapid response to the
target key (66% of trials), whereas the red color cue signaled a visually
guided reach toward the illuminated LED (33% of trials).
In this new procedure, such explicit coding of action should produce
high levels of activity in neural populations, and hence the effects should
be more pronounced. It is easy to predict the effect of this manipulation
on saccades. The increased activity of the saccade to the potential LED
target will produce greater reactive inhibition to prevent a saccade to the
LED than observed in experiment 1. Therefore our model predicts that
saccade deviations away from the LED cue will increase.
Unfortunately, it is not possible to predict the effect of this new proce-
dure on reaching trajectory. We have argued that the weakly activated
reaching response does not trigger reactive inhibition. Rather, selection
can be resolved via lateral inhibition between cells in the activated popu-
lations. We simply did not know whether the increased salience of the
reaching response to the LED would be sufficient to trigger reactive inhi-
bition, and thus reduce deviations toward, or even cause deviations away
from, the LED.
Experiment 2
Subjects Twenty-one new members of our subject panel took part (14
females; 7 males), all with normal vision. The mean age was 22.1 (age
range: 17 to 32). The design and procedure were as for experiment 1A,
Tipper, Howard, and Houghton
Figure 10.5 Mean trajectories from experiment 2 (LED cues are potential targets). H a n d
trajectories (panel A) a n d eye trajectories (panel B). The approximate location of the near
cues (left panels) and far cues (right panels) are shown.
except that subjects were instructed to reach out a n d touch the surface of
the LED if it flashed red, as happened on one in three trials (randomized
in blocks of 12 trials). When the LED flashed green, they were to reach to
the central target, as before. The apparatus was as for experiment 1B.
Data Collection and Analyses Only data for those trials on which sub-
jects reached for the target are reported. H a n d movement data were
available for all 21 subjects, a n d eye movement data were available for 6
of these. Of the h a n d movement data, 3% of trials were excluded d u e to
Selection from Neural Population Codes
recording or subject errors, and a further 20% were excluded because the
trajectories did not show a consistently increasing velocity, as described
in experiment 1A. (The same effects were produced even when these
trials were included.) Of the eye movement data, 10% of trials were
excluded because fixation had not been maintained or because the sac-
cade could not be determined.
Results and Discussion
Hand Trajectory Mean and standard error trajectory scores are shown
in table 10.1. Averaged trajectories are shown in figure 10.5. There was a
main effect of cue distance on hand trajectory, indicating that reaches
were more to the right when the cues were in the far locations: F(1, 20) =
8.75, p < 0.01. There was also a significant stage effect, with reaches mov-
ing more to the left as they progressed: F(2,40) =246.96, p< 0.0001. Of
most importance, there was also a significant side effect, with the hand
deviating toward the side on which the cue appeared: F(1, 20) = 4.44,
p < 0.05. The interaction of side and stage was significant: F(2,40) = 3.65,
p < 0.05. Post hoc ANOVAs showed that the side effect was greater in
the middle and end of the reach than in the beginning: p < 0.005 versus
p = 0.01 (figure 10.5A).
As in experiment 1A, we found that hand trajectory veers toward the
LED cue. The main difference is that we no longer observed the asym-
metry observed in experiment 1A. In experiment 1A, the reaching hand
veered toward the cue only when the cue was relatively near the hand, in
conformity with our view that visual stimuli are encoded in hand-
centered frames for reaching. In experiment 2, this contrast was no longer
observed. The amount of veering toward the attended LED was, if any-
thing, greater in the far than in the near conditions. This difference is
confirmed by a significant three-way interaction between side of cue (left
or right), distance from hand (near or far) and experiment (1A or 2):
F(1, 30) = 5.33, p < 0.05.
Although, as discussed, a priori predictions of hand trajectory were not
possible, we can provide the following post hoc suggestions. In experi-
ment 1A, the hand deviated toward the near LEDs. In experiment 2, the
reach evoked by the LED was assumed to be more potent because it was
now a potential target on some trials. If no reactive inhibition was trig-
gered, then deviations toward the LED should have been greater. Clearly,
this was not the case. We therefore suggest that reactive inhibition was
evoked when attending to near LEDs. Indeed, there is a small trend for
the deviations to be smaller in experiment 2 than in experiment 1A. In
contrast, LEDs far from the hand had no effect in experiment 1A.
Increasing the salience of the LEDs in experiment 2 may thus have
increased their activation state, but not passed the threshold for trigger-
Tipper, Howard, and Houghton
ing reactive inhibition. This implies that hand trajectory deviations
should be greater in experiment 2 than in experiment 1A, and, indeed,
that is what we observed. Clearly, these are only speculations, and more
research is necessary to confirm the properties of our model.
Eye Trajectory Mean fixation scores were analyzed as in experiment 1B
(table 10.2). There was a nonsignificant effect of attended side on fixation
drift: F(1, 5) = 3.48 (left = 0.238; right = 0.273 volts), n.s. When saccades
were analyzed, there was a significant main effect of cue side on eye tra-
jectory, with the eyes deviating away from the side of the cues: F(1, 5) =
10.30, p < 0.05. This effect was greater at the middle and end of the eye
movement than at the start, as evidenced by the interaction of stage with
side: F(1, 5) = 6.06, p < 0.05 (figure 10.5B).
We predicted that when overt behavior toward the cue was required on
some trials, subjects would explicitly prepare a saccade to that location.
This explicit level of internal representation is assumed to correspond to
greater activation levels than when action is never overtly directed
toward the stimulus. Furthermore, the subsequent inhibition of the sac-
cade toward the LED, when the saccade has to be directed to the keypress
target, was predicted to be larger. These predictions were supported.
Comparing figure 10.5B with figure 10.4B, it can clearly be seen that the
saccade deviations away from LED cues were approximately twice as
large in experiment 2, where saccades were sometimes directed to the
LED, as in experiment 1B, where saccades were never directed to the
LED. The two-way interaction between experiment (1B or 2) and side
of LED (left or right) was significant: F(1, 11) = 4.72, p = 0.05.
As can be seen in figure 10.5, there is a trend for saccade deviations to
be larger when attending to LEDs near the hand, which are in the oppo-
site hemifield to the saccades directed to the target key. Recall that this
trend was also observed in experiment 1B. As discussed previously, this
is opposite to the predictions of our model, and to findings by Sheliga,
Riggio, and Rizzolatti (1994).
We suggest that eye and hand movement systems have to make quite
different computations based on different frames of reference. Separate
neural systems will therefore be necessary, and this is supported by neu-
rophysiological evidence: lateral intraparietal (LIP) to frontal eye fields
(FEF) and ventral intraparietal (VIP) to dorsolateral prefrontal cortex (F4)
control eye and hand, respectively. However, eye and hand have to also
be closely coordinated in real-world interactions, such as when rapidly
and accurately grasping an object (e.g., Abrams, Meyer, and Kornblum
1990; Jeannerod 1988). Various brain structures encode both eye and
hand, and hence there are multiple neural sites for such interactions, such
as the cerebellum (e.g., Brown et al. 1993), superior colliculus (e.g.,
Bekkering, Pratt, and Abrams 1996; Fries 1984, 1985; Werner 1993), and
Selection from Neural Population Codes
supplementary eye fields (SEFs; e.g., Mushiake, Fuji, a n d Tanji 1996). For
example, a population of cells in an SEF responds only when saccade a n d
reach are directed to the same object.
If the hand- a n d eye-centered frames are closely integrated, then there
may be crosstalk between them. The main goal of the subjects in the pres-
ent experiments was to reach to a n d depress the target key as fast as pos-
sible. Hand-centered frames are critical for achieving this behavioral
goal. As we saw in experiment 1A, effects on h a n d trajectory were greater
when attending to the nearer LEDs. It is therefore a reasonable assump-
tion that movements to the nearer LEDs are more actively represented. If
eye a n d h a n d movements are closely related in some neural systems,
then there could be spreading activation from the highly active reach rep-
resentations of locations near the hand, to the representations of the same
locations in the eye movement system. The result of this would be to
inflate the effects of the LEDs near the h a n d when making a saccade to a
target in the u p p e r hemifield.
Although the idea of crosstalk between different action systems is spec-
ulative, recent pilot data indicate that it is worth pursuing further. Thus
when subjects undertake a task very similar to that of experiments 1A
a n d 1B, but only make saccades to the target (never reaching), the asym-
metries predicted by our model are obtained. That is, large saccade devi-
ations are produced when attending to LEDs far from the hand, in the
same (upper) hemifield as the saccade; whereas very small saccade devi-
ations are observed when attending to LEDs near the h a n d in the oppo-
site (lower) hemifield to the target saccade. Thus when reaching is
removed from the experimental task, the pattern of saccade deviations
changes completely.
10.6 GENERAL DISCUSSION
Parietal-frontal circuits do not encode space in some general form, but
in ways relevant to the control of action. Neurophysiological studies
demonstrate separate circuits for h a n d a n d eye movements, rather than a
unified m a p of space. A single visual object can be represented in multi-
ple ways in different brain areas depending on the actions that may be
evoked, such as saccading toward (Goldberg a n d Colby 1989) or reaching
to a n d grasping (Fogassi et al. 1992; Graziano a n d Gross 1993, 1996;
Rizzolatti et al. 1981). Rizzolatti has argued that orienting attention will
activate these motor circuits. Because of the fundamental role of eye
movements in visual perception, orienting the fovea to the object of inter-
est is automatically evoked. Hence orienting attention triggers saccades
to the attended location.
Although it is less obvious that reaching to an attended location should
also be evoked when only eye movements are required, when the goal is
to reach rapidly a n d accurately, h a n d movement circuits become acti-
Tipper, Howard, and Houghton
vated a n d merely attending to a location can begin to evoke a reach. Thus
covertly attending to a position can activate separate visuomotor net-
works in parallel.
Action parameters can be represented by population codes, a n d
models for the resolution of competition within such populations lead to
predictions regarding the dynamics of effector movement. Clearly, this is
our preferred interpretation of the trajectory deviation effects. However,
an alternative account is offered by Duhamel, Colby, a n d Goldberg
(1992), w h o have shown that the intention to make a saccade can change
the location to which a neuron responds. Typically, the receptive field
moves to the location it would occupy following the saccade. Sheliga a n d
colleagues (Sheliga, Riggio, a n d Rizzolatti 1994, 1995; Sheliga et al. 1995,
1997) have used this finding to explain eye trajectory deviations away
from cues by assuming that changes in the receptive field of the eye will
also affect eye position information. That is, even though the eye stays at
the fixation point, eye position information shifts to the attended side.
When a vertical saccade has to be made, the eye trajectory will veer away
from the side of attention.
On the other hand, Colby (1996) compared intention to act with atten-
tion in monkeys a n d reported that the remapping of saccades takes place
only when the monkey intends to saccade. When the animal covertly
attends to a location without making a saccade to it, no remapping is
observed. The latter procedure is the same as that used in experiment 1,
in which subjects never saccaded to the covertly attended LED. As Colby
points out, remapping would be counterproductive if no saccade were
m a d e because it would introduce a mismatch between the external world
a n d the internal parietal image of it. (Indeed, it is possible that the inhi-
bition preventing the execution of the saccade also inhibits the remap-
ping process.) If remapping does not occur, then it cannot explain the
saccade deviations reported in this chapter a n d in the studies by Sheliga
a n d colleagues (Sheliga, Riggio, a n d Rizzolatti 1994, 1995; Sheliga et al.
1995, 1997).
In conclusion, we have confirmed that two visuomotor circuits can be
activated by covert attention and that these two systems (oculomotor a n d
manual) function in essentially the same way. The basic properties of
these systems are (1) action is represented in distributed neural popula-
tions; (2) both relevant a n d irrelevant action can be represented in the
same networks; (3) inhibitory control mechanisms (lateral a n d reactive
inhibition) are necessary for selective behavior; a n d (4) the potency of the
stimulus determines whether reactive inhibition is triggered a n d hence
the nature of the movement trajectory deviation.
NOTES
This work was supported by grant S07727 from Biotechnology and Biological Sciences
Research Council. We would like to thank Darren Whitehouse a n d Daniel Fraser for their
241 Selection from Neural Population Codes
assistance with data collection and Matthew Paul and Beccy Pleasant for their assistance
with data processing.
1. Greenhouse-Geisser a n d Huhn-Feldt corrections for sphericity appropriate to noninde-
pendent repeating measures were used.
REFERENCES
Abrams, R. A., Meyer, D. E., and Kornblum, S. (1990). Eye-hand co-ordination: Oculomotor
control in rapid aimed limb movements. Journal of Experimental Psychology: Human Percep-
tion and Performance, 16, 248–267.
Abrams, R. A., a n d Pratt, J. (Forthcoming). Retinal coding of inhibited eye movements to
recently attended locations.
Andersen, J. R., a n d Zipser, D. (1988). The role of the posterior parietal cortex in co-ordinate
transformations for visual-motor integration. Canadian Journal of Physiological Pharmacology,
66, 488–501.
Anderson, M. D., a n d Bjork, R. A. (1994). Mechanisms of inhibition in long-term memory:
A n e w taxonomy. In D. Dagenbach a n d T. H. Carr (Eds), Inhibitory processes in attention, mem-
ory, and language, p p . 265–326. N e w York: Academic Press.
Bekkering, H., Pratt, J., a n d Abrams, R. A. (1996). The gap effect for eye a n d h a n d move-
ments. Perception and Psychophysics, 58, 628–635.
Boysen, S. T. (1993). Counting in chimpanzees: N o n h u m a n principles and emergent prop-
erties of number. In S. T. Boysen and E. J. Capaldi (Eds), The development of numerical compe-
tence: Animal and human models, p p . 39–59. Hillsdale, NJ: Erlbaum.
Brown, S. H., Kessler, K. R., Hefter, H., Cooke, J. D., a n d Freund, H. J. (1993). Role of the
cerebellum in visuomotor coordination: 1. Delayed eye and arm initiation in patients with
mild cerebellar ataxia. Experimental Brain Research, 94, 478–488.
Colby, C. L. (1996). A neurophysiological distinction between attention and intention. In T.
Inui and J. L. McClelland (Eds), Attention and Performance XVI, p p . 157–178. Cambridge,
MA: MIT Press.
Coles, M. G., Gratton, G., Bashore, T. R., Eriksen, C. W., and Donchin, E. (1985). A psy-
chophysical investigation of the continuous flow model of h u m a n information processing.
Journal of Experimental Psychology: Human Perception and Performance, 11, 529–553.
de Graaf, J. B., Sittig, A. C., and Denier van der Gon, J. J. (1994). Misdirections in slow, goal-
directed arm movements are not primarily visually based. Experimental Brain Research, 99,
464–472.
Dell, G. S., and O’Seaghdha, P. G. (1994). Inhibition in interactive activation models of lin-
guistic selection and sequencing. In D. Dagenbach and T. H. Carr (Eds.), Inhibitory processes
in attention, memory, and language, p p . 409–454. New York: Academic Press.
Deubel, H., a n d Schneider, W. X. (1996). Saccade target selection a n d object recognition:
Evidence for a common attentional mechanism. Vision Research, 6, 1827–1837.
Diamond, A. (1990). Developmental time course in h u m a n infants a n d infant monkeys, a n d
the neural bases of inhibitory control in reaching. Annals of the New York Academy of Sciences,
608, 637–676.
Duhamel, J.-R., Colby, C. L., a n d Goldberg, M. E. (1992). The updating of the representation
of visual space in parietal cortex by intended eye movements. Science, 255, 90–92.
Tipper, Howard, and Houghton
Fogassi, L., Gallese, V., Di Pellegrino, G., Fadiga, L., Gentilucci, M., Luppino, G., Matelli, M.,
Pedotti, A., and Rizzolatti, G. (1992). Space coding by premotor cortex. Experimental Brain
Research, 89, 686–690.
Fries, W. (1984). Cortical projections to the superior colliculus in the macaque monkey: A
retrograde study using peroxidase. Journal of Computative Neurology, 230, 55–76.
Fries, W. (1985). Inputs from the motor and premotor cortex to the superior colliculus in the
rhesus monkey. Behavioural Brain Research, 18, 95–105.
Georgopoulos, A. P. (1990a). Neural coding of the direction of reaching and a comparison
with saccadic eye movements. Cold Spring Harbor Symposia on Quantitative Biology, 55,
849–859.
Georgopoulos, A. P. (1990b). Neurophysiology of reaching. In M. Jeannerod (Ed.), Attention
and Performance XIII, p p . 227–263. Hillsdale, NJ: Erlbaum.
Georgopoulos, A. P. (1995). Current issues in directional motor control. Trends in Neuro-
sciences, 18, 506–510.
Gernsbacher, M. A., and Faust, M. E. (1995). Skilled suppression. In F. N. Dempster a n d
C. J. Brainerd (Eds.), Interference and inhibition in cognition, p p . 296–327. San Diego, CA:
Academic Press.
Ghez, C., Hening, W., and Gordon, J. (1991). Organisation of voluntary movement. Current
Opinion in Neurobiology, 1, 664–671.
Goldberg, M. E., and Colby, C. L. (1989). The neurophysiology of spatial vision. In F. Boller
a n d J. Grafman (Eds.), Handbook of neurophysiology, vol. 2, p p . 301–315. Amsterdam:
Elsevier.
Graziano, M. S. A., and Gross, C. G. (1993). A bimodal m a p of space: Somatosensory recep-
tive fields in the macaque putamen with corresponding visual receptive fields. Experimental
Brain Research, 97, 96–109.
Graziano, M. S. A., and Gross, C. G. (1996). Multiple pathways for processing visual space.
In T. Inui a n d J. L. McClelland (Eds.), Attention and Performance XVI, p p . 181–208.
Cambridge, MA: MIT Press.
Hasher, L., a n d Zacks, R. T. (1988). Working memory, comprehension a n d aging: A review
a n d new view. Psychology of Learning and Motivation, 22, 193–225.
Henderson, J. M., and Ferreira, F. (1990). Effects of foveal processing difficulty on the per-
ceptual span in reading: Implications for attention and eye movement control. Journal of
Experimental Psychology: Human Perception and Performance, 16, 417–429.
Hinton, G. E., and Parsons, L. M. (1988). Scene-based a n d viewer-centered representations
for comparing shapes. Cognition, 30, 1–35.
Hoffman, J. E. (1998). Visual attention and eye movements. In H. Pashler (Ed.), Attention, p p .
119–153. Hove, East Sussex: Psychology Press.
Houghton, G., and Tipper, S. P. (1994). A model of inhibitory mechanisms in selective atten-
tion. In D. Dagenbach and T. Carr (Eds), Inhibitory mechanisms in attention, memory and lan-
guage, p p . 53–112. Orlando, FL: Academic Press.
Houghton, G., a n d Tipper, S. P. (1996). Inhibitory mechanisms of neural and cognitive con-
trol: Application to selective attention and sequential action. Brain and Cognition, 30, 20–43.
Houghton, G., and Tipper, S. P. (Forthcoming). Attention and the control of action: An inves-
tigation of the effects of selection on population coding of h a n d a n d eye movement. In D.
Heinke, G. W. Humphreys, a n d A. Olson (Eds.), Proceedings of the 5th neural computational
and psychological workshop. Springer-Verlag.
Selection from Neural Population Codes
Howard, L. A., a n d Tipper, S. P. (1997). H a n d deviations away from visual cues: Indirect evi-
dence for inhibition. Experimental Brain Research, 113, 144–152.
Jeannerod, M. (1988). The neural and behavioural organisation of goal-directed movements.
Oxford: Clarendon Press.
Kalaska, J. F. (1988). The representation of arm movements in postcentral and parietal cor-
tex. Canadian Journal of Physiological Pharmacology, 66, 455–463.
Kalaska, J. F., Caminiti, R., and Georgopoulos, A. P. (1983). Cortical mechanisms related to
the direction of two-dimensional arm movements: Relations in parietal area 5 and com-
parison with motor cortex. Experimental Brain Research, 51, 247–260.
Kugler, P. N., and Turvey, M. T. (1987). Information, natural law, and the self-assembly of rhyth-
mic movements. Hillsdale, NJ: Erlbaum.
Lhermitte, F. (1983). “Utilization behaviour’’ a n d its relation to lesions of the frontal lobes.
Brain, 106, 237–255.
Meegan, D., and Tipper, S. P. (1998). Reaching into cluttered visual environments: Spatial
a n d temporal influences of distracting objects. Quarterly Journal of Experimental Psychology,
51A, 225–249.
Moran, J., and Desimone, R. (1985). Selective attention gates visual processing in the extra-
striate cortex. Science, 229, 782–784.
Morrison, R. E. (1984). Manipulation of stimulus onset delay in reading: Evidence for par-
allel programming of saccades. Journal of Experimental Psychology: Human Perception and
Performance, 10, 667–682.
Mushiake, H., Fuji, N., and Tanji, J. (1996). Visually guided saccades versus eye-hand reach:
Contrasting neuronal activity in the cortical supplementary a n d frontal eye fields. Journal of
Neurophysiology, 75, 2181–2191.
Reichle, E. D., Pollatsek, A., Fisher, D. L., and Rayner, K. (1998). Toward a theory of eye
movement control in reading. Psychological Review, 105, 125–157.
Riddoch, M. J., Edwards, M. G., Humphreys, G. W., West, R., and Heafield, T. (1998). Visual
affordances direct action: Neuropsychological evidence from manual interference. Cognitive
Neuropsychology, 15, 645–683.
Rizzolatti, G., Riggio, L., Dascola, I., and Umiltà, C. (1987). Reorienting attention across the
horizontal and vertical meridians: Evidence in favor of a premotor theory of attention.
Neuropsychologia, 25, 31–40.
Rizzolatti, G., Scandolara, C., Matelli, M., and Gentilucci, M. (1981). Afferent properties of
periarcuate neurons in macaque monkey: 2. Visual responses. Behavioural Brain Research, 2,
147–163.
Schall, J. D., and Hanes, D. P. (1993). Neural basis of saccade target selection in frontal eye
fields during visual search. Nature, 366, 467–469.
Sheliga, B. M., Craighero, L., Riggio, L., a n d Rizzolatti, G. (1997). Effects of spatial attention
on directional manual a n d ocular responses. Experimental Brain Research, 114, 339–351.
Sheliga, B. M., Riggio, L., Craighero, L., and Rizzolatti, G. (1995). Spatial attention-
determined modifications in saccade trajectories. Neuroreport: Cognitive Neuroscience and
Neuropsychology, 6, 585–588.
Sheliga, B. M., Riggio, L., a n d Rizzolatti, G. (1994). Orienting of attention and eye move-
ments. Experimental Brain Research, 98, 507–522.
Tipper, Howard, and Houghton
Sheliga, B. M., Riggio, L., and Rizzolatti, G. (1995). Spatial attention a n d eye movements.
Experimental Brain Research, 105, 261–275.
Simon, J. (1969). Reactions toward the source of stimulation. Journal of Experimental
Psychology, 78, 344–346.
Soechting, J. F., and Flanders, M. (1989). Sensorimotor representations for pointing to targets
in three-dimensional space. Journal of Neurophysiology, 62, 582–594.
Sparks, D. L., Holland, R., a n d Guthrie, B. L. (1976). Size and distribution of movement
fields in the monkey superior colliculus. Brain Research, 113, 21–26.
Stins, J. F. (1998). Information-action compatibility. Ph.D. diss., Vrije University, Amsterdam.
Theeuwes, J., Kramer, A. F., Hahn, S., a n d Irwin, D. E. (1998). Our eyes do not always go
where we want them to go: Capture of the eyes by new objects. Psychological Science, 9,
379–385.
Tipper, S. P. (1985). The negative priming effect: Inhibitory priming by ignored objects.
Quarterly Journal of Experimental Psychology, 37A, 571–590.
Tipper, S. P. , Brehaut, J. C., a n d Driver, J. (1990). Selection of moving a n d static objects for
the control of spatially directed action. Journal of Experimental Psychology: Human Perception
and Performance, 16, 492–504.
Tipper, S. P., Howard, L. A., and Houghton, G. (1998). Action-based mechanisms of atten-
tion. Proceedings of the Royal Society of London, Series B, 353, 1385–1393.
Tipper, S. P. , Howard, L. A., and Jackson, S. R. (1997). Selective reaching to grasp: Evidence
for distractor interference effects. Visual Cognition, 4, 1–32.
Tipper, S. P. , Lortie, C., a n d Baylis, G. C. (1992). Selective reaching: Evidence for action-
centered attention. Journal of Experimental Psychology: Human Perception and Performance, 18,
891–905.
Tipper, S. P., Weaver, B., a n d Houghton, G. (1994). Behavioural goals determine inhibitory
mechanisms of selective attention. Quarterly Journal of Experimental Psychology, 47A,
809–840.
Werner, W. (1993). Neurons in the primate superior colliculus are active before a n d during
arm movements to visual targets. European Journal of Neuroscience, 5, 335–340.
245 Selection from Neural Population Codes
11 The Prepared Reflex: Automaticity and Control in Stimulus-Response Translation
Bernhard Hommel
ABSTRACT This chapter reviews a number of empirical a n d theoretical approaches to the
translation of stimulus information into action in choice reaction tasks. Abundant evidence
shows that stimulus-response (S-R) translation does not always conform to people’s inten-
tions, which rules out the notion that it is a highly selective control (or intentionally con-
trolled) operation. This has led to the conception of dual-route models, which view action
control as the outcome of a competition between intentional a n d automatic S-R translation
processes. Although these conceptions have m a n y advantages, they also have their limita-
tions. In particular, there is evidence for more than two routes from perception to action;
intention-related S-R translation can shown to be triggered automatically; and effects attrib-
uted to “automatic translation’’ often d e p e n d on the actor’s intentions. An alternative view
conceives of intentional and automatic processes, not as being different in kind, but rather
as taking place at different points in time, with intentional processes setting the stage for
automatic S-R translation.
Higher organisms exhibit an enormous flexibility in responding a n d
adapting to immediate changes in environmental conditions. Their
behavior is not only controlled by direct a n d persistent input-output
connections but mediated by internal states and modified through expe-
rience. A wealth of cognitive processes is involved in transforming sen-
sory inputs into observable muscle contractions. This chapter will focus
on a central stage in the transformation process—the interface between
perceptual processing and action selection—emphasizing the role inten-
tional and automatic processes play in translating stimulus information
into response activation.
Theories of h u m a n information processing commonly deal with this
interface under the heading of “stimulus-response translation’’ (or “S-R
translation’’), “response determination,’’ “response identification,’’ or
“response selection.’’ Although most models include a box carrying one
of these labels surprisingly little is k n o w n about h o w stimulus informa-
tion is actually translated into action plans. However, to speak of S-R
translation at least t w o requirements need to be met.
First, there has to be some indication, whatever the level of analysis—
that response-related functional codes or brain structures are activated, at
least to some degree. These indications may be relatively direct, such as
the increase in activation of some part of the motor cortex in a brain-
imaging study; or indirect, such as a reaction time pattern revealing
competition between alternative responses. The consequences of S-R
translation differ widely between situations, ranging from the unobserv-
able activation of a mild response tendency, overcome within a few milli-
seconds, to the actual execution of the activated response; clearly, these
differences are of great theoretical and practical moment. Yet, in this
chapter, all that counts is whether there is any indication of response
activation under particular stimulus conditions a n d task instructions.
The second requirement is that the measured arousal of response ten-
dencies, be systematically related to the present stimulus conditions.
Obviously, merely observing that some situations induce response ten-
dencies or increase the likelihood of responding does not yet allow one to
assume that some kind of stimulus information was translated into a cor-
responding response. To be sure that S-R translation actually took place
requires one to predict which response tendency w a s aroused as a func-
tion of which stimulus information. Its logical dependency on available
stimulus information already puts some constraints on the temporal a n d
functional locus of S-R translation in the sequence of stages in h u m a n
information processing. Indeed, most authors (e.g., De Jong 1993; Frith
a n d Done 1986; Kornblum, Hasbroucq, a n d Osman 1990; Meyer a n d
Kieras 1997; Pashler 1994) locate the S-R translation or response selection
stage in between what is commonly called the “perceptual’’ or “stimulus
identification’’ stage a n d those stages having to do with “response initia-
tion’’ and “response execution.’’ Although some stimulus processing is
required before the processed information can be translated into a re-
sponse, this does not mean that S-R translation has to await full process-
ing or identification of a stimulus. For instance, Miller (1988) a n d others
have argued that perceptual stages may pass partial output to response
stages before stimulus identification is complete. For our present pur-
poses, any specific, stimulus-related activation of response-related codes
or structures will count as evidence that S-R translation has taken place,
irrespective of the type of the corresponding stimulus information a n d
the degree to which it is processed.
Authors have characterized intentional a n d automatic processes in
many different ways (for overviews, see N e u m a n n 1984; Schweickert a n d
Boggs 1984): intentional translation processes have been characterized as
controlled (by whatever state or mechanism), working serially (implying
only one translation at a time), capacity limited, effortful, conditional (on
intentions), a n d conscious, whereas automatic processes have been char-
acterized as uncontrolled, working in parallel (implying more than one
translation at a time), capacity unlimited, effortless, unconditional, a n d
unconscious. However, most of the data to be discussed here speak only
to the issue of whether, or h o w much, translation processes depend on
the perceiver’s or actor’s intentions, apart from some preliminary hints
Hommel
about whether these processes work serially or in parallel (thus being
capacity limited or unlimited).1
From a phenomenological perspective, it may seem odd to ask whether
S-R translation depends on intentions. We commonly feel that we per-
ceive an environmental event, think about it, a n d then deliberately select
an appropriate action without further a d o . This view, which has so obvi-
ously motivated many stage models of information processing, strongly
suggests that S-R translation is a more or less direct reflection of the per-
ceiver’s or actor’s intentions. There is increasing empirical evidence,
however, for stimulus-induced and unwanted response activation, which
challenges the idea of S-R translation being under direct, immediate
intentional control.
11.1 THE DEMONSTRATION OF AUTOMATIC STIMULUS-
RESPONSE TRANSLATION
Under normal circumstances, we do not have the slightest doubt that the
actions we perform originate within ourselves, that we are the causal
agents in the process of transforming mere willing into actual moving.
Accordingly, many early psychological approaches to action control,
especially those based on the theorist’s introspection, assumed that
h u m a n action w a s guided and controlled by h u m a n will.
A well-known proponent of such an intentional view was Donders
(1868), w h o attributed the responsibility of translating perceptual infor-
mation into movement to an “organ of will’’ (wilsorgaan). To measure h o w
long this organ would need to make a decision, Donders manipulated
S-R uncertainty in a number of ways. In one experiment, subjects re-
sponded to the electrical stimulation of their left or right foot by mov-
ing their left or right hand, respectively. It turned out that subjects were
faster to respond correctly if they knew in advance which stimulus would
occur than when they did not, a n d Donders took this difference in reac-
tion time as an estimate for the combination of stimulus discrimination
a n d “determination of the will.’’ To further disentangle these two pro-
cesses, Donders employed a g o / n o - g o task that required a selective
response to a specified subset of the stimulus set, pairing stimulus uncer-
tainty with response certainty. He reasoned that such a task w o u l d not
require any further will determination processes (assuming that the
response could be selected in advance), so that their duration could then
be estimated by subtracting the g o / n o - g o reaction time from that
obtained in conditions requiring a response decision. He calculated will
determination to take 36 msec.
The outdated expression “will determination’’ easily translates into the
more fashionable “S-R translation’’ or “response selection’’ (Gottsdanker
a n d Shragg 1985). Indeed, despite marked changes in terminology, some
The Prepared Reflex in S-R Translation
information-processing models (e.g., Hasbroucq, Guiard, a n d Ottomani
1990; Pashler 1994; Sanders 1980; Teichner a n d Krebs 1974; Welford 1968)
are still based on the (sometimes implicit) idea of S-R translation as a
process that exclusively serves to realize the actor’s intention. Conceived
this way, S-R translation represents a control operation by means of
which the “will,’’ or some functional equivalent, decides what to do by
selecting one stimulus a n d activating the corresponding response. Fitting
well into this picture are claims (e.g., Pashler 1994; Welford 1952) that
S-R translation d r a w s heavily on mental resources and thus constitutes
a rather fixed, structural bottleneck in the flow of information through
the cognitive system. On the other hand, a number of robust empirical
findings cast doubt on whether an account of S-R translation as purely
intentional is tenable. These findings fall into four categories, each sug-
gesting a different type of nonintentional and sometimes even counter-
intentional S-R translation.
Compatibility: Effects of Stimulus-Response Similarity
Since the classical work of fitts a n d Seeger (1953), it is known that the
speed of S-R translation d e p e n d s not only on the stimulus or the response
but also on the relationship or mapping between stimuli a n d responses.2
If stimuli a n d responses vary on the same dimension, such as with left-
a n d right-hand responses to left- a n d right-side stimuli, then responses to
stimuli having the same value on the respective dimension (e.g., left
response to left stimulus) can be initiated faster than responses that do
not (e.g., left response to right stimulus).
Of greater interest for our purposes is that feature overlap between
stimulus a n d response affects performance even if this overlap is irrele-
vant to the task, as convincingly demonstrated by the Simon effect
(Simon a n d Small 1969; for an overview, see Lu a n d Proctor 1995). This is
observed when people make a spatial response, such as a left versus a
right keypress, to a nonspatial stimulus attribute, such as color. If the
location of the stimulus varies randomly, a n d if it does so on the same
spatial dimension as the response, performance is better if the stimulus
spatially corresponds to the response than if it does not. Importantly, this
is true not only for absolute spatial S-R correspondence, but also when
left and right stimuli appear within the same visual hemifield (Nicoletti
a n d Umiltà 1989; Umiltà and Liotti 1987) or when subjects respond with
two fingers of the same h a n d (Arend and Wandmacher 1987; Heister,
Ehrenstein, and Schroeder-Heister 1987). That is, anatomical linkage
between hemifield a n d h a n d is insufficient to account for the Simon
effect.
If S-R translation exclusively reflected the instructed S-R mapping
rules, stimulus location would have no effect. The location of the stimu-
Hommel
lus is obviously processed, however, which leads to at least partial acti-
vation of the spatially corresponding response. Presenting a left or right
stimulus can be shown to activate the corresponding response even when
the relevant stimulus feature calls for the alternate response—whether
response activation is assessed by means of lateralized readiness poten-
tials (De Jong, Liang, a n d Lauber 1994; Sommer, Leuthold, and Herma-
nutz 1993), electromyographical recordings (Zachay 1991), or registration
of subthreshold movements (Zachay 1991). Even symbolic stimuli with a
spatial meaning, such as left- or right-pointing arrows, can under certain
conditions automatically activate the corresponding response (Eimer
1995). Clearly, these findings provide strong evidence against S-R trans-
lation being purely under the control of intentions, all the more so
because the critical spatial stimulus feature is evidently not relevant for
the task at hand.
One might argue that, for some reason, the wrong stimulus feature is
“intentionally’’ translated into response activation, perhaps because the
(nonspatial) relevant stimulus dimension is sometimes confused with the
(spatial) relevant response dimension. Or S-R translation might always
need to take into account all the features of a relevant stimulus, so that
irrelevant features cannot be excluded. However, these attempts to save
the intentional translation notion are inconsistent with the observation of
Simon-type effects in tasks that, on a given trial, do not require any trans-
lation between attributes of the critical stimulus a n d the appropriate
response. For instance, if subjects are signaled to prepare a left- or right-
h a n d keypress in advance of a g o / n o - g o signal—so that all relevant S-R
translations can be completed before that signal appears—performance
is still better with spatial correspondence between go signal and response
(Hommel 1995a, exp. 1). Correspondence effects show up even with
100% go- signal probability, that is, in simple reactions, a n d even when
responses are blocked over 80 consecutive trials (Hommel 1996).
Altogether, these findings clearly undermine the idea that the transla-
tion of stimulus location into response activation is wholly u n d e r the con-
trol of intentional processes. There is more evidence from nonspatial
tasks. The best known example is the Stroop effect (Stroop 1935; for an
overview, see MacLeod 1991), which occurs when people verbally name
the color of ink in which color w o r d s are written. Performance is better if
the color word denotes the color of ink to be named (e.g., “RED’’ written
in red ink) than if it refers to another color (e.g., “GREEN’’ written in red
ink). On the one hand, the occurrence of the Stroop effect can be taken to
show that the meaning of the stimulus word cannot be ignored but is
automatically translated into a (congruent or incongruent) response.3 On
the other, requiring subjects to name or respond to the color of the word
clearly introduces color as a task-relevant dimension, and it may be
exactly this task relevance that makes the word so difficult to ignore.
The Prepared Reflex in S-R Translation
Habits: Effects of Overlearned Stimulus-Response Associations
From everyday life, we know h o w difficult it is to escape bad habits, that
is, to change or inhibit overlearned responses to particular stimuli
(Ouellette a n d Wood 1998). In what appears to be the first empirical
study of the interplay between will and habits, Ach (1910) argued that
h u m a n will can be studied best w h e n opposed by overlearned habits that
need to be overcome. In his “combined method,’’ he first had subjects
acquire particular S-R associations by asking them, for instance, to pro-
duce a rhyme to a stimulus syllable (e.g., “zup’’ “tup’’). After extensive
practice, he presented the same stimuli but asked for another response,
such as reading the syllable backward (e.g., “zup’’ “puz’’; cf. Hommel
2000). According to Ach, practice leads to direct associations between
stimuli and responses, so that presenting a stimulus later on will auto-
matically activate the corresponding response. If this response is not the
correct one, it is up to the will to counteract the n o w dysfunctional habit
a n d to make sure that the intended response is m a d e . This extra d e m a n d
should show up in two measures: (1) increased reaction time to stimuli
previously associated with a different response; and (2) increased occur-
rence of what Ach called “intended errors,’’ that is, production of the pre-
viously associated but n o w incorrect response.
Although the methodological standards of experiments in these early
days certainly do not meet today’s expectations—especially the lack of
inferential statistics and the small number of subjects per study—both
increased reaction times and increased frequency of “intended errors’’
after the task switch were replicated many times by Ach a n d several of
his students (summarized in Ach 1935). According to Ach, these findings
indicate that a stimulus event not only provokes an intentional trans-
lation into an appropriate response; it may also, and at the same time,
automatically retrieve a previously acquired S-R association, thereby
activating the previously associated response.
In a better-controlled study, MacLeod and Dunbar (1988) followed the
same logic as Ach in trying to manipulate the relative strength of S-R
associations through differential practice (cf., Stroop 1935 for a very simi-
lar approach). They first trained their subjects to give verbal color word
responses to the shapes of polygons. Then colored polygons were pre-
sented, a n d subjects either n a m e d the color (color color word, shape
being irrelevant) or responded to the shape (shape color word, color
being irrelevant). In congruent conditions, stimulus color and shape
called for the same response, a n d in incongruent conditions the implied
responses were different. As it turned out, testing after only a little prac-
tice produced substantial effects of congruence on shape naming but
not on color naming, suggesting that the associations between stimulus
colors a n d color word responses were stronger than those between the
shapes a n d the just acquired color word responses. However, after more
Hommel
practice, congruence also affected color naming; after even more practice,
incongruent shapes h a d a stronger effect on color naming than incongru-
ent colors had on shape naming. Obviously, then, the relative impact of
irrelevant stimuli on response selection varies with the relative strength
of S-R associations, which suggests that the speed or likelihood of auto-
matic S-R translation, or both, can be affected by learning.
A similar conclusion might be d r a w n from the findings of Proctor a n d
Lu (1999). Their subjects practiced a spatial compatibility task for three
sessions with either a compatible S-R mapping (left stimulus left
response; right stimulus right response) or an incompatible mapping
(left stimulus right response; right stimulus left response) before
performing a standard Simon task requiring left-right responses to letter
stimuli. After compatible mapping practice, a Simon effect of normal size
was obtained, but an inverted effect w a s observed after incompatible
mapping practice. Possibly, learning an incompatible mapping leads to
the formation of S-R associations that are then automatically activated in
the Simon task, too, a n d thus cancel out, a n d even overwrite the usual
benefits of spatial correspondence.
Rules: Effects of Involuntary Application of the Mapping-Rule
Thus far, the evidence for automatic S-R translation discussed has been
restricted to S-R pairs that were either compatible or highly overlearned.
However, indications of automatic translation have also been observed
in single-session experiments (with no opportunity for extensive S-R
learning) using arbitrary S-R mappings. In none of these studies were
the translation-inducing stimulus attributes really task irrelevant, nor
was the translation completely unrelated to the task or the subject’s
intentions. Nevertheless, the translation indicated by the results was in-
voluntary and inappropriate, either translating the wrong stimulus or
occurring at the wrong time—the right rules used in a wrong way.
If people make a discriminative response to a visual target, their reac-
tion time is strongly affected by irrelevant stimuli surrounding the target.
For instance, if a left versus right keypress is m a d e to the centrally pre-
sented letters H a n d S, which are flanked by other letters, performance is
better if target a n d flankers look the same (e.g., H flanked by Hs) than if
the flankers resemble the other, alternative target (e.g., H flanked by Ss;
Eriksen a n d Eriksen 1974). This is not just an effect of visual similarity or
distraction. If two dissimilar letters are assigned to each response,
flankers assigned to the same response as (but different from) the present
target produce better performance than flankers assigned to the alternate
response (Miller 1991). Such an effect suggests that both flankers and tar-
get are processed a n d activate their corresponding responses. Indeed,
incongruent flankers activate their assigned (incorrect) response to the
extent that the activation can be observed in the lateralized readiness
The Prepared Reflex in S-R Translation
potential (LRP; Coles et al. 1985), or in electrophysiological activity
(Eriksen et al. 1985) a n d overt, subthreshold movements of the wrong
h a n d (St. James 1990).
The flanker effect demonstrates that stimuli are not translated into
response activation only in strict conformity with the actor’s intention
a n d thus indicates some kind of automatic processing.4 It is also true,
however, that the incorrectly selected and translated flanker stimuli in a
flanker task are not completely irrelevant; after all, they are valid targets
that merely appear at a wrong location. On the one hand, S-R translation
in a flanker task is intentional in the sense that it realizes the intention to
respond to stimuli in a particular way. On the other, it seems that not
every aspect of the resulting translation can be controlled, so that, some-
what paradoxically, intended S-R rules are automatically applied.
A very similar picture emerges from studies on task-switching per-
formance: moving from one task to another does not switch off the pre-
viously used S-R mapping rules completely. Consider, for instance,
Sudevan a n d Taylor 1987, whose subjects responded to single digits
ranging from 2 to 9 by pressing a left or right key. There were two dif-
ferent S-R m a p p i n g rules, varying randomly from trial to trial, that were
signaled by a letter cue preceding the stimulus. According to one rule,
o d d digits were assigned to one response key a n d even digits to the other,
while the alternate rule assigned low digits (2–5) to one key a n d high dig-
its (6–9) to the other. Obviously, such mappings introduce conditions of
rule or intertask S-R congruence and incongruence, inasmuch as some
stimuli require the same response u n d e r either S-R assignment (e.g., “3’’
if “odd’’ a n d “low’’ stimuli were assigned the same key), while other
stimuli imply different responses (e.g., “2’’). In fact, intertask congruence
h a d a strong impact on performance, with response-congruent stimuli
(i.e., stimuli that in the alternate task w o u l d require the same response)
speeding up performance even if the mapping rule was precued as early
as four seconds before the stimulus set in. Similar effects have been
observed in Rogers and Monsell 1995, in Meiran 1996, and in several
experiments in our lab, suggesting that cross talk between tasks is a reli-
able phenomenon (cf., Allport a n d Wylie, chap. 2, this volume). As
observed by Otten et al. (1996), this cross talk can have far-reaching con-
sequences, with stimuli belonging to the currently invalid task triggering
their associated response up to a level of response-related LRPs.
Note that cross talk between different tasks can occur only if the m a p -
ping rules of these tasks are concurrently applied to translate the stimu-
lus into response activation. In fact, participants in task-switching studies
seem not so much confused about what to do as uncertain about which
(of the simultaneously applied) rules to follow. For instance, Meiran a n d
Daichman (forthcoming) h a d people switch between tasks under high
time pressure, so that many errors were m a d e . Analyses a n d simulations
showed that the types of errors m a d e were not r a n d o m but rather
Hommel
reflected the correct use of the incorrect S-R mapping rule, which fits well
with the (commonly less pronounced) error patterns observed in other
task-switching studies. Obviously, then, we have here the same kind of
interplay between intentional and automatic processes as seen before. On
the one hand, we find evidence of S-R translation that is neither needed
nor helpful, which indicates a high degree of automaticity even in the
absence of extensive practice and S-R similarity. On the other, the out-
comes of these automatic processes do not seem erratic or habitlike, but
rather are strongly related to the actor’s intentions.
The same conclusion can be d r a w n from Hommel 1998a. Subjects
performed two tasks in a row (response order was strictly controlled), a
manual left-right keypressing response (R1) to the color (S1) of a stimulus,
followed by a verbal color name response (R2) to the form (S2) of the same
stimulus. As often found in such double tasks, the second response was
delayed relative to the first by a half second or more, hence there was a
“psychological refractory period’’ (PRP) effect (Telford 1931). However,
the type of R2 strongly affected reaction time in the primary manual task.
If the meaning of R2 corresponded to the color of S1 (e.g., S1 = red; R2
= “red’’) the response to S1 was much faster than if R2 and S1 did not
match (e.g., S1 = red; R2 = “green’’). This could only h a p p e n if R2 was acti-
vated before the primary task was completed, which again implies that
(at least some) S2-R2 translation must have taken place with or even
before the processing of S1 and R1. Obviously, then, S-R translation is
unlikely to be the “structural bottleneck’’ that is widely believed to be
responsible for dual-task costs and PRP effects (e.g., Pashler 1994, chap.
12, this volume; Welford 1952). Whatever or wherever this bottleneck
may be, it does not seem to prevent different stimuli from being trans-
lated into response activation at about the same time. Indeed, the appli-
cation of arbitrary S-R translation rules seems to be so automatic that
it occurs even if it or its timing produces unintended and inappropriate
results.
Integration: Aftereffects of Stimulus-Response Binding
The previous examples show that extensive learning may promote, but
is not always necessary to bring about, automatic S-R translation. Even
single-trial learning can produce stimulus-triggered response activation.
Hommel (1998b) used a task that required two responses (R1 and R2) to
two stimuli (S1 and S2) on each trial. Participants were presented with a
response cue that signaled the identity of R1 (e.g., left versus right key-
press). R1 was prepared but not performed until S1 was presented.
Although S1 varied randomly in shape, color, and location (e.g., green
versus red; X versus O; top versus bottom position), R1 did not depend
on or covary with any of the features of S1. About 1 sec later, S2 appeared;
it varied on the same dimensions as S1, with one feature (shape, say) sig-
The Prepared Reflex in S-R Translation
naling R2. That is, the already prepared, simple R1 was m a d e to the mere
onset of S1, and the binary forced-choice discrimination R2 was m a d e to
the relevant feature of S2. For example, a left-pointing arrow might cue a
left-hand response, which is then prepared and performed at S1 onset,
independent S1 being, say, a red X in the top position. After 1 sec, S2
would appear (e.g., a green X at the bottom position), with its shape
signaling a left-hand response. (Note that this example implies repetition
of stimulus shape a n d response, and alternation of stimulus color a n d
location.)
One might expect several kinds of repetition effects with a task like this,
such as better performance if a stimulus feature or the response is re-
peated. Indeed, repetition effects were obtained, although not very reli-
ably so and only in task versions with very short intervals between S1 a n d
S2 (Hommel forthcoming-a). Much more interesting, however, is the con-
sistent observation that stimulus- and response-related repetition effects
interacted. In particular, repeating stimulus shape or location w a s
beneficial only if the response was also repeated; if not, shape or location
repetition yielded interference instead (Hommel 1998b). Apparently, a
single co-occurrence of S1 a n d R1 resulted in an association or binding of
stimulus (features) and response (features). As a consequence, presenting
the same stimulus (feature) reactivated the associated response, which
caused a problem if this response was not the correct one, that is, if the
repeated stimulus required a new response. That automatic response acti-
vation is indeed involved is also suggested by experiments in which the
forced-choice R2 was replaced by a free-choice response to S2. Even if
urged to avoid any strategy a n d produce random behavior, participants
tend to repeat R1 if S1 is also repeated (Hommel forthcoming-b). Being
unintended, unwanted, a n d unhelpful, these S-R binding effects fulfill
the most common criteria for automaticity and hence represent a case
of automatic S-R translation. Interestingly, however, they clearly do not
result from practice or S-R compatibility, or from applying S-R rules in an
inappropriate way.
Automaticity: Types versus Degrees
The foregoing examples making the case for automatic translation stem
from a broad range of tasks and paradigms a n d may therefore seem to
indicate very different kinds of automaticity. However, it is tempting to
try ordering them on a common dimension, such as the length of the
learning history involved. Binding effects, which result from experienc-
ing a single S-R co-occurrence, clearly have the shortest history, followed
by effects indicating inappropriate rule use, which can be measured after
only 50 trials or less. Then we have effects of S-R associations that seem
to take several sessions of practice to emerge and, finally, effects of S-R
compatibility, which are sometimes attributed to extreme overlearning
Hommel
of S-R relationships (cf. Umiltà a n d Zorzi 1997). Indeed, the available
demonstrations of automatic S-R translation m a y differ only with respect
to the strength of the underlying S-R associations a n d thus indicate
merely different degrees or states, not different types of automaticity.
Although such an account is attractively parsimonious, it is not s u p -
ported by the (still few) findings that speak to this issue. First, there is no
evidence available as to whether binding effects increase with the n u m -
ber of consistent S-R occurrences, so that it is not clear whether binding
is the first stage of S-R associative learning or only a temporary phenom-
enon. Second, there is no indication that effects of inappropriate rule use
would increase over practice. On the contrary, whereas Hommel (1998a)
found no systematic relationship between effects of automatic, inappro-
priate rule use a n d practice within a single session, Sudevan and Taylor
(1987) observed a general decrease of such effects over 20 sessions of
task-switching practice. Third, whereas there is strong evidence for the
impact of task-irrelevant S-R associations on performance increasing with
practice (MacLeod a n d Dunbar 1988), the studies on automatic rule
use (Hommel 1998a; Sudevan and Taylor 1987) have found no evidence
of such a relationship, suggesting that the two kinds of effect are of dif-
ferent origin.
Fourth, up to now there is no convincing evidence that S-R compatibil-
ity effects are d u e to S-R learning. Of course, testing this assumption is
difficult—if we are talking about lifelong experience (e.g., responding
with the right h a n d to objects on the right side or verbally responding to
objects with their name), it would be unethical to prevent subjects from
having this experience and impractical to experimentally induce an equi-
valent number of (counter-) practice trials. Nevertheless, several studies
have investigated whether S-R compatibility effects could be eliminated
through extensive practice. For instance, Fitts and Seeger (1953) found
better performance with spatially compatible than with incompatible
S-R mappings even after 32 sessions of practice. Later studies all showed
the same pattern of results. During the very first trials, subjects have
much more difficulty getting into the task with an incompatible than
with a compatible mapping, but then the difference between compatible
a n d incompatible conditions stabilizes very quickly a n d is more or less
unaffected by further practice (e.g., Brebner, Shephard, a n d Cairney
1972; Dutta a n d Proctor 1992; Morin and Grant 1955). A similar pattern
has been observed in Simon tasks. Although Simon, Craft, a n d Webster
(1973) did find a reduction during 5 sessions of 216 trials each, a pro-
nounced Simon effect was still observed in the final session (see also
Proctor a n d Lu 1999). Even 30 sessions of 210 trials each do not suffice to
eliminate the effect, as demonstrated by the performance of a single,
heroic subject in Hommel 1995b.
To s u m u p , the available findings do not support the assumption of a
single dimension of automaticity or associative S-R strength on which the
The Prepared Reflex in S-R Translation
observed phenomena could be easily ordered. However, given that some
relationships between significant phenomena have not yet been in-
vestigated a n d that some of these relationships are difficult to investi-
gate in any meaningful way, it w o u l d be premature to d r a w definitive
conclusions.
11.2 MULTIPLE ROUTES FROM STIMULUS TO RESPONSE
We have seen substantial evidence against the intuitive, but perhaps
naive idea that S-R translation is a control operation that realizes the
intentions of a perceiver or actor, thereby shielding the action system
against unwelcome stimulus-induced action tendencies. The insight that
S-R translation is only partially under intentional control has led to the
formulation of several models that assume both an intentional and an
automatic route from perception to action. I shall review some of the
most influential dual-route models, considered state-of-the art in many
domains of information-processing psychology, pointing out limitations
that need to be overcome if we are to achieve a comprehensive model of
S-R translation.
Dual-Route Models
Part of the reasoning behind today’s dual-route models can already be
found in Ach 1910, which distinguished between will, a capacity-limited
mechanism in charge of S-R translation and action control, and habits,
S-R associations that result from a n d become stronger with S-R learning.
Habits are assumed to lead to fully automatic S-R translation, that is, to
the activation of the response most often associated with the given stim-
ulus in the past. If the outcome of this translation is in agreement with
(i.e., functional for reaching) the intended action goal, only minimal
effort (or will power) needs to be applied, a n d execution is facilitated. If
an existing habit activates a counterproductive tendency, however, this
needs to be overcome by an increase in effort deployed.
Although current dual-route models are often more specific as to the
processes involved a n d the conditions that need to be fulfilled, the gen-
eral idea that habit and will compete for action control is still alive—even
if habitual S-R translation is now called “automatic’’ or “unconditional’’
a n d willed translation referred to as “intentional’’ or “conditional.’’ A
model that has much in common with Ach’s has been suggested by
Logan (1988), w h o assumes that each experience of a S-R episode leaves
a memory trace of an “instance.’’ Attended stimulus events necessarily
retrieve the instances associated with them and, through that retrieval,
activate the associated response. The more S-R co-occurrences experi-
enced in the past, the more instances retrieved; the more instances
Hommel
retrieved, the more likely it is that the corresponding response will be
activated, which then will compete with intentional, rule-governed S-R
translation processes for action control. Although the two models imple-
ment habits in different ways—through the strengthening of single S-R
associations (Ach) a n d through a separate trace for each experience
(Logan)—the general way they characterize the relationship between
intentional a n d automatic S-R translation is very similar.
Perhaps the most general of dual-route models, Kornblum, Hasbroucq,
a n d Osman’s “dimensional overlap model’’ (1990) attributes S-R compat-
ibility effects to a competition between automatic response activation a n d
voluntary S-R translation. If, a n d only if, a stimulus event shares features
with a response, such as with spatial S-R correspondence in a Simon task,
the stimulus activates the corresponding response automatically and in
parallel to the controlled translation of the relevant stimulus feature into
the correct response. If the automatically activated response happens to
be appropriate, response execution is faster a n d performance better. If
not, the system must suppress the misleading response tendency before
the correct response can be issued—a time-consuming process. This basic
architecture is shared by other, less general models of S-R compatibility
(e.g., De Jong, Liang, a n d Lauber 1994; Hommel 1993a; Lu 1997; Virzi a n d
Egeth 1985).
In the last decade, computational parallel distributed processing (PDP)
or neural network models of S-R compatibility have spelled out the dual
routes in increasing detail, often implementing intentional a n d automatic
routes in very similar w a y s . Typically, stimulus feature codes are
assumed to be permanently connected to codes of responses they share
features with, such as a left stimulus code and a left response code (e.g.,
Barber and O’Leary 1997; Kornblum et al. 1999; Zorzi and Umiltà 1995).
Consequently, registering and coding a stimulus leads to a spreading of
activation to the feature-overlapping response, hence to automatic S-R
translation. In contrast, intentional translation is modeled by introducing
temporary, short-term associations connecting codes of the relevant stim-
ulus feature or features a n d the respective response. These associations
are task specific a n d intention dependent a n d may be taken to represent
something like S-R rules temporarily stored in working memory.
The notion of dual routes from perception to action has advanced our
basic understanding of S-R compatibility and motivated a wealth of
empirical investigations. It has played a crucial role in explaining, among
other things, the consistent observation that the Simon effect decreases
with increasing task difficulty (De Jong, Liang, a n d Lauber 1994; Hommel
1993a) a n d the dependence of spatial compatibility effects on task prepa-
ration (De Jong 1997; Shaffer 1965). There are several reasons, however,
w h y the basic idea and architecture of dual-route models may fail to fully
capture the essence and diversity of S-R translation. I shall discuss three.
The Prepared Reflex in S-R Translation
Multiple Routes to Action
Constructed to serve rather specialized purposes, such as accounting for
practice effects or effects of S-R compatibility, existing dual-route models
emphasize one particular type of automaticity a n d neglect others. In-
asmuch as there is more than one type or cause of automatic S-R
translation, however, none of the available models seems sufficiently
developed to serve as a comprehensive model of S-R translation. Such a
model would need more than two routes or pathways from perception
to action. To model such multiple pathways, we need to understand the
relationships between the various phenomena indicative of automatic
translation.
First, we need to know whether S-R binding is only a process for short-
term temporary integration or whether it also represents the mechanism
that forms long-term S-R associations—what Logan (1988) has called
“instances.’’ Second, we need to know when, how, a n d w h y S-R rules,
stored in working memory to guide current behavior, can be accessed
a n d used by other, inappropriate or irrelevant stimuli to activate the cor-
responding responses, a n d what roles short-term binding a n d long-term
learning play in this context. Third, we need to know more clearly what
the relationship is between habits or overlearned S-R associations a n d
S-R compatibility. Take, for instance, MacLeod and Dunbar’s finding
(1988) that practicing at naming shapes with color words results in
Stroop-like interference with naming colors. If this effect indicates some
kind of acquired compatibility between irrelevant stimulus shape a n d
response (which are defined on nonoverlapping dimensions), this would
seem to argue against, say, the dimensional overlap model of Kornblum,
Hasbroucq, and Osman (1990). Alternatively, if the effect is assumed to be
mediated by different mechanisms a n d simply to mimic compatibility
effects, we need to specify these mechanisms and h o w they differ from
those mediating compatibility effects. This in turn requires compatibility
models to be specific as to why similarity between stimulus and response
sets lead to automatic S-R translation—an issue commonly neglected in
dual-route models (but see Eimer, Hommel, and Prinz 1995; Hommel
1997).
Automaticity of Intentional Translation
Obviously, people can respond to the same stimulus in many different
ways, depending on the task or context and, most important, depending
on their intentions and strategies. To account for this enormous degree of
flexibility in S-R translation, dual-route models have been equipped with
“intentional’’ or “controlled’’ pathways, that is, with perception-action
links that are u n d e r full control of the perceiver’s or actor’s intentional
states. On the other hand, we have already seen that intentional or con-
Hommel
trolled translation is not always as intended a n d controlled as it should
be: irrelevant flankers activate arbitrarily assigned responses, a n d task-
specific S-R rules are inappropriately applied while performing another
task. This means that stimuli can activate responses automatically not
only via the automatic pathway proposed by dual-route models but also
by the intentional route. If so, it cannot be the process of S-R translation
that is under intentional control, but rather the implementation of the
underlying S-R rules. That is, although intentional states may determine
which rules are selected, formed, a n d implemented, once they are estab-
lished, stimuli seem to have direct a n d uncontrolled access to these rules,
leading to automatic translation via intentional routes.
This conclusion has important theoretical implications. First, as far as
S-R translation is concerned, it shifts the time point of intentional control
from the interval between stimulus perception and response selection to
the beginning of a task. In a sense, such a view stands in contrast to
Donders’s idea (1868) that “will determination’’ follows perception—an
idea that has m a d e its way into many modern information-processing
models. In fact, if the preconditions for S-R translation are already set
before a stimulus comes u p , at least part of the will has already been
determined in advance, a consideration I will develop in the section 11.3.
Second, if intentional S-R translation is really as automatic as the avail-
able findings suggest, it is unlikely to represent the processing bottleneck
that has always been associated with it by single-channel models of dual-
task performance since Welford 1952. Obviously, if more than one stimu-
lus at a time can be translated into a response, there is no reason w h y
costs observed in dual-task performance should have something to do
with S-R translation. Rather, it may be the automaticity of intentional
translation, not the lack of it, that causes the trouble. If more than one
stimulus at a time is translated into its response, the system may need to
find out which response belongs to which stimulus, and in what order
the responses are to be carried out. This may be called a problem of
“response selection,’’ but not one of S-R translation (Hommel 1998a).
Intentionality of Automatic Translation
Although exact criteria for automaticity are still under debate (e.g., Bargh
1989; Hasher a n d Zacks 1979; N e u m a n n 1984), dual-route models explic-
itly or implicitly share the definition of Kornblum, Hasbroucq, a n d
Osman (1990, 261) that the automatic route can “under some conditions
be attenuated or enhanced’’ but “under no conditions . . . ignored or by-
passed,’’ and that, accordingly, people “whether instructed to use or to
suppress an automatized process w o u l d therefore produce evidence of its
operation in their performance.’’ There are reasons to believe, however,
that automatic S-R translation is not independent of the task at h a n d a n d
the instructions given to acting participants. In particular, it has been
The Prepared Reflex in S-R Translation
shown that the occurrence of effects attributed to automatic translation
d e p e n d s on attention (i.e., the way stimuli are selected and coded), inten-
tion (i.e., the way responses are prepared a n d coded), a n d on task-specific
strategies.
Attention and Stimulus Coding A first demonstration of the impact of
instructions on “automatic’’ S-R translation comes from the observation
that the Simon effect occurs not only with unilateral, but also with sym-
metrical, bilateral stimulation. That is, even w h e n people are presented
with a left a n d a right stimulus at the same time, with the relevant one
defined by its form (Grice, Boroughs, and Canham 1984), color (Hommel
1993b; Proctor and Lu 1994), or meaning (O’Leary a n d Barber 1993), they
are faster if the relevant stimulus comes up on the same side as the
required response. Thus it is not the spatial correspondence between any
stimulus and the response that matters for the Simon effect, but the
spatial relationship between the attended stimulus of a display and the
response (Stoffer a n d Umiltà 1997). Given that the task instruction
specifies which stimulus to attend to, this implies that there is no Simon
effect without specific task instructions, at least when more than one stim-
ulus is presented at a time. Inasmuch as the Simon effect is attributed to
automatic S-R translation, this kind of translation cannot be completely
independent from the task.
There are more challenging findings. Consider, for example, Eimer’s
observation (1995) that response-irrelevant arrows automatically activate
corresponding responses, a finding consistent with dual-route models of
S-R compatibility. In a recent lateralized readiness potential study, Eimer
a n d Schlaghecken (1998) showed that even subliminal (i.e., not con-
sciously perceivable) arrowheads preceding a target arrow activated the
corresponding response. However, as soon as the relevant arrow stimuli
were replaced by letters without any spatial meaning, arrow primes no
longer produced “automatic activation.’’ Obviously, the translation of
stimulus information into the activation of spatially congruent responses
can depend critically on what relevant information a perceiver or actor
intends to translate—hence automatic translation depends on intentions.
A very similar conclusion is suggested by the findings of Cohen a n d
Shoup (1997), w h o modified the standard flanker task by manipulating
targets and distractors on two dimensions: color and orientation. For
instance, one response key could be assigned to a red vertical line a n d a
blue right diagonal line a n d the other key to a green vertical line and a
blue left diagonal line. If target a n d flankers were defined on the same
dimension (e.g., red vertical line flanked by red vertical lines versus green
vertical lines), the standard flanker effect was obtained, that is, congruent
flankers produced better performance than incongruent flankers. If, how-
ever, target a n d flankers were defined on different dimensions (e.g., red
vertical line flanked by blue right diagonal lines versus blue left diagonal
Hommel
lines), there w a s no congruence effect—an observation also m a d e by
Fournier, Eriksen, a n d Bowd (1998) in a speeded feature judgment task.
It seems that, although incongruent flankers are unintentionally trans-
lated into corresponding response activation, the probability of this trans-
lation is strongly determined by what is defined a n d identified as target,
that is, by task-specific, attentional a n d intentional processes. This fits
nicely with the results of Bauer and Besner (1997), w h o showed that
Stroop words affect keypressing responses only if participants classify the
ink of the words, but not if they judge whether a given color is present
or absent (even if RT levels are comparable). Obviously, automatic pro-
cesses are (or at least can be) task dependent.
Intention and Response Coding Evidence for a role of response sets in
S-R translation comes from Hommel’s 1996, study on spatial S-R com-
patibility in simple, prepared responses. One major outcome was that
effects of S-R compatibility are not restricted to situations involving
response uncertainty, as commonly believed (e.g., Berlucchi et al. 1977),
but also occur if a completely prepared response is m a d e to a spatially
compatible or incompatible go stimulus. It also turned out that the size
of the compatibility effect depended strongly on the task relevance of
the responses. For instance, if the same (left- or right-hand) response
was used throughout a long block of trials, the effect of spatial corre-
spondence between response and go signal was very small a n d often
insignificant. Interestingly, though, much larger a n d more reliable corre-
spondence effects showed up w h e n another spatial (i.e., right- or left-
hand) response w a s used in a secondary task performed in between the
trials of the compatibility task. Apparently, the overlap of stimulus a n d
response features is not a sufficient predictor of automatic S-R translation.
Whether a particular response possesses a particular feature and whether
this feature overlaps with those of the stimulus are of little consequence
if the task at hand does not require use of the response feature to dis-
criminate the response from another one. In other words, similarity
between a stimulus a n d a response produces “automatic’’ S-R translation
only (or at least mainly) if the respective feature dimension is important
to the given task context.
If this is so, one should be able to manipulate the kind of “automatic’’
S-R translation by asking the participant to attend more to some response
features than to others. This is what Hommel (1993c) did in a version of
the Simon task, where people responded to the pitch of a tone heard ran-
domly on the left or right side by pressing a left- or right-hand key.
Pressing a particular key flashed a light on the opposite side, so that each
response h a d two spatial features: the location of the finger or key and the
location of the action-contingent light. When subjects were instructed, as
in a standard Simon task, to “press the left/right key in response to the
l o w / h i g h pitch,’’ left-hand keypresses were faster to left-side tones a n d
The Prepared Reflex in S-R Translation
right-hand keypresses were faster to right-side tones—a standard Simon
effect. When, however, subjects were instructed to “flash the right/left
light in response to the l o w / h i g h pitch,’’ left-hand keypresses were faster
to right-side tones a n d right-hand keypresses were faster to left-side
tones. Obviously, the instruction not only had a strong impact on auto-
matic S-R translation; it actually determined its outcome. Merely describ-
ing the task in terms of keypressing led the participants to code their
responses with respect to the locations of the response keys, whereas
describing the very same task in terms of light flashing persuaded them
to code their responses with respect to the locations of the lights. If we
attribute the Simon effect to automatic S-R translation, this is further evi-
dence that automatic translation is not independent of h o w participants
interpret the task a n d h o w they intend to solve it.
Strategies and Implementation of Stimulus-Response Rules Apart
from stimulus- and response-related factors, automatic translation can
also be affected by task-specific strategies and expectations. Evidence for
this comes from variations of the relative frequency or likelihood of
stimulus-stimulus-congruent or stimulus-response-compatible trials
in Stroop tasks (Logan 1980; Logan a n d Zbrodoff 1979), flanker tasks
(Gratton, Coles, a n d Donchin 1992), a n d Simon tasks (Hommel 1994; Toth
et al. 1995), that is, from manipulations of the utility of irrelevant, but
response-related information. Whatever the task, increasing the fre-
quency of congruent or compatible trials increased, and decreasing the
frequency decreased or even eliminated, the effect. In the same vein,
Proctor, Lu, a n d Van Zandt (1992) found that the Simon effect gets larger
if the likely response is precued a n d can be prepared in advance. Clearly,
these observations suggest that the degree and outcome of automatic
translation is modified by, a n d sometimes even d e p e n d s on, task-specific
strategies and preparatory processes.
Further evidence for a role of task preparation has been reported by
Valle-Inclán a n d Redondo (1998), w h o measured response activation in a
Simon task by means of LRPs. The relevant S-R mapping was not fixed in
this study, but varied randomly from trial to trial, as did the temporal
order in which mapping and stimulus were presented. When the m a p -
ping was presented before the stimulus, the stimulus immediately acti-
vated the spatially corresponding response, independently of which
response w a s correct. That is, there was evidence of automatic S-R trans-
lation. On the other hand, when the stimulus appeared before the S-R
m a p p i n g , automatic response activation w a s no longer observed.
Apparently, although automatic S-R translation did not follow the rele-
vant S-R rules, it required their implementation or at least, as Valle-Inclán
a n d Redondo suggest, some degree of readiness to react. Whatever the
correct answer may be, it seems clear that automatic routes proposed by
dual-route models can be “ignored or bypassed,’’ which stands in con-
tradiction to h o w these routes are typically defined a n d characterized.
264 Hommel
11.3 PROSPECTS: STIMULUS-RESPONSE TRANSLATION AS
PREPARED REFLEX
The abundant evidence for several kinds of automatic access of stimuli
to action control calls for a translation model with more than just one,
highly controlled pathway from perception to action. As a consequence,
several dual-route models have been developed to account for different
aspects of the available evidence, and these models are quite successful
in their respective empirical domains. On the other hand, if we want a
comprehensive S-R translation model not restricted to particular experi-
mental effects, we still have some way to go.
I have sketched three major theoretical problems that need to be
solved. First, a comprehensive model is likely to comprise more than two
routes. There is evidence of at least four kinds of automatic S-R transla-
tion, a n d the ways they differ do not suggest that they originate in the
same type of process. It thus seems insufficient to distinguish just one
intentional and one automatic route. We need more complex, multiroute
models. Second, observations of inappropriate rule use suggest that the
intentional route from perception to action is not very tightly controlled,
but can be automatically accessed by task-related stimuli. This raises
doubts about the usefulness of distinguishing between controlled a n d
uncontrolled routes, or at least requires that we specify exactly when a n d
h o w control is exerted. Third, phenomena that current dual-route models
attribute to automatic S-R translation strongly depend on attentional set
a n d action intentions, suggesting that the supposedly automatic route is
not uncontrollable. Thus, all in all, there are reasons to doubt that the
roles of, a n d the interplay between, control and automaticity in S-R trans-
lation are best captured by the distinction between intentional a n d auto-
matic routes.
A more suitable approach to the control-automaticity relationship
might be derived from consideration of Exner 1879. On the basis of his
introspections in “reaction time’’ experiments (a term he had introduced
to psychology six years earlier), Exner explicitly rejected the notion that
intentional control (or the will) intervenes between stimulus a n d
response—a notion that seemed quite natural to Donders a n d that still
does to his followers. Exner argued that preparing for a task is accom-
plished by setting oneself, long before the first stimulus comes u p , into a
state that ensures that responses are carried out efficiently and as in-
tended. Although evoking that state is a voluntary act requiring atten-
tion, once the state is created, the response is actually involuntary, that is,
no further effort of will is needed to translate the upcoming stimulus into
the response. In fact, stimuli trigger their respective response unless the
mediating state is actively deactivated or inhibited. According to this
conception, intentional processes do not actually carry out S-R transla-
tion, but only configure the cognitive system to do so automatically, once
the defined target stimulus arrives—that is, as a “prepared reflex’’
265 The Prepared Reflex in S-R Translation
(Woodworth 1938). Interestingly, the old idea of theoretically distin-
guishing between intentional set implementation a n d set-dependent, but
automatic S-R translation is currently experiencing a revival (see the
overview by Monsell 1996), a n d recent models such as those of Cohen
a n d Huston (1994) or Meyer and Kieras (1997; Kieras et al., chap. 30, this
volume) can be viewed as first, systematic attempts to implement the
major aspects of this distinction into a computational framework.
From a prepared reflex perspective, it is not so surprising to find evi-
dence of both automaticity of intended S-R translation a n d intentional
control of automatic routes. Obviously, a prepared cognitive reflex is nei-
ther exclusively automatic nor exclusively voluntary. On the one hand, it
is implemented as a consequence of, a n d does express a voluntary deci-
sion to perform an action under particular circumstances in a particular
way a n d thus necessarily d e p e n d s on task and intention. If so, the result-
ing task set is likely to reflect the way the task is understood and inter-
preted by the perceiver or actor, a n d hence determines how stimuli are
coded (e.g., which stimulus features are attended and linked to response
features), h o w responses are coded (e.g., which response features are
attended a n d linked to stimulus features), when stimulus information is
expected, and when actions are prepared a n d issued. As we have seen, all
these decisions have a strong impact on the occurrence of automatic
processes, a n d therefore can be regarded as both implementing arbitrary,
transient S-R connections (the intentional route) a n d directly or indirectly
enabling learning- or compatibility-related S-R associations (the auto-
matic route).
Once a task set is implemented (and automatic routes enabled), how-
ever, the whole system is prepared to act in an automatic fashion—and
this may sometimes produce undesirable side effects. It is certainly an
advantage that the cognitive system is able to automatize itself, so to
speak, so that the onset of a stimulus immediately triggers the corre-
sponding prepared action without (much) further a d o . On the other
hand, the price to pay for this economical solution is that u n w a n t e d infor-
mation will sometimes lead to troublesome consequences, especially if an
irrelevant stimulus fits the internal description of the triggering stimulus,
such as in flanker or Stroop tasks, or in task-switching experiments.
Nevertheless, even unhelpful and misleading S-R translations of this
sort strictly depend on, a n d thus in some sense represent, the actor’s
intention.
Such a prepared reflex view may be helpful in developing a compre-
hensive theory of S-R translation. Indeed, it complements and extends
recent attempts at computational modeling of S-R translation processes in
compatibility a n d related tasks. Take, for instance, the models of Barber
a n d O’Leary (1997) a n d of Zorzi a n d Umiltà (1995), which distinguish
between transient S-R associations reflecting the instructed S-R mapping
a n d permanent links that can be hard-wired or acquired through learn-
Hommel
ing. Although this distinction m a p s onto that of intentional and auto-
matic routes, once the transient links are implemented, they work in a
purely stimulus-triggered fashion like their permanent counterparts.
That is, the two types of pathway differ only in history a n d durability, not
in automaticity. The same can be said of the model proposed by Cohen,
Dunbar, and McClelland (1990) and Cohen a n d Huston (1994), w h o went
one step further in attempting to deal with the process of route imple-
mentation itself (also treated in Meyer a n d Kieras 1997). To do so, task
d e m a n d representations are postulated, the activation of which (e.g.,
through presenting task instructions) can directly modify the flow of
information from stimulus to response codes. In this case, S-R links differ
neither in permanence nor automaticity, but in task-specific strength only.
Although it is clear that more work needs to be done to understand a n d
model in greater detail h o w S-R associations are acquired in the first
place, h o w stimulus a n d response coding can affect the implementation
or use of S-R links, and how the preparation to act influences the likeli-
hood of automatic S-R translation, current modeling attempts are very
much in line with the idea of S-R translation as a prepared cognitive
reflex.
To summarize, we have seen that S-R translation is not just a direct
expression of h u m a n will, nor is it satisfactorily sketched as a competition
between fully automatic, stimulus-triggered processes and autonomous
control operations representing an on-line realization of task intentions.
S-R translation is almost always modulated by the intentions of the per-
ceiving or acting person. Rather than directly intervening between stim-
ulus perception a n d response selection, a n d thus actually performing the
translation, intentional processes seem merely to set the stage for later
S-R translation and to leave the rest to the dynamic interplay between
intentionally implemented a n d nonintentionally enabled automatic
processes. Even though this kind of interplay may sometimes produce
unwanted side effects, we must not forget that intentions usually refer to
behavioral outcomes, not to processes realizing them. Therefore, the
functionality of our intentionally controlled automatic processes should
be judged in terms not of reaction times but of behavioral outcome. Given
that, with sufficient time, no subject in a Stroop task would ever name the
color word, this surely provides a much brighter perspective on our
capacity for self-control.
NOTES
1. Some evidence pertaining to the relationship between conscious awareness a n d the con-
trol of manual pointing and grasping is reviewed by Milner (chap. 9, this volume), although
the distinction made there between processing streams for conscious perception and for
visuomotor control does not easily m a p onto the distinction between intentional a n d auto-
matic S-R translation discussed here.
The Prepared Reflex in S-R Translation
2. In this chapter, the terms compatible a n d incompatible refer to the relationship or mapping
between stimuli and responses, whereas the terms congruence a n d incongruence refer to the
relationship between stimuli or between responses.
3. The Stroop effect has also been observed with manual keypressing responses (e.g., in the
absence of S-R feature overlap; Keele 1972), which might be taken to suggest a contribution
of stimulus-stimulus (in)congruence to the overall Stroop effect (e.g., Kornblum 1994). Even
if this were so, however, the robust finding that switching from manual to verbal responses
substantially increases the effect (e.g., Redding and Gerjets 1977) shows that S-R compati-
bility makes an important contribution of its own.
4. Note that this conclusion in no way depends on the actual cause of the flanker effect.
Whether the effect is d u e to a conflict between target- and flanker-activated responses
(Eriksen and Schultz 1979) or to interactions between target- and flanker-coding processes
(Kornblum et al. 1999)—implying that response activation only reflects, but does not pro-
duce, the flanker effect—it is clear that (1) flanker information is translated into response
activation and (2) this particular translation is not intended.
REFERENCES
Ach, N. (1910). Über den Willensakt und das Temperament. Leipzig: Quelle and Meyer.
Ach, N. (1935). Analyse des Willens. In E. Abderhalden (Ed.), Handbuch der biologischen
Arbeitsmethoden. Vol. 4, Berlin: Urban a n d Schwarzenberg.
Arend, U., a n d Wandmacher, J. (1987). On the generality of logical recoding in spatial inter-
ference tasks. Acta Psychologica, 65, 193–210.
Barber, P. J., and O’Leary, M. J. (1997). The relevance of salience: Towards an activational
account of irrelevant stimulus-response compatibility effects. In B. Hommel and W. Prinz
(Eds.), Theoretical issues in stimulus-response compatibility, p p . 135–172. Amsterdam: Elsevier.
Bargh, J. A. (1989). Conditional automaticity: Varieties of automatic influence in social per-
ception and cognition. In J. S. Uleman, and J. A. Bargh (Eds.), Unintended thought, p p . 3–51.
London: Guilford Press.
Bauer, B., and Besner, D. (1997). Processing in the Stroop task: Mental set as a determinant
of performance. Canadian Journal of Experimental Psychology, 51, 61–68.
Berlucchi, G., Crea, F., Di Stefano, M., and Tassinari, G. (1977). Influence of spatial stimulus-
response compatibility on reaction time of ipsilateral and contralateral h a n d to lateralized
light stimuli. Journal of Experimental Psychology: Human Perception and Performance, 3,
505–517.
Brebner, J., Shephard, M., a n d Cairney, P. (1972). Spatial relationships a n d S-R compatibil-
ity. Acta Psychologica, 37, 93–106.
Cohen, A., and Shoup, R. (1997). Perceptual dimensional constraints in response selection
processes. Cognitive Psychology, 32, 128–181.
Cohen, J. D., Dunbar, K., a n d McClelland, J. L. (1990). On the control of automatic pro-
cesses: A parallel distributed processing account of the Stroop effect. Psychological Review,
97, 332–361.
Cohen, J. D., a n d Huston, T. A. (1994). Progress in the use of interactive models for under-
standing attention and performance. In C. Umiltà, and M. Moscovitch (Eds.), Attention and
Performance XV: Conscious and nonconscious information processing, p p . 453–476. Cambridge,
MA: MIT Press.
Hommel
Coles, M. G. H., Gratton, G., Bashore, T. R., Eriksen, C. W., a n d Donchin, E. (1985). A psy-
chophysiological investigation of the continuous flow model of h u m a n information pro-
cessing. Journal of Experimental Psychology: Human Perception and Performance, 11, 529–553.
De Jong, R. (1993). Multiple bottlenecks in overlapping task performance. Journal of
Experimental Psychology: Human Perception and Performance, 19, 965–980.
De Jong, R. (1997). Compatibility effects on performance a n d executive control in dynamic
task settings. In B. Hommel and W. Prinz (Eds.), Theoretical issues in stimulus-response com-
patibility, p p . 223–239. Amsterdam: Elsevier.
De Jong, R., Liang, C.-C., a n d Lauber, E. (1994). Conditional a n d unconditional automa-
ticity: A dual-process model of effects of spatial stimulus-response correspondence. Journal
of Experimental Psychology: Human Perception and Performance, 20, 731–750.
Donders, F. C. (1868). Over de snelheid van psychische processen. Onderzoekingen, gedann in
het physiologisch laboratorium der Utrechtsche hoogeschool, series 2, vol. 2, p p . 92–120.
Dutta, A., a n d Proctor, R. W. (1992). Persistence of stimulus-response compatibility effects
with extended practice. Journal of Experimental Psychology: Learning, Memory, and Cognition,
18, 801–809.
Eimer, M. (1995). Stimulus-response compatibility and automatic response activation:
Evidence from psychophysiological studies. Journal of Experimental Psychology: Human
Perception and Performance, 21, 837–854.
Eimer, M., Hommel, B., a n d Prinz, W. (1995). S-R compatibility a n d response selection. Acta
Psychologica, 90, 301–313.
Eimer, M., a n d Schlaghecken, F. (1998). Effects of masked stimuli on motor activation:
Behavioral and electrophysiological evidence. Journal of Experimental Psychology: Human
Perception and Performance, 24, 1737–1747.
Eriksen, B. A., and Eriksen, C. W. (1974). Effects of noise letters upon the identification of a
target letter in a nonsearch task. Perception and Psychophysics, 16, 143–149.
Eriksen, C. W., Coles, M. G. H., Morris, L. R., and O’Hara, W. P. (1985). An electromyo-
graphic examination of response competition. Bulletin of the Psychonomic Society, 23,
165–168.
Eriksen, C. W., a n d Schultz, D. W. (1979). Information processing in visual search: A contin-
u o u s flow conception and experimental results. Perception and Psychophysics, 25, 249–263.
Exner, S. (1879). Physiologie der Grosshirnrinde. In L. Hermann (Ed.), Handbuch der
Physiologie, vol. 2, part 2, p p . 189–350. Leipzig: Vogel.
Fitts, P. M., and Seeger, C. M. (1953). S-R compatibility: Spatial characteristics of stimulus
a n d response codes. Journal of Experimental Psychology, 46, 199–210.
Fournier, L. R., Eriksen, C. W., a n d Bowd, C. (1998). Multiple-feature discrimination faster
than single-feature discrimination within the same object? Perception and Psychophysics, 60,
1384–1405.
Frith, C. D., a n d Done, D. J. (1986). Routes to action in reaction time tasks. Psychological
Research, 48, 169–177.
Gottsdanker, R., and Shragg, G. P. (1985). Verification of Donders’ subtraction method.
Journal of Experimental Psychology: Human Perception and Performance, 11, 765–776.
Gratton, G., Coles, M. G. H., and Donchin, E. (1992). Optimizing the use of information:
Strategic control of activation of responses. Journal of Experimental Psychology: General, 121,
480–506.
The Prepared Reflex in S-R Translation
Grice, G. R., Boroughs, J. M., and Canham, L. (1984). Temporal dynamics of associative
interference a n d facilitation produced by visual context. Perception and Psychophysics, 36,
499–507.
Hasbroucq, T., Guiard, Y., and Ottomani, L. (1990). Principles of response determination:
The list-rule model of SR compatibility. Bulletin of the Psychonomic Society, 28, 327–330.
Hasher, L., and Zacks, R. T. (1979). Automatic and effortful processes in memory. Journal of
Experimental Psychology: General, 108, 356–388.
Heister, G., Ehrenstein, W. H., and Schroeder-Heister, P. (1987). Spatial S-R compatibility
with unimanual two-finger choice reactions: Effects of irrelevant stimulus location.
Perception and Psychophysics, 42, 195–201.
Hommel, B. (1993a). The relationship between stimulus processing and response selection
in the Simon task: Evidence for a temporal overlap. Psychological Research, 55, 280–290.
Hommel, B. (1993b). The role of attention for the Simon effect. Psychological Research, 55,
208–222.
Hommel, B. (1993c). Inverting the Simon effect by intention. Psychological Research, 55,
270–279.
Hommel, B. (1994). Spontaneous decay of response code activation. Psychological Research,
56, 261–268.
Hommel, B. (1995a). Stimulus-response compatibility and the Simon effect: Toward an
empirical clarification. Journal of Experimental Psychology: Human Perception and Performance,
21, 764–775.
Hommel, B. (1995b). Unpublished study reported in W. Prinz, G. Aschersleben, B. Hommel,
a n d S. Vogt, Handlungen als Ereignisse. In D. Dörner a n d E. van der Meer (Eds.), Das
Gedächtnis: Trends, Probleme, Perspektiven, p p . 129–168. Göttingen: Hogrefe, 1995.
Hommel, B. (1996). S-R compatibility effects without response uncertainty. Quarterly Journal
of Experimental Psychology, 49A, 546–571.
Hommel, B. (1997). Toward an action-concept model of stimulus-response compatibility. In
B. Hommel and W. Prinz (Eds.), Theoretical issues in stimulus-response compatibility, p p .
281–320. Amsterdam: Elsevier.
Hommel, B. (1998a). Automatic stimulus-response translation in dual-task performance.
Journal of Experimental Psychology: Human Perception and Performance, 24, 1368–1384.
Hommel, B. (1998b). Event files: Evidence for automatic integration of stimulus-response
episodes. Visual Cognition, 5, 183–216.
Hommel, B. (2000). Intentional control of automatic stimulus-response translation. In Y.
Rossetti and A. Revonsuo (Eds.), Interaction between dissociable conscious and nonconscious
processes, p p . 223–244. Amsterdam: John Benjamins Publishing Company.
Hommel, B. (Forthcoming-a). Time course of feature binding.
Hommel, B. (Forthcoming-b). Single stimulus-response co-occurrences affect subsequent
free-choice behavior.
Keele, S. W. (1972). Attention d e m a n d s of memory retrieval. Journal of Experimental
Psychology, 93, 245–248.
Kornblum, S. (1994). The way irrelevant dimensions are processed depends on what
they overlap with: The case of Stroop- and Simon-like stimuli. Psychological Research, 56,
130–135.
Hommel
Kornblum, S., Hasbroucq, T., a n d Osman, A. (1990). Dimensional overlap: Cognitive basis
for stimulus-response compatibility—a model a n d taxonomy. Psychological Review, 97,
253–270.
Kornblum, S., Stevens, G, Whipple, A., and Requin, J. (1999). The effects of irrelevant stim-
uli: 1. The time-course of stimulus-stimulus and stimulus-response consistency effects with
Stroop-like stimuli, Simon-like tasks, a n d their factorial combinations. Journal of Experi-
mental Psychology: Human Perception and Performance, 25, 688–714.
Logan, G. D. (1980). Attention a n d automaticity in Stroop a n d priming tasks: Theory a n d
data. Cognitive Psychology, 12, 523–553.
Logan, G. D. (1988). Toward an instance theory of automatization. Psychological Review, 95,
492–527.
Logan, G. D., and Zbrodoff, N. J. (1979). When it helps to be misled: Facilitative effects of
increasing the frequence of conflicting stimuli in a Stroop-like task. Memory and Cognition, 7,
166–174.
Lu, C.-H. (1997). Correspondence effects for irrelevant information in choice-reaction tasks:
Characterizing the stimulus-response relations a n d the processing dynamics. In B. Hommel
a n d W. Prinz (Eds.), Theoretical issues in stimulus-response compatibility, p p . 85– 117.
Amsterdam: Elsevier.
Lu, C.-H., and Proctor, R. W. (1995). The influence of irrelevant location information on per-
formance: A review of the Simon a n d spatial Stroop effects. Psychonomic Bulletin and Review,
2, 174–207.
MacLeod, C. M. (1991). Half a century of research on the Stroop effect: An integrative
review. Psychological Bulletin, 109, 163–203.
MacLeod, C. M., and Dunbar, K. (1988). Training and Stroop-like interference: Evidence for
a continuum of automaticity. Journal of Experimental Psychology: Learning, Memory, and
Cognition, 14, 126–135.
Meiran, N. (1996). Reconfiguration of processing mode prior to task performance. Journal of
Experimental Psychology: Learning, Memory, and Cognition, 22, 1423–1442.
Meiran, N., a n d Daichman, A. (Forthcoming). Parallel processing in rapid switching
between tasks.
Meyer, D. E., a n d Kieras, E. D. (1997). A computational theory of executive cognitive
processes and multiple task performance: 1. Basic mechanisms. Psychological Review, 104,
3–75.
Miller, J. (1988). Discrete a n d continuous models of h u m a n information processing:
Theoretical distinctions and empirical results. Acta Psychologica, 67, 191–257.
Miller, J. (1991). The flanker compatibility effect as a function of visual angle, attentional
focus, visual transients, and perceptual load: A search for boundary conditions. Perception
and Psychophysics, 49, 270–288.
Monsell, S. (1996). Control of mental processes. In V. Bruce (Ed.), Unsolved mysteries of the
mind, p p . 93–148. Hove, U.K.: Erlbaum.
Morin, R. E., and Grant, D. A. (1955). Learning and performance on a key-pressing task as
function of the degree of spatial stimulus-response correspondence. Journal of Experimental
Psychology, 49, 39–47.
Neumann, O. (1984). Automatic processing: A review of recent findings and a plea for an
old theory. In W. Prinz, and A. F. Sanders (Eds.), Cognition and motor processes, p p . 255–293.
Berlin: Springer.
The Prepared Reflex in S-R Translation
Nicoletti, R., a n d Umiltà, C. (1989). Splitting visual space with attention. Journal of
Experimental Psychology: Human Perception and Performance, 15, 164–169.
O’Leary, M. J., and Barber, P. J. (1993). Interference effects in the Stroop and Simon para-
digms. Journal of Experimental Psychology: Human Perception and Performance, 19, 830–844.
Otten, L. J., Sudevan, P. , Logan, G. D., and Coles, M. G. H. (1996). Magnitude versus parity
in numerical judgements: Event-related brain potentials implicate response conflict as the
source of interference. Acta Psychologica, 94, 21–40.
Ouellette, J. A., and Wood, W. (1998). Habit and intention in everyday life: The multiple
processes by which past behavior predicts future behavior. Psychological Bulletin, 124, 54–74.
Pashler, H. (1994). Dual-task interference in simple tasks: Data and theory. Psychological
Bulletin, 116, 220–244.
Proctor, R. W., a n d Lu, C.-H. (1994). Referential coding and attention shifting accounts of the
Simon effect. Psychological Research, 56, 185–195.
Proctor, R. W., a n d Lu, C.-H. (1999). Processing irrelevant location information: Practice and
transfer effects in choice-reaction tasks. Memory and Cognition, 27, 63–77.
Proctor, R. W., Lu, C.-H., a n d Van Zandt, T. (1992). Enhancement of the Simon effect by
response precuing. Acta Psychologica, 81, 53–74.
Redding, G. M., a n d Gerjets, D. A. (1977). Stroop effect: Interference and facilitation with
verbal and manual responses. Perceptual and Motor Skills, 45, 11–17.
Rogers, R. D., and Monsell, S. (1995). Costs of a predictable switch between simple cogni-
tive tasks. Journal of Experimental Psychology: General, 124, 207–231.
Sanders, A. F. (1980). Stage analysis of reaction processes. In G. E. Stelmach, and J. Requin
(Eds.), Tutorials in motor behavior, p p . 331–354. Amsterdam: Elsevier.
Schweickert, R., a n d Boggs, G. J. (1984). Models of central capacity and concurrency. Journal
of Mathematical Psychology, 28, 223–281.
Shaffer, L. H. (1965). Choice reaction with variable S-R m a p p i n g . Journal of Experimental
Psychology, 70, 284–288.
Simon, J. R., Craft, J. L., a n d Webster, J. B. (1973). Reactions toward the stimulus source:
Analysis of correct responses a n d errors over a five-day period. Journal of Experimental
Psychology, 101, 175–178.
Simon, J. R., a n d Small, A. M. (1969). Processing auditory information: Interference from an
irrelevant cue. Journal of Applied Psychology, 53, 433–435.
Sommer, W., Leuthold, H., and Hermanutz, M. (1993). Covert effects of alcohol revealed by
event-related potentials. Perception and Psychophysics, 54, 127–135.
St. James, J. D. (1990). Observations on the microstructure of response conflict. Perception and
Psychophysics, 48, 517–524.
Stoffer, T. H., and Umiltà, C. (1997). Spatial stimulus coding and the focus of attention in
S-R compatibility and the Simon effect. In B. Hommel and W. Prinz (Eds.), Theoretical issues
in stimulus-response compatibility, p p . 181–208. Amsterdam: Elsevier.
Stroop, J. R. (1935). Studies of interference in serial verbal reactions. Journal of Experimental
Psychology, 28, 643–662.
Sudevan, P., a n d Taylor, D. A. (1987). The cuing a n d priming of cognitive operations. Journal
of Experimental Psychology: Human Perception and Performance, 13, 89–103.
Teichner, W. H., and Krebs, M. J. (1974). Laws of visual choice reaction time. Psychological
Review, 81, 75–98.
272 Hommel
Telford, C. W. (1931). The refractory phase of voluntary a n d associative responses. Journal of
Experimental Psychology, 14, 1–36.
Toth, J. P., Levine, B., Stuss, D. T., Oh, A., Winocur, G., and Meiran, N. (1995). Dissociation
of processes underlying spatial S-R compatibility: Evidence for the independent influence
of what and where. Consciousness and Cognition, 4, 483–501.
Umiltà, C., a n d Liotti, M. (1987). Egocentric a n d relative spatial codes in S-R compatibility.
Psychological Research, 49, 81–90.
Umiltà, C., a n d Zorzi, M. (1997). Commentary on Barber and O’Leary: Learning a n d atten-
tion in S-R compatibility. In B. Hommel a n d W. Prinz (Eds.), Theoretical issues in stimulus-
response compatibility, p p . 173–178. Amsterdam: Elsevier.
Valle-Inclán, F., and Redondo, M. (1998). On the automaticity of ipsilateral response activa-
tion in the Simon effect. Psychophysiology, 35, 366–371.
Virzi, R. A., a n d Egeth, H. E. (1985). Toward a translational model of Stroop interference.
Memory and Cognition, 13, 304–319.
Welford, A. T. (1952). The “psychological refractory period’’ a n d the timing of high-speed
performance: A review and a theory. British Journal of Psychology, 43, 2–19.
Welford, A. T. (1968). Fundamentals of skill. London: Methuen.
Woodworth, R. S. (1938). Experimental psychology. N e w York: Holt, Rinehart a n d Winston.
Zachay, A. (1991). Diskrete u n d kontinuierliche Informationsverarbeitungsmodelle zur
Erklärung von Reiz-Reaktions-Inkompatibilitäten: Evidenz für einen Antwortkonflikt beim
Simon-Effekt. Master’s thesis, University of Tübingen.
Zorzi, M., a n d Umiltà, C. (1995). A computational model of the Simon effect. Psychological
Research, 58, 193–205.
273 The Prepared Reflex in S-R Translation
III Task Switching and Multitask Performance
12 Task Switching and Multitask Performance
Harold Pashler
ABSTRACT Research on task switching and dual-task performance has spawned two lit-
eratures that have, to a surprising extent, developed independently. This tutorial reviews
the principal findings of each tradition a n d considers how these phenomena may be re-
lated. Beginning with Jersild 1927, task-switching studies reveal that w h e n people per-
form two tasks in succession, with each task requiring different responses to the same set of
stimuli, substantial slowing occurs. Recent research suggests that while this slowing can be
partially ameliorated by allowing sufficient time between tasks, advance reconfiguration is
almost always incomplete. In studies of dual-task performance, stimuli are presented very
close together in time, and subjects attempt concurrently to perform two wholly distinct
tasks. A substantial slowing of one or both tasks is usually observed. The most stubborn
source of this slowing appears to be queuing of central processing stages, sometimes sup-
plemented by other kinds of interference. This queuing occurs even when the tasks are
highly dissimilar a n d is unlikely to reflect voluntary strategies. A number of possibilities for
h o w task switching a n d dual-task queuing might be related are discussed critically, includ-
ing the possibility that queuing might stem from an inability to maintain two distinct task
sets at the same time.
What happens w h e n people try to switch rapidly between one task a n d
another? What happens when they try to do more than one task at the
same time? The first of these two fundamental questions is chiefly dis-
cussed in a modest-sized literature u n d e r the label “task switching’’ or
“mental set’’; the second, in a much larger literature u n d e r the label
“divided attention’’ or “dual-task performance.’’ The present chapter
reviews main phenomena a n d theoretical issues in both areas a n d tries to
d r a w some substantive connections between them.
12.1 TASK SWITCHING
In 1927, well before the modern era of information-processing psychol-
ogy, an educational psychologist named Arthur T. Jersild published a
pioneering study of people’s ability to alternate between different
tasks. Jersild measured the total time it took a person to work through a
printed list of stimuli, making a response of some kind to each individual
item in turn. In pure task blocks, subjects performed the same task on
each item (for example, subtracting three from each number on a list).
In alternating-task blocks, subjects performed one task on all the odd-
numbered stimuli, and another task on the even-numbered stimuli. In
some of the experiments, every stimulus was a potential input for either
task (following Fagot 1994, I will refer to this arrangement of tasks
and stimuli as a “bivalent’’ list or mapping). One of Jersild’s bivalent
alternating-task lists contained two-digit numbers; subjects were in-
structed to subtract three from the first number, a d d six to the second
number, subtract three from the third, and so forth. They were sub-
stantially slower (more than 0.5 sec per item) in bivalent alternating lists
than in pure lists, sometimes by more than 1 sec per item. This difference
between pure and alternating bivalent lists will be referred to as the
“alternation cost.’’
Jersild also examined the case of task alternation, where each stimulus
was a potential input only for the appropriate task (henceforth referred to
as a “univalent’’ list or mapping). For example, one univalent alternating
list contained two-digit numbers and words, numbers alternating with
words; subjects were instructed to subtract three from each number and
to say aloud the antonym of each word. Remarkably, Jersild found that
there was no alternation cost at all with these univalent lists; indeed, sub-
jects were actually slightly faster in responding to alternating lists than to
pure lists.
Some fifty years later, Spector and Biederman (1976) confirmed
Jersild’s basic results, finding a modest-sized benefit of alternation with
univalent lists. This occurred, however, only when the items were printed
as in Jersild’s studies, and subjects were allowed to preview items ahead
of the ones they were responding to.1 When the items were placed on
cards, so that subjects could not see the next stimulus until they turned a
card over, there was actually a small alternation cost; the same was true
when the experiment was run with a discrete-trials procedure. With the
alternating bivalent lists (adding three, then subtracting three, etc.),
Spector and Biederman found a large alternation cost (402 msec/item).
This was cut about in half, to 188 msec/item, when a visual task cue
(“+3’’ or “—3’’) was placed next to each item.
Several rather trivial potential explanations for the basic alternation
cost need to be considered.
One might propose that the alternation cost merely reflects a tendency
of subjects occasionally to forget what task they should perform next. If
this is correct, the slowing should largely be confined to a few, very slow
trials. This does not seem to be the case, however. Fagot (1994) had
subjects make button-push responses to either the color or the identity
of a letter (an A, B, or C in red, green, or blue). Figure 12.1 shows the
Vincentized reaction time (RT) distributions for a zero response-stimulus
interval (RSI) condition where the two tasks were performed in alterna-
tion.2 The slowing is by no means confined to the slowest responses.
Evidently, then, among the sources of the alternation cost are events that
occur on at least a significant number of trials.
Pashler
Figure 12.1 Vincentized reaction time (RT) distributions for a bivalent list alternating-task
design. Alternation cost appears even among the fastest responses. From Fagot 1994;
reprinted with permission.
One might also propose that the faster responses to p u r e as opposed to
alternating lists arise because alternating lists do not include any stimu-
lus repetitions. In any speeded-choice task, people respond much faster
to stimuli that match whatever was presented on the preceding trial
(Kornblum 1973; Pashler a n d Baylis 1991). This potential confound does
not explain the effect, however. In the experiment by Fagot (1994) shown
in figure 12.1, lists were selected with the constraint that there be no item
repetitions, but the alternation cost was still found; the same was proba-
bly done informally in some of the earlier studies.3
What, then, accounts for the alternation cost with bivalent lists, a n d
w h y is this cost sometimes virtually absent with univalent lists? Perhaps
the most obvious interpretation is that depicted in figure 12.2. According
to this “task set reconfiguration’’ (TSR) view (Monsell 1996), preparing to
perform a task involves linking a n d / o r configuring different processing
modules. Different modules are assumed to be responsible for different
aspects or stages of the task (e.g., perception, response selection, etc.).
With bivalent lists, task alternation requires changing the links, settings,
or both between when the central processing of one stimulus is com-
pleted a n d when the central processing of the next begins. In some
cases, changes in the configuration of perceptual modules may also be
involved. Given the conflicting response selection rules in the case of
bivalent stimuli, the module responsible for response selection cannot be
set the same way throughout the block of trials. At first blush in this
account, one would assume that the alternation cost simply reflects the
time needed to complete the switch. As for univalent alternating tasks, it
should be possible for the t w o task mappings to coexist more or less
Task Switching a n d Multitask Performance
Figure 12.2 Discrete conception of task set switching. The response selection machinery is
prepared at any one time to perform either of the two incompatible mappings, but not both.
In the alternating-task blocks, one mapping is switched out a n d the other inserted, some-
what as a crystal in early radio sets.
happily, so that the union of the two mappings could simply be loaded
into the response selection module. This may explain w h y there should
be minimal cost in that situation, although of course by itself it does not
explain why there should ever be a benefit.
If this account is correct, allowing extra time between the response to
stimulus n and the presentation of stimulus n + 1 (RSI) might allow sub-
jects to complete the switch in advance, thereby reducing or eliminating
the alternation cost. Many recent studies of task alternation have found
some reduction. A notable example is Rogers and Monsell 1995, which
found an approximately 50% reduction as RSI was lengthened from 150
msec to 1,200 msec, so long as subjects could rely on having the long RSI.
In Fagot 1994, conducted in my own laboratory, subjects were instructed
to respond to colored letters by pushing a button to indicate either the
color of the letter or its identity. In alternating-task blocks, RSI varied
from 0 to 1.5 sec; the alternation cost fell from 314 msec at the zero RSI to
201 msec at the 1.5 sec RSI, with most of the reduction occurring over the
range of RSIs between zero and 400 msec; this pattern was confirmed in
several other experiments within that series.
280 Pashler
130CW
x ~.— ™
V %* AABB
(A) (B)
AAAA
T 1 1 1 '
1 2 3 4
Relative position in s e q u e n c e
Figure 12.3 Reaction time (RT) results from an AABB task (Fagot 1994), redrawn in Pashler
1997. Subjects are faster on second performance of a given task, but still not so fast as in a
pure block of trials.
As Rogers and Monsell (1995) point out, the alternation cost (difference
between pure and alternating-task blocks) is likely to include several
factors in addition to reconfiguration time. For example, there might be
slowing d u e to the processing “overhead’’ needed to maintain and imple-
ment the intention to alternate. Because concurrent memory loads gener-
ally slow performance in reaction time tasks (Logan 1978), it seems
reasonable to expect that holding onto a plan for alternating would
impose a memory load of its own. In addition, differences in effort or
arousal cannot be ruled out.
To help tease apart these factors, Rogers and Monsell used an
“alternating-runs’’ procedure, wherein subjects performed first one task
a number of times, then the other, and so forth. A pair of characters
was presented on each trial, one a letter and the other a digit. Subjects
either classified the letter as a vowel versus consonant, or the digit as odd
versus even. Sometimes each task was performed twice in succession
(AABB). The first response within a run of a given task (AABB) was sub-
stantially slower than the second (AABB), even at the long RSI. This was
later confirmed by Fagot (1994) using the color and letter tasks described
above. In AABB lists, subjects were required to perform the color task
twice, the letter task twice, and so forth. Fagot also included pure blocks
of trials and alternating (ABAB) blocks for comparison. As seen in figure
Task Switching a n d Multitask Performance
12.3, the first performance of a given task (AABB) was close to the ABAB
blocks, but slower than the second performance (AABB), as in Rogers a n d
Monsell’s data. The second performance w a s still quite a bit slower than
the pure block (AAAA), however, suggesting that the overhead cost is
nontrivial.
In a further example of the stubbornness of the residual task switch
cost at long RSIs, Goschke (chap. 14, this volume) allowed subjects 1.5 sec
between two colored letters, each of which was to be classified by color
or shape, and found responses were substantially slower when a differ-
ent task h a d to be performed on the second letter.
Thus it appears, as Rogers a n d Monsell argued, that several factors
play a role in the basic Jersild alternation effect. From the standpoint of
conventional thinking in information-processing psychology, probably
the most surprising of these factors is the switch cost, which persists even
after ample time has been provided for reconfiguration. Some clues about
the nature of this residual switch cost come from an additional experi-
ment by Rogers a n d Monsell (1995), in which subjects performed a task
four times in succession, then switched and performed the other task four
times, and so forth. Performing a given task initially produced a substan-
tial speedup for the second response, but over the next two responses,
no additional improvement was detected (see figure 12.4). The authors
concluded that the gain observed from performing the task once could
not be attributed to “micropractice’’—a small dose of the same optimiza-
tion process that, over many trials, yields the familiar practice effect. After
all, they reasoned, such a process could hardly reach an abrupt a n d final
asymptote after one trial, as these data seem to show. The empirical basis
for this conclusion has recently been challenged, however, by Salthouse
et al. (1998), w h o h a d subjects switch tasks a n d then perform fairly long
runs of a different task. They found RTs for the second trial within a r u n
h a d still not reached a baseline in their data, and argued that Rogers a n d
Monsell may have h a d insufficient experimental power to detect this con-
tinuing decline.
Cuing the Task Set
Control over task set is also illuminated by experimental designs in
which the subject cannot tell which task to perform until a task cue is pro-
vided. Following in the footsteps of Shaffer (1965), Sudevan a n d Taylor
(1987) h a d subjects perform one of two different tasks involving a digit.
One task required classifying the digit as o d d or even, while the other
required classifying it as less than six or greater than five (bivalent m a p -
ping). The cue preceded the digit by an interval ranging between 400
msec and 4 sec. Responses became faster a n d more accurate as the inter-
val w a s lengthened to about 2 or 3 sec. In his color/identity design, Fagot
(1994) examined cue-target intervals ranging from zero to 4 seconds, a n d
Pashler
Figure 12.4 Reaction times (RT) a n d error rates in Rogers a n d Monsell 1995, experiment 6,
as a function of position in a r u n of four trials (subjects performed one task four times, then
switched to the other task; redrawn from Rogers and Monsell 1995, fig. 5).
found that the bulk of the benefit (over 200 msec) occurred over the range
from 0 to 500 msec, with some further improvement out to about 1 sec;
thereafter, performance w a s little changed. Other studies (e.g., Logan a n d
Zbrodoff 1982) have also found a similar time course using cues that are
helpful but not strictly necessary in performing the task. As Rogers a n d
Monsell (1995) point out, one cannot directly derive an estimate of the
time needed for reconfiguration based on these kinds of experiments,
because at the shortest cue-target interval, the time needed to read a n d
interpret the cue is presumably slowing responses, along with the re-
quirement to reconfigure.
Recall that in the alternating-task procedure described earlier, the first
response within a r u n of t w o successive instances of the same task is
slower than the second response, even with an ample RSI. Based on that
result, we would naturally expect that in the cuing procedure, no matter
Task Switching a n d Multitask Performance
h o w long the cue-target interval, responses w o u l d be slower when the
previous trial involved the other task. This is indeed the case. For exam-
ple, Fagot (1994) presented task cues in blocks with a r a n d o m or a fixed
task sequence (either alternating or nonalternating). Even when subjects
h a d four seconds to use the task cue, there was still a benefit of having
performed the same task on the preceding trial; as expected, there was an
additional benefit of having a fixed sequence.
In a clever recent study, Meiran (1996) cued subjects to respond to the
vertical or horizontal position of a disk; the task varied from trial to trial
within a block. Task cues (arrows pointing either up a n d d o w n or left a n d
right) appeared about 200 or 1,400 msec prior to onset of the imperative
stimulus. Subjects were slower when they had to perform a different task
from one trial to the next. This difference w a s substantially greater at the
short cue-stimulus interval than at the long interval, but did not disap-
pear at the longer interval. Meiran argued that the reduced task alterna-
tion effect produced by increasing the cue-target interval did not occur
merely because lengthening this interval m a d e the previous task more
distant in time, reducing its impact by passive decay. When the interval
between the previous response and the cue was decreased to make up for
the increase in the cue-target interval, thereby holding the RSI constant,
the longer cue-target interval still reduced the effect of a task switch. This
strongly suggests that some, albeit incomplete, advance reconfiguration
is indeed taking place.
Incompleteness of Reconfiguration
We have seen that in both the alternating-task procedure a n d the task-
cuing procedure, providing subjects plenty of time to prepare reduces the
cost of having to perform a task different from the one they just per-
formed (in the bivalent situation), but it does not allow them to respond
as quickly as if no switch of task h a d been required. This residual differ-
ence cannot be attributed to overhead cost because it appears also with
the alternating-runs procedure (e.g., Rogers a n d Monsell 1995) as well as
with the task-cuing procedure (e.g., Meiran 1996).
Why should there be a residual switch cost? Why is reconfiguration
incomplete? De Jong (chap. 15, this volume) asked whether the residual
switch cost stems from a constant slowing that appears on all trials or
from a slowing that arises on only a fraction of the trials. Using the
alternating-runs procedure of Rogers and Monsell, he had subjects clas-
sify colored letters according to either color or identity (consonant versus
vowel). He found little evidence for a constant slowing component at the
long RSI, a n d argued that incompleteness of reconfiguration is at least
avoidable under certain conditions. His results may not rule out the pos-
sibility, however, that residual cost is always present, but imposes a delay
whose magnitude varies from trial to trial.
Pashler
Meiran (chap. 16, this volume) proposes that residual shift depends on
a feature of certain switching designs not discussed thus far, namely,
“ambiguity of responses,’’ the use of an overlapping set of responses in
the two tasks. As in his earlier experiments described above, Meiran used
a design in which subjects respond to either the vertical or the horizontal
position of a disk placed in one of four quadrants of the display. When
the t w o tasks used the same two response keys (ambiguous responses),
there was a positive residual task switch cost; when the responses were
separate, the residual cost was reduced or absent.
According to Meiran, task preparation may involve not only the selec-
tive amplification or enabling of particular stimulus-response links, as
depicted in figure 12.2, but also the selection of a response set, which
can only be achieved by actually performing the task. This proposal is
intriguing, and receives support from the data reported in this volume,
although there are cases in the literature where residual task-switching
costs have been observed even when two tasks did not involve “ambigu-
ous’’ responses. For example, Fagot (1994) observed residual switch costs
for mappings both with the same keys and with corresponding keys of
different h a n d s .
An alternative view of the residual cost of a task switch is that it results,
not from the need to perform a time-consuming control process on the
switch trial (as the authors described above have assumed), but from a
prolongation on switch trials of the response selection process that hap-
pens on all trials. This prolongation, is caused by competition d u e to posi-
tive or negative priming of task sets or of S-R associations from previous
trials on which the other task was performed. Such a view was first pro-
posed by Allport, Styles, and Hsieh (1994), a n d a new version of it is pre-
sented by Allport a n d Wylie’s chapter (chap. 2, this volume), to which the
reader is referred for arguments a n d evidence. It seems clear from Allport
a n d Wylie’s work that there are carryover effects from recently perform-
ing the alternative task in response to the same stimulus or class of stim-
ulus. What is not clear, however, is whether these carryover effects are
sufficient to account for the dramatic d r o p in RT from the first to the sec-
on d trial after a task swtich. Further, the notions of priming effects a n d
control processes are by no means mutually exclusive.
Task Congruity Effects
The incompleteness of reconfiguration is revealed, not only by residual
switch costs that persist despite long RSIs, but also by persisting effects of
the purportedly disengaged mapping. Recall that Rogers and Monsell
(1995) h a d subjects respond to either the letter or the digit in a letter-digit
pair, using an alternating-runs procedure. The authors examined reaction
times as a function of whether the irrelevant item in the pair would,
according to the irrelevant (supposedly inactive) task mapping, yield the
Task Switching and Multitask Performance
same response as that required on the current trial. The trial was called
“congruent’’ when it did, a n d “incongruent’’ when it did not. There was
a modest but significant tendency for slower responses on incongruent
trials than on congruent trials, although responses trials with neutral
stimuli were faster still. Similarly, in Fagot’s color/letter design, where
subjects responded either to color or to identity, responses were about 90
msec slower when the other feature w a s associated with a response inap-
propriate on the current trial. These congruency effects imply that the
“competing task set is not entirely disabled’’ (Rogers a n d Monsell 1995,
216).
There is some controversy about whether the competing task set can be
disabled when a sufficiently long RSI is provided. In their experiment 3,
Rogers and Monsell found no significant reduction in the congruence
effect (measured, as usual, in RTs) when they increased the R-S interval,
although there was a marginally significant interaction in the error rates.
Similarly, in three experiments, Fagot (1994) found only a weak reduction
in congruency effects. By contrast, Meiran (1996), using his location but-
ton tasks, found a strong interaction, with congruence effects reduced but
not eliminated. Finally, Sudevan and Taylor (1987) reported that congru-
ence effects with their digit task disappeared at long cue-target intervals,
while Goschke (chap. 14, this volume) reports having nearly eliminated
the effect of task congruence with a long, unfilled RSI and after practice.
Unhappily, then, the results r u n the full gamut from complete persistence
of the congruence effect at a long RSI all the way to virtual disappearance.
This issue remains to be sorted out.
Conclusions
Evidently, w h e n subjects anticipate the need to perform a task incompat-
ible with the one they just performed (as in the case of a bivalent list),
whether this anticipation is based on the requirement to alternate (as in
the Jersild paradigm a n d its spin-offs), or on the perception of a cue
telling them to perform a task different from the one they just performed,
some advance reconfiguration can occur, as depicted in figure 12.2. With
the sorts of simple but arbitrary tasks studied in this literature, this
reconfiguration usually seems to take under 0.5 sec when subjects have
no other intervening task to perform. Reconfiguration may be accompa-
nied by verbalization, usually covert, of the instructions for the upcoming
trial.
The notion of advance reconfiguration illustrated in figure 12.2 seems
to have some validity, but it misses important aspects of task switching.
First, advance reconfiguration usually fails to eliminate the costs of hav-
ing just performed a different task. Even with ample RSIs or cue-target
intervals, subjects are still typically slower w h e n they must perform a
task different from the one they just performed (although Meiran’s design
Pashler
reveals at least some exceptions). Actually performing a task once allows
a significant amount of additional reconfiguration or tuning to take place.
Rogers a n d Monsell refer to the tuning before first performing a task as
the “endogenous component’’ of task preparation, and to that after per-
forming the task as the “exogenous component.’’ Although their data h a d
suggested that exogenous reconfiguration is complete after one trial, sub-
sequent data (Salthouse et al. 1998) suggest it may not be entirely com-
plete until two trials.
Perhaps the most intriguing aspect of task switching is the lingering
effect of the irrelevant m a p p i n g — t h e “task congruity effect.’’ Not only
advance reconfiguration, but indeed all reconfiguration accomplished up
to the point of selecting a response in the new task seems incapable of
wholly disabling the old mapping. While task congruity effects have on
some occasions been observed to disappear with adequate preparation
time, as noted above, more commonly they seem to persist, at least to
some extent (an issue discussed in detail by Allport a n d Wylie, chap. 2,
this volume).
12.2 DUAL-TASK PERFORMANCE
We turn n o w to the limitations that arise w h e n people attempt to perform
two different tasks at the same time. Our focus here will be on discrete
tasks; with more continuous tasks, interference and switching are easily
disguised for reasons that will emerge clearly below. Not surprisingly,
limitations on simultaneous mental operations evidently arise at various
different functional loci. Perceptual analysis of multiple stimuli often
takes place in parallel, with capacity limitations sometimes becoming evi-
dent when perceptual d e m a n d s exceed a certain threshold (Pashler 1997)
although nonperceptual factors (such as statistical noise in search
designs) often masquerade as capacity limitations (Palmer, 1995). These
limitations appear largely, but probably not entirely, modality specific
(Treisman a n d Davies 1973; Duncan, Mertens, a n d Ward 1997). Similarly,
response conflicts arise when responses must be produced close together
in time. These perceptual limitations are often most acute when similar or
linked effectors are used, such as the two h a n d s (Heuer 1985).
The most intriguing, a n d for the present topic most relevant, limita-
tions arise in central stages of decision, memory retrieval, and response
selection. Intuitively, most laymen assume that the cognitive aspects of
two tasks can be performed simultaneously unless one or both are intel-
lectually demanding. That this is not the case, however, is most clearly
seen w h e n people try to carry out two speeded but relatively simple
tasks, each requiring a response to a separate individual stimulus. As
Telford (1931) first observed, people almost invariably respond more
slowly to the second stimulus when the interval between the two stimuli
is reduced. Telford called this the “psychological refractory period’’ (PRP)
Task Switching and Multitask Performance
Figure 12.5 Schematic diagram of the psychological refractory period (PRP) design and
idealized pattern of data (hypothetical numbers).
effect, by analogy to the refractory period of neurons. Though the anal-
ogy is probably not very apt, the label has stuck. In the PRP design, two
stimuli (S1 and S2) are presented, their onsets separated by some stimu-
lus onset asynchrony (SOA). The person makes a separate response to
each stimulus (R1 and R2, respectively). Figure 12.5 (idealized data)
shows the type of result usually obtained; the reaction time between S2
and R2 (RT2) grows as the SOA is shortened. Meanwhile, the reaction
time between S1 and R1 (RT1) is usually relatively constant, although this
depends on the instructions (see below). In some cases, the slope relating
RT2 to SOA is as extreme as —1, which means that any reduction in SOA
beyond a certain point merely increases RT2 by the same amount. To p u t
it differently, presenting S1 and S2 closer together in time (once the
interval reaches some minimum value) often fails to result in R2 being
produced any earlier. Another important observation is that while pro-
cessing required by the two tasks resists being “compressed’’ beyond a
certain point, at short SOAs, the total time required to carry out both
tasks (the interval between S1 and R2) is often substantially less than the
sum of the times required to complete each task separately. In short, there
is a saving in the total time for completing the two tasks, suggesting over-
lap in some aspects of processing.
The PRP effect has been observed in many different tasks, including
simple reaction time (as in Telford’s studies) and choice reaction time
tasks (starting with Creamer 1963). Although early PRP experiments
Pashler
mostly used pairs of manual responses, sometimes m a d e with the same
finger, the effect can also be found w h e n the two tasks involve com-
pletely different effectors. For example, PRP effects have been found with
tasks combining manual and oculomotor responses (Pashler, Carrier, a n d
Hoffman 1993), manual and vocal responses (Pashler 1990), manual a n d
foot responses (Osman a n d Moore 1993), a n d vocal and foot responses
(Pashler a n d Christian 1994). Thus it is not necessary for two tasks to use
a common motor control system in order for a PRP effect to be observed.
The PRP effect is also found w h e n the two stimuli involve different sen-
sory modalities. For example, Borger (1963) a n d Creamer (1963) found
PRP effects with visual a n d auditory stimuli, as have many more recent
researchers. It is not clear whether the PRP effect is greater when S1 a n d
S2 are presented in the same modality; this is hard to determine because
changes in input modality are typically confounded with differences in
the compatibility of the task mapping.
Limits of the Psychological Refractory Period Effect
The PRP effect is very robust, but over the past twenty-five years or so, a
number of exceptions have emerged. Greenwald and Shulman (1973;
Greenwald 1972) found that the effect of SOA on second-task RTs was
virtually eliminated when one task involved repeating a spoken word
(shadowing) and the other involved a highly compatible visuomanual
task. They hypothesized that “ideomotor compatibility,’’ the fact that the
stimulus mimics the feedback produced by the response, might be criti-
cal. Although McLeod and Posner (1984) demonstrated noninterference
with combinations of shadowing and other tasks in ways that seemed
consistent with this proposal, other research suggests ideomotor compat-
ibility is probably not sufficient to eliminate interference. For example,
Brebner (1977) devised a novel ideomotor-compatible task, requiring
subjects to press a button in response to u p w a r d pressure from a solenoid
located u n d e r the corresponding finger. When task 1 involved left-hand
stimulation and task 2 involved right-hand stimulation, a clear-cut PRP
effect was observed. Tasks requiring a saccadic eye movement toward a
single spot, or even the generation of an eye movement in response to
a single stimulus based on its color, seem not to generate PRP effects
(Pashler, Carrier, a n d Hoffman 1993). Visuomanual tasks with very high
spatial stimulus-response compatibility may also be free of central inter-
ference (Koch 1994). At present, then, the conditions under which the
PRP effect disappears are not well characterized. Indeed, it seems that
dual-task interference in pairs of punctate tasks can be eliminated only
with tasks that are, intuitively speaking, extremely natural and easy.
Whether the critical factor is the existence of prewired neural circuits that
bypass central machinery, a high degree of practice, or some combination
of these factors remains u n k n o w n . Perhaps the more significant point is
Task Switching and Multitask Performance
Figure 12.6 Central bottleneck account of the psychological refractory period (PRP) effect.
that it is easy to find tasks with minimal cognitive d e m a n d s that produce
robust PRP effects.
Sources of Dual-Task Slowing
Based largely on observations of PRP interference even where there is
no overlap in stimulus or response modality, Welford (1952, 1980) pro-
posed that dual-task slowing arises from a bottleneck in what he called
“stimulus-response translation’’—in more modern parlance, the stage of
“response selection.’’ The basic idea is illustrated in figure 12.6.
According to this hypothesis, each task is composed of three broad stages
(perception, response selection, a n d response execution); any stage of
task 1 can overlap any stage of task 2, except for the shaded stage of
response selection: while one response is being selected, selection of the
other response must wait. As formulated, however, the hypothesis does
not say what should happen in tasks more complicated than choice RT,
where one often encounters mental operations that do not obviously fall
into any one of the three categories.
From this account, one can derive very specific predictions for the
results of dual-task experiments in which different stages of task 1 or task
2 are selectively prolonged. Increases in the duration of stages of task 1
up to a n d including the shaded stage should, at short SOAs, propagate
a n d slow task 2 as well as task 1. Increasing the duration of the post-
bottleneck stages of task 1, on the other hand, should slow only task 1,
regardless of the SOA. Increasing the duration of stages in task 2 prior to
the bottleneck should correspondingly slow the second response at long
SOAs. At short SOAs, on the other hand, there is “slack’’ because the
Pashler
response selection in task 2 is not waiting for completion of perception in
task 2, but rather for the completion of response selection in task 1. The
result of the slack is that, at short (but not at long) SOAs, the perceptual
slowing should cease to affect RT2. The prediction, then, is that manipu-
lations of the prebottleneck processing stages in task 2 should exhibit
underadditive interaction with SOA (see Jolicoeur, Dell’Acqua, a n d
Crebolder, chap. 13, this volume, for further details a n d examples).
Lengthening the duration of stages at or after the shaded portion of task
2, on the other hand, should always slow R2 to the same extent, regard-
less of SOA.
These predictions have been confirmed in many experiments involving
fairly elementary choice RT tasks (for a review, see Pashler 1997). The pre-
dictions are distinctive in the sense that they not only favor the central
bottleneck, but also rule out accounts that would place the bottleneck ear-
lier or later in the sequence of processing stages. Several of the results also
seem unfavorable to graded capacity-sharing models, especially the fact
that increases in first-task response selection difficulty have at least as
large an effect on RT2 as it has on RT1 (e.g., Broadbent and Gregory 1967).
If task 1 were being performed with depleted capacity, and the manipu-
lations increased the capacity required to carry out the stage in question,
one would expect to see a greater effect on RT1 than on RT2 (see Pashler
a n d Johnston 1998 for discussion).
Much recent work within the bottleneck framework has focused on the
question of exactly which processes are subject to this limitation, a n d
which are not. Manipulations of the duration of sensory processing in
task 2 (e.g., contrast) show the underadditive pattern indicating that the
stages affected are not subject to the bottleneck (Pashler 1984; De Jong
1993). Johnston and McCann (forthcoming) degraded letters by making
them very squat or very narrow without altering stroke widths a n d con-
trast. In another experiment, they altered the tilt of strokes composing the
letters (for instance, rotating the diagonal segments in the letter A inward
so that the character looked something like a teepee). At long SOAs, these
distortions slowed RT2 by about 30 msec. At short SOAs, however, RTs
for distorted and undistorted were indistinguishable, suggesting absorp-
tion into slack. It seems likely, therefore, that letter identification, not
merely visual feature extraction, can occur on task 2 while critical stages
of task 1 are u n d e r way. On the other hand, when perceptual processing
d e m a n d s on task 2 include not just identifying stimuli, but performing
additional manipulations such as mental rotation or comparisons, these
operations are usually subject to the central bottleneck (Ruthruff, Miller,
a n d Lachman 1995).
Recent evidence suggests that, not merely the planning of actions
based on task-mapping instructions or difficult perceptual manipula-
tions, but memory retrieval overall is subject to queuing. Carrier a n d
Pashler (1996) combined a manual response to a tone (task 1) with paired
Task Switching and Multitask Performance
associate retrieval cued by a visually presented word (task 2) in a PRP
design. The duration of the memory retrieval w a s manipulated by vary-
ing the amount of practice subjects h a d carrying out any particular
retrieval. Second-task RTs were, not surprisingly, faster for better-learned
pairs. In the dual-task situation, this difference appeared additive with
SOA (Carrier and Pashler 1996). Following the logic described above, this
implies that memory retrieval was postponed by first-task processing
a n d refutes the claim that only the execution of the motor response is
delayed. The latter point seems especially clear because of the greater
difficulty of task 2 compared to task 1. In the short (50 msec) SOA condi-
tion, subjects responded to the tone about 600 msec after it was presented;
the paired-associate task was far more challenging, however: on average,
the paired-associate response did not occur until about 1,100 msec later.
If all interference were response related, it is hard to see what could be
postponing a second response so temporally remote from the first. The
results are to be expected, however, if one assumes that the central bottle-
neck encompasses both response selection in task 1 a n d memory retrieval
in task 2 (and perhaps response selection as well, if that is a separate stage
in this sort of task).
It seems to me a reasonable conjecture that the inability to select
two responses at the same time, which is apparent in choice RT tasks
(Welford’s response selection bottleneck), may be just a special case of a
broader constraint, namely, that two retrievals cannot be carried out at
the same time. Within the confines of the choice RT experiment, it is an
action plan that is to be retrieved, whereas in other situations, it may, for
example, be a word or concept or episode. While the proposed constraint
can be expressed very simply, it stands in great need of explication. For
example, what is meant by “two retrievals’’? If two stimuli are presented,
each associated with the same single response, does the lookup of that
single response based on the t w o stimuli constitute two retrievals or one?
In choice RT tasks, two redundant stimuli produce what Miller (1982)
calls “coactivation,’’ a particularly strong form of parallel processing. The
same is almost surely true of more time-consuming memory retrieval
operations. What about one stimulus associated with two responses?
Timothy Rickard and I (Rickard and Pashler 1998) trained subjects in one
phase of training to associate each item on a list of ten words with a cor-
responding verbal paired associate, a n d then, in a second phase of train-
ing, to associate each item on the same list with a manual response.4 In a
final test phase, subjects were sometimes instructed to carry out both
retrievals at once. Whichever response was produced second had on
average a latency that was about twice as long as the single-task control.
Other aspects of the data also argued that the retrievals were carried out
sequentially. Thus, for the purpose of the proposed constraint, it is the
number of outputs, not the number of inputs, that determines whether a
single retrieval or multiple retrievals are required.
Pashler
The term retrieval also needs clarification. A priori, one might have
described letter identification, for example, as involving the retrieval of
the letter identity corresponding to a visually presented character. Yet I
have argued that object identification is not subject to the bottleneck.
What differentiates retrieval from classification or identification? At this
point, the answer must be vague: it seems that the operations subject to
queuing involve retrieving some mental contents that are distinct from
the input in that they are not an internal description of the input but some
separate contents. Sharpening up this description will require at the very
least testing a broader range of different types of retrievals in different
dual-task contexts; conceivably, it will also require a better understanding
of the neural substrates of these processes.
Strategic Interpretations
The apparent inability to execute the central stages of even fairly easy
tasks concurrently is surprising from both an intuitive and a computa-
tional standpoint. It has recently been argued that postponement of cen-
tral processing in the PRP design stems not from a fundamental inability
to carry out the two tasks at the same time, but rather represents a strate-
gic response to the explicit or implicit d e m a n d s of the experiment. This
idea has been developed in detail by Meyer a n d Kieras (1997), w h o pro-
posed an ambitious theory of h u m a n performance (“executive process
interactive control’’ or EPIC), discussed in detail by Kieras et al. (chap. 30,
this volume). According to EPIC, there are no intrinsic limitations what-
ever in the ability to select responses or carry out memory retrievals con-
currently. There are, however, structural limitations in the initiation a n d
execution of responses. In addition, postponement of central processing
(i.e., queuing of processing stages) may occur whenever subjects perceive
this to be advantageous.
Why would subjects adopt a queuing strategy in a dual-task design
when doing so means responding more slowly in one or both tasks? As
Meyer and Kieras note, in m a n y PRP experiments, subjects have been
told to produce R1 as fast as possible (and even, in a few cases, to produce
R1 before R2). Primarily, this has been done in order to avert the “group-
ing’’ strategy that people naturally fall into, whereby R1 is buffered a n d
then emitted shortly before R2 (Borger 1963; Pashler a n d Johnston 1989,
exp. 2). Given a strong emphasis on first-task speed, subjects might
choose not to select the two responses in parallel because doing so might
result in responding to task 2 before task 1.
One obvious question, then, is what happens when there is no empha-
sis on the speed of the first task and subjects try to respond to both tasks
as quickly as possible. A number of studies that did not emphasize first-
task speed have nevertheless shown evidence of central postponement.
For example, in Carrier a n d Pashler 1996, even though subjects were not
Task Switching and Multitask Performance
told to emphasize the speed of the first response, both slowing of R2 a n d
postponement of central processing were observed. Similarly, in one of
their experiments, Ruthruff, Miller, a n d Lachmann (1995) did not em-
phasize first-task speed but nonetheless found evidence of a central
bottleneck.
There are also some other, rarely cited studies in which investigators
looked at performance of t w o serial choice RT tasks, where subjects are
instructed to respond to a train of signals in each task, rather than to two
discrete signals, as in the PRP design. Here the order of responses is
entirely up to the subjects, w h o simply attempt to achieve as much
“throughput’’ as possible in each task. Gladstones, Regan, a n d Lee (1989),
for example, h a d subjects perform serial tasks paced by the experimenter
(e.g., pressing a key in response to the position of a light and pronounc-
ing a letter in response to the color of a light). In some conditions, subjects
performed just one such task, whereas in others, they performed two
concurrently. The total rate at which information was processed s u m m e d
over the two tasks (which corresponds roughly to the total number of
responses in either task per unit time) was the same whether one task was
performed or t w o . This w a s true even after considerable practice, a n d
regardless of whether the tasks used the same or different input a n d out-
p u t modalities. Similar findings were reported by Fisher (1975a,b) a n d
Schouten, Kalsbeek, a n d Leopold (1960). Although, following Meyer a n d
Kieras (1997), some interference might be expected d u e to conflicts in the
initiation of responses, a bottleneck confined to response-related process-
ing should allow a dramatic increase in total throughput rate to be
achieved when two tasks are performed, instead of one.
My colleagues a n d I recently carried out other kinds of studies using
discrete tasks to examine whether central queuing is strategic in origin. In
one study, Eric Ruthruff, Alwin Klaassen, a n d I instructed subjects to per-
form two tasks and group the responses close together in time, a require-
ment subjects find quite natural. One task required judging whether a
figure was a normal or a mirror image letter a n d making a corresponding
keypress response. The other task, which could be performed more
quickly, involved discriminating between a single 17 msec tone a n d a
rapid-fire sequence of t w o 17 msec tones separated by 50 msec, with a
vocal response (saying “one’’ or “two’’). The first tone a n d the letter
began simultaneously.
The instruction to group the two responses obviously does not provide
any incentive to perform one task before the other. If there is no interfer-
ence between the decision or response selection phases of the two tasks,
the response should almost always be selected more quickly in the easier
task, normally the tone judgment. Thus the grouped response should
only be a bit slower than the response for the letter task alone, d u e to
occasional trials in which the letter task happens to take longer than the
tone task, plus any cost associated with grouping. In fact, there w a s very
Pashler
substantial slowing of mean RTs (1,475 msec for the dual-task grouped
response, compared to 917 msec for the letter task alone). Monte Carlo
simulations disclosed that this slowing could not be accounted for by the
fact that the tone task was occasionally slower than the letter task. It is
also not likely to reflect extra time taken to produce a grouped response;
costs of producing grouped responses can be assessed directly, a n d prove
negligible (e.g., Pashler a n d Johnston 1989, exp. 2).
As a further test, the difficulty of response selection in the easier task
was varied: in compatible blocks, subjects responded by saying “one’’ to
the single tone pulse, a n d “two’’ to the two pulses; in incompatible
blocks, the mapping was reversed, producing about 200 msec of slowing.
If central processing on the easier task were carried out in parallel with
central processing on the harder task, much of the slowing of the tone
task should be absorbed in “slack,’’ a n d thus have minimal effect on the
time to produce the grouped responses. In reality, compatibility had at
least as large an effect on the grouped response in the dual-task context
as it h a d on performance of the tone task by itself. Thus the whole pattern
of results in this experiment favors the idea that central queuing was
occurring in a situation where parallel processing would clearly have
been advantageous.
In another recent study, Levy and I required subjects to make a three-
alternative button-push response to the color of a large disk presented on
a monitor screen, a n d to make a vocal response to its position (saying
“one’’, “two’’ or “three’’ for left, middle, or central position). Here, rather
than using grouping, we provided explicit payoffs designed to promote
parallel processing a n d to place equal emphasis on the speed of each task.
On blocks where both stimuli were presented, average reaction times for
both tasks exhibited substantial slowing. Again it appears that encour-
agement to prioritize one task more than the other is by no means a
necessary condition for dual-task interference to occur.
12.3 RELATING DUAL-TASK INTERFERENCE AND TASK SET
Having very briefly and selectively reviewed some of the main phenom-
ena in the area of task switching a n d central limitations in simple dual-
task performance, let us consider possible relations between the two
topics. The research on dual-task interference bears on the issue of task
set a n d task switching in several interesting respects. Two of these will be
discussed here. The first is a very broad question of cognitive architec-
ture: Do the phenomena of task set reconfiguration a n d dual-task inter-
ference (and specifically the sort of central queuing argued for in section
12.2) singly or jointly imply the existence of a “central executive’’ or
“supervisory attention system’’? The second question is narrower: Does
the bottleneck itself reflect a limitation in task set, and perhaps the same
limitation as is responsible for task-shifting costs, in which case the phe-
Task Switching and Multitask Performance
nomena of dual-task queuing a n d task switching might really be one a n d
the same?
Many writers have assumed that cognitive control requires the exis-
tence of a specific controlling mechanism whose function is to program
(other) cognitive machinery. As discussed in several chapters in this vol-
ume, this controlling function is often associated with the frontal lobes or
specific parts thereof. Several well-known theoretical frameworks in cog-
nitive psychology, such as Baddeley’s dissection (1986) of working mem-
ory a n d Norman a n d Shallice’s theory (1986) of attention a n d control,
famously invoked the idea of a “central executive.’’ For present pur-
poses, we can p u t aside the common criticism that invoking an execu-
tive as an account of mental control creates a sort of infinite regress (does
the executive contain its o w n executive?). Rather, let us simply ask
whether the phenomena of set and dual-task interference provide any
sort of evidence for such a conception.
As several authors have pointed out (e.g., Allport 1987; Monsell 1996),
the alternative is a scheme in which executive control emerges from the
interaction of the very same machinery that ordinarily carries out the
mental processes being controlled. The brute phenomena of executive
control (e.g., that we can decide to perform one task or another; that
verbal instructions can, if their recipient chooses to comply, completely
determine which stimuli evoke which responses) emphatically do not
require the existence of machinery dedicated for the purpose of control.
Mutual competition between distributed mechanisms for the control of
thought a n d action may well account for task set–switching phenomena.
Indeed, work on “multiagent planning’’ in artificial intelligence suggests
such a mechanism is capable of much more than that (e.g., Suarez,
Winstanley, a n d Griffiths 1998). Furthermore, some of the phenomena of
task set described above, such as the need to perform at least one trial of
a new task in order to fully reconfigure processing machinery for that
task, seem slightly more congenial to a distributed control architecture
than to the notion of a distinct executive mechanism.
It is also commonly suggested that the idea of an all-or-none pro-
cessing bottleneck (particularly a single bottleneck that spans diverse
cognitive contents, as argued for above) naturally implies or at least
suggests the existence of a single mechanism that carries out whatever
cognitive operations are subject to queuing. Noting this, some writers
(e.g., Kinsbourne 1981) have pointed out that the notion of a single-
channel bottleneck seems hard to reconcile with the highly distributed
processing that characterizes the h u m a n cerebral cortex.
It is certainly true that one very natural explanation for obligatory
queuing of any given operation is the possibility that there is only a sin-
gle device capable of carrying out the operation. That may not be the only
explanation, however, let alone the correct one. Consider, for example,
recent studies of processing bottlenecks in commisurotomy (“split-
Pashler
brain’’) patients. If the central bottleneck described above has a defined
cortical locus, split-brain patients should show no PRP effect whenever
each task is confined to a separate hemisphere (assuming they are capa-
ble of performing the tasks u n d e r such conditions). However, using lat-
eralized stimuli and responses, Pashler et al. (1994) observed relatively
normal performance and a relatively normal PRP effect in four split-brain
patients. We concluded that the queuing underlying the PRP bottleneck
must have a subcortical source because connections at these brain levels
remain intact in split-brain patients (but see Ivry and Haseltine, chap. 17,
this volume, for another view based on later studies conducted with one
of these patients). It seems very unlikely that a brain stem mechanism
would be responsible for actually carrying out memory retrieval a n d
response selection. The natural alternative, then, is that the operations
subject to queuing are themselves distributed a n d subcortical mecha-
nisms trigger or control the queuing.
Is Queuing a Consequence of Task Set Limitations?
Is it possible that difficulties in selecting two responses at the same time
(resulting in the PRP effect) stem from an inability to simultaneously
maintain the task set for the two separate tasks? Although this idea has
been suggested from time to time (e.g., Gottsdanker 1980), such a reduc-
tion seems h a r d to reconcile with the task-switching p h e n o m e n a
described earlier in this chapter.5 Recall that in the Jersild paradigm, peo-
ple usually incur only a fairly modest cost (and sometimes none at all) in
shifting from one task to another so long as the mapping is univalent (i.e.,
where no stimulus is ever m a p p e d onto different responses in the two
tasks). Because, in the typical PRP task, the stimulus sets for the t w o tasks
are nonoverlapping, the problem of concurrent task set maintenance
should be comparable to that found with the univalent Jersild task, not
with the bivalent task. Based on the results described earlier, one would
therefore expect to find only a fairly modest slowing, presumably
because both tasks sets can simultaneously coexist. Because the PRP effect
often reaches several h u n d r e d milliseconds, presumably this concurrent
maintenance problem cannot be the whole source of it.
On the other hand, one need not rely on indirect inferences; the con-
current maintenance contribution to PRP slowing can be assessed fairly
directly, with a control seldom used until recently, by introducing to the
PRP experiment blocks in which subjects prepare for both tasks, but are
presented only one stimulus and are unable to predict which one this will
be. In one unpublished study, Eric Ruthruff and I had subjects make a
verbal response to a color patch, a manual response to a tone, or both. In
the “or’’ task, subjects performed one task or the other, but not both (only
one stimulus w a s presented). The “and’’ task was basically a PRP task
with a zero SOA. There was some slowing in the “or’’ task compared to
Task Switching and Multitask Performance
pure task blocks, but much more slowing on top of that in the “and’’
task.
The preparatory limitation responsible for the slowing in the “or’’ task
as compared to a pure single task is likely to be responsible for slowing
found in various single-task designs, as Gottsdanker (1980) pointed out.
In a choice RT task, a greater number of stimulus-response (S-R) pairs is
associated with longer RTs (Hick 1952)—an effect that d e p e n d s chiefly on
the number of alternatives subjects must prepare for, rather than the
number of different alternatives they were exposed to during the current
block of trials (Dixon 1981). Presumably, the need to prepare more S-R
“links’’ means that each link cannot be prepared as fully, causing per-
formance to be slowed (Gottsdanker 1980; Logan 1978). It is not merely
the number of links that matters, however; the more conceptually co-
hesive the set of stimuli m a p p e d onto any single response, the faster
the task can be performed (Greenwald, McGhee, a n d Schwartz 1998;
Seymour 1973). What is not clear is h o w preparatory costs should be
understood. For example, does poorer preparation for larger or more het-
erogeneous mappings reflect more time having elapsed since a given link
was prepared, or is “preparatory capacity’’ subject to continuous sharing,
as proposed by Gottsdanker (1980)?
A Modified Reduction Hypothesis
Even though dual-task slowing is not reducible to the preparatory limi-
tation for the reasons just discussed, one could still try to explain the PRP
effect in terms of a limitation in task set. Consider the following hypoth-
esis. In the “or’’ task experiments just described, the response selection
module might not be preset at all, or it might be set in a “neutral’’ fash-
ion. The shift from this unprogrammed state to the appropriate task set
might occur very quickly, producing only a minor cost. Suppose, counter
to what we have been assuming throughout this chapter, that, in the PRP
design, despite a univalent mapping, the first task set must be disen-
gaged a n d the second task set loaded before the second task can be
processed. To explain w h y the dual-task case (“and’’ task) produces more
slowing than the u n k n o w n single-task case (“or’’ task), one merely has to
suppose that the response selection machinery cannot be reprogrammed
while it is being used. This does not seem like an unreasonable supposi-
tion. The only problem is that because this account presumes that task set
reconfiguration is necessary even with univalent mappings, it fails to
explain w h y bivalent lists exhibit so much more alternation cost than
univalent lists, although, with some ingenuity, it could probably be m a d e
to explain this as well.
Fortunately, however, we do not need to rely on such arguments. What
would provide a critical test of the hypothesis that the bottleneck reflects
a limitation in maintaining the set for each task? If the bottleneck re-
Pashler
flects an inability to prepare the two task mappings simultaneously, then
it should disappear when two or more tasks use the same mapping. That
is, if the stimulus-response mapping rule remains fixed, and several
stimuli must be processed, parallel central processing should be possible,
unlike in the normal PRP case. One possible test of this claim would use
a PRP task in which two distinct stimuli are presented and the response
rule is the same.6 Another method in which the mapping remains con-
stant but subjects attempt to perform more than one task at the same time
is the serial RT task, where subjects respond to a whole string of stimuli.
In a recent study, we had subjects carry out a self-paced serial task, with
and without preview (Pashler 1994). Letters unfolded from left to right,
and subjects m a d e a button-push response to the identity of each letter
(four possible keys and four possible letters); ten letters unfolded, so that
at the completion of the trial, there were ten letters on the screen and
subjects had made ten responses. In the no-preview condition, the exper-
iment began with the presentation of a single letter; stimulus n + 1 was
presented as soon as subjects responded to stimulus n. In the preview
condition, the experiment began with two letters on the screen; stimulus
n + 1 was presented on the response to stimulus n — 1. Due to the pre-
view, subjects could potentially begin processing stimulus n + 1 while
still processing stimulus n. Is this logical possibility also a psychological
possibility?
The rate of responding in the preview condition was greater than in the
no-preview condition. First noted by Cattell (1886) and confirmed by
Leonard (1953), this finding strongly suggests that some overlap of pro-
cessing stages does indeed occur in the preview condition (as it does in
the conventional PRP situation, too; see figure 12.7). The key question
was whether the response selection stages associated with successive
stimuli could overlap. To answer this question, several different task
difficulty manipulations were used: targeting perception, response selec-
tion, and response production. When the mapping was made less natu-
ral, thereby increasing response selection duration (the manipulation was
applied for the whole list of ten stimuli), the time between each response
in the r u n was increased. The slowing was the same with or without pre-
view. On the other hand, when perceptual processing was made more
difficult, the time between the first stimulus and the first response length-
ened, b u t the rate of responding thereafter was virtually unaffected. The
results can be summarized by saying that response selection (but not per-
ception or response production) seems to be rate limiting for serial per-
formance even when stimuli are presented well before they are needed.
Evidently, only one response can be selected at a time even if the rule for
selecting responses does not change.
If the need to select new responses without any need to change task set
is sufficient to produce response selection queuing, it seems gratuitous
(or at least unparsimonious) to attribute the bottleneck in selecting com-
Task Switching and Multitask Performance
Figure 12.7 Effects of preview, stimulus quality a n d S-R compatibility on serial reaction
time task. From Pashler 1994.
pletely distinct responses to an inability to maintain nonoverlapping
(univalent) mappings simultaneously prepared. In view of this finding,
plus the minimal cost of shifting in univalent lists (Jersild a n d others), it
seems likely the limitation on carrying out two response selections at
once cannot be reduced to a limitation on maintaining the two task sets
at once. Presumably, because the mappings are univalent, the response
selection module is loaded with both mappings (although not without
cost, a n d not necessarily to the same degree at all times throughout the
trial). That would suggest that the order of task performance in the PRP
situation is probably not preplanned, a view that has been challenged by
De Jong (1995). Logically speaking, there is no contradiction between say-
ing that the t w o task mappings are simultaneously loaded a n d saying
300 Pashler
that the order of processing is planned or anticipated, although De Jong’s
evidence for preplanning of order involved tasks with two manual
responses, and may therefore represent a rather special form of response
selection.7
Alternative Explanations for Bottlenecks
We have considered two possible reasons for w h y a bottleneck might
arise in the process of action planning (and, it w a s suggested above,
memory retrieval as well). One explanation suggested that the bottleneck
reflects strategic choices in scheduling mental operations, rather than a
structural limitation: the other, that it reflects a limitation in simultane-
ously maintaining the two mappings in an active state. The evidence
described above, although not fully conclusive, suggests that neither
of these explanations is likely to be correct. If so, h o w else might one
account for this puzzling limitation?
One intuitively very appealing idea, proposed by Allport (1987, 1993)
a n d endorsed by De Jong (chap. 15, this volume), is that a bottleneck in
planning might serve a positive function of preventing incompatible
actions, thus maintaining the overall coherence of our behavior. The PRP
effect, which appears as an obstacle to optimal performance within the
contrived constraints of the dual-task experiment, might therefore be
adaptive—in computer parlance, a “feature, not a bug.’’ This proposal
does not explain, however, w h y even time-consuming memory retrievals
should be subject to queuing, as argued above. Nor, as formulated, does
it specify exactly what sort of incoherence is meant to be prevented by
queuing. One idea might be that preventing unrelated actions from being
selected simultaneously w o u l d prevent the simultaneous execution of
motor responses created by different action plans. This, it might be
argued, would help maintain the coherence of behavior because a single
planning operation will seldom (one might assume) generate behaviors
that are mutually disruptive. The problem with this idea is that we are
actually quite capable of simultaneously executing responses reflecting
two or more independent planning operations. Casual observation of
ordinary h u m a n activities reveals many examples. In a café, for example,
a patron will lower a coffee cup while simultaneously beginning to speak;
in a store, a clerk greets a customer while simultaneously putting the pre-
vious customer’s groceries in a bag. It seems far-fetched to suppose that
the speech a n d the h a n d movement, or the greeting a n d the hand move-
ment, result from a single p l a n . These informal observations are
confirmed by objective data. Van Galen a n d ten Hoopen (1976), for exam-
ple, h a d people pronounce multisyllabic words in response to a letter a n d
then make a button-push response to a second letter that followed soon
after. The button-push response often occurred while the vocal response
was still in progress; w h e n this happened, there was no detectable
interference.
301 Task Switching and Multitask Performance
One might suggest that what the brain is engineered to avoid is not the
overlapping execution of independently selected responses, but rather
the planning of an action that would terminate or disrupt a previously
selected action. Such a constraint might, in de Jong’s words (chap. 15, this
volume), “protect task performance in progress from interference.’’ Here
again, there is little reason to believe that the constraint envisioned really
exists. People can cancel actions that have just been launched, even when
these are highly practiced. For example, Logan a n d Burkell (1983)
showed that skilled typists could rapidly stop typing when an auditory
stop signal was presented. In simple terms, action planning a n d the
earliest stages of execution are not “ballistic.’’ If they were, it might
lend a certain form of coherence to our behavior, but probably a sort of
coherence we should be glad not to possess.
The obvious alternative to accounts that view queuing as a positive
benefit are accounts that claim the computational requirements of paral-
lel retrieval would exceed available resources. This is somewhat p u z -
zling, though, in view of the rather elementary kinds of task mappings
that elicit queuing. The possibility of cross talk between tasks may help
explain the ubiquity of queuing, if not quite as directly as some writers
have supposed. Because similarity of tasks seems not to be a necessary
condition for dual-task interference or queuing, attributing dual-task
interference to content-specific cross talk within a given task combination
seems rather unpromising (Pashler 1997). It is possible, however, that the
system is wired up to require queuing as a general policy (conceivably
one that can be overcome with sufficient practice) to prevent cross talk
from unpredictably degrading performance in certain cases. Such an
account seems consistent with several findings described earlier, includ-
ing the proposed unity of limitations in action selection a n d memory
retrieval, a n d the evidence from split-brain patients that anatomically
distributed processing can be subject to queuing.
Open Questions
The study of task set is in its relative infancy, a n d the suggestions offered
here about h o w we might relate task set to dual-task limitations are mod-
est a n d preliminary. Many very basic questions remain to be addressed.
One obvious question is whether the process of task reconfiguration itself
can be carried out in parallel with another task. Goschke (chap. 14, this
volume) finds that people are able to achieve the usual (partial) degree of
reconfiguration if required concurrently to verbalize a description of the
task they are about to perform. On the other hand, producing an irrele-
vant verbalization interfered with reconfiguration. What is not clear is
whether carrying out an unrelated nonverbal task would interfere. This
issue seems quite amenable to chronometric study.
Another open question is how the concepts useful in thinking about
arbitrary choice reaction time tasks that have been the focus of the
302 Pashler
research described here might generalize to the more ordinary activities
of everyday life. In activities like driving and conversing, one may speak
of “task schemata’’ or “goals,’’ but the notion of “mapping’’ seems inapt
or contrived. Unfortunately, the implications of many of the concepts
described here for such tasks remain to be clarified. This statement is not
intended as a criticism of researchers w h o have, reasonably enough,
started by studying relatively tractable cases. One area where some steps
have been taken toward greater “ecological validity’’ is bilingual lexical
production. Several investigators have given bilingual subjects cues
telling them to name stimuli such as numbers in one language or another,
a n d examined the effects of RSI and related variables. Thus far, the results
with this task seem encouragingly similar to those found with non-
linguistic laboratory tasks described above (MacNamara, Krauthammer,
a n d Bolgar 1968; Meuter and Allport 1999). It is to be hoped that
further efforts to examine tasks of this sort, as well as classic laboratory
tasks, may shed greater light on the issues of task control and dual-task
performance.
NOTES
This work was supported by National Institute of Mental Health grant 1-R01-MH45584 a n d
by National Science Foundation grant SBR9729778.
1. Why preview should produce a switch benefit remains an open question. Conceivably,
people can overlap more of the processing of each successive task when the mapping is
changing.
2. In Vicentized distributions, the values for different percentiles are determined separately
for each subject, then averaged across subjects; the results represent the typical shape of
individuals’ distributions, even if their speed of responding differs greatly.
3. In some cases (e.g., Rogers and Monsell 1995, exp. 4), a significant switch cost has been
found with univalent lists that use compound stimuli, where the irrelevant stimulus was
neutral (i.e., associated with no response).
4. The stimulus terms were color names a n d the verbal response terms were digits. During
testing, single- and dual-task blocks were interspersed.
5. Note that the issue here is not whether the PRP effect arises merely as a consequence of
temporal uncertainty about when S2 will arrive. This idea is clearly refuted by the finding
that when the temporal parameters are unchanged, but subjects need not respond to S1, no
PRP slowing occurs (e.g., Pashler a n d Johnston 1989).
6. One would naturally assume that sensory- or effector-specific interference would poten-
tially contaminate such a study. If, however, the duration of central processing substantially
exceeded that of more peripheral processing, reuse of the same sense and effector mecha-
nisms should make very little difference; this deserves testing.
7. Manual response selection may ordinarily choose a spatial location, rather than a finger.
If both a left-hand response and a right-hand response must be selected, the potential set of
spatial locations may be unwieldy. As a strategy, the response-selection machinery might
therefore choose within-hand spatial locations for each hand in turn, requiring a planned
order (Pashler 1990). If this explains De Jong’s findings, evidence for preplanning ought to
disappear when one task is manual and the other vocal.
Task Switching and Multitask Performance
REFERENCES
Allport, A. (1987). Selection for action: Some behavioral a n d neurophysiological considera-
tions of attention a n d action. In H. Heuer a n d A. F. Sanders (Eds.), Perspectives on perception
and action, p p . 395–419. Hillsdale, NJ: Erlbaum.
Allport, A. (1993). Attention and control: Have we been asking the wrong questions? A
critical review of twenty-five years. In D. Meyer a n d S. Kornblum (Eds.), Attention and
Performance XIV: Synergies in experimental psychology, artificial intelligence, and cognitive neuro-
science, p p . 183–218. Cambridge, MA: MIT Press.
Allport, A., Styles, E. A., a n d Hsieh, S. (1994). Shifting intentional set: Exploring the dynam-
ic control of tasks. In C. Umiltà a n d M. Moscovitch (Eds.), Attention and Performance XV:
Conscious and nonconscious information processing, p p . 421–452. Cambridge, MA: MIT Press.
Baddeley, A. D. (1986). Working memory. Oxford: Oxford University Press.
Biederman, I. (1973). Mental set a n d mental arithmetic. Memory and Cognition, 1, 383–386.
Borger, R. (1963). The refractory period a n d serial choice reactions. Quarterly Journal of
Experimental Psychology, 15, 1–12.
Brebner, J. (1968). The search for exceptions to the psychological refractory period. In S.
Dornic (Ed.), Attention a n d Performance V1, p p . 63–78. Hillsdale, NJ: Erlbaum.
Broadbent, D. E., and Gregory, M. (1967). Psychological refractory period and the length of
time required to make a decision. Proceedings of the Royal Society of London B168, 181–193.
Carrier, M., and Pashler, H. (1996). The attention d e m a n d s of memory retrieval. Journal of
Experimental Psychology: Learning, Memory, and Cognition, 21, 1339–1348.
Cattell, J. M. (1886). The time it takes to see and name objects. Mind, 11, 63–65.
Creamer, L. R. (1963). Event uncertainty, psychological refractory period, a n d h u m a n data
processing. Journal of Experimental Psychology, 66, 187–194.
De Jong, R. (1993). Multiple bottlenecks in overlapping task performance. Journal of
Experimental Psychology: Human Perception and Performance, 19, 965–980.
De Jong, R. (1995). The role of preparation in overlapping-task performance. Quarterly
Journal of Experimental Psychology: Human Experimental Psychology, 48A, 2–25.
Dixon, P. (1981). Algorithms and selective attention. Memory and Cognition, 9, 177–184.
Duncan, J., Martens, S., and Ward, R. (1997). Restricted attentional capacity within but not
between sensory modalities. Nature, 387, 808–810.
Fagot, C. (1994). Chronometric investigations of task switching. Ph.D. diss., University of
California, San Diego.
Fisher, S. (1975a). The microstructure of dual-task interaction: 1. The patterning of main-task
responses within secondary-task intervals. Perception, 4, 267–290.
Fisher, S. (1975b). The microstructure of dual task interaction: 2. The effect of task instruc-
tions on attentional allocation and a model of attention-switching. Perception, 4, 459–474.
Gladstones, W. H., Regan, M. A., and Lee, R. B. (1989). Division of attention: The single-
channel hypothesis revisited. Quarterly Journal of Experimental Psychology: Human
Experimental Psychology, 41A, 1–17.
Gottsdanker, R. (1980). The ubiquitous role of preparation. In G. E. Stelmach and J. Requin
(Eds.), Tutorials in motor behavior, p p . 355–371. Amsterdam: Elsevier.
Pashler
Greenwald, A. G. (1972). On doing two things at once: I. Time-sharing as a function of ideo-
motor compatibility. Journal of Experimental Psychology, 100, 52–57.
Greenwald, A., and Shulman, H. (1973). On doing two things at once: II. Elimination of the
psychological refractory period. Journal of Experimental Psychology, 101, 70–76.
Greenwald, A. G., McGhee, D. E., and Schwartz, J. L. K. (1998). Measuring individual dif-
ferences in implicit cognition: The implicit association test. Journal of Personality and Social
Psychology, 74, 1464–1480.
Heuer, H. (1985). Intermanual interactions during simultaneous execution and program-
ming of finger movements. Journal of Motor Behavior, 17, 335–354.
Hick, W. E. (1952). On the rate of gain of information. Quarterly Journal of Experimental
Psychology, 4, 11–26.
Jersild, A. T. (1927). Mental set a n d shift. Archives of Psychology, no. 89.
Johnston, J. C., and McCann, R. S. (forthcoming). On the locus of dual-task interference: Is
there a bottleneck at the stimulus classification stage?
Kinsbourne, M. (1981). Single channel theory. In D. Holding (Ed.), Human skills, p p . 65–89.
Chichester, U.K.: Wiley.
Kornblum, S. (1973). Sequential effects in choice reaction time: A tutorial review. In S.
Kornblum (Ed.), Attention and Performance IV, p p . 259–288. New York: Academic Press.
Leonard, J. A. (1953). Advance information in sensorimotor skills. Quarterly Journal of
Experimental Psychology, 5, 141–149.
Logan, G. D. (1978). Attention in character classification tasks: Evidence for the automatic-
ity of component stages. Journal of Experimental Psychology: General, 107, 32–63.
Logan, G. D., a n d Burkell, J. (1986). Dependence and independence in responding to
double stimulation: A comparison of stop, change and dual-task paradigms. Journal of
Experimental Psychology: Human Perception and Performance, 12, 549–563.
Logan, G. D., and Zbrodoff, N. J. (1982). Constraints on strategy construction in a speeded
discrimination task. Journal of Experimental Psychology: Human Perception and Performance, 8,
502–520.
MacNamara, J., Krauthammer, M., a n d Bolgar, M. (1968). Language switching in bilinguals
as a function of stimulus a n d response uncertainty. Journal of Experimental Psychology, 78,
208–213.
McLeod, P., and Posner, M. I. (1984). Privileged loops from percept to act. In H. Bouma a n d
D. G. Bouwhuis, (Eds.), Attention and Performance X, p p . 55–66. London: Erlbaum.
Meiran, N. (1996). Reconfiguration of processing mode prior to task performance. Journal of
Experimental Psychology: Learning, Memory, and Cognition, 22, 1423–1442.
Meuter, R. F. I., a n d Allport, A. (1999). Bilingual language switching in naming:
Asymmetrical costs of language selection. Journal of Memory and Language, 40, 25–40.
Meyer, D. E., a n d Kieras, D. E. (1997). A computational theory of executive cognitive
processes a n d multiple-task performance: 2. Accounts of psychological refractory-period
phenomena. Psychological Review, 104, 749–791.
Miller, J. (1982). Divided attention: Evidence for coactivation with redundant signals.
Cognitive Psychology, 14, 247–279.
Monsell, S. (1996). Control of mental processes. In V. Bruce (Ed.), Unsolved mysteries of the
mind: Tutorial essays in cognition, p p . 93–148. Hove, U.K.: Erlbaum and Taylor and Francis.
Task Switching and Multitask Performance
Norman, D. A., and Shallice, T. (1986). Attention to action: Willed and automatic control of
behavior. In R. J. Davidson, G. E. Schwartz, a n d D. Shapiro (Eds.), Consciousness and self-
regulation, vol 4, p p . 1–18. New York: Plenum Press.
Osman, A., and Moore, C. (1993). The locus of dual-task interference: Psychological refrac-
tory effects on movement-related brain potentials. Journal of Experimental Psychology: Human
Perception and Performance, 19, 1292–1312.
Palmer, J. (1995). Attention in visual search: Distinguishing four causes of a set-size effect.
Current Directions in Psychological Science, 4, 118–123.
Pashler, H. (1984). Processing stages in overlapping tasks: Evidence for a central bottleneck.
Journal of Experimental Psychology: Human Perception and Performance, 10, 358–377.
Pashler, H. (1990). Do response modality effects support multiprocessor models of divided
attention? Journal of Experimental Psychology: Human Perception and Performance, 16, 826–840.
Pashler, H. (1994). Overlapping mental operations in serial performance with preview.
Quarterly Journal of Experimental Psychology, 47, 161–191.
Pashler, H. (1997). The psychology of attention. Cambridge, MA: MIT Press.
Pashler, H., a n d Baylis, G. C. (1991). Procedural learning: 2. Intertrial repetition effects in
speeded-choice tasks. Journal of Experimental Psychology: Learning, Memory, and Cognition, 17,
33–48.
Pashler, H., Carrier, M., a n d Hoffman, J. (1993). Saccadic eye movements a n d dual-task
interference. Quarterly Journal of Experimental Psychology, 46A, 51–82.
Pashler, H., and Christian, C. (n.d.). Dual-Task interference and motor response production.
Pashler, H., and Johnston, J. C. (1989). Interference between temporally overlapping tasks:
Chronometric evidence for central postponement with or without response grouping.
Quarterly Journal of Experimental Psychology, 41A, 19–45.
Pashler, H., and Johnston, J. C. (1998). Attentional limitations in dual-task performance. In
H. Pashler (Ed.), Attention, p p . 155–189. Hove, U.K.: Psychology Press, Erlbaum, and Taylor
a n d Francis.
Pashler, H., Luck, S. J., Hillyard, S. A., Mangun, G. R., a n d Gazzaniga, M. (1994). Sequential
operation of disconnected cerebral hemispheres in split-brain patients. Neuroreport, 5,
2381–2384.
Rickard, T., a n d Pashler, H. (1998). A bottleneck in retrieval from a single cue. Talk pre-
sented at Thirty-ninth Annual Meeting of the Society for Psychonomics, Dallas, November.
Rogers, R. D., and Monsell, S. (1995). Costs of a predictable switch between simple cogni-
tive tasks. Journal of Experimental Psychology: General, 124, 207–231.
Ruthruff, E., Miller, J., and Lachmann, T. (1995). Does mental rotation require central mech-
anisms? Journal of Experimental Psychology: Human Perception and Performance, 21, 552–570.
Salthouse, T. A., Fristoe, N., McGuthry, K. E., a n d Hambrick, D. Z. (1998). Relation of task
switching to speed, age, a n d fluid intelligence. Psychology and Aging, 13, 445–461.
Schvaneveldt, R. W., a n d Staudenmayer, H. (1970). Mental arithmetic and the uncertainty
effect in choice reaction time. Journal of Experimental Psychology, 85, 111–117.
Schouten, J. F., Kalsbeek, J. W. H., a n d Leopold, F. F. (1960). On the evaluation of perceptual
a n d mental load. Ergonomics, 5, 251–260.
Seymour, P. H. (1973). Rule identity classification of name and shape stimuli. Acta
Psychologica, 37, 131–138.
Pashler
Shaffer, L. H. (1965). Choice reaction with variable S-R m a p p i n g . Journal of Experimental
Psychology, 70, 284–288.
Simon, J. R., Webster, J. B., a n d Craft, J. L. (1981). Effects of changing compatibility relation-
ships on reaction time to the first stimulus in a double-stimulation task. Journal of Motor
Behavior, 13, 301–312.
Spector, A., and Biederman, I. (1976). Mental set and mental shift revisited. American Journal
of Psychology, 89, 669–679.
Suarez, J. N., Winstanley, G. a n d Griffiths, R. N. (1998). A reluctance-based cost distribution
strategy for multiagent planning. Applied Intelligence, 9, 39–55.
Sudevan, P., a n d Taylor, D. A. (1987). The cueing and priming of cognitive operations.
Journal of Experimental Psychology: Human Perception and Performance, 13, 89–103.
Telford, C. W. (1931). The refractory phase of voluntary a n d associative responses. Journal of
Experimental Psychology, 14, 1–36.
Treisman, A., and Davies, A. (1973). Dividing attention to ear and eye. In S. Kornblum (Ed.),
Attention and Performance IV, p p . 101–117. N e w York: Academic Press.
Van Galen, G. P. , and ten Hoopen, G. (1976). Speech control and single channelness. Acta
Psychologica, 40, 245–255.
Welford, A. T. (1952). The “psychological refractory period’’ and the timing of high speed
performance: A review and a theory. British Journal of Psychology, 43, 2–19.
Welford, A. T. (1967). Single-channel operation in the brain. Acta Psychologica, 27, 5–22.
Welford, A. T. (1980). The single-channel hypothesis. In A. T. Welford (Ed.), Reaction time, p p .
215–252. N e w York: Academic Press.
307 Task Switching and Multitask Performance
13 Multitasking Performance Deficits: Forging Links between the Attentional Blink and
the Psychological Refractory Period
Pierre Jolicœur, Roberto Dell’Acqua, a n d
Jacquelyn Crebolder
ABSTRACT This chapter presents new empirical work that bears on the issue of whether
multitasking performance deficits are a by-product of strategic control as opposed to struc-
tural capacity limitations. Accuracy in reporting the identity of a masked visual target was
measured at various delays following an auditory stimulus that required an immediate
speeded response. A larger attentional blink (AB) effect was found when the auditory task
h a d four possible stimuli and response alternatives than when it had only two. In a psy-
chological refractory period (PRP) experiment, two speeded responses were required to
stimuli presented in rapid succession. The auditory task used as the first task of the AB
experiment served as the second task in the PRP experiment. Effects of number of response
alternatives in the second task were additive with stimulus onset asynchrony (SOA), sug-
gesting that the manipulation had an effect at or after the locus of PRP interference, and that
a locus of AB interference existed at or after a locus of interference causing the PRP effect.
Implications for the role of control versus structural limitations are discussed.
This chapter explores the relationship between two popular paradigms
that require different responses to two stimuli presented in rapid succes-
sion: the “psychological refractory period’’ (PRP) paradigm (see Pashler
1994 for a review) a n d the “attentional blink’’ (AB) paradigm (see Shapiro
a n d Raymond 1994 for a review). In the PRP paradigm, two speeded
responses are m a d e in rapid succession, usually to two distinct a n d
unmasked stimuli. The PRP effect is a slowing of the second response as
the stimulus onset asynchrony (SOA) between the two stimuli is reduced.
In the AB paradigm, two stimuli are also shown in rapid succession, a n d
usually both are masked. The AB phenomenon is a decrease in report
accuracy of the second stimulus at short SOAs.
With some notable exceptions, most researchers have concluded that
performance deficits in these paradigms arise because of capacity limita-
tions in central processing mechanisms. These mechanisms are assumed
to perform such operations as memory encoding a n d retrieval, response
selection, and other complex cognitive operations that occur after per-
ceptual encoding but before motor output.
As noted by Monsell (1996), discussion of these paradigms has focused
largely on the locus of interference effects or on the specific combinations
of tasks that result in performance limitations. Recently, however, the
issue of control has been brought to the fore by theoretical arguments that
relate major findings in both these paradigms to causal influences of
strategic control. We consider two examples.
First, using their “executive process interactive control’’ (EPIC) model,
Meyer and Kieras (1997a,b; see also Kieras and Meyer, chap. 30, this vol-
ume) propose that dual-task slowing in the PRP paradigm results from
adaptive executive control processes designed to guarantee that the
response to the first stimulus occurs before the response to the second
stimulus (subjects are often instructed to perform the responses in this
order). And second, finding evidence of a cross-modal AB deficit only
when a switch in task (Rogers and Monsell 1995) associated with the first
a n d second targets to be reported was required, Potter et al. (1998) pro-
pose that the AB deficit may have a “strategic’’ source, such as pre-
paration (see De Jong, chap. 15, this volume; De Jong and Sweet 1994),
different from the structural source of AB interference w h e n the stimuli
are presented in the same modality.
In the AB paradigm, the response to the second target stimulus is not
speeded. The control-based account of Meyer a n d Kieras (1997a,b; Kieras
a n d Meyer, chap. 30, this volume) for dual-task slowing in the PRP para-
digm w o u l d therefore not appear to apply to the AB paradigm, regard-
less of whether stimulation is within modality or between modalities.
Indeed, given their account, it would seem most natural to think of PRP
a n d AB effects as two rather different manifestations of multitasking per-
formance deficits. While the PRP effect would reflect central control
required to sequence rapid responses, AB effects would presumably be
taken to reflect some other form of system overload that occurs at a more
peripheral level (e.g., perceptual, motor, or both) because, according to
the EPIC model, the central operations required to perform the AB task
are not likely to induce central interference. However, this hypothesis
does not agree well with recent results suggesting that the AB effect may
be caused by central interference (see “Locus of Factor Effects in
Psychological Refractory Period a n d Attentional Blink Paradigms’’ in sec-
tion 13.1).
The empirical work presented below explores the relationship between
AB and PRP paradigms, a n d asks whether the sources of these multi-
tasking deficits may share some fundamental functional similarity, which
would in turn have implications for the role of control processes in caus-
ing these effects. We first consider the interpretation of factor effects in
these paradigms.
13.1 THE ATTENTIONAL BLINK PHENOMENON
The AB phenomenon is a decrease in the accuracy of report of a second
target (T2), when that target follows rapidly after a first target (T1) that
must also be reported. The paradigm most commonly used to study the
phenomenon embeds the two targets within a stream of other stimuli
Jolicœur, Dell’Acqua, Crebolder
presented using rapid serial visual presentation (RSVP; e.g., Broadbent
a n d Broadbent 1987; Chun a n d Potter 1995; Raymond, Shapiro, a n d
Arnell 1992, 1995; Jolicœur 1998; but see Duncan, Ward, and Shapiro
1994).
For example, Jolicœur (1998, exp. 1) presented a red first target (T1, H
or S) embedded in an RSVP stream of white letters. The second target (T2)
was an X or a Y. T2 occurred on every trial, but T1 was presented on only
half of the trials. The most interesting results concern the accuracy of
report of the second target. In control trials (T1 absent), the mean accura-
cy was about 85%. In the experimental condition (T1 present), accuracy
was at about 73% w h e n T2 followed T1 immediately (lag 1); about 64%
at lags 2–3; and about 7 1 % at lag 4, with a continued recovery to near-
baseline levels as lag w a s increased further. Raymond, Shapiro, a n d
Arnell (1992) labeled the loss of accuracy of report for T2, as a function of
the lag between T1 a n d T2, the “attentional blink’’ (AB), and this label is
n o w widely used.
Locus of Factor Effects in Psychological Refractory Period and
Attentional Blink Paradigms
Much of the work on the AB phenomenon has focused on two issues:
. Where is the interference between task 1 a n d task 2 taking place?
. What is the nature of this interference?
This chapter primarily addresses the first issue. There is growing agree-
ment that the locus of the interference is relatively late in processing,
probably after stimuli have activated semantic-level representations
(Shapiro et al. 1997; Luck, Vogel, a n d Shapiro 1996; Jolicœur 1998b, 1999c;
Duncan, Ward, and Shapiro 1994; Chun and Potter 1995; see also Jolicœur
a n d Dell’Acqua 1998, 1999, forthcoming; Dell’Acqua a n d Jolicœur 1998).
The simplest model of dual-task interactions assumes that some mech-
anisms cannot be shared across two tasks (Welford 1952). When two tasks
both need the same mechanism, interference results—the mechanism
constitutes a processing bottleneck. The top stage diagram in figure 13.1A
represents the processing required to perform the first of two tasks in a
PRP paradigm. The presentation of the first target (T1) triggers the stages
labeled A1, which represent all stages before the bottleneck. Processing
stages that require the bottleneck are labeled B1. Finally, stages after the
bottleneck are labeled C1. The sum of prebottleneck, bottleneck, a n d post-
bottleneck stage durations equals the response time in task 1, or RT1.
When the SOA between T1 a n d T2 is short (figure 13.1A) the prebottle-
neck processing in task 2 can proceed without interference. This is
illustrated by A2 in figure 13.1A. When prebottleneck processing is com-
pleted, the processing of T2 is ready to engage the mechanisms that con-
stitute the processing bottleneck, but these mechanisms are busy with
The Attentional Blink a n d the PRP
Figure 13.1 Stage diagrams showing the predicted task interactions in dual-task paradigms.
task 1. The result is a period of waiting, represented by three dots, during
which no further processing of T2 takes place. When task 1 no longer
requires the bottleneck, processing of T2 resumes. The initiation of bottle-
neck processing in task 2 (B2) thus coincides with the termination of
bottleneck processing in task 1 (B1). RT2 is the s u m of stage durations plus
the period of waiting (slack).
Now, suppose that a factor manipulated in task 2 increases the dura-
tion of a prebottleneck stage, as represented by an increase in the length
of A2. When the SOA is very short, as shown in figure 13.1A, this reduces
the period of waiting before the initiation of bottleneck processing, but
has no effect on RT2. The effect of the factor is said to have been absorbed
into the period of slack.
At longer SOAs (panel B), changing the duration of prebottleneck pro-
cessing has the expected effect of increasing RT2. Thus the effect of the fac-
tor should decrease as SOA is reduced. The resulting interaction is often
described as underadditive with decreasing SOA (or with increasing task
overlap).
If the factor manipulated in task 2 affects the duration of the bottleneck
stage, additive effects of the factor a n d SOA are expected. As shown in
figure 13.1C (short SOA) a n d 13.1D (long SOA). Although a period of
312 Jolicœur, Dell’Acqua, Crebolder
waiting (slack) is created by the contention for the bottleneck at short
SOA, the effects of the factor are not absorbed into slack because the fac-
tor affects a stage of processing that occurs after the period of waiting.
The increased duration of processing through the bottleneck stage is fully
a n d equally reflected in RT2 at both short a n d long SOA. Additivity is
also predicted if the factor affects the duration of processing after the
bottleneck (not shown).
This analysis, developed by Pashler and Johnston (1989), can be used
to interpret second-task factor effects on RT2 in the PRP paradigm. If the
factor effects are additive with SOA, the factor must be affecting a stage
in or after the bottleneck. If the factor effects decrease in magnitude as
SOA is reduced, then the factor must be affecting the duration of a stage
before the bottleneck. This analysis is sometimes called “locus-of-slack
logic,’’ and strong support for the method has been provided in numer-
ous studies (e.g., Pashler a n d Johnston 1989; McCann and Johnston 1992;
see Pashler 1994a for a review).
Figure 13.1E–F illustrates another prediction of the postponement
model of the PRP effect, concerning a factor manipulated in task 1. If the
effect of the factor is to lengthen the duration of processing at the bottle-
neck (figure 13.1E) or before, then the effect should carry over to response
times in task 2 as well. The longer bottleneck duration in the bottom pair
postpones the onset of processing at the bottleneck stage in task 2, which
results in a longer RT2. At a very short SOA, as illustrated, the effect of the
first-task factor should be the same on RT1 and RT2. Support for this pre-
diction can be found in Smith 1967, Van Selst, Ruthruff, and Johnston
1999, Williams 1974, and Pashler, 1994b. In figure 13.1F, a first-task factor
affecting processing after the bottleneck is assumed. While this factor
would affect RT1, no effect should be observed on RT2 (see Pashler 1994b
for some supporting evidence). The conclusion, therefore, is that a factor
manipulated in task 1, whose effects carry over to RT2, must affect a stage
in or before—but not after—the bottleneck.
In general, effects of first-task variables on RT2 such as the one in figure
13.1E are expected only at short SOAs. At longer SOAs, response times in
task 2 are not predicted to be influenced by first-task variables because
these effects are mediated by the competition for the bottleneck, and no
such competition takes place if the SOA is long enough.
For the AB paradigm, we are concerned with the effects of factor
manipulations in task 1 on performance in task 2. The analysis of factor
effects shown in figure 13.1E–F also applies to the AB paradigm,
although predictions are n o w m a d e for accuracy in task 2, rather than for
RT2. In the AB paradigm, response times in task 2 are not measured;
instead, the paradigm focuses on report accuracy to a masked target. For
a wide range of possible models, including all extant models, a factor
manipulated in task 1 of an AB experiment is not expected to affect accu-
racy in task 2 if the variable affects processing after the locus or loci of
interference causing the AB effect. Therefore, if a first-task factor m o d u -
313 The Attentional Blink a n d the PRP
lates the magnitude of the AB effect, which is measured as a change of
accuracy in task 2, the factor must be affecting a stage of processing that
is in or before the locus of AB interference (see Jolicœur 1998). Such effects
are expected only at shorter SOAs; at longer SOAs, there is no competi-
tion for processing capacity, a n d thus no expected dual-task interactions.
Clearly, some caution is required here because it is not difficult to think of
events that could occur after the critical task interactions causing the
AB effect that could cause a significant loss of information about T2.
The argument is sound, however, as long as the deficits in task 2 remain
clearly time locked to the occurrence of T1 at short SOAs, with a recovery
to baseline conditions at long SOAs, and as long as we remain within the
boundary conditions of the paradigms usually used to study the AB
phenomenon.
To account for the AB effect using postponement models, we must also
assume that there is a loss of information about T2 during the period of
waiting, with greater loss for longer waits (Jolicœur 1998). The results of
Jolicœur (1999-b) a n d Giesbrecht a n d DiLollo (1998) suggest that such
loss does not occur if T2 is not masked, presumably because sensory per-
sistence provides a form of storage of the information that can bridge the
period of waiting.
Locus of Attentional Blink Interference Relative to the Psychological
Refractory Period Bottleneck
Experiment 1 w a s a cross-modal AB experiment in which the first target
was a pure tone and the second target a visually presented letter in an
RSVP stream. The main factor manipulated in task 1 was the number of
stimulus a n d response alternatives. This manipulation had a large effect
on the magnitude of the AB effect. The conclusion is that this factor must
have its effect in or before a locus of interference causing the AB phe-
nomenon. In experiment 2, the same manipulation w a s performed in task
2 of a PRP experiment, a n d the effects were additive with SOA. The con-
clusion is that this factor must have its effects in or after the PRP bottle-
neck. Together, these results lead to the conclusion that at least one locus
of AB interference must be in or after the PRP bottleneck.
13.2 EXPERIMENT 1
Experiment 1 used a cross-modal speeded AB paradigm. T1 was a pure
tone and task 1 was an immediate speeded-choice response based on
pitch. T2 was a visual stimulus, the letter X or Y, presented on every trial,
embedded within an RSVP stream. Task 2 was a deferred a n d unspeeded
discrimination between X and Y (see also Jolicœur 1998, 1999a, 1999c).
The tone w a s presented concurrently with one of the letters in the RSVP
stream.
Jolicœur, Dell’Acqua, Crebolder
There were two versions of task 1. One had two tone frequencies a n d
two responses; the other, four frequencies and four responses. Based
on earlier work, we hypothesized that the two-alternative task 1 would
be associated with a shorter period of central processing than the four-
alternative task 1 (Van Selst a n d Jolicœur 1997; Schubert 1999). According
to Van Selst a n d Jolicœur (1997), this difference in processing should
occur in or after the PRP bottleneck. This assumption is verified in exper-
iment 2.
Strong claims have been m a d e regarding the outcome of experiments
like this. Pashler (1993) has influentially claimed that a speeded task will
not produce an SOA-dependent deficit on a closely following unspeeded
task. Duncan, Ward, a n d Shapiro (1994) have claimed that there is abso-
lutely no cross-modal AB. Yet earlier work in our laboratory has shown
that a speeded task 1 response to a tone can cause a significant AB effect
in a concurrent visual encoding task 2 (Jolicœur 1999a; Jolicœur a n d
Dell’Acqua 1999). Experiment 1 repeated Jolicœur’s experiment (1999a)
but also included two levels of first-task difficulty designed to influence
the duration of central processing.
Subjects
Twenty-six undergraduates at the University of Waterloo participated for
pay. All reported having normal or corrected-to-normal vision a n d nor-
mal hearing.
Stimuli
The auditory stimuli were pure tones presented well above threshold for
100 msec at a frequency of 200, 363, 660, or 1,200 Hz, using an internal
computer speaker. The middle two frequencies (363 and 660 Hz) were
used in the two-alternative condition.
The visual stimuli were white uppercase letters on a black background
presented in RSVP at the center of a computer screen, at a rate of 10
letters/sec (100 msec each with no blank interstimulus interval).
Between 6 a n d 9 letters were presented prior to the letter concurrent
with the tone, a n d 12 to 15 after the tone. The X or Y could occur with
equal probability at positions 1, 3, 5, 7, 9, or 11, following the tone. Thus
even the last target position had 1 to 4 letters following it, ensuring that
T2 was always effectively masked (Giesbrecht and Di Lollo 1998; Jolicœur
1999b).
On every trial, the background stream items were selected at random,
without replacement, from the letters of the alphabet, excluding H, S, X,
a n d Y. Each letter subtended about 1 degree of visual angle a n d had a
luminance of about 25 c d / m 2 and CIE(x, y) coordinates of (0.278, 0.306).
The background was black with a luminance of less than 1 c d / m 2 .
The Attentional Blink a n d the PRP
Procedure
Each trial began with t w o symbols at the center of the screen, which pro-
vided both fixation markers and performance feedback for the previous
trial. Pressing the space bar eliminated the fixation a n d feedback symbols
a n d initiated the RSVP sequence. A tone (T1) w a s presented on half of the
trials. Trials with no tone served as control trials, in which preparation
was equivalent to that in experimental trials. Tone-present a n d tone-
absent trials were intermixed at r a n d o m within each test session.
The experiment was divided into two sessions separated by a short
break. In one session task 1 involved two choices (363 Hz “ > ’ ’ ; 660 Hz
“?’’). In the other session, task 1 involved four choices (200 Hz “M’’;
363 Hz “< ’’; 660 Hz “ > ’ ’ 1,200 Hz “?’’). The response buttons
were contiguous on the bottom right row of the keyboard, a n d responses
were m a d e with the index, middle, ring, a n d little fingers of the right
hand. The index a n d middle fingers were used for the two-alternative
task 1. The instructions were to press the correct response button as
quickly as possible after hearing the tone, while keeping errors to a
minimum. A message asking subjects to respond more quickly to the tone
was presented if the RT to the tone was greater than 1,300 msec.
At the end of every trial, after the response to the tone, a prompt asked
subjects to indicate which visual target h a d been shown (X or Y). The “X’’
key was used to respond “X,’’ a n d the “C’’ key was used to respond “Y.’’
This response was not speeded.
The two-alternative discrimination session consisted of one block of 48
practice trials, followed by 3 blocks of 96 trials. The four-alternative dis-
crimination session consisted of two block of 48 practice trials, followed
by 3 blocks of 96 trials. The order of sessions was counterbalanced across
subjects.
Each block of experimental trials in the four-alternative task 1 con-
tained a full crossing of T1–T2 SOA (100, 300, 500, 700, 900, or 1,100 msec),
T1 frequency, T2 present versus absent, and T2 identity (X versus Y).
When T1 was not presented, a corresponding position in the RSVP stream
was selected nonetheless. This m a d e it possible to create control trials in
which the absolute position of T2 in the RSVP stream was equated across
T1-present a n d T1-absent trials. Each block of experimental trials in the
two-alternative task 1 contained two full crossings of the experimental
variables. Different random orders of the trials were used for each block
a n d for each subject. Performance feedback was given in the form of a
plus or minus sign for each response, at fixation, following each trial.
Results
The data from three subjects were eliminated. Two were less than 69%
correct in the control condition for task 2 in one or both sessions, a n d one
Jolicœur, Dell’Acqua, Crebolder
Figure 13.2 Mean proportion correct in task 2 of experiment 1. A. Results for each stimu-
lus onset asynchrony (SOA), each level of first-task difficulty (two alternatives: circular
symbols; four alternatives: square symbols), for trials on which T1 was presented (filled
symbols, solid lines), or omitted (unfilled symbols, dashed lines). B. Results for each SOA,
each level of first-task difficulty (two alternatives: circular symbols, dashed lines; four alter-
natives: square symbols, solid lines), for trials in which T1 was presented, depending on
the duration of RT1 (RT1 below the median: open symbols; RT1 above the median: filled
symbols).
w a s correct on only 54% of four-alternative tone task trials. The analyses
presented below are based on 13,248 pairs of responses generated by the
remaining 23 subjects.
Correct trials in tone-present trials were screened for outliers using a
close variant of the Van Selst a n d Jolicœur (1994) procedure (e.g., Jolicœur
1998, 1999a,b,c). Less than 1.9% of the trials were rejected. Analyses based
on data that included outliers produced the same patterns of results.
Task 2 Figure 13.2A displays mean accuracy in task 2. Both versions of
task 1 produced large AB effects. In an analysis of the two-alternative con-
317 The Attentional Blink a n d the PRP
dition, the interaction between SOA and T1 present or absent was highly
significant: F(5,110) = 6 . 9 1 , p< 0.0001; as were the two main effects:
p < 0.0001 in both cases. In a separate analysis of the four-alternative con-
dition, the SOA by T1 (present/absent) interaction was highly significant:
F(5,110) = 10.94, p< 0.0001; as were the two main effects; p< 0.0001 in
both cases.
A separate analysis of the data from the T1-present trials revealed a
significant interaction between the number of first-task response alterna-
tives and SOA: F(5,110) = 2.44, p < 0.04. There was also a large main effect
of number of Task 1 alternatives, F(1, 22) =43.53, p< 0.0001, corroborat-
ing what can be seen in figure 13.2, namely, that accuracy in task 2 was
lower when task 1 had four response alternatives than when it had only
two. A companion analysis examining the control trials showed that the
control conditions differed in overall levels of performance across the
two- and four-alternative versions of task 1: F(1, 22) = 9.47, p< 0.006.
However, there was no main effect of SOA and no interaction for the con-
trol conditions (p>0.05 in both cases).
The three-way interaction between SOA, T1 (present versus absent),
and number of first-task responses was not significant in the omnibus
analysis of variance (ANOVA), but the interaction between number of
first-task responses and T1 (present versus absent) was highly significant,
reflecting the larger difference between the experimental and control con-
ditions for the four-alternative than for the two-alternative first-task con-
dition: F(1, 22) = 24.42, p< 0.0001. The AB effect was reliably larger in the
four-alternative condition than in the two-alternative condition.
We also compared the difference between control and experimental
performance during the blink versus after the blink, as a more direct test
of the difference in AB effects across conditions (this more sensitive test
is justified by a priori expectations; see Jolicœur 1998). The difference
between average control performance and the average of the first four
SOAs (during the blink) was contrasted with the difference between the
control condition and the average of the last two SOAs (after the blink).
This difference was significantly larger for the four-alternative condition
than for the two-alternative condition: F(1, 22) = 15.66, p< 0.0007.
Task 1 As expected, mean RT1 was longer for four alternatives (691
msec) than for two (530 msec); F(1, 22) =217.37, p< 0.0001. Neither the
main effect of SOA nor the interaction between SOA and number of first-
task alternatives was significant: F(5,110) = 1.55, p > 0.18; F(5,110) = 1.41,
p > 0.22, respectively.
First-task accuracy was higher for two alternatives (93.2%) than for
four (77.4%); F(1, 22) = 105.84, p< 0.0001. There was no significant effect
of SOA and no interaction between SOA and number of first-task alter-
natives (p > 0.29 in both cases).
Jolicœur, Dell’Acqua, Crebolder
Task 2 as a Function of RT1 Accuracy in task 2 was also examined, as a
function of the speed of processing in task 1. The trials on which T1 was
presented were divided into cells for each subject, each SOA, whether
task 1 had two or four alternatives, and for each of the three blocks of
trials within each session. For each of these cells, the trials were sorted
further into two more cells depending on whether RT1 was above or
below the median RT1 in that bin. (Mean RT1 was 456 msec for faster two-
alternative responses, 596 msec for slower two-alternative responses, 606
msec for faster four-alternative responses, and 792 msec for slower four-
alternative responses.) For each resulting cell, the mean accuracy in task
2 was computed and submitted to an ANOVA with SOA, number of first-
task alternatives, block, and short and long RT1s as within-subject factors.
The means are displayed in figure 13.2B.
Accuracy in task 2 was higher (0.786) when RT1 was shorter than the
median and lower (0.737) when RT1 was longer than the median; F(1, 22)
= 23.09, p < 0.0001. The interaction between short and long RT1s and SOA
was significant, F(5,110) =5.03, p< 0.0003. This effect was modulated by
the number of response alternatives, as shown in figure 13.2B: F(5,110)
= 2.78, p< 0.025. The interaction between SOA and RT1 appears to have
the following form. Accuracy in task 2 is similar across short and long
RT1s at very short SOAs, accuracy for short and long RT1s diverge for
intermediate SOAs, with lower accuracy for long RT1s, followed by a con-
vergence of accuracy levels across short and long RT1s at the longest
SOAs.
The main effect of block was significant, with accuracy remaining
about the same from block 1 (0.756) to block 2 (0.745), and then improv-
ing in block 3 (0.784): F(2,44) = 3.61, p < 0.0355. The observed relationship
between RT1 and accuracy in task 2 was not an artifact of a general
improvement in performance in both tasks, as subjects became more
practiced, because lower accuracy in task 2 (for trials with a longer RT1)
was observed within each block or trials and because there was no over-
all increase in accuracy across blocks 1 and 2.
There was also little change in response times across blocks. Mean RT1
was 619 msec in block 1, 613 msec in block 2, and 605 msec in block 3; and
these means were not significantly different: F(2,44) = 1.56, p < 0.22. Thus
it is unlikely that the association between RT1 and accuracy in task 2
could be d u e to correlated changes in overall performance levels with
practice.
Discussion
The results were clear-cut. A larger AB effect was produced when task 1
involved four alternatives rather than two alternatives (figure 13.2).
Furthermore, within each first-task condition, a larger and longer AB
The Attentional Blink a n d the PRP
effect w a s found when processing of T1 took longer. Both of these results
support the hypothesis that a longer period of processing in task 1 in one
or more stages of processing carried over into accuracy scores in task 2.
13.3 EXPERIMENT 2
The interpretation of the results of experiment 1 hinges critically on the
locus of effect of the number of alternatives in task 1. On the one hand,
Van Selst and Jolicœur (1997) a n d Schubert (1999) both provided evi-
dence that the locus is in or after the PRP bottleneck, finding that n u m -
ber of alternatives (two or more), when manipulated in task 2 of a PRP
experiment, produced additive effects with SOA. On the other hand,
Schumacher et al. (1999) showed that underadditive interactions of n u m -
ber of alternatives a n d SOA can be found u n d e r certain conditions. The
conditions used in task 1 of experiment 1 do not match exactly the condi-
tions of any of these previous experiments, making it difficult to extrapo-
late from earlier work.
In experiment 2, the manipulation used in task 1 of experiment 1 was
applied to task 2 of a PRP experiment. According to the locus-of-slack
logic outlined earlier (figure 13.1), if the manipulation used in experiment
1 is at or after the stage of processing that causes PRP interference, then
additive effects of this variable should be observed with SOA. If some or
all of the effect is at an earlier stage, then an underadditive interaction
with SOA would result, as SOA is reduced.
Subjects
Thirty-three undergraduates at the University of Waterloo participated
for pay. All reported having normal or corrected-to-normal vision, a n d
normal hearing.
Stimuli
The auditory stimuli were identical to those used in experiment 1. The
visual stimuli were three letters, H, O, and S, presented at the same size
a n d luminance as the stream items in experiment 1. The exposure dura-
tion of the letter was 100 msec. The letter w a s not masked.
Procedure
Pressing the space bar removed the fixation symbols and initiated the
trial sequence. After a delay of 750 msec, a letter w a s presented, requir-
ing a speeded, three-alternative choice response to be m a d e with the left
hand: H ring finger (“Z’’ key); O middle finger (“X’’ key); S index
finger (“C’’ key). The key m a p ping was described on a piece of paper that
Jolicœur, Dell’Acqua, Crebolder
was at the top of the keyboard to facilitate learning during the practice
trials.
After an SOA of 50, 200, 500, or 1,100 msec, chosen pseudorandomly at
r u n time, the tone was presented. The frequencies and frequency
response mappings (right hand) were identical to those in experiment 1.
Each subject was tested in two sessions separated by a short break, one
for the two-alternative and one for the four-alternative version of task 2,
in counterbalanced order. Each session began with 48 practice trials, fol-
lowed by 6 blocks of 48 trials. Each block contained one or two full cross-
ings of the independent variables. The order of trials was randomized
anew for each block. Feedback was given in the form of a plus or minus
sign at fixation, for each response, following each trial.
Results
The data from 6 subjects were excluded because of accuracy less than 50%
in one or more cells in one or both tone tasks. The remaining 27 subjects
produced 15,552 experimental response pairs. Prior to RT analyses, the
correct trials were first screened for outliers using the same algorithm as
in experiment 1. Screening based on RT1 resulted in a loss of 2.8% of the
trials. The surviving trials were then screened for outliers on RT2, elimi-
nating an additional 2.2%. Analyses performed on the unscreened data
produced the same patterns of results as those reported below.
Task 2 The most important results concern mean RT2, as a function of
SOA and number of second-task alternatives, as shown in figure 13.3.
The m a i n effect of SOA w a s highly significant: F(3, 78) = 248.96,
p< 0.0001; as was the main effect of number of second-task alternatives:
F(1, 26) = 157.99, p< 0.0001. However, the interaction between these two
factors was not significant: F(3, 78) = 0.19, MSE = 1698.24, p>0.90. The
difference between the four-alternative condition and the two-alternative
condition was 199 msec at 50 msec SOA, 210 msec at 200 msec SOA.
200 msec at 500 msec SOA, and 203 msec at 1,100 msec SOA.
Accuracy in task 2 varied slightly as SOA increased (0.845, 0.859, 0.874,
and 0.866): F(3, 78) = 5.22, p< 0.0024. Accuracy was higher for two
(0.933) than for four alternatives (0.789): F(1, 26) = 160.65, p< 0.0001.
There w a s no interaction between these two variables, however:
F(3, 78) = 0.11, MSE = 0.001483 , p > 0.95.
Task 1 The mean RT1 for each SOA and each number of alternatives in
task 2 is also shown in figure 13.3. The main effect of SOA was significant:
F(3, 78) = 7.94, p < 0.0001. Mean RT1 was also longer when there were four
second-task alternatives (590 msec) than when there were two (544 msec):
F(1, 26) = 27.05, p < 0.0001. These two variables also interacted, as shown
in the figure: F(3, 78) = 13.66, p< 0.0001. The mean RT1 was constant
The Attentional Blink a n d the PRP
1100
1000
s
o
a
CO
0)
S
a
900 r
800
700
600
500
400 I-H
50 200 500
S O A ( m s )
1100
Figure 13.3 Results from experiment 2. Mean response time in task 2 (filled symbols, solid
lines) and in task 1 (open symbols, dashed lines), for each stimulus onset asynchrony (SOA),
and each level of second-task difficulty (two alternatives: circular symbols; four alternatives:
square symbols).
across SOAs for the two-alternative second-task condition, but it declined
with increasing SOA for the four-alternative second-task condition. It is
not clear to what these effects on RT1 should be attributed, but their small
magnitudes suggest that, in the main, the assumptions of the postpone-
ment model of PRP were not badly violated.
Mean first-task accuracy was slightly higher when there were two
alternatives in task 2 (0.962) than w h e n there were four (0.950):
F(1, 26) = 11.43, p < 0.0025. Neither the main effect of SOA nor the interac-
tion between SOA and n u m b e r of second-task alternatives was
significant (p < 1 in both cases).
Discussion
The results were clear-cut. The effects of varying the number of alterna-
tives in task 2 were additive with SOA. The implication is that this
manipulation had an effect that was in or after the PRP bottleneck. Given
that the manipulation in experiment 2 was identical to the one used in
task 1 of experiment 1, it is reasonable to assume that effects of number of
322 Jolicœur, Dell’Acqua, Crebolder
response alternatives in experiment 1 also took place at or after the stages
of processing constituting the PRP bottleneck.
It was important to test directly whether the number of alternatives has
its principal effect at or after the PRP bottleneck. A priori, one might have
expected some of the effect to occur relatively early, for example, d u e to a
greater difficulty of discrimination for four stimuli than for two stimuli.
The frequencies were equally spaced on a log scale (200, 363, 660, a n d
1,200 Hz), in an attempt to produce approximately equal steps in per-
ceived pitch; the two tones used in the two-alternative condition were
adjacent (363 and 660 Hz) in the sequence, in an attempt to equate the
degree of perceptual difficulty across the two conditions. It appears that,
under these conditions, the degree of perceptual difficulty in the two con-
ditions was very similar, such that the main difference between them was
later in processing, perhaps at response selection.
13.4 GENERAL DISCUSSION
In experiment 1, varying the number of stimulus-response alternatives in
task 1 of a speeded attentional blink paradigm produced a large modula-
tion of the AB effect. Response times in task 1 were clearly longer when
there were more response alternatives in task 1 than when there were
fewer. Changes in the duration of the stages of processing affected by this
manipulation carried over into accuracy in task 2. The manipulation
therefore h a d its effects at or before the locus of dual-task interaction in
the AB paradigm.
Experiment 2 showed that the manipulation of number of alternatives
h a d its effect in or after the PRP bottleneck, given that number of alterna-
tives in task 2 and SOA were additive in a PRP experiment (figure 13.1;
Pashler a n d Johnston 1989; Pashler 1994a; McCann a n d Johnston 1992).
Together, these two experiments lead to the conclusion that at least one
locus of interference contributing to the AB phenomenon is in or after the
PRP bottleneck.
The results converge nicely with those of Jolicœur (1998, 1999a,b,c) in
suggesting a close connection between the dual-task interference
observed in the AB a n d PRP paradigms. In experiment 1, large AB effects
were obtained using stimuli presented in different sensory modalities,
replicating a n d extending those of Jolicœur (1999a).
Attentional Blink versus Task Switch Costs
Potter et al. (1998) argued that there are two distinct attentional deficits in
serial target search tasks such as the one used in our experiment 1. One
deficit, the attentional blink (AB) hypothesized by Raymond, Shapiro,
a n d Arnell (1992), was claimed to occur only when both target stimuli are
The Attentional Blink a n d the PRP
visual, and not w h e n one or both are auditory. The other deficit, an
amodal effect, was hypothesized to be caused by capacity d e m a n d s of
task switching (as discussed in several other chapters in this volume).
If Potter et al. (1998) are correct, then one could argue that the observed
deficits in task 2 of experiment 1 were d u e to task switch costs, rather
than to the within-modality AB effect studied by several researchers (e.g.,
Raymond, Shapiro, a n d Arnell 1992; Ward, Duncan, and Shapiro 1996). It
could be that task switch costs take place later in the system than the dis-
tinct within-modality AB effect postulated by Potter et al. (1998). If so, the
evidence provided in our experiments 1–2 may apply only to the amodal
AB effect, a n d not to the within-modality AB effect.
The present results suggests that at least some component of the AB
effect occurs relatively late in the information-processing stream (i.e., at
or after the PRP bottleneck). Additional research will be required to deter-
mine whether our results apply to the within-modality AB effect, to the
postulated amodal effect, or to both.
Attentional Blink and Short-Term Consolidation
Jolicœur a n d Dell’Acqua (1998) showed that encoding information into
memory causes responses in a subsequent speeded task to be delayed. In
their experiment 7, every trial began with the presentation of one or three
letters exposed for 250 msec and followed by a pattern mask (100 msec).
On “encode’’ trials, the letters had to be reported, without speed pres-
sure, at the end of the trial. On “ignore’’ trials, the letters could be
ignored. On every trial, the second stimulus was a tone to which the sub-
jects m a d e a speeded pitch discrimination response (two-alternatives).
The SOA between the letter display and the tone w a s varied between 350
a n d 1,600 msec.
The response times to the tones are shown in figure 13.4 (solid lines,
filled symbols). Responses to the tone were delayed as the SOA between
the letters a n d the tone w a s reduced, but only w h e n the information h a d
to be encoded (top two functions). Minimal effects of SOA were found
when the letters could be ignored (bottom function). A larger effect of
SOA was found when more information h a d to be subjected to short-term
consolidation (encode-3) than w h e n less information had to be encoded
(encode-1).
The results of Jolicœur a n d Dell’Acqua (1998) did not constrain the
nature of the interference causing the delay in responses to the tone (i.e.,
postponement versus capacity sharing). Computer simulations, however,
showed that the results could be approximated reasonably well by
assuming that some stage of processing in the tone task (e.g., response
selection) was postponed for some time while short-term consolidation of
the letters w a s taking place (see figure 13.4; Jolicœur and Dell’Acqua
1998). These results support the view that the short-term consolidation of
Jolicœur, Dell’Acqua, Crebolder
Figure 13.4 Cost of short-term consolidation. Results from experiment 7 of Jolicœur a n d
Dell’Acqua 1998. Mean response time (RT1) to the tone (in milliseconds) for each stimulus
onset asynchrony (SOA), by whether the visual information was encoded (top t w o func-
tions) or ignored (bottom function). The results from the encode condition are further split
depending on the number of letters to be encoded (1, middle function; 3, top function). The
unfilled symbols joined by dotted lines show the results of a simulation in which it was
assumed that response selection in the tone task was postponed for some period of time
while the short-term consolidation of the information to be remembered was taking place.
information into a durable form of memory is a capacity-demanding
operation that can delay or slow d o w n other cognitive processes. C h u n
a n d Potter (1995) a n d Jolicœur (1998; Jolicœur a n d Dell’Acqua 1998, 1999;
Crebolder a n d Jolicœur forthcoming) argue that short-term consoli-
dation is a likely locus of the dual-task interference causing the AB
phenomenon.
Summary, Conclusions, and Implications for Control
We have presented t w o n e w experiments designed to provide constraints
on possible loci of interference contributing to the AB phenomenon.
Experiment 1 showed that a large AB effect in an RSVP scanning task
with a deferred response can be caused by performing a speeded
response to a p u r e tone (see also Crebolder a n d Jolicœur forthcoming;
Jolicœur, 1998, 1999a,b; Jolicœur a n d Dell’Acqua forthcoming). Fur-
thermore, larger effects resulted when the tone task h a d more stimulus-
response alternatives. Also, within each version of task 1, a larger AB
effect was found w h e n RT1 was longer. The manipulation of first-task
difficulty carried over strongly onto task 2, as expected if the manipula-
tion in task 1 h a d its effect in or before a locus of interference involved in
the AB phenomenon.
In experiment 2, the number of stimulus-response alternatives used in
experiment 1 (task 1) w a s n o w used in task 2 of a PRP experiment, a n d
The Attentional Blink a n d the PRP
the effects were additive with SOA. Thus the manipulation h a d its effects
in or after the PRP bottleneck. Consequently, at least one locus of inter-
ference causing the AB effect must be in or after the PRP bottleneck.
Crebolder a n d Jolicœur (forthcoming) performed a series of experi-
ments that had the same logical structure as those in this chapter. Rather
than manipulating number of alternatives in task 1, they varied the rela-
tive frequency of T1 in AB experiments a n d of T2 in PRP experiments. In
the AB experiments, T1 a n d T2 were both letters, and less frequent T1 sig-
nals caused larger AB effects. Hence the first-task manipulation carried
over onto task 2. These effects were found both when task 1 was speeded
a n d when task 1 was unspeeded. Furthermore, effects of the frequency of
T2 were additive with SOA in PRP experiments. These results show that
the conclusions based on experiments 1–2 extend to within-modality AB
paradigms, a n d to AB paradigms in which task 1 is deferred.
The results also suggest a closer connection between interference in the
AB paradigm and in the PRP paradigm than has heretofore been s u p -
posed (e.g., Shapiro a n d Raymond 1994; C h u n a n d Potter 1995; Ward,
Duncan, and Shapiro 1996). At least one major source of AB interference
appears to be at or after the same stage as the PRP bottleneck. Although
a locus after the PRP bottleneck is logically possible, this alternative
seems less likely than loci of interference that coincide in the two para-
digms. While, this contention needs to be further tested, experiment 5 of
Jolicœur 1999-b already provides evidence against a very late locus
involving motor codes.
Additional evidence for a similarity between AB interference and PRP
interference w a s also reviewed. The results of Jolicœur a n d Dell’Acqua
(1998, forthcoming) suggest that the short-term consolidation of visual
information into memory causes responses in a concurrent tone task
to be delayed, suggesting that short-term consolidation requires central
capacity-limited mechanisms.
We began this chapter by noting that the issue of control has figured
prominently in some recent theoretical work on the PRP phenomenon
(Meyer and Kieras 1997b), and of the AB phenomenon (Potter et al. 1998).
Our results suggest that an effect at or after the PRP bottleneck also con-
tributes substantially to the AB phenomenon. This effect, in the AB para-
digm, could not be d u e to the need to control order of responses because
the second response in that paradigm is not speeded. Given that there is
good evidence ruling out late (motor-coding) accounts of the interference
in such paradigms (e.g., Jolicœur 1999b), the most natural explanation of
AB effects in the Meyer-Kieras framework would be to suppose that
interference takes place early, in mechanisms that operate before central
processing. Such an account, however, runs into difficulty given that a
substantial component of the AB effect appears to be in or after the PRP
bottleneck. A likely locus of effect for number of response alternatives
(the main manipulation in our experiment 2) is response selection, a n d
Jolicœur, Dell’Acqua, Crebolder
that locus is clearly beyond the early locus of interference that would be
most easily incorporated into the Meyer-Kieras (1997) framework. This
suggests to us that there may be more significant sources of structural
central capacity limitations than are allowed for in that framework.
Indeed, the results suggest to us that structural central capacity limita-
tions, rather than the need to control response order, may be contributing
causes of both AB a n d PRP dual-task interference.
The consolidation of information into memory is one important pro-
cess required to perform task 2 in the AB paradigm (Chun a n d Potter
1995; Jolicœur 1998). The results of Jolicœur a n d Dell’Acqua (1998); (see
figure 13.4) strongly suggest that short-term consolidation of letters
causes dual-task slowing in a concurrent cross-modal speeded task.
Jolicœur a n d Dell’Acqua (1997) showed that encoding a r a n d o m polygon
also causes dual-task slowing. This latter result is important because ran-
d o m polygons do not have names in long-term memory, thus ruling out
explanations that hinge on the use of names to represent stimuli. The
results of Jolicœur and Dell’Acqua (1997, 1998) show that dual-task
slowing occurs even w h e n only one response is speeded. This slowing
cannot therefore be d u e to the need to control the order of output of two
rapidly produced responses.
Although we argue that dual-task slowing in the Jolicœur a n d
Dell’Acqua (1997, 1998) experiments was not caused by the consequences
of strategic control, we want to highlight the important role of control
for the results in that paradigm and in the AB paradigm. The key point is
that short-term consolidation is not obligatory but under active con-
trol. Indeed, in many AB experiments, the control condition consisted of
trials in which a salient target was shown, but could be ignored (e.g.,
Raymond, Shapiro, a n d Arnell 1992). These trials do not show the time-
locked performance deficit that characterizes the AB effect. Similarly,
“ignore’’ trials in the short-term consolidation experiments of Jolicœur
a n d Dell’Acqua (1997, 1998; see figure 13.4) do not exhibit the dual-task
slowing found w h e n the information has to be consolidated.
The selection of information to be consolidated a n d the onset of the
consolidation process itself are both controlled operations. Given that
short-term consolidation appears to involve a significant cost in terms
of the concomitant capacity d e m a n d s , a key role for control processes is
to minimize such costs by engaging capacity-demanding processes only
when they are necessary.
Clearly, we are still quite far from having achieved a complete under-
standing of the AB and PRP phenomena. The present results and the
evidence reviewed suggest, however, that a closer consideration of
the similarities a n d differences between the patterns of interference in
the AB paradigm a n d in the PRP paradigm is likely to provide useful
constraints on theorizing in both domains.
The Attentional Blink a n d the PRP
NOTE
This work was supported by a research grant from the Natural Sciences a n d Engineering
Research Council of Canada (NSERC). We thank Margaret Ingleton for technical assistance
a n d Stephen Monsell, Molly Potter, and an anonymous reviewer for their very helpful com-
ments and criticisms. Roberto Dell’Acqua received postdoctoral support from Fondation
Fyssen, a n d Jacquelyn Crebolder was supported by an NSERC doctoral fellowship.
REFERENCES
Allport, D. A., Styles, E. A., a n d Hsieh, S. (1994). Shifting intentional set: Exploring the
dynamic control of tasks. In C. Umiltà a n d M. Moscovitch (Eds), Attention and Performance
XV: Conscious and nonconscious information processing, p p . 421–452. Cambridge, MA: MIT
Press.
Broadbent, D. E., and Broadbent, M. H. P. (1987). From detection to identification: Response
to multiple targets in rapid serial visual presentation. Perception and Psychophysics, 42,
105–113.
Chun, M. M., and Potter, M. C. (1995). A two-stage model for multiple target detection in
rapid serial visual presentation. Journal of Experimental Psychology: Human Perception and
Performance, 21, 109–127.
Crebolder, J., and Jolicœur, P. (Forthcoming). On the locus of the attentional blink bottle-
neck: Evidence from signal probability effects in the AB a n d PRP paradigms.
De Jong, R., and Sweet, J. B. (1994). Preparatory strategies in overlapping-task performance.
Perception and Psychophysics, 55, 142–151.
Dell’Acqua, R., and Jolicœur, P. (1998). Evidence for dual-task interference on visual en-
coding. Poster presented at the Eighteenth International Symposium on Attention and
Performance, Windsor Great Park, England.
Duncan, J., Ward, R., and Shapiro, K. L. (1994). Direct measurement of attentional dwell
time in h u m a n vision. Nature, 369, 313–315.
Giesbrecht, B. L., and Di Lollo, V. (1998). Beyond the attentional blink: Visual masking by
item substitution. Journal of Experimental Psychology: Human Perception and Performance, 24,
1454–1466.
Jolicœur, P. (1998). Modulation of the attentional blink by on-line response selection:
Evidence from speeded a n d unspeeded task 1 decisions. Memory and Cognition, 26,
1014–1032.
Jolicœur, P. (1999a). Restricted attentional capacity between sensory modalities. Psycho-
nomic Bulletin and Review, 6, 87–92.
Jolicœur, P. (1999b). Dual-task interference a n d visual encoding. Journal of Experimental
Psychology: Human Perception and Performance, 25, 596–616.
Jolicœur, P. (1999c). Concurrent response selection d e m a n d s modulate the attentional blink.
Journal of Experimental Psychology: Human Perception and Performance, 25, 1097–1113.
Jolicœur, P. , and Dell’Acqua (1997). Short-term consolidation of random polygons causes
dual-task slowing. Paper presented at the annual meeting of the Psychonomic Society,
Philadelphia.
Jolicœur, P., and Dell’Acqua, R. (1998). The demonstration of short-term consolidation.
Cognitive Psychology, 36, 138–202.
Jolicœur, Dell’Acqua, Crebolder
Jolicœur, P. , and Dell’Acqua, R. (1999). Attentional and structural constraints on visual
encoding. Psychological Research/Psychologische Forschung, 62, 154–164.
Jolicœur, P. , a n d Dell’Acqua, R. (Forthcoming). Selective influence of second target exposure
duration and task 1 load effects in the attentional blink phenomenon. Psychonomic Bulletin
a n d Review.
Luck, S. J., Vogel, E. K., and Shapiro, K. L. (1996). Word meaning can be accessed but not
reported during the attentional blink. Nature, 382, 616–618.
McCann, R. S., a n d Johnston, J. C. (1992). Locus of the single-channel bottleneck in dual-
task interference. Journal of Experimental Psychology: Human Perception and Performance, 18,
471–484.
Meyer, D. E., and Kieras, D. E. (1997a). A computational theory of executive cognitive
processes and human multiple-task performance: 1. Basic mechanisms Psychological Review,
104, 3–65.
Meyer, D. E., a n d Kieras, D. E. (1997b). A computational theory of executive cognitive
processes and h u m a n multiple-task performance: 2. Accounts of psychological refractory-
period phenomena. Psychological Review, 104, 749–791.
Monsell, S. (1996). Control of mental processes. In V. Bruce (Ed.), Unsolved mysteries of the
mind: Tutorial essays in cognition, p p . 93–148. Hove, U.K.: Erlbaum and Taylor and Francis.
Pashler, H. (1993). Dual-task interference a n d elementary mental mechanisms. In D. E.
Meyer and S. Kornblum (Eds.), Attention and Performance XIV: Synergies in experimental psy-
chology, artificial intelligence, and cognitive neuroscience, p p . 245–264. Cambridge, MA: MIT
Press.
Pashler, H. (1994a). Dual-task interference in simple tasks: Data and theory. Psychological
Bulletin, 116, 220–244.
Pashler, H. (1994b). Overlapping mental operations in serial performance with preview.
Quarterly Journal of Experimental Psychology, 47A, 161–191.
Pashler, H., a n d Johnston, J. C. (1989). Chronometric evidence for central postponement in
temporally overlapping tasks. Quarterly Journal of Experimental Psychology, 41A, 19–46.
Potter, M. C., Chun, M. M., Banks, B. S., and Muckenhoupt, M. (1998). Two attentional
deficits in serial target search: The visual attentional blink and an amodal task-switch
deficit. Journal of Experimental Psychology: Learning, Memory, and Cognition, 24, 979–992.
Raymond, J. E., Shapiro, and Arnell, K. M. (1992). Temporary suppression of visual pro-
cessing in an RSVP task: An attentional blink? Journal of Experimental Psychology: Human
Perception and Performance, 18, 849–860.
Raymond, J. E., Shapiro, K. L., and Arnell, K. M. (1995). Similarity determines the atten-
tional blink. Journal of Experimental Psychology: Human Perception and Performance, 21,
653–662.
Rogers, R. D., and Monsell, S. (1995). Costs of a predictable switch between simple cogni-
tive tasks. Journal of Experimental Psychology: General, 124, 207–231.
Schubert, T. (1999). Some more evidence for a central bottleneck in dual-task performance.
Journal of Experimental Psychology: Human Perception and Performance, 25, 408–425.
Schumacher, E. H., Lauber, E. J., Glass, J. M., Zurbriggen, E. L., Gmeindl, L., Kieras, D. E.,
a n d Meyer, D. E. (1999). Concurrent response selection in dual-task performance: Evidence
for adaptive executive control of task scheduling. Journal of Experimental Psychology: Human
Perception and Performance, 25, 791–814.
The Attentional Blink a n d the PRP
Shapiro, K. L., Driver, J., Ward, R., and Sorensen, R. E. (1997). Priming from the attentional
blink: A failure to extract visual tokens but not visual types. Psychological Science, 8, 95–100.
Shapiro, K. L., and Raymond, J. E. (1994). Temporal allocation of visual attention: Inhibition
or interference? In D. Dagenbach and T. H. Carr (Eds.), Inhibitory processes in attention, mem-
ory, and language, p p . 151–188. San Diego, CA: Academic Press.
Shapiro, K. L., Raymond, J. E., and Arnell, K. M. (1994). Attention to visual pattern infor-
mation produces the attentional blink in rapid serial visual presentation. Journal of Experi-
mental Psychology: Human Perception and Performance, 20, 357–371.
Smith, M. C. (1967). The psychological refractory period as a function of performance of a
first response. Quarterly Journal of Experimental Psychology, 19, 350–352.
Van Selst, M., a n d Jolicœur, P. (1994). A solution to the effect of sample size on outlier elim-
ination. Quarterly Journal of Experimental Psychology, 47A, 631–650.
Van Selst, M., and Jolicœur, P. (1997). Decision a n d response in dual-task interference.
Cognitive Psychology, 33, 266–307.
Van Selst, M., Ruthruff, E., and Johnston, J. C. (1999). Can practice eliminate the psycho-
logical refractory period effect? Journal of Experimental Psychology: Human Perception and
Performance, 25, 1268–1283.
Ward, R., Duncan, J., a n d Shapiro, K. L. (1996). The slow time-course of visual attention.
Cognitive Psychology, 30, 79–109.
Welford, A. T. (1952). The “psychological refractory period’’ a n d the timing of high-speed
performance: A review and theory. British Journal of Psychology, 43, 2–19.
Williams, L. R. T. (1974). Effects of number of alternatives on the psychological refractori-
ness of an extended movement. Journal of Motor Behavior, 6, 227–234.
330 Jolicœur, Dell’Acqua, Crebolder
14 Intentional Reconfiguration and Involuntary Persistence in Task Set
Switching
Thomas Goschke
ABSTRACT Switching between different tasks often increases response time compared to
repeated performance of a task. This switch cost has been thought to reflect either an exec-
utive process of task set reconfiguration or proactive interference from competing task sets.
This chapter tries to reconcile these views by showing that switch costs are influenced both
by voluntary preparation and involuntary carry-over of inhibition and stimulus-response-
bindings from the previous trial. Three experiments are reported in which participants
switched between responding to the color and responding to the identity of letters. Switch
costs were reduced when participants verbalized each task before the stimulus, compared
to when they performed a verbal distractor task, suggesting that intention retrieval sup-
ported advance reconfiguration. Switch costs increased when the two stimulus dimensions
activated incongruent responses and when task switches followed incongruent trials, indi-
cating persisting activation of preceding task sets a n d persisting inhibition of irrelevant
perceptual dimensions, S-R mappings, or both. Findings suggest that voluntary actions are
not controlled by a unitary central executive, but emerge from the interaction of separable
component processes, some maintaining intentions, others reconfiguring task sets.
According to the proposed model, seemingly dysfunctional aspects of cognitive control are
manifestations of adaptive mechanisms that have evolved to satisfy partially incompatible
constraints on action control.
14.1 INTENTIONAL RECONFIGURATION AND COGNITIVE
CONTROL
A remarkable property of willed action is its flexibility: by receiving an
instruction or forming an intention, we can transiently couple almost any
response to almost any stimulus or aspect of a stimulus, even when there
are neither innate nor acquired connections between stimulus and re-
sponse. For instance, if you, as a participant in a psychological experi-
ment, are instructed to press
