The value of connectionist models in neuropsychology: A case study of neglect - MSc Thesis J.H. Fabius Neuroscience & Cognition (Cognitive Neuroscience), Life Sciences, Utrecht University Supervisor: J.M.J. Murre Brain and Cognition, University of Amsterdam 1. Introduction Neuropsychology is concerned with the relationship between the injured brain and consequent abnormal behaviour (Deelman & Eling, 2004; Heilman & Valenstein, 2011; Lezak, 2004). Recent developments have contributed to fine-tuned neuropsychological assessments, in which the diagnosis, prognosis and rehabilitation program can be adapted to the clinical picture of an individual patient (Braun et al., 2011). Ultimately, the goal of neuropsychological science is to understand the neurobiology underlying cognitive symptoms (Lezak, 2004), and to construct effective rehabilitation programs (Cicerone, 2012). Despite the availability of cognitive rehabilitation programs for patients with neuropsychological disorders, studies of their effectiveness are not unanimously supportive (Cicerone et al., 2011; Hoffman, Bennett, Koh & McKenna, 2011; Fierro, Brighina & Bisiach, 2006; Powel et al., 2012; Rohling, Faust, Beverly & Demakis, 2009). Arguably, the scarcity of effective rehabilitation programs is partially the result of an insufficient understanding of the neurobiology underlying the disturbed cognitive function (Cernich, Kurtz, Mordecai & Ryan, 2010). In order to successfully set up rehabilitation programs, it is necessary to have a better understanding of how the brain works under normal conditions and how it gives rise to cognition (Bressler & Tognoli, 2006; Goldrick, 2008; Mogensen & Malá, 2009; Schierwagen, 2012). Unfortunately, although the way cognition arises from its underlying neurobiological structure is being clarified more and 1 more, modern neuropsychology is still unable to explain many cognitive disorders from a purely neurobiological perspective (Bilder, 2011; Vakil, 2012). Therefore, models are being used to be able to work with patients with neurological damage. This thesis will elaborate on the contribution that connectionist models can make to the development of neuropsychological methods. However, since appraising connectionism for all known neuropsychological disorders would be a very extensive read, this thesis will be limited to the neuropsychology of unilateral spatial neglect. To investigate what connectionism might be able to contribute, or has already contributed to the understanding of unilateral spatial neglect, is interesting for two reasons. First, many aspects of this disorder are still debated (Corbetta & Shulman, 2002, 2011; Hillis, 2006), yet it is a frequently observed and seriously disabling disorder (Bartolomeo, 2007; Gillen, Tennen & McKee, 2005; Nys et al., 2005; Ringman, Saver, Woolson, Clarcke & Adams, 2004). Moreover, although progress is being made (Hesse, Sparing & Fink, 2011), no effective rehabilitation programs for neglect have been established yet (Bowen, Lincoln & Dewey, 2002; Bowen & Wenman, 2002; Lincoln & Bowen, 2006). Connectionist models might be able to shed new light on some of the ambiguities. Second, models of neglect have not yet been reviewed, unlike models of disorders in other cognitive domains such as language or memory (see e.g. Bechtel & Abrahamsen, 2002; or Thomas & McClelland, 2008). Hence, since neglect both raises several important clinical questions and its connectionist models have not been reviewed thoroughly, this thesis will focus on this particular disorder. This will not be a historical overview of all contributions connectionism has made in the past to a better understanding of cognitive processes (for such overviews see e.g. Bechtel & Abrahamsen, 2002, or O’Reilly & Munakata, 2000), but rather a pragmatic approach of how neuropsychology could view these contributions, as exemplified with the case of neglect. Before the existing models of neglect will be reviewed, first the current contributions of connectionism to cognitive neuroscience will be illustrated. Subsequently, a short introduction to connectionist modelling will be given, covering some general concepts that are important to understand in order to discuss the existing models. Next, the current understanding of unilateral spatial neglect will be discussed. Then, the connectionist models of neglect will be reviewed individually, followed by a general discussion, in which the models will be compared and an evaluation will be given of the overall contribution of the models. 2. Why connectionism? Since the dynamics of the entire brain are still unknown, simplified models of cognitive functions have been used and localized to cortical areas in order to construct more accurate diagnoses and prognoses for patients with neurological damage. Classically, behavioural experiments gave rise to models of cognitive functions, which were then applied to patients with neurological damage (Deelman et al., 2004; Gazzaniga, Ivry & Mangun, 2009a). The models of cognitive functions that are applied to neuropsychological patients consisted of boxes and arrows (Bechtel & Abrahamsen, 2002; Gazzaniga et al., 2009a). At the starting point of a model is an external input. The input goes into a (series of) box(es) in which cognitive operations are performed. The operations, consequently, yield a new representation, i.e. the output. Bechtel and Abrahamsen (2002) refer to this approach of cognition as the ‘symbolic paradigm’, considering the central role the ‘symbols’ (i.e. the input of a box) play. The authors argue that with this approach, a cognitive function is defined by one or 2 several operations that can be performed on a symbol. The problem with the approach of boxand-arrow models is that the definitions of different operations can be quite abstract (e.g. ‘grammatical encoding’), and, thus, a topic of debate. Moreover, the abstract definition causes problems in linking an operation to neuronal activity (Mogensen & Malá, 2009; Price & Friston, 2005). Indeed, a more modern view of cognitive neuroscience on the relationship between brain structure and cognition is shifting from a localization paradigm (i.e. discrete anatomical modules for different cognitive functions) to a large-scale network paradigm. The network paradigm states that the human brain consists of numerous networks, made up of multiple cortical areas that are functionally connected to each other (Bressler & Menon, 2010; Seghier, Zeidman, Neufeld, Leff & Price, 2010; Zamora-López, Zhou & Kurths, 2010). Furthermore, a cognitive module is not (necessarily) represented by a single anatomical structure. In other words, there is no one-on-one correspondence between cognitive and anatomical modularity (Bressler & Tognoli, 2006; Medler, Dawson & Kingstone, 2005; Mogensen & Malá, 2009; Price & Friston, 2005). In fact, exactly how these networks give rise to cognition has yet to be unravelled. However, even though this coupling needs further research, it has been argued that it is highly unlikely that the brain contains neurons for operations such as ‘grammatical encoding’, and that it should therefore be the aim of cognitive neuroscience to elaborate on the contributions of cortical modules in terms of ‘simple’ manipulations on their input (Mogensen & Malá, 2009; Price & Friston, 2005). A subdivision of cognitive neuroscience that has specialized in elaborating on the bridge from simple computations to complex behaviour is connectionism, using artificial neural networks to model cognitive processes (Bechtel & Abrahamsen, 2002). In its relatively short history as a field of research within the cognitive neuroscience, connectionism has shown to be able to provide arguments in debates on whether or not a set of assumptions is sufficient to explain a complex theory. This quality is referred to as existence proof, meaning that a model does not aim at falsifying a theory, but rather showing that a certain set of assumptions is sufficient to address the characteristics of a concept that a cognitive theory tries to explain (McClelland, 2008; Meeter, Jehee & Murre, 2007). Perhaps the best known example of this has been provided by the model of Rumelhart and McClelland, on the transformation of English verbs to their past tense forms (Rumelhart & McClelland, 1986, in Thomas & McClelland, 2008). This model shed new light on the at that time predominant linguistic theory that children acquire the correct past tense of English verbs by learning a rule for the regular verbs and by learning the association between the present and past tense of irregular verbs by experience. The model showed that is also possible to learn a correct past tense transformation without any assumptions on the distinction between regular and irregular verbs (see Thomas & McClelland, 2008 for more explicit details and the consequences of this model for linguistic theory). Although this is not the case for the past tense model, many connectionist models aim to be biologically plausible, making them particularly interesting for neuropsychology because then they might explain or even predict human performance given a specific configuration (Meeter et al., 2007). For neuropsychologists, the models would be even more interesting if they could be lesioned and show the same pathological behaviour as a particular group of patients. Thus, the models could give insight into the specific cognitive processes that have been damaged in those patients. For example, connectionists have constructed models of cognitive disorders such as aphasia (e.g. Dell, Chang & Griffin, 1999; Plaut, 2002) amnesia (e.g. Meeter & Murre, 2005; Nadel, Samsonovich, Ryan & Moscovitch, 2000) and even of complex psychiatric disorders such as schizophrenia (e.g. Aakerlund & Hemmingsen, 1998; Cohen, Braver & O’Reilly, 1996; Cohen & Servan-Schreiber, 1992; Rolls et al., 2008). With the knowledge provided by these lesioned A1 A2 0.6 -0.3 -0.3 0.6 B1 0.6 3. Basic modelling C1 B2 0.6 principles of connectionist In connectionism an artificial neural network is constructed of simple units, representing neurons. Like neurons, the units can be either transmitting their activity or not (i.e. firing or not), depending on the input they receive. There are several ways by which it can be determined whether or not a unit ‘fires’ or not. A simple way is having a unit become activated when it receives enough input to rise above a certain threshold. Units are connected to each other with connections of varying strengths, called weights. The weight of a connection between two units can be either positive (excitatory) or negative (inhibitory), typically ranging from -∞ to ∞. To get a grasp of what a connectionist model looks like, a simplistic model is depicted in figure 1A. This network consists of five units (A1 ... C1), spread over three layers (A, B and C) with weights of either 0.6 or -0.3. The activity of each unit is updated synchronously per layer. When the sum of the input exceeds the threshold of a unit, it is activated. For example, all units have a threshold (θ) of 0.6. If only unit A1 is activated, then B1 will A1 Figure 1a. Simple three layer network, containing five nodes. C1 will only be activated if either A1 or A2 is activated, not both or neither. 3 networks, performance of the patient and network could be compared. Then, it would theoretically be possible to define to what extent the network is still intact, and, subsequently, to construct a rehabilitation program, focussing on either the recovery of the damaged operations, or on the training of the operations that are still intact (Robertson & Murre, 1999). Which of these two different rehabilitation approaches would be favourable for an individual patient could be determined with knowledge of the network. Hence, connectionism could contribute to a more fundamental understanding of neuropsychological symptoms and, eventually, to the construction of more effective rehabilitation programs. A2 0.6 0.3 -0.3 0.3 B1 0.6 C1 B2 0.6 Figure 1b. Network with a design similar to 1a, but with different weights. This changes the way by which C1 will be activated. become activated and consequently C1. B2 will not be activated because A1 inhibits B2. With this design, C1 is only activated when either A1 or A2 is activated, but not when both or neither are activated. However, if the weights would be adjusted, C1 could, for example, become active if either A1 alone or both A1 and A2 were activated, but not when A2 alone was activated (see fig. 1B). Adjustment of the weights is central to many connectionist networks, because this allows the network to ‘learn’ associations between input and output patterns. Thus, information can be stored in the way the units are connected to each other. For neurobiological plausibility, units should not be activated indefinitely. Therefore, once a unit has become activated, there are two main ways by which it can become inactive again. First, the unit could become inactive when its input is below its threshold. Second, similar to real neurons, the activation level of the unit could decay over time from the point when it became activated. Time in this sense is a model of the ‘real’ time. One unit of time, called an iteration, is defined as a single update of the activations of the units. Furthermore, many physiological features of real neurons can be implemented in the behaviour of the units, such as the dynamics of an action potential, instead of having the unit being artificially ‘on’ or ‘off’ (O’Reilly & Munakata, 2000). Of course, more complex networks can store more complex information (such as accurate discriminating between different input patterns). In the simple model presented in figure 1 only feedforward connections are implemented. However, networks could also contain (indirect) feedbackward connections to provide top-down feedback, and lateral connections (i.e. connections within the same layer) to enable ‘competition’ between units. Competition is favourable when, in a layer only a single unit is supposed to be active, called a winner-takes-all principle. This, in its simplest form, implies a gating mechanism which ensures that the unit (or group of units) with the highest input will be activated, in contrast to the units in the same layer with a lower input (Maass, 2000). This is, for example, used to model word recognition based on the identification of simple features of letters (see e.g. the network in fig. 2 from the classical work of McClelland & Rumelhart, 1981). The units in the first layer representing the simple features (i.e. line configuration at a specific location in the word presented to the model), have excitatory connections to the letters, represented in the next layer, that contain those features (in fig. 2 those connections have arrow endpoints) and inhibitory connections to the letters that do not contain those features (in fig. 2, circular endpoints). The units in the second layer have lateral inhibitory connections. So, when one letter is processed, some features may activate multiple units in the second layer, but due to the lateral inhibition, the unit representing the presented letter will quickly inhibit the units that correspond only to some of the features of the presented letter. The same applies to the third layer. This concludes some of the basic principles that are found in most of the models that will be discussed further on. 4. The neuropsychology of unilateral spatial neglect 4.1 Clinical presentation Unilateral spatial neglect refers to an impaired ability to direct attention to the contralesional space without primary sensory deficits (Heilman, Watson & Valenstein, 2011). Patients frequently recover from neglect within the first twelve months after a stroke (Karnath, Rennig, Johannsen & Rorden, 2011). Nevertheless, the implications of neglect are severe: it disrupts daily Figure 2. McClelland and Rumelharts interactive model of context effects in letter perception. (Adopted from McClelland & Rumelhart, 1981). 4 activities (Bartolomeo, 2007) and is one of the best negative predictors for stroke recovery (Gillen et al., 2005; Nys et al., 2005). It has been found that the disorder can potentially affect all sensory modalities, but it is most frequently observed in the visual domain, possibly because in that form it is most disabling (Brozzoli, Dematte, Pavani, Frassinetti & Farne, 2006; Hillis, 2006). Therefore, this thesis will focus on unilateral visuospatial neglect. Hence, from here on, “neglect” will refer to unilateral visuospatial neglect. In the following paragraphs the behavioural characteristics and neurobiology of neglect will be discussed. 4.2 Characteristics of neglect Clinical assessment tools – In clinical settings, neglect is most frequently assessed using one out of two tasks that are easy to conduct: the cancellation task and the line bisection task (Parton, Malhotra & Husain, 2004). The typical performance of a neglect patient is a failure to cancel out the targets at the contralesional side of a scene and a deviation to the right on the line bisection task. In other more experimental tasks, patients with neglect often fail to fixate attention at targets on the contralesional side of a scene during a visual search task, show spontaneous rightward eye movements, fail to copy the left side of a picture, orienting task (Bartolomeo, 2007, Corbetta et al., 2005; Corbetta & Shulman, 2011; Fruhmann-Berger & Karnath, 2005; Molenberghs & Sale, 2011). Despite the common use of the line bisection and cancellation task it is still under debate whether or not the tasks are sensitive to the same deficit (Ferber & Karnath, 2010; Molenberghs & Sale, 2011; Molenberghs, Sale & Mattingley, 2012; Rorden, Fruhmann-Berger & Karnath, 2006). Attentional gradient – One of the most accepted characteristics of neglect is that inability to direct attention to stimuli follows a gradient across the visual space (Kinsbourne, 1993). This gradient implies that the probability of selecting a stimulus is high for stimuli located at the right side of space (ipsilesional), and decreasing for stimuli located more to the left (contralesional). However, the gradient is not monotonically increasing from 5 the extreme left to the extreme right, but rather has a peak somewhere in the right visual field and decreases again to the extreme right (Behrmann, Watt, Black & Barton, 1997; Karnath, Mandler & Clavagnier 2011; Müri, Cazzoli, Nyffeler & Pflugshaupt, 2009;). Frames of reference – Neglect is classically associated with egocentric frames of reference, i.e. neglect of stimuli with respect to the body of the patient (Brozzoli et al., 2006; Corbetta & Shulman, 2011; Hillis, 2006). Nevertheless, patients who showed neglect within an allocentric Frame of reference, i.e. neglecting a part of the stimulus, independent of the egocentric location of the stimulus, have been observed as well (Brozzoli et al., 2006; Corbetta & Shulman, 2011; Hillis, 2006). This has been investigated using cancellation tasks with stimuli consisting of small circles. The targets were circles with an opening at either the left or right side, distractors were closed circles. Targets and distractors were evenly distributed across the scene. Some patients omitted targets, independent of the lateralization of their opening, when they were located on the contralesional side of the scene with respect to themselves (egocentric Frame of reference), whereas other patients omitted the targets with an opening on the contralesional side, independent of their egocentric location (allocentric Frame of reference). The two types of Frames of reference in neglect seem to be both behaviourally and anatomically dissociable (Corbetta & Shulman, 2011). In addition, egocentric neglect occurs more often than allocentric neglect (Marsh & Hillis, 2008). However, it has been suggested that the dissociation between egocentric and allocentric neglect can be explained from a purely egocentric perspective, considering the attentional gradient (Driver & Pouget, 2000). By assuming that neglect is most severe at the egocentric left and decreasingly less severe toward the egocentric right, the left side of both the entire visual space and individual objects would be more neglected than the right. The theory of Driver and Pouget (2000) has lead to the hypothesis that the severity of allocentric neglect is associated with the severity of egocentric neglect, which has been shown by several studies (e.g. Karnath et al., 2011; Rorden et al., 2012). Hence the theory questions the suggested behavioural and anatomical dissociations. According to Karnath and Rorden (2012), egocentric neglect is the core deficit in neglect, and neglect severity can be estimated by calculating the ‘centre of cancellation’ on a cancellation task (Rorden & Karnath, 2010). Nevertheless, different lesion sites have been suggested to underlie the different deficits (Chechlacz, Rothstein, Bickerton, Hansen, Deb & Humphreys, 2010; Yue, Weiqun, Huo & Wang, 2012). Extinction – Another attentional phenomenon associated with neglect is extinction. Extinction is a condition in which a patient can direct his/her attention toward stimuli on either side of his body, but when two equally salient stimuli are presented simultaneously, the stimulus on the ipsilesional side of visual space ‘wins’ the attention competition (Heilman et al., 2011). Extinction is thought to be associated with a different type of lesion than neglect, or at least, less extensive (Karnath & Rorden, 2012; Umarova et al., 2011; Vossel, Eschenbeck, Weiss, Saliger, Karbe & Fink, 2011), as will be discussed in the next paragraph. 4.3 Neurobiology Right hemisphere lateralization – Neglect is observed, mainly, after a stroke in the right hemisphere (Bartolomeo, Thiebaut de Schotten & Doricchi, 2007; Kleinman, Newhart, Davis, HeidlerGary, Gottesman & Hillis, 2007; Ringman et al., 2004). It has been estimated that up to 90% of all patients with neglect has had a right hemisphere stroke (Corbetta, Kincade, Lewis, Snyder & Sapir, 2005). Moreover, approximately 25% of the patients with a stroke in the right hemisphere has been diagnosed with neglect (Ringman et al., 2004; Vossel et al., 2011). Given that not all patients with a right-hemispheric stroke suffer from neglect much research has been done investigating the anatomical structures that are commonly damaged in neglect patients. Major theories – (1) Heilman and Van den Abell (1980) and Mesulam (1981, 1991) suggest that parietal and frontal regions in the right hemisphere direct spatial attention towards both the ipsi- and contralateral side of space, while the same regions 6 in the left hemisphere direct attention exclusively to the contralateral space. A right hemispheric lesion would therefore result in the absence of a system orienting attention towards the left side of space. (2) Kinsbourne (1977) argues that both hemispheres are capable of directing attention to either side of space, but each hemisphere is biased to direct attention toward the contralateral side. In addition, the left hemisphere is more heavily biased toward the contralesional space than the right hemisphere, resulting in bias toward the right side of space in the healthy brain. According to this theory, a right hemispheric lesion would be followed by an increased bias to direct attention toward the right visual field. Integration of lesions and theories – The influential works of Corbetta and Shulman (2002; 2011) and Karstner and Ungerleider (2000) state that both hemispheres contain networks for visual selection that are biased toward the contralateral space and that are mutually inhibitory. To accomplish visual selection, they integrate topdown and bottom-up information to create salience values for all presented stimuli. These networks are thought to consist of dorsal frontoparietal areas. The networks compete with each other to direct attention to the stimulus with the highest salience in their receptive field. In contrast to the bilateral dorsal networks, only the right hemisphere contains a network consisting of ventral frontoparietal areas that is specialized for reorientation to novel or behaviourally relevant stimuli, independent of their location. However, to direct reorientation of attention to a novel or relevant stimulus, coactivation of the dorsal network is required. Hence, it is hypothesized that the ventral network interrupts the ongoing activity of the dorsal network, thus forcing the dorsal frontoparietal network to reorient and select the novel stimulus. Corbetta and Shulman (2002, 2011) distinguish between spatial and non-spatial deficits of neglect. The spatial deficit comprises the egocentric bias in spatial attention, and arises from damage to the right dorsal network. As mentioned earlier, the dorsal networks in both hemispheres compete with each other in the selection of a visual stimulus. Both networks are biased to the contralateral visual field. Hence, when competition is not equal anymore (as a consequence of a weakened right dorsal network), the left dorsal network will almost always ‘win’ the competition. An interesting line of evidence for this distorted competition lies in non-invasive brain stimulation. It has been shown that neglect can virtually be induced in healthy subjects by stimulating the right hemisphere with transcranial magnetic stimulation (TMS; Sack, 2010). Moreover, neglect symptoms have been shown to be ameliorated when TMS is performed on the left, intact, hemisphere, and thereby hypothetically decreasing the superiority of the left dorsal network in the competition for visual selection (Fierro, Brighina & Bisiach, 2006; Hesse et al., 2011; Koch et al, 2012; Oliveri, 2011). The nonspatial deficits are an impaired attentional disengagement and reorientation, and impaired detection of behaviourally relevant stimuli. Damage to the ventral network underlies the non-spatial deficits of neglect. Over the last decade there has been a growing consensus, in favour of the third theory. As shown by lesion studies, neglect most frequently occurs after lesions to the right temporoparietal junction (TPJ), the inferior parietal lobule (IPL), the superior and middle temporal gyri (STG and MTG, resp.) and the ventral lateral prefrontal cortex (vlPFC; Bartolomeo et al., 2007; Corbetta et al., 2005; Karnath & Rorden, 2012; Molenberghs et al., 2012; Urbanski et al., 2011). Especially the disruption of the connections between these areas (indicating damage to the superior longitudinal fascicle subunits II/III) could cause the symptoms of neglect (Bartolomeo et al, 2007). Except for the IPL, all areas were localized in the ventral frontoparietal network by Corbetta and Shulman (2002, 2011). Arguably, the non-spatial deficits that are related to a lesioned ventral frontoparietal network seem to correspond with the clinical description of extinction, which is just what lesion studies suggest. Several studies have shown that patients with extinction, but without neglect have more focal lesions, located in the areas that comprise the ventral frontoparietal network, while their dorsal frontoparietal network is relatively spared (Karnath & Rorden, 2012; Vossel et al, 2011). In contrast, the spatial deficits of neglect were associated with damage to the dorsal frontoparietal network 7 (Karnath & Rorden, 2012; Umarova et al., 2011; Vossel et al, 2011). 5. Connectionist models of neglect In the next section, eight connectionist models will be summarized and appraised for both the symptoms of neglect they explain and the method by which the symptoms are induced. The models will be presented in a chronological order of publication, because the authors often seem to be influenced by each other. By presenting the models in a chronological order it may become clear how the models complement or improve each other. If the authors have not named their model, it will be referred to as the name of the first author between brackets (for example, [Fabius]). For each model several aspects will be discussed. Each description starts with a brief overview of the architecture of the model (in the section ‘design’). The technical details of, for example, update rules and connections between individual units will not be discussed unless they have significant importance for the manifestation of neglect. These details can be found in the papers cited in the headings of the model’s paragraph. After the design, the way the performance of the model is observed will be discussed (‘performance’). Then, the method by which the model is virtually lesioned is explained (‘lesion’), and how that affects the model’s performance (‘results’). Finally, an appraisal of the strengths and limitations of the model is given (‘evaluation’). 5.1 Basis function model (Pouget & Sejnowski, 1996, 1997a, 1997b, 2001). Design – The basis function model of Pouget and Sejnowski was originally designed to model cortical processing of object location by integrating sensory input of different modalities, without explicitly encoding frames of references. The architecture of the model is based on the observation that the brain integrates information of different modalities that all code for object position. In their model, Pouget and Sejnowski used retinal and eye-position coordinates. The model (see fig. 3) consists of two input maps, a retinotopic (V1) and an eye-position map (thalamus). These maps project to an intermediate map, the ‘basis function map’ (area 7a of the macaque cortex). This map is the core of the model. There is one unit in the map for one of all possible combinations of retinal and eye-position coordinates. In the first paper (1996) there is one basis function map. In the third paper (1997b), it is specifically stated that there is one basis function map for each hemisphere. The units in the basis function maps are tuned in such a way that a contralateral activity gradient is induced, e.g. in the left hemispheric basis function map, the units are on average activated most strongly when an object appears on the right side of the retina (or V1) and when the eyes are turned to the right. After this transformation, the basis function map projects to two motor maps, one to direct saccadic eye movements based on retinotopic coordinates (superior colliculus) and another to direct reaching movements based on head-position coordinates (premotor cortex). When multiple stimuli are presented, selection is based on the salience value of the presented stimuli. The salience value of a stimulus is defined as “the sum of activities of all the basis function units that have their receptive field centered on the retinal position of the stimulus” (1997b). The stimulus that will be selected is the one with the highest salience (i.e. winner-takes-all). At the next time step, the salience of the previous winner is set to zero (inhibition of return; for further information on this principle see box 1.) in the basis function map. To observe which stimulus had been selected, the profile of activity of the module representing superior colliculus (see fig. 3) was monitored, since the units in that map are activated according to a mathematically equivalent method. In the third paper (1997b), reaction times were also monitored to observe the effect of distractors. Reaction time was determined by target selection and target processing. Target selection was arbitrarily set to take 50 ms per iteration. Target processing was set to be inversely proportional to the stimulus salience. Performance – The goal of the model is to have the basis function map select a stimulus for action, i.e. reaching or a saccadic eye movement. Lesion – In the first paper (1996), neglect is induced by tuning more neurons to right retinal and eye-position coordinates. However, since in the third and fourth paper (1997b), both basis function maps are already biased to the contralateral side, the entire right basis function map is removed, leaving the model in the same state as in the first paper, i.e. one map with a contralateral activity gradient. Figure 3. The basis function model. Each dot represents a unit. Input enters the model in the retinotopic map, which represents V1. The eye position cells provide another input, representing the thalamus. (Adopted from Pouget & Sejnowski, 2001). 8 Results – Neglect behaviour was observed in a line cancellation (1996, 1997b) and a line bisection task (1997b). (a) In the line cancellation the model was presented with a stimulus display consisting of multiple small lines. A cancellation was modelled with the selection of a line. Due to the activity gradient in the basis function map, only the lines on the right half of the display were selected. Stimulus selection started on the outer most right side of the display (as a consequence of the higher salience of the right stimuli and the winner-takes-all principle), progressing to the left (as a consequence of the inhibition of return). How far the stimuli to the left of the display will be selected depends on the inhibition of return recovery rate. Which the authors, apparently, set to a value that when the Box 1. Inhibition of return In many of the models described in this thesis, the authors have implemented an ‘inhibition of return’ principle. Which most frequently refers to oculomotor inhition of return, rather than the classical inhibition of return, as proposed by Posner and his colleagues (Gazzaniga et al., 2009b; Klein, 2000; Smith & Henderson, 2011; Wang & Klein, 2009). Inhibition of return implies that fixations, due to saccadic eye-movements during the scanning of a visual scene, are not able to return to the previous point of fixation. One mechanism on which this principle is thought to be grounded is the visual short term memory (VSTM; Corbetta & Shulman, 2011). The VSTM temporarily stores the locations of fixation and thereby decreasing the salience of those locations, ensuring that they will not be visited until they are no longer stored in the VSTM. However, it has also been argued that this oculomotor inhibition of return is not so much an inhibition of the salience of the previous location of fixation, but rather a result of a bias of saccades to be made in the same direction as the previous saccade (Smith & Henderson, 2011). However, as will be seen, the implementation of this principle varies over the models. basis function map selected lines at the centre of the display, the salience of the stimuli on the outer right side was higher than the salience of the stimuli to the left of the centre. (b) In the line bisection, the centre of the line was estimated by computing the centre of mass of activity in the basis function map after stimulus presentation. The activity gradient induces an bias in the estimation of the line centre, leading to a ‘response’ that is slightly to the right of the centre of the line. This error increased proportionally with line length, and a cosine function of line orientation, when the line was rotated. (c) In a third experiment, the model was presented with four stimuli, one was the target, the other three distractors. The stimuli were horizontally arranged, such that the target was either to the left or the right of the distractors. Due to the activity gradient in the basis function map, the target was fastest detected when it was located on the right side of the distractors, independent of the retinal lateralization of the target. However, the 9 salience of the stimuli increased monotonically (due to the gradient) when their location moved from the left to the right. (d) To investigate the frames of reference used in neglect, the eye-position coordinates were replaced by head-position coordinates. That is, the output of the superior colliculus (used in the majority of the simulation) are coded using retinotopic coordinates, provided by the basis function map. The premotor cortex of the basis function model (see fig 3.), extracts head-centered coordinates from the basis function map. So, in this final simulation, the authors monitored the output of the premotor cortex instead of the superior colliculus. Following this adjustment, the model was presented with a stimulus to the left or right of the centre of the display, with the head oriented straight ahead or turned to the right. Stimulus identification was modelled as the total amount of activity in the basis function map evoked by the stimulus. The amount of activity was higher for the right than for the left stimulus, independent of head orientation. Additionally, The amount of activity was higher when the head was oriented toward the right, independent of the retinal position of the stimulus. Evaluation – (Strengths) The basis function model has a neurobiological background, that is based on the theory of sensorimotor integration in the parietal cortex (Pouget & Sejnowski, 1997a), although mainly inspired by the parietal areas of the macaque, the authors state that the same integration has been shown in human parietal cortices. However, it is not one of the major goals of this model to be fitted exactly with neurobiological principles. As is consistent with human data (although far from generally accepted, see e.g. Corbetta & Shulman, 2011), damage to the modelled parietal cortex leads to neglect like behaviour, as shown by the performance on the cancellation and line bisection tasks. Furthermore, the model elegantly addresses the problem of the frames of reference in neglect. It does not explicitly implement the frames of reference, but they are an emerging property of the input transformations in the basis function map. As a result, effects of neglect can be found in both egocentric and allocentric frames of reference, with the allocentric symptoms being more severe in the contralesional egocentric space. This finding is congruent with the patient data of Karnath, Mandler and Clavagnier (2011). However, the two types of neglect (i.e. alloand egocentric neglect) rarely co-occur simultaneously and are thought to be the consequences of lesions to different anatomical regions (Corbetta & Shulman, 2011). Interestingly, as a result of the same lesion that was used to model neglect, the model would show extinction. This extinction behaviour was in an allocentric form, as shown by the third experiment (i.e. when two stimuli are presented to the model), the stimulus located most the right is always selected, independent of the egocentric locations of the stimuli. In the version of the model with two basis function maps, the neuronal activity gradient increasing from the ipsi- to the contralateral side, implemented in the ‘healthy’ model, gracefully gets around the problem of a gradient lesion (a type of lesion found in some of the following models to account for the attentional gradient). Consequently, the entire right parietal cortex is removed, which is more biologically plausible than inducing a gradient lesion (as will be discussed later on with some of the other models), for which there seems to be no mention in the literature. (Limitations) Removing the entire right basis function map is used to simulate damage to the right parietal cortex, inducing left-sided neglectlike behaviour, congruent with human neglect patients. However, removing the left basis function map would just as easily result in right-sided neglect, which is inconsistent with the observations that 90% of the neglect patients have right hemisphere damage, resulting primarily in contralateral neglect (Corbetta et al., 2005). Another limitation is that the lesioned model will always initially select the stimulus located far to the right of the display, since, when all stimuli are identical, the salience of the targets most to the right is the highest, due to the activity gradient of the basis function map. The next stimulus that is selected will always be more to the left of the former target. However, there is no clear indication of this kind of sequentially organized behaviour found in patient data (Kettunen, Nurmi, Dastidar & Jehkonen, 2012). Furthermore, the principle of inhibition of return, as argued by the authors, is 10 necessary for the basis function map to produce human like behaviour, i.e. to not select the same stimulus twice in a row. Nonetheless, the artificial way by which their inhibition of return is implemented has consequences for the lesioned model. That is, the time (i.e. number of iterations) it takes before the salience of the initially selected target is recovered enough to be selected again, depends on the size of a recovery factor. This size, regrettably, is not justified in any of the papers, and specified only in the fourth paper. One could imagine that, if indeed there is an inhibition of return recovery factor in humans, its value would probably vary among neglect patients and thus be a contributing factor to neglect severity. Whether this is actually the case has yet to be investigated, for example with eye-tracking methods. With the initial cancellation being the stimulus most to the right, the inhibition of return principle leads to the observation that saccades (or cancellations) are always made to the left of the previous stimulus. This pattern is repeated until the salience of the stimulus that was selected at first (at the right side of the display) is recovered. These patterns are somewhat similar, though not identical to patient data (Husain et al. 2001; Mannan et al., 2005; Müri et al. 2009; Nys, Stuart & Dijkerman, 2010; Parton et al., 2006). Moreover, since the centre of cancellation is supposed to be a measure of neglect severity (Rorden & Karnath, 2010), the severity of neglect in the basis function model depends on the speed of recovery of inhibition of return and the activity gradient in the basis function map. Therefore, the severity of neglect as modelled in the third article seems arbitrary. 5.2 MORSEL (Mozer, Halligan & Marshall, 1997; Mozer, 2002). Design – MORSEL (short for “Multiple Object Recognition and attentional SELection”) was originally designed to analyze complex scenes with multiple visual stimuli, more specifically letters and words, and to account for multiple psychological data, such as perceptual errors, facilitating effects of context, and attentional phenomena (Mozer, 1990). The modules in MORSEL (see fig. 4) are explicitly not based on cortical structures, but rather considered inevitable to model the intended psychological data. The input enters the model through the ‘retina’. In this module, there are 5 feature detectors per retinal location, 4 for line orientation and 1 for line termination. The activity pattern of the retina projects to ‘BLIRNET’, a collection of processing modules, originally designed for letter and word identification. Each module processes a different property of the retinal input. BLIRNET modules in turn projects to the ‘pull out network’, a module that selects activity patterns of BLIRNET with a semantic content. The module of MORSEL that is central to modelling neglect is the attentional mechanism (AM). The AM has one unit per retinal location. It serves to gate the input of the BLIRNET, by selecting the spatial locations of the input that are needed for processing. It receives input from the retina and from higher cognitive areas. These are not ‘physically’ modelled in MORSEL, but rather implemented as an artificial bias to the units in the AM. Active units in the AM bias the transmission from the retina to the BLIRNET by mediating the probability by which activity from the retina is transmitted to BLIRNET. The presence of the AM is necessary because BLIRNET has a limited processing capacity. decreasing gradient, most severe at the left and least severe at the right side. Damage to the connections leads to a decrease in the probability that the retinal input is transmitted to BLIRNET. Mozer and his colleagues (1997) state that damaging the units in the AM directly, instead of their input, would yield the same deficit. Results – In the first paper (1997), the model is used to explain line bisection data. In the second paper (2002), a large body of other experimental paradigms were used to explain the frames of reference in neglect based on the lesioned MORSEL. In the line bisection task, the line is assumed to be transected at the centre of the active AM units. Because the lesion is modelled as a graded impairment and the activity of the units in the AM depends on their retinal input, the activation of their neighbours and the mean activity of the AM, the lesion results in the transmission of more than half of the line. Due to the linearly Performance – The goal of the AM is to select spatial regions of the retinal input that have to be processed by BLIRNET. The update rule for the units in the AM depends on three factors: the input by the spatially congruent retinal location, the activity of their neighbours, their relative activation as compared to the mean activity of the AM (i.e. “units below the mean activity are encouraged to turn off, units below the mean activity are encouraged to turn on”; Mozer et al., 1997). This update rule ensures that the AM selects the spatial region that consists of the location of an object. The update rules for the units in BLIRNET and the pull out network are not discussed in the papers because they have no influence on neglect, the way it is modelled with MORSEL. Thus, performance of MORSEL is observed through the activity patterns of the AM. Lesion – To induce neglect in the performance of MORSEL, the connections from the retina to the AM are damaged with a monotonically 11 Figure 4. MORSEL. For each retinal location the retina contains five feature detectors, depicted as five separate layers. BLIRNET and the pull out network were not used in the simulation of neglect. (Adopted from Mozer, Halligan & Marshall, 1997). damaged connections to the AM, the lines were bisected to the right of the true centre by the lesioned model, because the activity transmission from the retina to the AM was opposite to the left part of the line. When a line is present on the retina of the ‘healthy’ model, a 0.9 probability of transmission from a retinal position to a congruent AM position is assumed to account for varying responses in healthy human subjects. Thus, the variability of the responses of the healthy model increases when line length increases. This increase in the variability of the response as a function of line length is also observed in the lesioned model. To model the cross-over effect found in neglect patients, an additional term was used in determining the centre of the line, as judged by the AM. This effect describes the phenomenon where neglect patients with a rightward bias on the line bisection task, show a leftward bias when the line is of a short length (Marshall & Halligan, 1989; Savazzi, Posteraro, Veronesi & Mancini, 2007). The additional term used for this effect consists of a small, fixed leftward bias, which is overruled by the rightward bias (that depends on line length), when the line length is sufficiently large. Furthermore, neglect in the line bisection task was allocentric, as pointed out by positioning the line at different locations across the retina. At all positions the right side of the line was processed by the AM, in contrast to the left side. Although neglect appeared to be less severe for positions on the right of the retina. In the second paper (2002), this emerging property (i.e. allocentric neglect) of MORSEL was further elaborated. The lesioned model was subjected to a set of experiments by Behrman and Tipper (1994, 1996, 1999, in Mozer 2002), Pavlovskaya, Glass, Soroker, Blum and Groswasser (1997), Arguin and Bub (1993, in Mozer 2002) and Driver and various colleagues (1991, 1994, 1999, in Mozer 2002). The experiments aimed to elaborate on the frames of reference in neglect. They used rotated, rotating, cued, and differently lateralized targets. With the exception of some effects found in the original experiments that “could readily be incorporated in the model” (Mozer, 2002), the majority of the data could be replicated by the lesioned model without making any adjustments to it. 12 Evaluation – (Strengths) The design of MORSEL is well suited to explain the behavioural phenomena of neglect, as indicated by the large bulk of experimental data of neglect patients that can be explained by lesioning the model. Moreover, no adjustments had to be made to the original design of MORSEL in order to replicate the data. Among the tasks on which MORSEL can replicate human neglect, the line bisection task is particularly interesting since it is used in many clinical settings as an indicator for neglect. Hence, as stated by the authors, it may be possible to use the model as a diagnostic tool to indicate the severity of neglect, by the means of a transmission probability curve (Mozer et al., 1997). Furthermore, MORSEL addresses the allocentric frame of reference in neglect without it being explicitly implemented, but rather with it being an emergent property of the egocentric frame of reference. Although this is, as mentioned with the basis function model, not conform patient data, since it is thought that the two forms of neglect rarely co-occur. (Limitations) As it is not an objective of MORSEL to be biologically plausible, there are some questions about neglect that cannot be answered by MORSEL. It assumes a graded lesion, which is justified by the authors because it is present in behavioural data. Although lesioning the model this way is effective in explaining behavioural data, it does not address the origin of this lesion. On the other hand, this issue could be dismissed by noting that the gradients are already present in the healthy brain, cf. the basis function model of Pouget and Sejnowski (1996, 1997a, 1997b, 2001). Nonetheless, it would then still remain unexplained why 90% of the patients with neglect have right hemisphere damage (Corbetta et al., 2005). Summarized, MORSEL does fairly well in explaining behavioural data. It is probably also able to predict performance of neglect patients in various new tasks. Therefore, the model could be used as a clinical instrument to indicate the severity of neglect. However, it does not address the origin of neglect, other than in a conceptual manner. Questions such as why neglect occurs mostly after right hemisphere damage, or how the attentional gradient in neglect arises remain unanswered. 5.3 [Hilgetag] (Hilgetag, 2000; Hilgetag, Kötter & Young, 1999). Design – [Hilgetag] was designed to explain the paradoxical effect on visual orienting, of contralesional lesioning (Sprague effect). This effect has been investigated mostly in cats. Hence, the architecture of [Hilgetag] is based on the anatomical connectivity of the superior colliculus of the cat. Moreover, the connectivity and physiology of the midbrain structures of the cat have been explored quite thoroughly (Hilgetag, 2000). Additionally, when lesioned, these structures produces symptoms similar to the symptoms observed in human patients suffering from neglect. The model consists only of two retinotopic modules, Ml and Mr (M = midbrain, l/r = left/right). Each module covers the entire retinal input, which, in the model, ranges from -90° to 90° visual angle. As can be seen in figure 5, both modules are tuned with a skewed Gaussian distribution, in favour of the visual hemifield contralateral to each module. Furthermore, the modules mutually inhibit each other in a mirrored fashion (see fig. 5). Performance – The input of the model was very basic. There was a stimulus either to the left, to the right, or in both hemifields. When input was presented to the model, a saccade was assumed to be made to a new location, determined by the average input pattern of the two modules. The activity of Ml promotes a rightward saccade, Mr a leftward saccade. The amount of activity per module determined the final length of the saccade is the result of the amount of activation of the highest activated module. Lesion – To induce neglect, the input of Mr was reduced with 90%. If the input to Mr was reduced with 70%, the model exhibited extinction. Results – The severe lesion (90% reduced input) prevented Mr to increase its activation to a level, high enough to produce a leftward saccade, because the activity of Mr did not rise above its threshold when a stimulus was presented in the left visual field. The activity level of Mr did not rise above the threshold due to a combination of the reduced input and the inhibition of Ml. However, if, following the lesion to Mr, Ml was also lesioned, stimuli close to the central visual field (i.e. were the responsiveness of the two modules is the highest), appropriate saccades could be made again. Because the activity level of Mr was no longer inhibited by Ml. This resembles the Sprague effect, observed in cats and perhaps also in a human patient (Weddell, 2004). When the inputs of Mr were reduced only 70% a leftward saccade could be made to a left stimulus, however, only in the absence of a right stimulus and, again, when the stimulus was close to the central visual field. Evaluation – (Strengths) The model is based on anatomical, physiological and behavioural Figure 5. [Hilgetag]. ML and MR represent the left and the right part of a Midbrain structure of the cat. The numbers ranging from -90 to 90 correspond to the retinal location that is represented by that location in the midbrain structure. The ‘I’-s stand for Input, where the length of the arrow indicates the relative sensitivity of the midbrain structure to input at that specific location. The mutually, reciprocal inhibitory connections between the two halves of the midbrain are indicated with kC/K-C. (Adopted from Hilgetag, 2000). 13 data of the cat. Although the model is quite small, its biological plausibility is relatively high, as compared to the previous two models, but it should be noted that this plausibility is when the model is compared to the brain of a cat, not a human. Moreover, the monotonic neuronal gradient assumed in the previous two models, is in [Hilgetag] a skewed Gaussian distribution, which is more biological plausible, as shown by the data from the cat, and human neglect patients, as will be discussed with the next model (the ISO-map of Niemeier and Karnath, 2002). Additionally, the model explains effects of various types of lesions (Hilgetag et al., 1999). The strongest point of this model is that it accounts for the Sprague effect, which can probably be extrapolated to the effects of TMS in humans (Dambeck et al., 2006; Fierro et al., 2006). By damaging the healthy left midbrain structure (Ml), when the right structure has already been damaged, leads to partial recovery of the neglect symptoms. This effect is not only in found in animal lesion studies, but it also seems to be present following TMS of the left parietal cortex in humans (Fierro et al., 2006; Hesse et al., 2011; Koch et al., 2008; Koch et al., 2012; Oliveri, 2011). Moreover, the behavioural result of a bilateral lesion (i.e. relatively normal attention to targets in the central visual field), is quite similar to the symptoms found in a case study of a neglect patient with a bilateral stroke (Cazzoli et al., 2012). (Limitations) – The design of [Hilgetag] is based on the anatomy and physiology of the brain of the cat. However, these data are only used as guidelines and greatly simplified in the model. Moreover, given that the model is based on the brain of the cat leaves the question whether this model also applies to human neglect unanswered. On the other hand, the behavioural results of [Hilgetag] bear resemblance with the aforementioned case study. Furthermore, in contrast with the number of lesion effects that are explained, the number of behavioural tasks that have been presented to the model is restricted to one. However, the authors argue that it the explanation of neglect could easily be extended to the human performance on the line bisection task. On the issue on which frames of reference neglect operates, [Hilgetag] makes use of only egocentric coordinates. To be fair, the authors (1999) mention 14 that the model only uses egocentric coordinates, because there were no known experimental paradigms for addressing allocentric frame of reference in a cat. Still, this is in contrast with the basis function model and MORSEL, in which the frames of reference were both ego- and allocentric. 5.4 ISO-map (Niemeier & Karnath, 2002). Design – The integrated space (ISO) map is the only module in the model of Niemeier and Karnath (2002). Input and output modules are not explicitly modelled, but could be incorporated easily. The model offers an explanation of the process by which allocentric neglect can be ameliorated by the egocentric positions, an empirical finding described by the authors. The units in the ISO-map integrate object-based (i.e. allocentric) and trunk-centered (i.e. egocentric) coordinates. As can be seen in figure 6, a stimulus always covers the entire ‘allocentric axis’, but only a part of the ‘egocentric axis’. That part is proportional to the size of the stimulus. Each ISOunit assigns a salience value to a point of the presented stimulus, with salience being the probability of orienting to a certain location. Performance – The task that was used to observe the performance of the model was a visual Figure 6. The ISO-map. Retinal input is mapped onto this ISOmap. The object covers the entire vertical axis, and only a portion of the horizontal axis (depending on its size). (Adopted from Niemeier & Karnath, 2002. Note: the image was also blurry in the original paper). search task, similar to task used to test the basis function map. The goal was to have the ISO map select a stimulus for action. For simulation purposes, action was modelled by an additional algorithm, in order to generate saccades. This algorithm was based on the algorithm used by Pouget and Sejnowski (1996, 1997a, 1997b, 2001) in the basis function model. Hence, saccades followed a winner-takes-all and inhibition of return principle, similar to the winner-takes-all and inhibition of return principles in the basis function model. The location for the next saccade was the location with the highest salience. The salience of the previous location was temporarily reduced. With the algorithm, the model selects locations for fixation based on the salience value, assigned by the ISOmap. The model was presented with three stimuli, a visual scene without a target, a visual scene with a target area to the right of the centre, and a target area somewhat further to the right. Lesion – The lesion, used to model neglect, induces a Gaussian distribution of salience values along the egocentric axis and a linear graded salience distribution along the allocentric axis (monotonically increasing from the left to the right; see fig. 6). Results – The authors made a distinction between the global and local exploration of the model, in order to be able to make statements about ego- and allocentric neglect. Global refers to the exploration of the entire visual scene. Local refers to the exploration of a specific area of that scene. Globally, the saccades produced by the lesioned model tended to be clustered within the right visual field. Locally, the right part of an area was investigated. However, the more the investigated area was to the right, the more the left part of that area was investigated. Evaluation – (Strengths) The model seems to behave similar to the basis function map of Pouget and Sejnowski (1996, 1997a, 1997b, 2001), although a bit more simplified, as the allocentric are explicitly modelled, in contrast to the basis function map. The most notable difference between the basis function map and the ISO-map is the lesion. The lesion of the ISO-map produces behaviour that better resembles patient data, as shown by directly 15 comparing the performance of the ISO-map to the performance four neglect patients. The distribution of saccades produced by the model was similar to the distribution of saccades produced by the patients. Karnath and Niemeier (2002) note that this type of performance results from a Gaussian distribution of salience values in egocentric coordinates, and will never result from a linear lesion (cf. basis function map and MORSEL). (Limitations) Except for the shape of the salience curve induced by the lesion, the model provides no additional insights to the previous models. Moreover, by explicitly modelling the allocentric coordinates without stating how they might arise in the human cortex, the ISO-map is less elegant than the basis function map. It could be argued that by tuning the units in the basis function map, not according to a sigmoid, as it was tuned by Pouget and Sejnowski, but to a Gaussian function, the basis function map would probably perform just as good as the ISO-map. 5.5 [Deco] (Deco & Rolls, 2002; Deco & Zihl, 2004; Heinke, Deco, Zihl & Humphreys, 2002). Design – [Deco] was originally designed to construct a biological plausible model for visual attention, accounting for the ventral and dorsal pathways (Goodale & Milner, 1992). The dynamics of the model are based on the physiological dynamics of neuronal pools. Input is presented to the model by the retina (or ‘visual field’). The model first receives the input in V1, a retinotopic module. For each retinal location, there are twenty-four feature sensitive units in V1, with three spatial frequencies and eight orientations. V1 has bidirectional connections with the parietal module (PP), which represents the dorsal stream. Units in PP represent the spatial location of a stimulus in the scene presented to the object. When a unit in PP is active, the retinal location it represents can be thought of as being paid attention to. Because the PP not only receives input from V1, but also projects back to V1, a certain location can be biased by directing spatial attention toward that location. V1 also has bidirectional connections with a module for object recognition, called the inferior temporal cortex (IT), which represents the ventral stream. IT contains one unit per object that can be presented to [Deco]. It should be noted that the connections from IT back to V1 were absent in most simulations, in order to account only for bottom-up effects of spatial attention. As can be seen in figure 7, the model also contains several ‘inhibitory pools’. When these inhibitory pools are activated they inhibit the pool they are connected to (i.e. V1, PP or IT). The more active V1, PP or IT is, the more their inhibitory pool is activated, and thus, the inhibition of V1, PP or IT is increased. Consequently, only the most active units will survive the inhibition, which will thus be able to activate their connected units in the other modules. The inhibitory modules are not present in the first paper (Deco & Rolls, 2002), in which [Deco] is used specifically to model allocentric neglect, because the inhibitory pools are causing only one stimulus to be selected by PP. Therefore, Deco and Rolls (2002), provided V1 and PP with local lateral inhibition, which produces local peaks of activity in both maps. These multiple, local peaks are necessary to account for allocentric neglect when multiple stimuli are presented. Performance – The inputs consisted of visual scenes containing a stimulus, located at different positions within the scene (in Deco & Zihl, 2004; and Heinke, Deco, Zihl & Humphreys, 2002), or two stimuli at various distances from each other (in Deco & Rolls, 2002; and Deco & Zihl, 2004). To observe how the model performed, activated locations of PP were registered. According to the authors, these locations can be thought of as ‘paid attention to’. Hence, by observing the active PP locations, one can see which locations are attended and which are neglected. The model also accounts for extinction. To accomplish this, a visual discrimination task was used in which either one or two stimuli were presented. One stimuli was the target, the other a distractor. The difference between target and distractor was defined as a topdown bias from IT to V1. This task consisted of four conditions (1) target in the ipsilesional hemifield; (2) target in the ipsilesional hemifield and distractor in the contralesional hemifield; (3) target in the contralesional hemifield; (4) target in the contralesional hemifield and distractor in the ipsilesional hemifield. Lesion – Two types of lesions were used to induce neglect in [Deco], one for allocentric neglect, and one for egocentric neglect/extinction. To Figure 7. [Deco]. The arrows indicated the directions of the connections. Only connections with inhibitory pools are inhibitory, all others are excitatory. The dots represent single units, but there is no one-on-one correspondence between the number of dots depicted and the number of units modeled. (Adopted from Deco & Zihl, 2004). 16 induce allocentric neglect, the activity of the units in PP was reduced. The magnitude of the activity reduction was graded linearly, increasing from the left to the right of PP (i.e. comparable to the way in which neglect was induced to MORSEL). In order to induce egocentric neglect/extinction, PP was divided into two halves. The activity of the right half was reduced with 40% (i.e. a unilateral damage). Results – The gradient lesion over the entire PP accounted for allocentric neglect. When a stimulus was presented, the PP units that represented the location of the stimulus became active, only when they represented the right half of the stimulus. The PP units that represented the left half of the stimulus were inhibited as the activity of the inhibitory pools increased along with the overall activity of the PP. This was observed independent of the egocentric location of the stimulus. The overall activity of the PP was larger when the stimulus appeared at the right side (ipsilesional) of the scene, as compared to stimuli on the left side (contralesional). In the first paper (Deco & Rolls, 2002), allocentric neglect was also observed when two stimuli were presented simultaneously, at either a short or a long distance from each other. Note that in that paper the inhibitory pools (as depicted in fig. 7) were replaced by local lateral inhibition. When the two stimuli were at a short distance from each other, the PP units representing the right side of the right stimulus, inhibited all other units that represented either the left part of the right stimulus, or any part of the left stimulus, as a result of the local lateral inhibition in combination with the gradient lesion. However, when the stimuli were at sufficient distance of each other, the local lateral inhibition was not strong enough to inhibit the activity of the units representing the left stimulus. Consequently, the units representing the right side of both stimuli and, due to the gradient impairment, inhibited the units representing the left sides. The unilateral lesion accounted for egocentric neglect, as shown by the activity of the PP units when a stimulus was presented at different sites in the visual field. When the stimulus appeared at the right side the PP units became active. When it appeared at the left side, no PP units were activated. The ‘egocentric lesion’ combined with 17 the top-down bias accounted for extinction. The top-down bias was strong enough to overrule the inhibition when a target was presented in the contralesional hemifield. However, the bias was not strong enough to overrule the inhibition when a distractor was present in the ipsilesional hemifield. Evaluation – (Strengths) The authors have implemented several physiological features of spatial processing that have been found to be present in the human brain, including topographical organization of cortical areas and the spiking behaviour and receptive fields of individual neurons (Fitzpatrick, 2008). The search behaviour produced by the [Deco] is quite similar to the search behaviour observed in humans. This makes [Deco], in its non-lesioned form the most biological plausible model, as compared to the previous models. Later, the neurophysiology was implemented to an even further extent in a small model of Mavritsaki, Heinke, Deco and Humphreys (2009). This model will not be discussed in this thesis, because it mainly elaborates on the function of the healthy model, but not on neglect. (Limitations) The number of different lesions to the same network, that is required to induce allo-, egocentric neglect and extinction is the main limitation of [Deco]. On the other hand, as mentioned before, it has been stressed that indeed these symptoms are clinically and anatomically dissociated (Corbetta & Shulman, 2011; Karnath & Rorden, 2012; Umarova et al., 2011; Vossel et al, 2011). Nonetheless, these different lesions are not different lesions to the same structure, but rather the same lesions to different structures. Although the healthy model functions quite well in simulating human behaviour in visual search tasks, the model requires additional assumptions about the type of lesion or top-down bias to mimic symptoms of neglect. Furthermore, the search tasks used to test the model do not correspond to the tasks often used to diagnose neglect, in contrast with MORSEL for example. Moreover, as with most of the previous models, [Deco] does not account for the high prevalence of neglect after right hemisphere damage or the ameliorating effects that TMS has on neglect. If TMS would be simulated by a lesion to the left PP, when the right PP was already unilaterally lesioned, PP would show no activity at all, unless IT provides a top-down bias. In sum, [Deco] takes into account many neurophysiological data, which makes the non-lesioned model fairly biological plausible for human visual attention. However, to induce neglect a number of assumptions had to be made, which lower the explanatory power (concerning neglect) of this model. 5.6 SAIM (Heinke & Humphreys, 2003). Design – The selective attention for identification model (SAIM) was designed to simulate ‘translation invariant object recognition’, not be confused with view-invariant recognition. Translation invariant means: high-order object recognition irrespective of the original retinal location of the object. SAIM consists of six modules (see fig. 8) The retina is the module to which input is presented. The information is then transmitted to the contents network (a matrix specifying all possible combinations between the retina and the following module, the FOA) and the selection network. The selection network gates activity of the contents network, resulting in a single stimulus to be transmitted from the contents network to the FOA. The units in the FOA (focus of attention) are Figure 8. Architecture of SAIM. (Adopted from Heinke & Humphreys, 2003). 18 activated bottom-up. The FOA contains a single stimulus, that has to be transmitted to the knowledge network. The knowledge network is comparable to IT in [Deco] and the pull-out network in MORSEL, i.e. it contains template units. These units contain representations of all known stimuli, based on the stimulus feature. The knowledge network generates a top-down bias, modulating the activity of the selection network in favour of stimuli that are ‘known’ to the network. When a unit in the knowledge network is activated, it becomes temporarily inhibited, in order to simulate a shift in attention from an identified to an unidentified object. The last module is the location map, a retinotopic map that has bidirectional connections with the selection network. Units in this map represent the retinal locations that are activated by a stimulus. The number of active units in the location map decrease according to a winner-takesall principle, ultimately resulting in the activation of only a single unit. This unit corresponds to the location of the target. When only one unit is active, the location map temporarily inhibits the corresponding units in the selection network, leading to an inhibition of return principle, as seen in the basis function model and the ISO-map. Performance – The performance of the healthy model was evaluated thoroughly by the authors before a lesion was implemented in the model. This evaluation was done using either one or two stimuli (a ‘+’ and a ‘2’), located at varying positions on the retina. The stimuli that were used were equal, in the sense that the number of pixels per stimulus was identical. The performance of the model was monitored by looking at the activation pattern of the FOA. To model reaction time, the number of iterations before a stable state was reached by the network, was monitored. The performance of the lesioned model was monitored with the same task. In addition, a line bisection task was presented to the model. The bisection point was determined by the part of the line that was represented in the middle of the FOA. Lesion – In order to induce neglect in SAIM, the selection network was lesioned. The authors described three types of lesions. The first lesion damaged the bottom up connections from the retina to the selection network with a linear gradient. In line with their visualization of SAIM, this lesion was called the ‘vertical lesion’. The vertical lesion did not entirely remove the connections, but reduced their strength. The second lesion, the horizontal lesion, damaged the connections that project to the right side of the FOA (i.e. the side of the FOA representing the left side of the visual input). The third lesion was a combination of the first two lesions, i.e. both linearly graded damage to the connections projecting to the selection network and unilateral damage to the selection network itself. Results – Obviously, when one stimulus was presented to the healthy model it was selected for attentional processing and identified correctly. When two stimuli were presented to the healthy model, the ‘+ ‘ was selected by the FOA, over the ‘2’. This was the case because the pixels of the ‘+’ are spread more uniformly with respect to its centroid, which results in faster identification, enabling top-down modulation, in favour of the ’+’ to quicken the competition of the two stimuli. Lesioning the model with the vertical lesion had no effect on the mapping of that stimulus on the FOA, also when the position of the stimulus was on the most affected side. However, when two stimuli were presented, the stimulus that was presented in the least affected field always won the competition, indicating extinction. This was observed for both the ‘2’ and the ‘+’, however, extinction could be ameliorated when the damage was not severe and the stimulus on the most affected side was more salient (e.g. larger) than the stimulus on the least affected side. The presence of the knowledge network ameliorated the neglect symptoms as well. When the model was lesioned using the horizontal lesion, the centre of the stimulus was mapped too far to the left of the FOA, missing the pixels at the left. This represents allocentric neglect because missing the left part was independent of the retinal location of the stimulus. The authors explain the rightward deviation on the line bisection task of neglect patients with this horizontal lesion, because with this lesion the left part of the line is not mapped onto the FOA. Finally, by combining the horizontal and the vertical lesion, the model shows extinction, but also allocentric neglect. Moreover, the model 19 can simultaneously show both left-sided extinction and right-sided allocentric neglect. Evaluation – (Strength) As the authors note, SAIM shows several emergent properties of features that have been found to be of influence on human behaviour. Among these features are stimulus familiarity, stimulus salience and the number of presented stimuli. These properties of SAIM also lead to a naturalistic presentation of extinction. As shown by both SAIM and experiments with human patients, extinction can be ameliorated by prior knowledge of the stimuli and the salience of it (Driver & Vuilleumier, 2001). Furthermore, in contrast with [Deco], SAIM needs lesions at different locations to show symptoms of extinction or neglect, which, as mentioned before, have been shown in patients. Additionally, like [Hilgetag], it seems that SAIM is able to explain the ameliorating effects of TMS on extinction (Oliveri, 2011). By damaging the contralesional connections to the selection network, or the contralesional connections to the FOA, the connections are equally strong again, and thereby eliminating the bottomup effects of neglect. (Limitations) On the basis of this model lie several case studies of a patients with bilateral damage, who showed left-sided egocentric neglect, and at the same time right-sided allocentric neglect. As has been mentioned, SAIM is capable of simulating these deficits. However, these are not common symptoms when neglect is observed after a bilateral stroke (Cazzoli, et al., 2012). Another limitation of the case studies as a basis for SAIM, is that the lesions used do not explain egocentric neglect, the core deficit of neglect (Karnath & Rorden, 2012), nor do they explain that the severity of allocentric neglect varies with the egocentric position of a stimulus (Driver & Pouget, 2000; Karnath et al., 2011). Moreover, most tasks used to observe neglect symptoms of SAIM differ from the tasks used in clinical neuropsychology, except for the line bisection task. Not that this is a limitation per se, but, for example, MORSEL demonstrates its strength by explaining many data of neglect patients. Furthermore, as with all previous models the lesions can be applied just as easy to the right side of the model as to the left side, in contrast with the prevalence of neglect (Corbetta et al., 2005). Finally, as noted by the authors themselves, the behaviour of individual units is very basic, in contrast with for example [Deco] or the spiking search over time and space model (not described here; Mavritiski, Heinke, Deco & Humphreys, 2009). In sum, SAIM simulates extinction quite elegantly, and unlike most of the other models, it can potentially explain the effects of TMS (at least partially), however, its explanation of neglect is very limited. 5.7 [Monaghan] (Monaghan & Shillcock, 2004). Design – The model of Monaghan and Shillcock was constructed in explicit response to all previously summarized models. The authors argued that one of the most striking features of neglect remained inexplicable by all the other models. This feature, as mentioned before, is the observation that neglect occurs much more often after right hemispheric damage than after left hemispheric damage. All the models described above have not attempted to make a distinction between a lesion to the left or right hemisphere. For [Monaghan] on the other hand, explaining this difference is its primary goal. The authors investigate three potential hemispheric asymmetries. The model consists of four modules: a retina, a left hemisphere (LH), a right hemisphere (RH) and an output module (see fig. 9). Input is presented to the retina. As can be seen in figure 9, due to the design of the retina, only horizontal lines can be ‘perceived’. The retina projects to both hemispheres. The hemispheres are comparable to the basis function maps of the basis function model. Each hemisphere is tuned to toward the contralesional space, similar to the central modules of the basis function model, [Hilgetag] and the ISO-map. The output module consists of a module with a size identical to the retina. The potential hemispheric asymmetries are modelled by sequentially implementing three differences between the LH and the RH. The first difference concerns the steepness of the contralateral gradient. As proposed by Pouget and Sejnowski (2001), the gradient by which the LH is tuned toward the contralateral hemifield is steeper, as compared to the gradient in the RH. With the second asymmetry, the units in the LH have narrow 20 receptive field, whereas units in the RH have broad receptive field, based on the concept of local versus global processing (Gazzaniga, Ivry & Mangun, 2009c). With the third asymmetry, the RH was tuned to both hemifields, while the LH kept its contralateral bias, based on the theory of Heilman and Van den Abell (1980, in Hillis, 2006) and Mesulam (1981, 1991, in Hillis, 2006). The implementation of this third asymmetry was based on a mathematical model of line bisection behaviour in neglect patients (Anderson, 1996). The three asymmetries were modelled separately. Their performances were compare by the authors. Performance – [Monaghan] was designed to be able to perform the line bisection task, and to explain several distinctive features of a right or left hemispheric lesion on the line bisection task (see the features of neglect described in table 1). The bisection point was determined to be the centre of the active output units. The lines used in the task were of varying lengths, ranging from 2 to 16 (i.e. the entire retina). Thus, on average the activity pattern of the output module of the healthy model is identical to activity pattern of the retina (the input module). Lesion – The lesion applied to the model was a reduction the activity of either the RH or the LH with 10, 50 or 100 percent. The learned weights of the connections projecting to and from the Figure 9. [Monaghan]. The input and output layer contain units that can be either activated or inactive (i.e. 1 or 0). The input layer can be considered as a representation of the retina, the output layer as a final internal representation of the visual input. The layers ‘left hidden’ en ‘right hidden’ both encode the entire visual input. (Adopted from Monaghan & Shillcock, 2004). Table 1. The authors summarized their simulations in the this table. Y = behaviour reproduced. N = behaviour not reproduced. (N) = not reproduced with conservative parameter (see original paper for details). (Adopted from Monaghan and Shillcock, 2004). 1. 2. 3. 4. 5. 6. 7. 8. Feature of neglect RH lesions reduce bisection accuracy RH lesions produce contralateral neglect RH lesions cause larger displacements and more deviation for longer stimuli. No relationship for LH lesions RH lesions crossover effect RH lesions cause larger bisection errors than LH lesions LH lesions cause greater variance in bisections RH lesions cause greater cuing effect RH lesions result in slower recovery lesioned hemisphere remained intact. Results – In their article, Monaghan and Shillcock provided a table that summarizes the features of neglect that the different asymmetries accounted for (see table 1). As can be seen, the different sizes of receptive fields per hemisphere accounted for all the feature that the authors tried to explain, whereas the two other asymmetries lacked the explanation of approximately two features. In sum, the model showed neglect when the RH of model with different receptive field sizes was lesioned. When the LH of the same model was lesioned, the variability of the response increased. Evaluation – (Strengths) [Monaghan] is the first model that elaborated on the hemispheric asymmetries of neglect. In doing so, it adopted some features of the previously described models (e.g. the neuronal gradient of the basis function map). Furthermore, it did not implement one theory, but rather compared three of the main theories on hemispheric asymmetry and compared the results. Thus, simulations with [Monaghan] argue in favour of the theory concerning differing sizes of receptive fields between the two hemispheres and against both the plausibility of the theory of Heilman and Van den Abell and Mesulam (Hillis, 2006) and the theory on the steepness of the neuronal gradient of Pouget and Sejnowski (2001). In addition, like MORSEL, [Monaghan] explained many features of line bisection behaviour as seen in neglect patients. In addition to the simulations with MORSEL, [Monaghan] did not require an additional assumption in order to account for the cross-over effect. (Limitations) On the other hand not all line bisection data were simulated entirely accurately. [Monaghan] showed decreasing variability in the 21 Receptive field Y Y Model Steeper LH gradient Y Y Bilateral RH Y Y Y Y Y Y Y Y N Y (N) Y N Y Y (N) N Y Y N Y Y performance of the model when the RH was damaged, whereas the MORSEL and human data show that variability increases with line length. The main limitation of [Monaghan] is its small size. Beside the limited biological plausibility, the limited size of the model disables it to explain only more data than that of the line bisection task. Actually, that is the only task it can replicate. The line bisection, although frequently used in clinical settings, is a frequent topic of debate in whether or not it is truly sensitive for neglect (Ferber & Karnath, 2010; Molenberghs & Sale, 2011; Molenberghs et al., 2012; Rorden et al., 2006). Whether the model can be extended to explain more behavioural, human data remains unanswered. Furthermore, in contrast to most of the previous models, the current model did not aim on elaboration of the frames of reference damaged in neglect. However, it would have been enlightening if the simulations would have included this topic, because literature on neglect patients with left hemispheric lesions report that there seems to be a difference in the frequency of allocentric and egocentric neglect between patients with a left, or a right hemispheric lesion (Kleinman et al., 2007; Molenberghs & Sale, 2011). It has been observed that patients with a left hemispheric lesion suffer from allocentric neglect twice as much as egocentric neglect, whereas this ratio seems to be inversed in patients with right hemispheric lesions. Perhaps, the lack of an explanation for these data is because simulation of the performance of patients with a left hemispheric lesion was largely based on a single study (i.e. Mennemeier, Vezey, Chatterjee, Rapcsak, & Heilman, 1997). In sum, the explanatory value of [Monaghan] seems small, however, it provides an interesting view on neglect in relation to the difference hemispheric lesions. between left and right 5.8 [Lanyon] (Lanyon & Denham, 2004a, 2004b, 2010). Design – [Lanyon] was originally designed to simulate the effects of attention on visual search, based on neuroanatomical and -physiological data of a macaque (2004a, 2004b). In the third article (2010), the model was lesioned in order to replicate neglect. The authors examined the difference between the effects of frontal and parietal lesions on visual search behaviour. [Lanyon] contains eight modules (see fig. 10), mainly named after anatomical structures of the macaque brain: the retina, V1, V4, the inferior temporal area (IT), the ventral prefrontal cortex (vPFC), the covert spatial attention window (AW), the parietal module (in the first two articles this module was named after the lateral bank of the intraparietal sulcus; LIP), and the orbitofrontal cortex (OFC). The design and dynamics are fairly comparable to the basis function model and [Deco]. Input is presented to the retina, where form and colour processing takes place by simple centre-surround processing and concentric opponent processing, respectively, as modelled by Grossberg and Raizada (2000, in Lanyon & Denham, 2004a). For simulation purposes, the size of the retina is variable, and all subsequent retinotopic modules have the same size as the retina. However, in most simulations, the size of the retina was fixed. The retina projects to V1, a retinotopically organized module with 4 feature detectors per retinal location, 2 for line orientation and 2 for colour. In order to process line orientation, simple cells were used, for colour, double opponent cells, as modelled by Grossberg and Raizada (2000, in Lanyon & Denham, 2004a). In double opponent cells, the centre of the receptive fields is excitatory when its preferred colour is presented (red or green in this model), the surrounding receptor area is inhibitory when 22 the same colour is presented to it. After V1, information is processed in a model of the dorsal and a model of the ventral route (Milner & Goodale, 2008), where the dorsal stream is mainly concerned with object location, while the ventral stream more with object identity. The dorsal route consists of the AW, the parietal module and the OFC. The AW modulates the form V1 to the parietal module by adjusting the retinal image according to the stimulus density around the point of fixation, i.e. a larger part of the scene if transmitted to the parietal module when stimulus density is low. The parietal module is a retinotopically organized module, which receives retinal input, modulated by the AW. The function of the parietal module is to select a location for the next fixation, based on bottom-up information (through the AW) and by top-down bias (from the OFC and V4). The OFC generates temporarily Figure 10. Architecture of [Lanyon]. The broad arrows indicated connections between units in different modules but at similar retinal locations. (Adopted from Lanyon & Denham, 2010). inhibits the current location of fixation in the parietal module, thus creating inhibition of return. Beside, the current location of fixation all adjacent location are also, though to a lesser degree, temporarily inhibited. The OFC stores the inhibition values in a retinotopic map. The inhibition values of a visited location decrease with every new fixation. The ventral stream, comprising V1, V4, IT and the vPFC, is implemented for object identification and to generate a top-down bias in visual search. V1 projects to V4, a retinotopically organized module with 4 feature detectors per retinal location, one for each of the 4 stimuli that can be presented to the model (described in ‘Performance’). Two sets of inhibitory neurons mediate the competition within V4, one set for the competition of stimulus orientation, and another for stimulus colour. V4 projects, as mentioned, to the parietal module to generate a top-down bias, based on stimulus features. Conversely, V4 receives a spatial bias by the parietal module, enhancing the processing of stimuli in the attended region. Furthermore, V4 has bidirectional connections with IT. IT contains 4 units (one for each possible stimulus) for object identification. The vPFC can select one of the stimuli as a target, by biasing IT. In turn, IT biases stimulus processing in V4 in favour of the target that was selected by the vPFC. Performance – [Lanyon] was designed to simulate visual search task, and the influence of visual attention on that task. The task consisted of a large scene in which numerous small bars were present. The bars varied in orientation (horizontal or vertical) and colour (red or green). The task for the model was to identify all the bars that were instructed as target (e.g. all red horizontal bars). In order to do so the parietal module and V4 cooperate extensively. When a scene is presented the initial fixation point is at the centre of the scene. Then, a part of the scene, with the size of the retina, is transmitted further into the network. The parietal module operates on a part of the retinal selection, because of the modulation of the AW. V4 also shows increased activation of the area selected by the AW, due to the extensive interaction with the parietal module. In V4, the units that represent the behaviourally relevant stimuli (i.e. targets) are becoming more and more active, due to interaction 23 with IT and indirectly the vPFC. Finally, the location of a target will be most active in the parietal cortex, and is thus the location for the next fixation. When multiple stimuli are equally relevant, the probability by which they will be selected for the next fixation is also equal. In the next iteration, the previously fixated location is inhibited by the OFC, making it highly unlikely that the previous target will be selected for the next fixation. Lesion – As mentioned before, one of the goals of Lanyon and Denham was to model the effects of different lesion locations. Hence, the authors lesioned the OFC uni- and bilaterally, and the parietal module, unilaterally and with a linear gradient. The uni- and bilateral damage to the OFC implied that none of the units in the affected area could be activated. The same follows for the unilateral lesion to the parietal module. The gradient lesion to the parietal module is comparable to the lesions in MORSEL, [Deco] and SAIM. The damage ranges from 100 to 0 % from the left to the right. Results – The unilateral parietal lesion resulted in a complete neglect of the left visual hemifield. Since the left part of the parietal module had been lesioned, the location of the next fixation was always a location to the right of the current fixation. This lead to a scanpath that consisted solely of rightward saccades. Hence, the scanpath became stuck at the right border of the scene. Therefore, a reset saccade was implemented, bringing the fixation back to the centre of the scene. When the gradient lesion was applied, the more a stimulus was located to the left, the more severe neglect of that stimulus was (i.e. the less likely it was that a stimulus was selected for fixation). When the OFC was damaged unilaterally, the scanpath showed the a neglect pattern similar to the pattern following a parietal lesion, although less severe. A bilateral lesion completely impaired the inhibition of return, thus, disabling the model to completely explore the scene. Locations with a target were frequently revisited. Lesions to the parietal module lowered the activity in V4 and V1, whereas lesions to the OFC left the activity of those modules unaltered. Evaluation – (Strenghts) [Lanyon] covers many of the symptoms that are thought to be central to neglect, as proposed by Corbetta and Shulman in their model (2002, 2011). Namely, the model distinguishes different lesion locations for spatial (neglect of the left hemifield) and nonspatial (frequent re-fixation) deficits, a dissociation that has also been found in human patients (Husain et al., 2001; Karnath & Rorden, 2012; Molenberghs et al., 2012; Umarova et al., 2011; Vossel et al, 2011). Although it should be noted that the impairments of model, following lesions to the OFC, were more severe than the impairments following damage the corresponding regions in humans (Hodgson, Mort, Chamberlain, Hutton, O’Neill & Kennard, 2002; Husain et al., 2001) Moreover, the model captures several well-established features of visual search behaviour of healthy humans, including spatial and temporal characteristics of the relationship between saccades and stimuli abundance or salience (for details see Lanyon & Denham, 2004a). Also, many neurobiological data were implemented in [Lanyon], similar to [Deco], but more focussed on cortical visual processing. Another interesting feature of [Lanyon], is that it shows that parietal lesions alter the activity in V4 and V1, which has also been observed in neglect patients (Corbetta et al., 2005). Furthermore, the ameliorating effects of TMS, could arguably be reproduced by [Lanyon], by lesioning the right half of the parietal module. However, visual search would be quite uncontrolled, since the parietal module and OFC could not exercise their effects on the search behaviour, thus leaving only the ventral areas to the task. However, the disadvantageous effects of a unilateral or gradient lesions would be ameliorated (Limitations) Like, [Deco] the model captures many aspects of human visual search behaviour, but simulating neglect behaviour raises some problems. In the first place, the authors test the effects of both a unilateral and a gradient lesion. The gradient lesion was used to make neglect less severe than following a unilateral lesion. This distinction, however, is not useful. Since there is no biologically founded justification for a gradient lesion, the gradient of neglect could arise from a neuronal tuning gradient that is present in healthy brains, as argued by Driver and Pouget 24 (2000), Pouget and Sejnowski (1997) and Niemeier and Karnath (2002). Thus, a unilateral lesion leads to a gradient impairment. Second, following a unilateral parietal lesion (and in a milder form following a gradient lesion), the model only made rightward saccades. Although it has been noted that neglect patients tend to make spontaneous rightward saccades (i.e. seemingly in the absence of salient stimuli), this does not indicate an inability to make a leftward saccade (Fruhmann-Berger & Karnath, 2006; Husain et al., 2001). To overcome this problem, the authors implemented an artificial reset saccade. Third, the model only captured a basic visual search task to investigate neglect behaviour, in contrast to the basis function model, MORSEL, [Deco] and SAIM, that have all been shown to be able to explain a variety of tasks of which impaired performance is associated with neglect. With the visual search task used with [Lanyon], egocentric neglect is produced. Whether or not the neglect simulated by the model also applies to allocentric neglect or other clinical assessment tools, such as the line bisection task, remains untested. Furthermore, like many previous models, [Lanyon] does not account for the hemispheric asymmetry of neglect lesions. 6. General discussion Connectionism is a relatively young field of research that has been able to provide significant contributions to debates within the cognitive neurosciences, as illustrated by the example of the past tense model of Rumelhart and McClelland (Thomas & McClelland, 2008). In thesis the value of connectionist models for neuropsychology has been appraised, with the hypothesis that connectionist models can also contribute to the understanding of neuropsychological disorders, arising from neurological damage. Eight connectionist models that explain several features of neglect have been reviewed. In table 2 a summary is given of those aspects. In the table the models are ordered according to the number of features of neglect they account for. It is then visible that, remarkably, the basis function model is the most comprehensive model, yet also the oldest model reviewed here. Table 2. Accounts of the models for features of neglect described in section three. 0 = the model did not account for this feature. 1 = the model accounted for the feature. Explanation of the features: 1. The probability by which a stimulus is neglected decreases from the left to the right. 2. TMS ameliorates neglect in human patients. 3. The models explanation of neglect has been shown to be applicable to more than one task. 4. Either two different lesions account for neglect and extinction, or extinction is a milder form of neglect. 5. Does the model correspond to visuospatial processing in healthy human subjects? 6. Does the lesion require other assumptions (e.g. graded damage) than the degradation of a specific module or connections? 7. The model explains the allo- and egocentric frames of reference of neglect with a single lesion. 8. Neglect occurs more often after a right than after a left hemispheric stroke. *BF-model = basis function model BFmodel* [Hilgetag] SAIM [Monaghan] [Deco] MORSEL [Lanyon] ISO-map Total number of models Attentional gradient1 TMS effects2 Multiple clinical data explained3 Extinction4 Plausibility healthy model5 Plausibility of the lesion6 1 1 1 1 1 1 1 1 8 0 1 1 1 0 0 1 0 4 1 0 1 0 1 1 0 0 4 0 1 1 0 1 0 0 0 3 1 0 0 0 1 0 1 0 3 1 1 0 1 0 0 0 0 3 However, conclusions from table 2 should be drawn carefully, because whether a model is a ‘good’ model depends on the features accounted for by the model are considered ‘good’ or relevant, which in turn depends on the interest of the reader. Therefore, in this discussion, mainly the number of features that can be explained by a model will be considered, leaving the reader to assign his/her value to a model, considering the specific features. As can be seen in the last row, where it is shown how many models account for a specific feature, all models accounted for the attentional gradient, which is not very surprising since it is a cardinal symptom of neglect. Nevertheless, only the basis function model, [Hilgetag] and [Monaghan] did so using a lesion that requires no assumption about a gradient in the lesion itself, for which, as mentioned in the individual analyses, is no indication in the literature. Therefore, the attentional gradient of neglect is best accounted for by unilateral (and uniform) damage to a single hemisphere containing neurons tuned toward the contralateral side of egocentric space. This is of course in accordance both with the theory of Heilman and Van den Abell, and Mesulam, and with the theory of Kinsbourne (Hillis, 2006). Monaghan and Shillcock (2004) explicitly state that they used Kinsbournes approach as the base of their model. However, it would not be fair to lower the explanatory value of a model on the type of lesion, without taking a closer look at the justification for a gradient lesion. As the basis function model, [Hilgetag] and [Monaghan] showed, when each hemisphere contains a retinotopic map that is 25 Multiple FoR with single lesion7 1 0 0 0 0 1 0 0 2 Hemispheric asymmetry8 Total number of features explained 0 0 0 1 0 0 0 0 1 5 4 4 4 4 3 3 1 tuned in favour of the contralateral hemifield, unilateral damage results in a single map tuned in favour of the contralateral field, directly corresponding to the attentional gradient of neglect patients. Since the other models did not implement a contralateral tuning, they were thus forced to introduce the gradient in the form of the lesion, in order to account for the attentional gradient. Hence, at least in the cases of MORSEL, SAIM and [Lanyon], this issue could probably be solved if the neuronal gradients would be implemented into the healthy model. In contrast, the lesions used in [Deco] and the ISO-map required additional assumptions beside the gradient, concerning different types of lesions to the same module to induce allo- and egocentric neglect. Although empirical studies are ambiguous about whether or not the different frames of reference of neglect arise from different lesions (Chechlacz et al., 2010; Corbetta & Shulman, 2011; Karnath & Rorden, 2012; Umarova et al., 2011; Vossel et al, 2011; Yue et al., 2012), it is implausible that they arise from different types of lesions at the same anatomical location. It should be noted that this argument only holds if the damaged module directly corresponds to a single anatomical structure. This one-on-one correspondence is actually suggested by the authors of both models. Nonetheless, it is a frequent topic of debate in which frames of reference neglect is presented (only in egocentric coordinates? Only in allocentric? Or in both?). The table might give a flawed view of the number of models addressing this topic, since only the models that accounted for both allo- and egocentric neglect using a single lesion received a “1”. Despite the “0”-s, SAIM, [Deco] and the ISO-map also address this issue, but they use different types of lesions to a single module to account for the different frames of reference. Moreover, in the case of the ISO-map, two different lesions have been applied to the same module simultaneously. How this is compatible with neurological data remains unanswered by the authors of those models. Conversely, the facevalidity of the method of lesioning and stimulus processing used in the basis function model, to account for the different frames of reference with a single lesion is relatively large. The method was later formalized by Driver and Pouget (2000), and elaborated on by the ISOmap. The ISO-map was designed with the primary goal to elaborate on the shape of the attentional gradient, which has shown to be more similar to a Gaussian function than a linear one. This was based on the finding that, in human patients, neglect did not continually decrease from the left to the right, but started increasing again when stimuli appeared in the right peripheral field. Hence, the other models might be more accurate if they adopted this shape of neuronal tuning from the ISO-map. However, it should be noted that this tuning distribution applies to a visual scene ranging from 140° to 140° in egocentric coordinates, whereas most models were designed to apply to visual scenes that are in sizes comparable to the sizes of most tasks used in clinical settings. Another topic of debate in the neuropsychology of neglect is the phenomenon of extinction (i.e. is it part of the same disorder, but in milder form? Or is a distinct disorder?). Extinction has been accounted for by three models ([Hilgetag], SAIM and [Deco]). They all accounted for extinction as being a milder form of unilateral neglect. [Deco] most elegantly explains extinction as a milder form of neglect, by attributing the difference between them to a less extensive lesion. [Deco] states that the egocentric damage in patients with neglect or extinction is the same, but in patients with extinction a top-down bias can still bias visuospatial processing, whereas this bias is diminished in patients with neglect (Deco & Zihl, 2004). The other two models, [Hilgetag] and SAIM, accounted for 26 extinction by lowering the intensity of the damage. Although this may be appropriate to describe extinction at a behavioural level, it is hard to imagine how this represents neurological damage, since milder damage in neurology usually implies a less extensive region that has been affected, not damage to an area of similar size but just less damage (?) to individual neurons. The number of clinical tasks that can be performed by a model, yielding the same performance as neglect patients, is appraised in the fourth column: “multiple clinical data explained”. All models explaining the data of more than one task were assigned a “1”. The other models explained the data of only a single task, and received a “0”. This does of course not imply that their explanation was inadequate, but rather that their explanation of neglect data has not (yet) been demonstrated to be generalizable to other tasks. Particularly, Heinke and Humphreys (2003), and Mozer (2001) described the performance of SAIM and MORSEL on a broad variety of tasks. They explain why neglect patients can fail on both the line bisection task and cancellation tasks, following a single lesion. No model provided an explanation for the apparent dissociation between the lesions that impair performance on the line bisection task, but not on a cancellation task, or the other way around (Ferber & Karnath, 2010; Rorden et al., 2006). The main difference between the simulations of SAIM and MORSEL was that SAIM was mainly tested to explore the characteristics of its own performance, whereas MORSEL was more explicitly tested in its ability to describe human performance in a wide variety of tasks to which human neglect patients have been submitted. Although, also with MORSEL, not only quantity, but also quality of its performance was monitored, especially for the performance on the line bisection task. Which resulted in the suggestion of Mozer and his colleagues (1997) to directly apply MORSEL in clinical settings, in order to characterize the transmission probability curve of an individual patient. This curve shows the probability by which a stimulus at different retinal locations will be detected and processed. Conversely, MORSEL has not been attempted to be made biologically plausible, e.g. by implementing many neurophysiological or -anatomical data. Indeed, it might be concluded that MORSEL only describes the behavioural characteristics of neglect, rather than explaining how they arise from cerebral damage. In contrast, [Deco] and the basis function model have clear neurobiological foundations but they are limited in the level of details of the explanation of neglect behaviour, as compared to MORSEL. Whether or not a model was considered to be biologically plausible depends on the number of neurophysiological or -anatomical features that have been implemented in the model. For example, [Deco] and [Lanyon] provide very detailed characteristics of small pools of neurons, such as taking into account the known characteristics of receptive fields of neurons in V1. The model of visuospatial processing by Mavritsaki, Heinke, Deco and Humphreys (2009), not discussed here, even takes into account characteristics of different neurotransmitters and their receptors are implemented. Pouget and Sejnowski (1997b, 2001) did not describe the individual neurons in the basis function model to this detail, but they named the modules in their model after anatomical structures that have been shown to contain neurons that carry out the same functions as the units in their modules, which has been argued to be more favourable than simulating the cellular features of neurons (like the model of Mavritsaki et al., 2009), given the goal of the basis function model, namely explaining behaviour following neurological damage (Meeter et al., 2007). As for the basis function model, this also applies to [Hilgetag]. However, this model was based on the neuroanatomy of a cat, making it difficult to extrapolate its explanations to human patients. [Hilgetag] does have another, interesting property, namely mutually inhibitory hemispheres. With this property it is able to explain the ameliorating effects of TMS over the unlesioned hemisphere (Fierro et al., 2006; Hesse et al., 2011; Koch et al, 2012; Oliveri, 2011), as explained in its individual evaluation. These effects really add something to the other models, because, as can be seen in the table, four out of the eight models cannot account for this effect. By taking a closer look at the dynamics of the modules of those models, it can be concluded that the model will 27 perform even worse when both hemispheres would be damaged (as is opposite of what the effect of TMS is thought to result in). As for SAIM, [Monaghan] and [Lanyon], the authors do not state anything about the effects of TMS, but by analyzing the dynamics of these models, it can be concluded that damaging the left half of the damaged module might decrease the symptoms of neglect, when the right half is damaged. Damaging the right or left half in most models is arbitrary and exchangeable. However, this is actually one of the striking features of neglect: it occurs mainly after right hemisphere damage. The only model that takes this feature into account is [Monaghan]. Although this model is limited in explaining many of the other features, it does address the hemispheric asymmetry. However, the asymmetry concerning receptive field sizes (LH fine coded, RH coarse coded), which the authors appraised as most powerful, still yielded data that was inconsistent with human data, but perhaps this is consequence of the limited size of the model. It could be quite interesting to see the what would happen to the other models when they would incorporate a hemispheric difference like this. Pouget and Sejnowksi (2001) state that the basis function model could easily be updated with hemispheric asymmetry, instantly accounting for the hemispheric asymmetry. Though, they did not provide any simulations to support their claim. 7. Conclusion The connectionist models of neglect demonstrate a trade-off between explaining the characteristics of neglect from a neurobiological perspective and being able to accurately simulate the behavioural data of neglect patients. However, as claimed by some of the authors, improvements could be made to the models, combining the power of different models and thus increasing the explanatory value of the model (Pouget & Sejnowski, 2001). Yet, so far, no model has been made using combinations of several of the models described here. Nonetheless, the models have shown to be able to address for example how it is possible that neglect seems to manifest in two different frames of reference, while only the egocentric coordinates need to be neurally encoded (Mozer, 2002; Pouget & Sejnowski, 1997b, 2001), or why neglect arises more frequently after right than left hemisphere damage (Monaghan & Shillcock, 2004). Still, as mentioned, all models consider only some aspects of neglect, while disregarding others. However, this should actually be regarded as a virtue, according to McClelland (2008). He states that, when appraising connectionist models, one must view simplicity of models as a virtue, because then they are able to breakdown complex concepts into comprehensible subconcepts. On the other hand, with increasing simplicity models may lose explanatory value, thus becoming less interesting to (in this case) neuropsychologists. Currently, neuropsychology has shown moderate interest in the models, considering the number of times the models are cited in other articles (ranging from 13 to 1 time(s) per year). Moreover, the models are often referred to with phrases, such as: “[...] as is consistent with computer simulations” (Bartolomeo, 2007, pp. 384), i.e. without elaborating on the design of the model. However, if empirical studies would use the existence proof, as provided by the models, as a basis for constructing hypotheses, the strength of the models could be confirmed or refuted. Moreover, the model would instantly provide an explanation for the data. See for example the study of Karnath and colleagues (2011) on the frames of reference of neglect. Unfortunately, such studies are quite rare. As for the models discussed in this thesis, they are able to provide some hypotheses. The basis function model, MORSEL and the ISO-map predict that the degree of allocentric neglect depends on the severity of egocentric neglect. Moreover, a single lesion would account for both types of neglect, according to the basis function model en MORSEL. Even more, the models make predictions about tasks used in the clinical assessment of neglect (see the [Monaghan], the basis function model and MORSEL). Also, eyemovement patterns are well predicted by the models (though not necessarily 100% accurate). 28 [Deco] and SAIM, for example, state that a partially neglected object would be more extensively explored when it is familiar to the patient, as compared to unfamiliar objects. Another interesting prediction comes from [Hilgetag], which states that the performances depending on targets in the right (i.e. unaffected) visual field should be better-thannormal performances. Because it is mainly the left hemisphere that is involved in processing that target, and the right hemisphere can no longer exert its inhibition upon that hemisphere (since it is lesioned), the left hemisphere would experience less interference in the lesioned model than in the healthy model. In sum, it seems as though neuropsychology uses connectionist models primarily as an addition to their experimental data in explaining cognitive disorders or symptoms. However, as shown by the models presented in this thesis (i.e. concerning neglect), connectionism could be of more value when more attention to the models would be paid. Models could direct the formation of hypotheses and the interpretation of behavioural data. On the other hand, connectionist models that elaborate on neuropsychological disorders are not a majority of all models. Perhaps, because lesioned models are only of interest to neuropsychology, whereas intact models can be of interest to both cognitive science and computer science (Abrahamsen & Bechtel, 2002). Therefore, neuropsychology has only limited options when they want to ‘consult’ a model. However, if models aim to be biologically plausible, lesioning them is an excellent method to test them, just like inferring the function of a brain area by lesion-mapping (Rorden & Karnath, 2004). [Lanyon] provides a good example to illustrate this. It was constructed to model the effects of attention on human visual search behaviour. When intact, the model could fairly well replicate this behaviour, but when lesioned to induce neglect, the model could only make rightward saccades. This finding is not congruent with the search behaviour of neglect patients, so the authors implemented a reset saccade when the model reached the border of a scene. A reset saccade was an artificial way to overcome their problem, but it was also an additional assumption to their model, decreasing the biological plausibility of [Lanyon]. If these limitations could be improved in future developments, perhaps with the help of neuropsychological data, the models biological plausibility and thereby its explanatory or predictive value would increase. Hence, both fields could benefit from a more extensive cooperation between neuropsychologists and connectionists. For future directions, lesioned models, explaining or describing features of neglect, may also be combined with connectionist approaches to model rehabilitation. Although so far, these approaches with connectionist models are still very limited (Harley, 1996; Plaut, 1996). As for neuropsychological research, studies could be performed, testing the hypotheses that were provided by the models. Thus, the models could increase the understanding of the deficit and thus revealing new targets for rehabilitation (Raymer et al., 2008), and contribute to the aforementioned goal of neuropsychology. References Aakerlund, L., & Hemmingsen, R. (1998). Neural networks as models of psychopathology. Biological psychiatry, 43 (7), 471-482 Anderson, B. (1996). A mathematical model of line bisection behavior in neglect. Brain, 119, 841-850. doi: 10.1093/brain/119.3.841 Bartolomeo, P. (2007). Visual neglect. Current Opinion in Neurology, 20, 381-386. doi: 10.1097/WCO.0b013e32816aa3a3 Bartolomeo, P., Thiebaut de Schotten, M., & Doricchi, F. (2007). Left unilateral neglect as a disconnection syndrome. Cerebral Cortex, 17, 2479-2490. doi: 10.1093/cercor/bhl181 Bechtel, W., & Abrahamsen, A. (2002). Connectionism and the Mind: Parallel Processing, Dynamics, and Evolution in Networks. Malden, MA: Blackwell Publishing. Behrmann, M., Watt, S., Black, S.E., & Barton, J.J.S. (1997). Impaired visual search in patients with unilateral neglect: an oculographic 29 analysis. Neuropsychologia, 35 (11), 14451458. Bilder, R.M. (2011). Neuropsychology 3.0: Evidencebased science and practice. Journal of the International Neuropsychological Society, 17 (1), 7-13. doi: 10.1017/S1355617710001396 Bowen, A., Lincoln, N.B., & Dewey, M.E. (2002). Spatial neglect: Is rehabilitation effective? Stroke, 33, 2728-2729. doi: 10.1161/01.STR.0000035747.03607.1A Bowen, A., & Wenman, R. (2002). The rehabilitation of unilateral neglect: A review of the evidence. Reviews in Clinical Gerontology, 12, 357-373. doi: 10.1017/S0959259802012479 Braun, M., Tupper, D., Kaufmann, P., McCrea, M., Postal, K., Westerveld, M., Wills, K., & Deer, T. (2011). Neuropsychological assessment: A valuable tool in the diagnosis and management of neurological, neurodevelopmental, medical, and psychiatric disorders. Cognitive and Behavioural Neurology, 24 (3), 107-114. doi: 10.1097/WNN.0b013e3182351289 Bressler, S.L., & Menon, V. (2010). Large-scale brain networks in cognition: Emerging methods and principles. Trends in Cognitive Sciences, 14, 277-290. doi: 10.1016/j.tics.2010.04.004 Bressler, S.L., & Tognoli, E. (2006). Operational principles of neurocognitive networks. International Journal of Psychophysiology, 60, 139-148. doi: 10.1016/j.ijpsycho.2005.12.008 Brozzoli, C., Dematte, M.L., Pavani, F., Frassinetti, F., & Farne, A. (2006). Neglect and extinction: Within and between sensory modalities. Restorative Neurology and Neuroscience, 24 (4-6), 217-232. Byrne, P., Becker, S., & Burgess, N. (2007). Remembering the past and imaging the future: A neural model of spatial memory and imagery. Psychological Review, 114 (2), 340-375. doi: 10.1037/0033-295X.114.2.340 Cazzoli, D., Schumacher, R., Baas, U., Müri, R.M., Wiest, R., Bohlhalter, S., Hess, C.W., & Nyffeler, T. (2012). Bilateral neglect after bihemispheric strokes. Cortex, 48, 504-508. doi: 10.1016/j.cortex.2011.09.006 Cernich, A.N., Kurtz, S.M., Mordecai, K.L., & Ryan, P.B. (2010). Cognitive rehabilitation in traumatic brain injury. Current Treatment Options in Neurology, 12, 412-423. doi: 10.1007/s11940-010-0085-6 Chechlacz, M., Rothstein, P., Bickerton, W., Hansen, P.C.., Deb, S., & Humphreys, G.W. (2010). Separating neural correlates of allocentric and egocentric neglect: distinct cortical sites and common white matter disconnections. Cognitive Neuropsychology, 27 (3), 277-303. doi: 10.1080/02643294.2010.519699 Cicerone, K.D. (2012). Facts, theories, values: Shaping the course of neurorehabilitation. The 60th John Stanley Coulter memorial lecture. Archives of Physical Medicine and Rehabilitation, 93, 188-191. doi: 10.1016/j.apmr.2011.12.003 Cicerone, K.D., Langenbahn, D.M., Braden, C., Malec, J.F., Kalmar, K., Fraas, M., Felicetti, T., Laatsch, L., Harley, J.P., Bergquist, T., Azulay, J., Cantor, J., & Ashman, T. (2011). Evidencebased cognitive rehabilitation: Updated review of the literature from 2003 through 2008. Archives of Physical Medicine and Rehabilitation, 92, 519-530. doi: 10.1016/j.apmr.2010.11.015 Cohen, J.D., Braver, T.S., & O’Reilly, R.C. (1996). A computational approach to prefrontal cortex, cognitive control and schizophrenia: recent developments and current challenges. Philosophical Transactions: Biological Sciences, 351 (1346), 1515-1527. Cohen, J.D., & Servan-Schreiber, D. (1992). Context, cortex and dopamine: A connectionist approach to behavior and biology in schizophrenia. Psychological Review, 99 (1), 45-77. Corbetta, M., Kincade, M.J., Lewis, C., Snyder, A.Z., & Sapir, A. (2005). Neural basis and recovery of spatial attention deficits in spatial neglect. 30 Nature Neuroscience, 8 (11), 1603-1610. doi: 10.1038/nn1574 Corbetta, M., & Shulman, G.L. (2002). Control of goal-directed and stimulus-driven attention in the brain. Nature Reviews Neuroscience, 3 (3), 201-215. doi: 10.1038/nrn755 Corbetta, M., & Shulman, G.L., (2011). Spatial neglect and attention networks. Annual Review of Neuroscience, 24, 569-599. doi: 10.1146/annurev-neuro-061010-113731 Deco, G., & Rolls, E.T. (2002). Object-based visual neglect: a computational hypothesis. European Journal of Neuroscience, 16, 19942002. doi: 10.1046/j.1460-9568.2002.02279.x Deco, G., & Zihl, J. (2004). A biased competition based neurodynamical model of visual neglect. Medical Engineering & Physics, 26, 733-743. doi: 10.1016/j.medengphy.2004.06.011 Deelman, B., & Eling, P. (2004). Klinische neuropsychologie. In Deelman, Eling, De Haan, & Van Zomeren (6th ed.). Klinische Neuropsychologie (pp. 17-42). Amsterdam, NL: Uitgeverij Boom. Dell, G.S., Chang, F., & Griffin, Z.M. (1999). Connectionist models of language production: Lexical access and grammatical encoding. Cognitive Science, 23 (4), 517-542. Driver, J., & Vuilleumier, P. (2001). Perceptual awareness and its loss in unilateral neglect and extinction. Cognition, 79 (1-2), 39-88. Driver, J., & Pouget, A. (2000). Object-centered visual neglect, or relative egocentric neglect? Ferber, S., & Karnath, H.-O. (2010). How to assess spatial neglect – Line bisection or cancellation tasks? Journal of Clinical and Experimental Neuropsychology, 23 (5), 599607. doi: 10.1076/jcen.23.5.599.1243 Fierro, B., Brighina, F., & Bisiach, E. (2006). Improving neglect by TMS. Behavioural Neurology, 17, 169-176. Fitzpatrick, D. (2008). Unit II: Sensation and sensory processing. In D. Purves, G.J. Augustine, D. Fitzpatrick, W.C. Hall, A.-S. LaMantia, J.O. McNamara & L.E. White (4th ed.). Neuroscience (pp. 205-394). Sunderland, MA: Sinauer Associates, Inc. Fruhmann-Berger, M., & Karnath, H.-O. (2005). Spontaneous eye and head position in patients with spatial neglect. Journal of Neurology, 252 (10), 1194-1200. doi: 10.1007/s00415-005-0831-y Gazzaniga, M.S., Ivry, R.B., & Mangun, G.R. (2009a). Methods of cognitive neuroscience. In M.S. Gazzaniga, R.B. Ivry & G.R. Mangun (3rd ed.). Cognitive Neuroscience: The Biology of the Mind (pp. 110-161). New York, NY: W.W. Norton & Company, Inc. Gazzaniga, M.S., Ivry, R.B., & Mangun, G.R. (2009b). Attention and consciousness. In M.S. Gazzaniga, R.B. Ivry & G.R. Mangun (3rd ed.). Cognitive Neuroscience: The Biology of the Mind (pp. 110-161). New York, NY: W.W. Norton & Company, Inc. Gazzaniga, M.S., Ivry, R.B., & Mangun, G.R. (2009c). Hemispheric specialization. In M.S. Gazzaniga, R.B. Ivry & G.R. Mangun (3rd ed.). Cognitive Neuroscience: The Biology of the Mind (pp. 110-161). New York, NY: W.W. Norton & Company, Inc. Gillen, R., Tennen, H., & McKee, T. (2005). Unilateral spatial neglect: Relation to rehabilitation outcomes in patients with right hemisphere stroke. Archives of Physical Medicine and Rehabilitation, 86, 763-767. doi: 10.1016/j.apmr.2004.10.029 Goodale, M.A. & Milner, A.D. (1992). Separate visual pathways for perception and action. Trends in Neurosciences, 15 (1), 20-25. doi: 10.1016/0166-2236(92)90344-8 Goldrick, M. (2008). Does like attract like? Exploring the relationship between errors and representational structure in connectionist networks. Cognitive Neuropsychology, 35 (2), 287-313. doi: 10.1080/02643290701417939 31 Harley, T.A. (1996). Connectionist modelling of the recovery of language functions following brain damage. Brain and Language, 52 (1), 724. doi: 10.1006/brln.1996.0003 Heilman, K.M., & Valenstein, E. (2011). Introduction. In Heilman, & Valenstein (5th ed.). Clinical Neuropsychology (pp. 3-21). New York, NY: Oxford University Press. Heilman, K.M., Watson, R.T. & Valenstein, E. (2011). Neglect and related disorders. In Heilman, & Valenstein (5th ed.). Clinical Neuropsychology (pp. 3-21). New York, NY: Oxford University Press. Heinke, D., Deco, G., Zihl, J., & Humphreys, G.W. (2002). A computational neuroscience account of visual neglect. Neurocomputing, 44 (46), 811-816. doi: 10.1016/S0925-2312(02)00477-0 Heinke, D., & Humphreys, G.W. (2003). Attention, spatial representation, and visual neglect: simulating emergent attention and spatial memory in the Selective Attention for Identification Model (SAIM). Psychological Review, 110, 29-87. doi: 10.1037/0033-295X.110.1.29 Hesse, M.D., Sparing, R., & Fink, G.R. (2011). Ameliorating spatial neglect with noninvasive brain stimulation: From pathophysiological concepts to novel treatment strategies. Neuropsychological Rehabilitation, 21 (5), 676-702. Hilgetag, C.C. (2000). Spatial neglect and paradoxical lesion effects in the cat – A model based on midbrain connectivity. Neurocomputing, 32-33, 793-799. doi: 10.1016/S0925-2312(00)00246-0 Hilgetag, C.C., Kötter, R., & Young, M.P. (1999) Inter-hemispheric competition of subcortical structures is a crucial mechanism in paradoxical lesion effects and spatial neglect. Progress in Brain Research, 121, 125-146. Hillis, A.E. (2006). Neurobiology of unilateral spatial neglect. The Neuroscientist, 12 (2), 153-163. doi: 10.1177/1073858405284257 Hillis, A.E., & Caramazza, A. (1991). Category specific naming and comprehension – a double dissociation. Brain, 114 (5), 20812094. doi: 10.1093/brain/114.5.2081 Hodgson, T.L., Mort, D., Chamberlain, M.M., Hutton, S.B., O’Neill, K.S., & Kennard, C. (2002). Orbitofrontal cortex mediates inhibition of return. Neuropsychologia, 40 (12), 1891-1901. Hoffmann, T., Bennett, S., Koh, C.-L., & McKenna, K.T. (2011). The Cochrane review of occupational therapy for cognitive impairment in stroke patients. European Journal of Physical and Rehabilition Medicine, 47 (3), 513-519. Husain, M., Mannan, S., Hodgson, T., Wojciulik, E., Driver, J., & Kennard (2001). Impaired spatial working memory across saccades contributes to abnormal search in parietal neglect. Brain, 124, 941-952. doi: 10.1093/brain/124.5.941 Kastner, S., & Ungerleider, L.G. (2000). Mechanisms of visual attention in the human visual cortex. Annual Review of Neuroscience, 23, 315-341. doi: 10.1146/annurev.neuro.23.1.315 Karnath, H.-O., Mandler, A., & Clavagnier, S. (2011). Object-based neglect varies with egocentric position. Journal of Cognitive Neuroscience, 23 (10), 2983-2993. doi: 10.1162/jocn_a_00005 Karnath, H.-O., Rennig, J., Johannsen, L., & Rorden, C. (2011). The anatomy underlying acute versus chronic spatial neglect: a longitudinal study. Brain, 134, 903-912. doi: 10.1093/brain/awq355 Karnath, H.-O., & Rorden, C. (2012). The anatomy of spatial neglect. Neuropsychologia, 50 (6), 1010-1017. doi: 10.1016/j.neuropsychologia.2011.06.027 Kinsbourne M. (1977). Hemi neglect and hemisphere rivalry. In E. Weinstein & R. Friedland (Eds.). Hemi inattention and hemispheric specialization: Advances in neurology (pp. 92-105). New York, NY: Raven. Kinsbourne, M. (1993). Orientational bias mode of unilateral neglect: Evidence from attentional gradients within hemispace. In I.H. Robertson & J.C. Marshall (Eds.). Unilateral Neglect: Clinical and Experimental Studies (pp. 63-86). East Sussex, UK: Lawrence Erlbaum Associates Ltd., Publishers. 32 Koch, G., Bonni, S., Giacobbe, V., Bucchi, G., Basile, B., Lupo, F., Versace, V., Bozzali, M, & Caltagirone (2012). Theta-burst stimulation of the left hemisphere accelerates recovery of hemispatial neglect. Neurology, 78, 24-30. doi: 10.1212/WNL.0b013e31823ed08f Koch, G., Oliveri, M., Cheeran, B., Ruge, D., Lo Gerfo, E., Salerno, S., Torriero, S., Marconi, B., Mori, F., Driver, J., Rothwell, J.C., & Caltagirone, C. (2008). Hyperexcitability of parietal-motor functional connections in the intact left-hemisphere of patients with neglect. Brain, 131, 3147-3155. doi: 10.1093/brain/awn273 Kettunen, J.E., Nurmi, M, Dastidar, P., & Jehkonen, M. (2012). Recovery from visual neglect after right hemisphere stroke: Does starting point in cancellation tasks change after 6 months? The Clinical Neuropsychologist, 26 (2), 305320. doi: 10.1080/13854046.2011.648213 Kinsbourne, M. (1993). Orientational bias model of unilateral neglect: evidence from attentional gradients within hemispace. In: I.H. Robertson, J.C. Marshall (Eds.). Unilateral Neglect: Clinical and Experimental Studie (pp. 63-86). Hove, UK: Lawrence Erlbaum Associates. Klein, R.M. (2000). Inhibition of return. Trends in Cognitive Sciences, 4 (4), 138-147. doi: 10.1016/S1364-6613(00)01452-2. Kleinman, J.T., Newhart, M., Davis, C., Heidler-Gary, J., Gottesman, R.F., & Hillis, A.E. (2007). Right hemispatial neglect: Frequency and characterization following acute left hemisphere stroke. Brain and Cognition, 64, 50-59. doi: 10.1016/j.bandc.2006.10.005 Lanyon L.J., & Denham, S.L. (2004a). A model of active visual search with object-based attention guiding scan paths. Neural Networks, 17, 873-897. doi: 10.1016/j.neunet.2004.03.012 Lanyon L.J., & Denham, S.L. (2004b). A biased competition computational model of spatial and object-based attention mediating active visual search. Neurocomputing, 58-60, 655662. doi: 10.1016/j.neucom.2004.01.110 Lanyon, L.J., & Denham, S.L. (2010). Modelling visual neglect: Computational insights into conscious perception. PLoS ONE, 5 (6), e11128. doi: 10.1371/journal.pone.0011128 Levelt, W.J.M. (1995) The ability to speak: From intentions to spoken words. European Review, 3 (1), 13-23. doi: 10.1017/S1062798700001290 Mennemeier, M., Vezey, E., Chatterjee, A., Rapcsak, S. Z., & Heilman, K. M. (1997). Contributions of the left and right cerebral hemispheres to line bisection. Neuropsychologia, 35, 703– 715. doi: 10.1016/S0028-3932(96)00114-5 Lezak, M.D. (2004). The theory and practice of neuropsychological assessment. In M Lezak (4th ed.). Neuropsychological Assessment (pp. 3-14). New York, NY: Oxford University Press. Milner, A.D, & Goodale, M.A. (2008). Two visual systems re-viewed. Neuropsychologia, 46 (3), 774-785. doi: 10.1016/j.neuropsychologia.2007.10.005 Lincoln, N.B., & Bowen, A. (2006). The need for randomised treatment studies in neglect research. Restorative Neurology and Neuroscience, 24, 401-408. Mogensen, J., & Malá, H. (2009). Post-traumatic functional recovery and reorganization in animal models: A theoretical and methodological challenge. Scandinavian Journal of Psyhcology, 50, 561-573. doi: 10.1111/j.1467-9450.2009.00781.x Maass, W. (2000). On the computational power of winner-take-all. Neural computation, 12 (11), 2519-2535. doi: 10.1162/089976600300014827 Mannan, S.K., Mort, D.J., Hodgson, T.L., Driver, J., Kennard, C., & Husain, M. (2005). Revisiting previously searched locations in visual neglect: Role of right parietal and frontal lesions in misjudging old locations as new. Journal of Cognitive Neuroscience, 17 (2), 340-354. doi: 10.1162/0898929053124983 Marsh, E.B., & Hillis, A.E. (2008). Dissociation between egocentric and allocentric visuospatial and tactile neglect in acute stroke. Cortex, 44, 1215–1220. doi: 10.1016/j.cortex.2006.02.002 Marshall, J.C., & Halligan, P.W. (1989). When right goes left: An investigation of line bisection in a case of visual neglect. Cortex, 25, 503–515. Mavritsaki, E., Heinke, D., Deco, G., & Humphreys, G. (2009). Simulating posterior parietal damage in a biologically plausible framework: Neuropsychological tests of the search over time and space model. Cognitive Neuropsychology, 26 (4), 343-390. doi: 10.1080/02643290903424329 33 Molenberghs, P., & Sale, M.V. (2011). Testing for spatial neglect with line bisection and target cancellation: are both tasks really unrelated? PloS one, 6 (7), e23017. doi: 10.1371/journal.pone.0023017 Molenberghs, P., Sale, M.V., & Mattingley, J.B. (2012). Is there a critical lesion site for unilateral spatial neglect? A meta-analysis using activation likelihood estimation. Frontiers in Human Neuroscience, 6, 1-10. doi: 10.3389/fnhum.2012.00078 Monaghan, P., & Shillcock, R. (2004). Hemispheric asymmetries in cognitive modelling: Connectionist modelling of unilateral visual neglect. Psychological review, 111 (2), 283308. doi: 10.1037/0033-295X.111.2.283 Mozer, M.C. (1991). The perception of multiple objects: A connectionist approach. Cambridge, MA: MIT Press/Bradford Books. Mozer, M.C., Halligan, P.W., & Marshall, J.C. (1997). The end of the line for a brain-damaged model of unilateral neglect. Journal of cognitive neuroscience, 9, 171-190. doi: 10.1162/jocn.1997.9.2.171 McClelland, J. L. & Rumelhart, D. E. (1981). An interactive activation model of context effects in letter perception: Part 1. An account of Basic Findings. Psychological Review, 88, 375-407. Mozer, M.C. (2002). Frames of reference in unilateral neglect and visual perception: a computational perspective. Psychological Review, 109, 156-185. doi: 10.1037//0033-295X.109.1.156 Meeter, M., & Murre, J.M.J. (2005). TraceLink: A model of consolidation and amnesia. Cognitive Neuropsychology, 22 (5), 559-587. doi: 10.1080/02643290442000194 Müri, R.M., Cazzoli, D., Nyffeler, T., & Pflugshaupt (2009). Visual exploration pattern in hemineglect. Psychological research, 73, 147-157. doi: 10.1007/s00426-008-0204-0 Nadel, L., Samsonovich, A., Ryan, L., & Moscovich, M. (2000). Multiple trace theory of human memory: Computational, neuroimaging, and neuropsychological results. Hippocampus, 10, 352-368. doi: 10.1002/10981063(2000)10:4<352::AID-HIPO2>3.0.CO;2-D Niemeier, M., & Karnath, H.-O. (2002). Simulating and testing visual exploration in spatial neglect based on a new model for cortical coordinate transformation. Experimental Brain Research, 145, 512-519. doi: 10.1007/s00221-002-1132-7 Nys, G.M.S., Stuart, M., & Dijkerman, H.C. (2010). Repetitive exploration towards locations that no longer carry a target in patients with neglect. Journal of Neuropsychology, 4, 3345. doi: 10.1348/174866408X402424 Nys, G.M.S., van Zandvoort, M.J.E., de Kort, P.L.M., van der Worp, H.B., Jansen, B.P.W., Algra, A., de Haan, E.H.F., & Kapelle, L.J. (2005). The prognostic value of domain-specific cognitive abilities in acute first-ever stroke. Neurology, 64, 821–827. doi: 10.1212/01.WNL.0000152984.28420.5A Oliveri, M. (2011). Brain stimulation procedures for treatment of contralesional spatial neglect. Restorative Neurology and Neuroscience, 29, 421-425. doi: 10.3233/RNN-2011-0613 O’Reilly, R.C. & Munakata, Y. (2000). Computational Explorations in Cognitive Neuroscience: Understanding the Mind by Simulating the Brain. Cambridge, MA: MIT Press. Patron, A., Malhotra, P., & Husain, M. (2004). Hemispatial neglect. Journal of Neurology, Neurosurgery, and Psychiatry, 75 (1), 13-21. Pavlovskaya, M., Glass, I., Soroker, N., Blum, B., & Groswasser, Z. (1997). Coordinate frame for pattern recognition in unilateral neglect. Cognitive Neuroscience, 9, 824–834. doi: 10.1162/jocn.1997.9.6.824 Parton, A., Malhotra, P., Nachev, P., Ames, D., Ball, J., Chataway, J., & Husain, M. (2006). Space re-exploration in hemispatial neglect. Neuroreport, 17 (8), 833-836. doi: 10.1097/01.wnr.0000220130.86349.a7 Plaut, D.C. (1996). Relearning after damage in connectionist networks: Toward a theory of rehabilitation. Brain and Language, 52 (1), 25-82. doi: 10.1006/brln.1996.0004 34 Plaut, D.C. (2002). Graded modality-specific specialisation in semantics: A computational account of optic aphasia. Cognitive Neuropsychology, 19 (7), 603-639. doi: 10.1080/02643290244000112 Pouget, A., & Sejnowski, T.J. (1996). A model of spatial representations in parietal cortex explains hemineglect. Advances in Neural Information Processing, 8, 10-16. Pouget, A., & Sejnowski, T.J. (1997a). Spatial transformations in the parietal cortex using basis functions. Journal of Cognitive Neuroscience, 9 (2), 222-237. doi: 10.1162/jocn.1997.9.2.222 Pouget, A., & Sejnowksi, T.J. (1997b). A new view of hemineglect based on the response properties of parietal neurones. Philosophical Transactions: Biological Sciences, 352, 1449-1459. Pouget, A., & Sejnowski, T.J. (2001). Simulating a lesion in a basis function model of spatial representations: Comparison with hemineglect. Psychological review, 108 (3). 653-673. Powel, L.E., Glang, A., Ettel, D., Todis, B., Sohlberg, M.M., & Albin, R. (2012). Systematic instruction for individuals with acquired brain injury: Results of a randomised controlled trial. Neuropsychological Rehabilitation: An International Journal, 22 (1), 85-112. doi: 10.1080/09602011.2011.640466 Price, C.J., & Friston, K.J. (2005). Functional ontologies for cognition: The systematic definition of structure and function. Cognitive Neuropsychology, 22 (3-4), 262275. doi: 10.1080/02643290442000095 Raymer, A.M., Beeson, P., Holland, A., Kendall, D., Maher, L.M., Martin, N., et al. (2008). Translational research in aphasia: From neuroscience to neurorehabilitation. Journal of speech, language, and hearing research, 51 (1). S259-S275. doi:10.1044/10924388(2008/020) Ringman, J.M., Saver, J.L., Woolson, R.F., Clarcke, W.R., & Adams, H.P. (2004) - Frequency, risk factors, anatomy, and course of unilateral neglect in an acute stroke cohort. Neurology, 63 (3), 468-474. Rorden, C., Fruhmann Berger, M., & Karnath, H-O. (2006). Disturbed line bisection is associated with posterior brain lesions. Brain Research, 1, 17-25. doi: 10.1016/j.brainres.2004.10.071 Rorden, C., & Karnath, H.-O. (2004). Using human brain lesions to infer function: a relic from a past era in the fMRI age? Nature Reviews Neuroscience, 5, 812-819. doi: 10.1038/nrn1521 Rorden, C., & Karnath, H.-O. (2010). A simple measure of neglect severity. Neuropsychologia, 48, 2758-2763. doi: 10.1016/j.neuropsychologia.2010.04.018 Rorden, C., Hjaltason, H., Fillmore, P., Fridriksson, J., Kjartansson, O., Magnusdottir, S., & Karnath, H.-O. (2012). Allocentric neglect strongly associated with egocentric neglect. Neuropsychologia, 50 (6), 1151-1157. doi: 10.1016/j.neuropsychologia.2012.03.031 Rohling, M.L., Faust, M.E., Beverly, B., & Damakis, G. (2009). Effectiveness of cognitive rehabilitation following acquired brain injury: A meta-analytic re-examination of Cicerone et al’s (2000,2005) systematic reviews. Neuropsyhcology, 23 (1), 20-39. doi: 10.1037/a0013659.supp Rolls, E.T., Loh, M., Deco, G., & Winterer, G. (2008). Computational models of schizophrenia and dopamine modulation in the prefrontal cortex. Nature Reviews Neuroscience, 9, 696709. doi: 10.1038/nrn2462 Sack, A.T. (2010). Using non-invasive brain interference as a tool for mimicking spatial neglect in healthy volunteers. Restorative Neurology and Neuroscience, 28, 485-497. doi: 10.3233/RNN-2010-0568 Savazzi, S., Posteraro, L., Veronesi, G., & Mancini, F. (2007). Rightward and leftward bisection biases in spatial neglect: Two sides of the same coin? Brain, 130, 2070-2084. doi: 10.1093/brain/awm143 35 Seghier, M.L., Zeidman, P., Neufeld, N.H., Leff, A.P., & Price, C.J. (2010). Indentifying abnormal connectivity in patients using dynamic causal modelling of fMRI responses. Frontiers in Systems Neuroscience, 4, 1-14. doi: 10.3389/fnsys.2010.00142 Schierwagen, A., (2012). On reverse engineering in the cognitive and brain sciences. Natural Computing, 11, 141-150. doi: 10.1007/s11047-012-9306-0 Smith, T.J., & Henderson, J.M. (2011). Does oculomotor inhibition of return influence fixation probability during scene search? Attention, Perception, & Psychophysics, 73, 2384-2398. doi: 10.3758/s13414-011-0191-x Thomas, M.S.C. & McClelland, J.L. (2008). Connectionist models of cognition. In R. Sun (Ed.). Cambridge handbook of computational psychology (pp. 23-58). New York, NY: Cambridge University Press. Umarova, R.M., Dorothee, S., Kaller, C.P., Vry, M.-S., Glauche, V., Mader, I., Hennig, J., & Weiller, C. (2011). Acute visual neglect and extinction: distinct functional state of the visuospatial attention system. Brain, 134 (11), 3310-3325. doi: 10.1093/brain/awr220 Urbanski, M., Thiebaut de Schotten, M., Rodrigo, S., Oppenheim, C., Touzé, E., Méder, J.-F., Moreau, K., Louper-Jeny, C., Dubois, B., & Bartolomeo, P. (2011). DTI-MR tractography of white matter damage in stroke patients with neglect. Experimental Brain Research, 208, 491-505. doi: 10.1007/s00221-010-2496-8 Vakil, E. (2011). Neuropsychological assessment: Principles, rationale and challenges. Journal of Clinical and Experimental Neuropsychology, 34 (2), 135-150. doi: 10.1080/13803395.2011.623121 Vossel, S., Eschenbeck, P., Weiss, P.H., Saliger, J., Karbe, H., & Fink, G.R. (2011). Visual extinction in relation to visuospatial neglect after right-hemisphere stroke: Quantitative assessment and statistical lesion-symptom mapping. Journal of Neurology, Neurosurgery & Psychiatry, 82 (8), 862-868 doi: 10.1136/jnnp.2010.224261 Wang, Z., & Klein, R.M. (2009). Searching for inhibition of return in visual search: A review. Vision Research, 50, 220-228. doi: 10.1016/j.visres.2009.11.013 Weddell, R.A. (2004). Subcortical modulation of spatial attention including evidence that the Sprague effect extends to man. Brain and Cognition, 55, 497-506. doi: 10.1016/j.bandc.2004.02.075 Yue, .Y, Song, W., Huo, S., & Wang, M. (2012). Study on the occurrence and neural bases of hemispatial neglect with different reference frames. Archives of physical medicine and rehabilitation, 93 (1), 156-162. doi: 10.1016/j.apmr.2011.07.192 Zamora-López, G., Zhou, C., & Kurths, J. (2010). Exploring brain function from anatomical connectivity. Frontiers in Neuroscience, 5, 111. doi: 10.3389/fnins.2011.00083 36