TY - JOUR
AU1 - Yu,, Qing
AU2 - Shim, Won, Mok
AB - Abstract The respective roles of occipital, parietal, and frontal cortices in visual working memory maintenance have long been under debate. Previous work on whether parietal and frontal regions convey mnemonic information has yielded mixed findings. One possibility for this variability is that the mnemonic representations in high-level frontoparietal regions are modulated by attentional priority, such as temporal order. To test this hypothesis, we examined whether the most recent item, which has a higher attentional priority in terms of temporal order, is preferentially encoded in frontoparietal regions. On each trial, participants viewed 2 gratings with different orientations in succession, and were cued to remember one of them. Using fMRI and an inverted encoding model, we reconstructed population-level, orientation representations in occipital (V1–V3), parietal (IPS), and frontal (FEF) areas during memory maintenance. Unlike early visual cortex where robust orientation representations were observed regardless of serial order, parietal, and frontal cortices showed stronger representations when participants remembered the second grating. A subsequent experiment using a change detection task on color rings excluded the possibilities of residual stimulus-driven signals or motor preparative signals for responses. These results suggest that mnemonic representations in parietal and frontal cortices are modulated by temporal-order-based attentional priority signals. attentional priority, early visual cortex, parietal cortex, temporal order, visual working memory Introduction Recent advances in visual working memory (VWM) research have demonstrated differential contributions of a broad brain network in VWM maintenance: with sensory cortex sharing the same representations between perception and working memory (i.e., sensory recruitment hypothesis) (Awh and Jonides 2001; Pasternak and Greenlee 2005; Postle 2006; D’Esposito 2007; Harrison and Tong 2009; Serences et al. 2009; Riggall and Postle 2012; Emrich et al. 2013), and higher-level parietal and frontal cortices exhibiting sustained elevated activity related to executive control (Fuster and Alexander 1971; Miller et al. 1996; Ungerleider et al. 1998; D’Esposito et al. 2000; Todd and Marois 2004; Srimal and Curtis 2008; Zanto et al. 2011; Sreenivasan et al. 2014). While mnemonic representations in early sensory cortex and executive control in high-level frontoparietal regions have been widely supported in VWM research, whether parietal and frontal cortices also represent stimulus-specific information during memory delay remains unclear. Previous fMRI work using multivariate decoding methods has shown mixed findings: some studies found decodable memory contents in frontal (Albers et al. 2013; Lee et al. 2013) and parietal (Christophel et al. 2012; Albers et al. 2013; Bettencourt and Xu 2016) areas, while others failed to decode the content of VWM in either frontal or parietal areas (Riggall and Postle 2012; Emrich et al. 2013), or failed at higher memory loads (Gosseries, Yu, et al. 2018). Recent work using inverted encoding models (IEMs) has provided evidence supporting mnemonic representations in frontoparietal regions. Ester et al. (2015) have demonstrated that orientation information held in working memory was maintained in a broad neural network, including occipital cortex and several subregions in parietal and frontal cortices, suggesting that parietal and frontal cortices can also maintain stimulus-specific information. More recent work (Yu and Shim 2017) further showed that color- and orientation-selective responses in parietal and frontal regions are modulated by task demand during VWM maintenance. Despite all the supporting evidence, much remains to be explored as to the nature of the VWM representations in parietal and frontal cortices. For instance, it still remains unclear whether the functional characteristics of the mnemonic representations in parietal and frontal cortices different from those in early visual cortex. Prior work on attention models proposed that attention is allocated based on a saliency map, which is a topographic map of activation that represents a weighted sum of the bottom-up (e.g., stimulus salience) and top-down (e.g., behavioral priority or relevance) factors, and frontoparietal network has been implicated in representing this attention priority map for target selection via both bottom-up and top-down mechanisms (Fecteau and Munoz 2006; Serences and Yantis 2006; Bisley and Goldberg 2010). Thus, one possibility is that mnemonic representations in parietal and frontal areas are more susceptible to attentional priority signals compared with early sensory cortices. Furthermore, most of the proposed priority maps are based on either visual features (e.g., contrast, color) or space, and few studies have examined how an attentional priority map operates on the temporal domain. Evidence from previous studies on visual memory has shown that not all items in a sequence are represented with equal strength and that the most recent item maintains a privileged attentional state during retention. Early behavioral work based on a change detection paradigm as well as more recent studies using a delay-estimation task have demonstrated superior memory performance (higher precision and/or faster reaction times) for the item presented last in a sequence (Phillips and Christie 1977; Broadbent and Broadbent 1981; Gorgoraptis et al. 2011; Zokaei et al. 2011). This recency effect could occur even when only 2 items are presented in the sequence (Gorgoraptis et al. 2011), and is resistant to perceptual interference (McElree and Dosher 1989; Nee and Jonides 2008). More interestingly, the recency effect appears to persist even when the focus of attention needs to be switched between multiple items in working memory (Vergauwe et al. 2016). This evidence suggests that the most recent item may be attentionally prioritized over other items in working memory. Neurally, the representation of temporal order information has been shown to engage posterior parietal and lateral prefrontal cortex (LPFC) (Marshuetz et al. 2000; Carpenter et al. 2018). We therefore hypothesize that attentional priority based on temporal order would impact feature-selective representations in parietal and frontal cortices, such that the item presented most recently is preferentially encoded. In the current study, by separately examining neural responses to each memory sample presented in different serial position, we investigated how attentional priority influences stimulus-specific mnemonic representations maintained in parietal and frontal cortices. Specifically, in Experiment 1, participants performed a delay-estimation task on 1 of 2 sequentially presented oriented gratings. Using an IEM of orientation (Brouwer and Heeger 2009, 2011; Ester et al. 2013, 2015; Yu and Shim 2017), we reconstructed population-level feature representations of memory items during the delay phase as a function of serial position. Our results reveal that delay period activity in the parietal and frontal regions, but not early visual cortex, retain enhanced neural representations of the most recent item, as compared with the preceding item. To exclude the possibility that the effect we observed in Experiment 1 was driven by alternative confounds (e.g., spatial attention or eye movements along the oriented lines of the most recent grating; preparation of motor responses toward the position of the target orientation on the probe wheel; residual stimulus-driven activity from the most recent sample), we conducted Experiment 2 in which participants performed a change detection task on 1 of 2 sequentially presented color rings. Using an IEM of color, we again demonstrated a similar neural recency effect in parietal but not early visual cortex. These 2 experiments together suggest that mnemonic representations in parietal and frontal cortices are modulated by attentional priority based on temporal order. Materials and Methods Participants A total of 15 participants from Dartmouth College (18–31 years of age, 5 males) participated in Experiment 1 — 9 of which also participated in the attention task of Experiment 1. Ten participants (18–21 years of age, 4 males) participated in the fMRI session of Experiment 2, and another 12 participants (18–27 years of age, 4 males) participated in the behavioral session of Experiment 2. All had normal or corrected-to-normal vision, normal color vision (in Experiment 2), and were neurologically intact. Participants were provided informed consent in accordance with the Institutional Review Board of Dartmouth College before the experiment and were compensated for their participation with cash or course credits. All participants were naïve to the purpose of the study. Stimuli and Procedure Experiment 1 Stimuli were generated and presented using Matlab and Psychtoolbox 3 (Brainard 1997; Pelli 1997). Participants performed an orientation delay-estimation task (Wilken and Ma 2004) in the scanner while fixating at the central dot (Fig. 1A). Each trial began with the sequential presentation of 2 sinusoidal gratings within an annulus (inner radius = 1.5°, outer radius = 10°, contrast = 0.6, spatial frequency = 0.5 cycles/deg with a phase angle randomized between 0° and 180°) on a gray background. The orientations of the 2 gratings were randomly chosen from 3 orthogonal pairs: 0° and 90°, 30° and 120°, and 60° and 150°, with a random jitter of 0–3° added to each orientation on each trial. Each grating was presented for 500 ms, with an interstimulus interval of 400 ms. The presentation order of the 2 orientations was randomized for each trial. The 800 ms after the presentation of the second grating, a digit cue was displayed at the center indicating which of the 2 gratings to remember (“1” or “2” corresponding to the first and the second grating, respectively). The likelihood of either grating being cued was equal (50%). The cue duration was 800 ms, followed by a retention period of 10.4 s. At the end of the delay period, a test grating appeared that participants rotated, via button presses, until it matched the remembered orientation of the cued grating as precisely as possible. The angular change brought by each button press (clockwise or counterclockwise rotation) was 5°, and 5 s was given to complete the adjustment. During retention, a red fixation dot was displayed instead of a white one in order to distinguish this period from the resting period. Each run consisted of twelve trials, with 2 trials for each remembered orientation and lasted 308 s. Intertrial intervals varied in 4, 6, 8, or 10 s, and there was an 8 s blank fixation period at the beginning of each run. Participants finished 12–14 runs in total, depending on the time limit. The memory recall error was measured for each trial as the absolute angular difference (in degrees) between the orientation of the cued grating and that of a participant’s response. Before scanning, each participant practiced the working memory task outside the scanner for about 20 min. Figure 1. View largeDownload slide Orientation delay-estimation and attention tasks in Experiment 1 and color change detection task in Experiment 2. (A) Participants (N = 15) performed an orientation delay-estimation task in the scanner. On each trial, they viewed 2 gratings presented in succession and were cued to remember the orientation of one grating in the 2-grating sequence (Remember First or Remember Second). After a prolonged delay of 10.4 s, participants adjusted a test grating until it matched the remembered orientation of the cued grating in 5 s. (B) Participants (N = 9) performed an orientation attention task in the scanner. On each trial, they viewed gratings flickering at 3 Hz at the center of the screen for 4 s and were asked to report the direction of the orientation change (clockwise or counterclockwise) in one of the frames. (C) Participants (N = 10) performed a color change detection task. On each trial, they viewed 2 concentric rings with different colors presented in succession and were cued to remember one of the colors (remember first or remember second). A color mask stimulus was presented briefly after the presentation of the 2 samples. After a prolonged delay of 9.3 s, a test color appeared, and participants were required to indicate whether the test color was the same as the remembered color in 2 s. Figure 1. View largeDownload slide Orientation delay-estimation and attention tasks in Experiment 1 and color change detection task in Experiment 2. (A) Participants (N = 15) performed an orientation delay-estimation task in the scanner. On each trial, they viewed 2 gratings presented in succession and were cued to remember the orientation of one grating in the 2-grating sequence (Remember First or Remember Second). After a prolonged delay of 10.4 s, participants adjusted a test grating until it matched the remembered orientation of the cued grating in 5 s. (B) Participants (N = 9) performed an orientation attention task in the scanner. On each trial, they viewed gratings flickering at 3 Hz at the center of the screen for 4 s and were asked to report the direction of the orientation change (clockwise or counterclockwise) in one of the frames. (C) Participants (N = 10) performed a color change detection task. On each trial, they viewed 2 concentric rings with different colors presented in succession and were cued to remember one of the colors (remember first or remember second). A color mask stimulus was presented briefly after the presentation of the 2 samples. After a prolonged delay of 9.3 s, a test color appeared, and participants were required to indicate whether the test color was the same as the remembered color in 2 s. In addition to the main experimental runs, participants also performed 2 visual field localizer runs. The visual field localizer consisted of blocked presentations of a flickering checkerboard within an annulus of 3°–9.5° eccentricity (block duration = 16 s). Participants fixated at the central fixation and pressed the button when it changed color. A retinotopic mapping session was also conducted following standard retinotopic mapping procedures (Engel et al. 1994; Sereno et al. 1995). Twelve participants performed 2 additional working memory localizer runs, which were identical to the main working memory runs, except that the retention period was short (2 s). Nine out of the fifteen participants performed an additional orientation discrimination task (attention task) in a separate scan session (Fig. 1B). On each trial, one grating (parameters identical to the grating used in the working memory task) flickered at 3 Hz at the center of the screen for 4 s (6 frames in total, each frame contains ~0.33 s of stimulus and ~0.33 s of blank, and the phase of the grating randomly changed on each frame). The orientation of the grating was randomly chosen from the 6 orientations (0°, 30°, 60°, 90°, 120°, and 150°) used in the working memory task. During one of the frames, the grating rotated clockwise or counterclockwise by 3°–5°, and participants were instructed to respond to the direction of the change by pressing corresponding buttons. The response interval was 2 s, and intertrial interval varied in 4, 6, and 8 s. A total of 18 trials were conducted in each run, with 3 trials for each orientation. There was an additional 8 s blank period at the beginning of each run. Each run lasted 248 s and each participant finished 10 runs in total. Experiment 2 Participants performed a color change detection task in the scanner while fixating at the central dot. Each trial began with a sequential presentation of 2 concentric rings (radius = 10°, spatial frequency = 1 cycles/deg, phase angle randomized between 0° and 180°) with different colors at the center of the screen. All colors were equiluminant to the background, determined by a minimum motion adjustment task (Cavanagh et al. 1984) prior to scanning for each individual participant. The cued color was randomly chosen from 9 colors that are equally spaced on the CIEL*a*b color space (L* = 45, a* = 0, b* = 0). The uncued color was always 160° apart from the cued color (+160° or −160°). A random jitter of 0–3° was added to each color on each trial. Each color was presented for 500 ms, with an interstimulus interval of 400 ms. The presentation order of the 2 colors was randomized for each trial. The 800 ms after the presentation of the second color, a digit cue was displayed at the center for 800 ms indicating which of the 2 colors to remember (“1” or “2” corresponding to the first and the second color, respectively). The likelihood of either color being cued was equal. The 1 s after the cue, a color dot mask with the same size as the sample was presented at the center for 200 ms to mask stimulus-driven activity from the sample colors. The dots in the mask (size of a dot = 0.52°, number of dots = 550) were displayed in concentric rings, and the color of each dot was randomly chosen from the 9 sample colors with a reduced (40%) saturation value. The mask was then followed by a retention period of 9.3 s. At the end of the delay period, a test color appeared for 500 ms, and participants reported whether the test color was identical to the cued color in 2 s. The angular color change of the test color relative to the sample color, if existed, was 10°–15° (Fig. 1C). Each run consisted of 18 trials, with 2 trials for each color, and lasted 386 s. Intertrial intervals varied in 4 or 6 s and there was an 8 s blank fixation period at the beginning of each run. Participants finished 10–13 runs depending on the time limit. Before scanning, each participant practiced the working memory task outside the scanner for about 30 min. In addition to the main experimental runs, participants also performed several localizer runs, including retinotopic mapping (see Experiment 1), visual field localizer and working memory localizer. The visual field localizer consisted of alternating, blocked presentations of flickering chromatic and achromatic random dot patches at the center of the screen (radius = 10°, block duration = 12 s). Participants fixated at the central fixation and pressed the button when the fixation checker flipped. The working memory localizer was identical to that of the main experimental runs except that the intertrial interval was fixed at 12 s and there was no mask after the presentation of 2 sample gratings. A new group of participants performed the same task outside the scanner in a separate session. The experimental task followed that of the fMRI session, except that the memory delay was 3 s instead of 9.3 s. Each run consisted of 36 trials, and each participant performed 5–7 runs depending on the time limit. Data Acquisition MRI data were acquired using a Philips Intera Achieva 3 T MRI scanner with a 32-channel head coil. Anatomical images were obtained using a standard T1-weighted MPRAGE sequence (voxel size = 1 × 1 × 1 mm3). Functional imaging was conducted using a gradient-echo echo-planar imaging (EPI) sequence (TR = 2 s, TE = 35 ms, FOV = 240 × 240 mm2, voxel size = 3 × 3 × 3 mm3, flip angle = 90°). The 33 slices oriented along the calcarine sulcus were acquired to primarily cover occipital, parietal, and frontal cortices. Data Preprocessing and ROI Definition Functional MRI data were preprocessed using AFNI (http://afni.nimh.nih.gov; Cox 1996). The data were first registered to the final volume of each scan and to the retinotopy session. The data were then motion corrected, linearly and quadratically detrended, and z-score normalized within each run. Localizer data were spatially smoothed with a 4-mm full width at half maximum (FWHM) Gaussian kernel. For the group-level whole-brain analyses in Experiment 1, the data were further normalized to the MNI-ICBM 152 space (Mazziotta et al. 2001). All functional regions-of-interest (ROIs) were defined on the cortical surface reconstructed with FreeSurfer (http://surfer.nmr.mgh.harvard.edu; Fischl et al. 1999, 2001) using independent functional localizer data. In Experiment 1, early visual areas were separated into V1, V2, and V3 using standard retinotopic mapping procedures (Engel et al. 1994; Sereno et al. 1995). We included the voxels in the V1, V2, and V3 ROIs that showed greater activation during presentation of visual stimuli compared with a fixation baseline in the visual field localizer runs (false discovery rate [FDR], q < 10−4). In Experiment 1, to define IPS and FEF for 12 participants who underwent separate working memory localizer scans, we first selected the voxels that showed significantly greater activation during the working memory task compared with the fixation baseline period (q < 0.01) in frontal and parietal regions. The activated voxels were further constrained using anatomical labels from the Destrieux Atlas (Destrieux et al. 2010). IPS was defined as activated voxels within intraparietal sulcus (IPS) and the inferior part of superior parietal lobule (SPL), and FEF was defined as activated voxels within superior precentral sulcus (sPCS) and the caudal part of superior frontal gyrus (SFG). For the 3 participants who did not complete working memory localizer scans, the procedure of defining IPS and FEF was identical, except that all working memory runs in the main experiment were used (Fig. 2). In Experiment 2, we defined V1 and IPS using the similar method as in Experiment 1. Figure 2. View largeDownload slide Demonstration of functionally defined ROIs (V1, V2, V3, IPS, and FEF) of one representative participant, on the left hemisphere surface. Figure 2. View largeDownload slide Demonstration of functionally defined ROIs (V1, V2, V3, IPS, and FEF) of one representative participant, on the left hemisphere surface. For participants in Experiment 1, in addition to the ROIs defined within each participant, we also performed a group-level whole-brain analysis to localize other brain regions that showed greater activation during the working memory task compared with a fixation baseline (q < 0.05). These ROIs include an occipitotemporal ROI near the fusiform gyrus (FG) (FG, MNI coordinates: [−38.7, −65.1, −9.9] and [39.0, −55.5, −13.5]), and a prefrontal ROI near the middle frontal gyrus (LPFC, MNI coordinates: [−42.6, 23.5, 26.2] and [48.5, 9.6, 31.2]). We then aligned these ROIs back to each individual’s brain space. Inverted Encoding Model IEM analyses were conducted using in-house Python and Matlab scripts. We reconstructed population-level feature representations within each ROI using an encoding model of color or orientation (Brouwer and Heeger 2009, 2011; Ester et al. 2013, 2015; Yu and Shim 2017). The feature selectivity of each voxel was characterized as a weighted sum of 6 (for orientation) or 9 (for color) hypothesized channels. The idealized feature tuning curve of each channel is a half-wave-rectified and squared sinusoid raised to the sixth power (for orientation, FWHM = 0.94 in radians) or to the eighth power (for color, FWHM = 1.64 in radians). Before feeding the preprocessed data into the IEM, a baseline from each voxel’s response was removed in each run using the equation (Brouwer and Heeger 2011): B = B – m(mTB), in which B represented the data matrix from each run with size v × c (v is the number of voxels in the ROI; c is the number of orientations/colors) and m represented the mean response across all stimulus conditions of length v. A constant of 100 was added to B to avoid matrix inversion problems. We then computed the weight matrix (W) that projects the hypothesized channel responses (C1) to actual measured fMRI signals in the training dataset (B1), and extracted the estimated channel responses (⁠ Cˆ2 ⁠) for the test dataset (B2) using this weight matrix. The relationship between the training dataset (B1, v × n, n: the number of repeated measurements) and the channel responses (C1, k × n) was characterized by the following equation: B1=WC1 where W was the weight matrix (v × k). Therefore, the least-squared estimate of the weight matrix (⁠ W^ ⁠) was calculated using linear regression: W^=B1C1T(C1C1T)−1 The channel responses (⁠ C^2 ⁠) for the test dataset (B2) was then estimated using the weight matrix (⁠ W^ ⁠): C^2=(W^TW^)−1W^TB2 For the orientation delay-estimation task of Experiment 1, we defined the time period from 8 to 14 s following each trial onset as the retention period. The time series of each voxel in each ROI during this period was extracted, and the average signals across these time points were used as input for the IEM. Next, we used a leave-one-run-out procedure to build the weight matrix and to calculate the estimated channel outputs for each of 6 orientations in the test dataset. The weight matrix of the encoding model was calculated based on all trials in the training dataset and applied to the first and second gratings separately in the test dataset. The estimated channel outputs obtained after each iteration were shifted to a common center, where 0° represented the cued orientation channel and 90° represented the uncued orientation channel on each trial. The shifted channel outputs were then averaged across all iterations. In order to improve the orientation selectivity in frontal and parietal regions, we conducted an additional analysis in which we selected only the top 50% of voxels within IPS and FEF that could best discriminate the different orientations (Brouwer and Heeger 2011). Specifically, on each iteration, we conducted a one-way ANOVA on the 6 orientations tested in the working memory runs, for each voxel in the training dataset. All voxels were then sorted by the P-values of the ANOVA and the top 50% of voxels were selected. The estimated channel outputs in the test dataset were extracted from these voxels only. The same procedure was repeated for all iterations and the results were averaged. For the analysis at each individual time point of the delay interval, we used the same leave-one-run-out procedure to train and test the IEM, except that the test dataset was not averaged across time points. On each iteration, the weight matrix was applied to each time point (8, 10, 12, and 14 s) in the retention period separately. For the attention task of Experiment 1, we defined the first 3 TRs of each trial as the attention period (taking a 6-s hemodynamic delay into account) and used the average signals during this period as input for the IEM. To test the generalization of the neural codes from the attention task to the orientation delay-estimation task, we built the IEM using all runs from the attention task, and applied the resulting weight matrix to the orientation delay-estimation task. The methodological details of the IEM in Experiment 2 followed those in Experiment 1. Again, we used a leave-one-run-out procedure to build the weight matrix and to calculate the estimated channel outputs for each of 9 colors in the test dataset. The IEM analysis was performed for 10–16 s, 10−12 s (early delay), and 14–16 s (late delay) after trial onset. Permutation Significance Test Since the hypothesized channels in the IEM were not independent, we performed permutation tests to determine statistical significance. For each ROI, we randomly shuffled the trial labels for the entire dataset, generated corresponding model channel outputs, and repeated this procedure 10 000 times. Next, we calculated the statistics (e.g., memory index in Results) on each output and obtained a null distribution of 10 000 values. The statistics for a specific comparison were compared with this null distribution, and the probability of obtaining the original statistics given this distribution was reported as the P-value of the permutation test. Results Experiment 1 Behavioral Recency Effect in VWM Participants (N = 15) viewed 2 gratings presented in succession and were cued to remember the orientation of 1 grating in the 2-grating sequence (either “Remember First” or “Remember Second”) over a prolonged delay (Fig. 1A). A comparison of average recall error revealed significantly higher precision for trials in which the second grating was remembered (Remember Second; M = 10.0°, SD = 3.8°) than for trials in which the first grating was remembered (remember first; M = 11.6°, SD = 3.6°), t(14) = 3.129, P = 0.007. This superior performance on the most recent item coincides with the previously reported recency effect in visual memory (Phillips and Christie 1977; Broadbent and Broadbent 1981; Gorgoraptis et al. 2011; Zokaei et al. 2011). Reconstructing Orientation Representations of the First and Second Targets During Delay We built an IEM to extract population-level orientation representations from each ROI over the delay period (Brouwer and Heeger 2009, 2011; Ester et al. 2013, 2015; Yu and Shim 2017). This model assumes that the response of each voxel can be characterized by the weighted sum of hypothesized orientation channels. By computing the weights that relate the voxel responses to the orientation channel responses in the training dataset and applying these weights to the test dataset (a leave-one-run-out procedure), we obtained the estimated BOLD responses for each orientation channel in each ROI for the Remember First and Remember Second conditions separately. In V1, V2, and V3, we observed graded orientation responses that peaked at the remembered orientation in both the Remember First and Remember Second conditions, indicating that the orientation of the remembered item was retained in early retinotopic visual cortex regardless of its serial position. Similar orientation response profiles were also observed in IPS and FEF—but only in the Remember Second condition. In the Remember First condition, however, we observed reduced channel responses to the cued-to-be-remembered first orientation, as well as an increase in the channel response to the second, uncued orientation (Fig. 3A). Figure 3. View largeDownload slide Population-level orientation reconstructions over delay in the Remember First and Remember Second conditions in Experiment 1. (A) Orientation reconstructions in the Remember First and Remember Second conditions in V1, V2, V3, IPS, and FEF, estimated from data averaged across 8–14 s after trial onset. Responses were shifted to a common center where 0° represented the cued orientation and 90° represented the uncued orientation on each trial, and then averaged across all participants. Channel responses are estimated BOLD responses in relative amplitude. (B) Estimated channel responses to the orientation of the cued and uncued gratings compared with other orientations (average channel response to the remaining 4 orientations that were not presented on a given trial), for the Remember First and Remember Second conditions in V1, V2, V3, IPS, and FEF. (C) Recency Index in V1, V2, V3, IPS, and FEF. Recency Index was calculated as ([Cued Orientation Remember Second−Uncued Orientation Remember Second]/Other Orientations Remember Second)−([Cued Orientation Remember First−Uncued Orientation Remember First]/Other Orientations Remember First). (D) Generalization of orientation representations from visual attention to visual working memory tasks in V1, V2, V3, IPS, and FEF. Error bars indicate ± 1 SEM. Figure 3. View largeDownload slide Population-level orientation reconstructions over delay in the Remember First and Remember Second conditions in Experiment 1. (A) Orientation reconstructions in the Remember First and Remember Second conditions in V1, V2, V3, IPS, and FEF, estimated from data averaged across 8–14 s after trial onset. Responses were shifted to a common center where 0° represented the cued orientation and 90° represented the uncued orientation on each trial, and then averaged across all participants. Channel responses are estimated BOLD responses in relative amplitude. (B) Estimated channel responses to the orientation of the cued and uncued gratings compared with other orientations (average channel response to the remaining 4 orientations that were not presented on a given trial), for the Remember First and Remember Second conditions in V1, V2, V3, IPS, and FEF. (C) Recency Index in V1, V2, V3, IPS, and FEF. Recency Index was calculated as ([Cued Orientation Remember Second−Uncued Orientation Remember Second]/Other Orientations Remember Second)−([Cued Orientation Remember First−Uncued Orientation Remember First]/Other Orientations Remember First). (D) Generalization of orientation representations from visual attention to visual working memory tasks in V1, V2, V3, IPS, and FEF. Error bars indicate ± 1 SEM. To further examine the strength of representations of the cued and uncued orientations as a function of the serial position of the target, we focused on the channel responses to the cued and uncued orientation, comparing them to the average of the remaining 4 orientations that were not presented on a given trial (which we defined as “baseline”). We employed a “Recency Index”: the difference between the channel responses to the cued and uncued orientations was divided by the baseline channel response to the remaining 4 orientations that were not presented on a given trial. This normalized difference score was computed for the Remember First and Remember Second conditions separately, and the difference between the scores in 2 target serial position conditions was defined as the Recency Index. A higher Recency Index reflects a larger difference between the 2 target serial position conditions in terms of differences in their channel responses to the cued and uncued gratings. A significant Recency Index was revealed in IPS and FEF (both P = 0.005), but not in V1–V3 (all P > 0.134), indicating that differential representations depending on the serial position of the target are only found in parietal and frontal regions (Fig. 3C). All P-values were FDR corrected, in this and all subsequent analyses. Moreover, pairwise comparisons showed that the orientation channel responses to the cued grating were significantly higher than those to the uncued grating (all P < 0.009) and those to the baseline (all P < 0.003), in all ROIs, when the second grating was remembered. In contrast, when the first grating was remembered, while a similar pattern was found in V1–V3 (all P < 0.002), no significant difference between the cued and uncued gratings was shown in IPS and FEF (both P > 0.577). Channel responses to the uncued second grating were significantly higher than baseline in IPS (P = 0.023) but not in FEF (P = 0.170), while channel responses to the cued first grating were marginally significantly different from the baseline in both IPS and FEF (P = 0.084 and 0.085, respectively) (Fig. 3B). These results suggest that the recency effect in working memory may arise from 2 possible reasons. First, the overall quality of mnemonic representations in the Remember First condition was reduced in IPS and FEF compared with those in the Remember Second condition. Second, in IPS, obligatory maintenance of the most recent item occurred even when this item was no longer task-relevant. Furthermore, the difference between early visual and high-level frontal and parietal areas is not likely due to the weaker orientation responses in higher-order cortical areas, since a similar effect was observed when we used only the top 50% of orientation-selective voxels in parietal and frontal ROIs (both P < 0.007). To investigate whether the same pattern was also preserved in other high-level cortical areas, we examined 2 other ROIs that were significantly activated during the working memory task in the group-level whole-brain analysis: a region in the occipitotemporal cortex near the FG and a prefrontal region near the middle frontal gyrus (LPFC). As in IPS and FEF, the Recency Index was significant in LPFC (P = 0.004), although the responses in target channels were degraded (significant response in only the target channel in the Remember Second condition, P = 0.031). On the other hand, the Recency index was not significant in FG (P = 0.289). These results suggest that the neural representations held in occipitotemporal cortex during working memory are qualitatively similar to the representations in early retinotopic visual cortex, but different from representations maintained in higher-order frontal and parietal regions, including IPS, FEF, and LPFC. To examine whether the neural recency effect in parietal and frontal cortex was specific to the functional contrast we used for defining ROIs, we repeated the analyses on 3 anatomically defined ROIs (including anatomical IPS, SPL, and sPCS from the Destrieux atlas). The Recency Index was still significant in all 3 ROIs (P = 0.004, 0.004, and < 0.00001), suggesting that the neural recency effect was not biased by our voxel selection criteria. Time-Resolved Analyses on Neural Representations of the First and Second Gratings During a Memory Delay Next, we extracted orientation responses separately at each time point during the delay to examine how the strength of the neural representation of the first and second gratings evolved over time during retention, depending on whether it was cued to be remembered or not. Again, we observed different patterns in parietal and frontal regions, as compared with early visual cortex. When the first grating was remembered, the difference between channel responses to the cued first and uncued second gratings was not significant in IPS or FEF over time (all P > 0.431). Channel responses to the task-irrelevant, uncued second grating were comparable to the responses to the remembered first grating, and these heightened responses to the second grating were robustly maintained throughout the entire delay period. However, when the second grating was remembered, a significant difference between the cued second and uncued first gratings was revealed in IPS and FEF. Specifically, the difference between the cued and uncued gratings started to emerge around 10 s after trial onset in IPS, (from 8 to 14 s after trial onset: P = 0.348, 0.016, 0.023, and 0.002) in the Remember Second condition, indicating that the response to the uncued first grating was reduced over time. No such patterns were observed in other ROIs. On the contrary, the difference between the cued and uncued gratings was always significant in V1–V3, regardless of the serial position of the target grating (all P < 0.039), confirming selective maintenance of the cued item only in the retinotopic visual cortex (Fig. 4). Figure 4. View largeDownload slide Orientation responses to the cued and uncued gratings at each time point during delay in Experiment 1 orientation channel responses to the cued and uncued gratings at each time point during the delay period (8–14 s after trial onset), for the Remember First and Remember Second conditions separately in V1, V2, V3, IPS, and FEF. Error bars indicate ± 1 SEM. Figure 4. View largeDownload slide Orientation responses to the cued and uncued gratings at each time point during delay in Experiment 1 orientation channel responses to the cued and uncued gratings at each time point during the delay period (8–14 s after trial onset), for the Remember First and Remember Second conditions separately in V1, V2, V3, IPS, and FEF. Error bars indicate ± 1 SEM. Generalization From the Attention Task to the VWM Task Lastly, we examined whether the neural recency effect was preserved when we used other tasks (e.g., a visual attention task) to build the IEM, and whether the neural codes underlying working memory were shared across different tasks. We first examined the robustness of orientation responses in the attention task where participants performed an orientation discrimination task on the same stimuli (Fig. 1B) by comparing the response in the target channel to the average response in all other channels. Our results revealed robust orientation representations in all ROIs we examined, all P < 0.007. Next, we used the data from the attention task to train the IEM and applied the resulting weight matrix to the Remember First and Remember Second conditions separately in the working memory task. The results showed that responses to the target orientation in early visual cortex were still robust (all P < 0.00001); moreover, there was no significant difference between Remember First and Remember Second conditions (all P > 0.760), consistent with previous results. However, responses to the target orientation in IPS and FEF were degraded notably (both P > 0.156). No significant difference was found between the orientation responses for the Remember First and Remember Second conditions (both P > 0.957) (Fig. 3D). These results suggest that neural codes in early visual cortex could be generalized between the attention and VWM tasks. That is, early visual cortex shares similar feature-specific representations when the stimulus is attended and the stimulus is held in memory, as suggested in previous studies (Harrison and Tong 2009; Albers et al. 2013). On the contrary, parietal and frontal cortices retain distinct neural representations depending on the type of information required in each task, with reduced generalizability between the tasks. Experiment 2 In Experiment 1, we showed that feature representations during VWM maintenance in parietal and frontal cortices are different from those in early visual cortex when representing mnemonic recency. However, there are several accounts that may alternatively explain this finding (Colby and Goldberg 1999; Schluppeck et al. 2005; Silver et al. 2005; Swisher et al. 2007; Silver and Kastner 2009; Jerde et al. 2012). For example, one may argue that this effect was caused by spatial attention, since participants may spread their spatial attention along the oriented lines of the most recent grating, resulting in enhanced representation of the most recent item in parietal and frontal cortices. A similar account is that this representation reflected subtle eye movements, or the preparation of motor responses toward the position of the target orientation on the probe wheel during memory maintenance. Other alternative accounts are that the effect was caused by the residual stimulus-driven activity from the most recent sample orientation or that, since the orientations were always presented in orthogonal pairs in Experiment 1, participants could infer the first orientation by remembering only the second orientation, resulting in better neural representation of the second orientation. In order to address these issues, we conducted Experiment 2 using a color working memory task. First, participants were asked to remember the color of concentric rings instead of the orientation of the gratings. The use of color stimuli can exclude the spatial attention account because color rings do not contain spatial structures on which spatial attention can be distributed differentially. Second, by using 2 different nontarget colors for each target color, the first color cannot be inferred from remembering only the second color. Third, in order to prevent potentially stronger residual stimulus-driven activities from the second stimulus in a sequence, a color mask was presented following the presentation of memory samples. Lastly, a change detection task was used instead of the delay-estimation task. Since a color or orientation wheel was not presented in the change detection task, there should be less confounds from execution or preparation of eye movements toward a target location on the probe wheel. If the stronger representation of the second item in parietal and frontal cortices found in Experiment 1 was due to preferential encoding of the most recent item, we should observe a similar effect with colors controlling for these confounding factors. Behavioral VWM Performance Since we used a change detection task of colors in Experiment 2 (Fig. 1C) we obtained the average accuracies for the first and second colors in the memory sequence instead of recall errors. As opposed to Experiment 1, no significant difference was observed between the memory accuracy for the first color (M = 64.4%, SD = 4.5%) and for that of the second color (M = 61.7%, SD = 6.4%), t(9) = 1.337, P = 0.214, perhaps due to the participants’ poor performance on the change detection task (average accuracy = 62.6%). We then recruited a new group of participants (N = 12) to perform the same change detection task (except that the memory delay was 3 s instead of 9.3 s) outside the scanner. This time we witnessed an increase in memory performance (average accuracy = 66.0%) and also significantly higher memory accuracy for the second color (M = 69.1%, SD = 9.3%) as compared with the first color (M = 62.8%, SD = 9.2%), t(11) = 4.879, P = 4.8 × 10−4. Reconstructing Color Representations of the First and Second Targets During Delay Following Experiment 1, we built an IEM to extract population-level color representations from each ROI over the delay period, for the Remember First and Remember Second conditions separately. The functional localizer in Experiment 2 failed to reveal significantly activated clusters in FEF of at least one hemisphere in many of our participants (8 out of 10 participants), probably due to the difference in cortical involvement in memory maintenance of different visual features (Yu and Shim 2017). Therefore, we focused our analysis on the comparison between V1 and IPS. We first examined the color representations during the delay period (10−16 s), in early visual cortex (V1), as well as in parietal cortex (IPS). Specifically, we defined the responses in the cued channel as the response to the cued color, and the responses in the uncued channel as the response to the uncued color. As an uncued color on each trial was separated by ±160° from the cued color, we flipped the channel responses of the −160° trials so that +160° represented the uncued channel in the subsequent analysis. Unlike Experiment 1, we failed to replicate the neural recency effect in IPS. There was no significant difference between the color responses for the Remember First and Remember Second conditions (both P > 0.246). One possibility is that the introduction of the mask following the memory samples might have changed the temporal dynamics of the memory representation. Our time point-by-time point analysis in Experiment 1 also suggested that the recency effect in IPS occurred at later time points during delay. Therefore, we separated the delay period into early delay (10−12 s) and late delay (14−16 s) periods and extracted color representations from the 2 delays separately. Again, we computed the recency index in each ROI we examined. During the early delay, as expected, the recency index was not significant in either V1 or IPS (both P > 0.953). Interestingly, however, we observed a trend of neural “primacy effect” in V1, which was indicated by a marginally negative recency index (P = 0.094). On the contrary, during the late delay—consistent with findings in Experiment 1—the recency index was significant in IPS (P = 0.048) but not in V1 (P = 0.662). In both target position conditions, target channel responses were significantly higher than baseline (P = 0.016 and 0.019) and nontarget channel responses (P = 0.016 and 0.028) in V1. However, in IPS the robustness of the target channel responses were weaker and were only significantly higher than baseline (P = 0.019) and marginally higher than the nontarget channel responses (P = 0.072) in the Remember Second condition but not in the Remember First condition (both P > 0.372) (Fig. 5). These results suggest an intriguing possibility that parietal and visual cortex are influenced differentially by a visual mask during working memory: in IPS, responses to the most recent memory item are initially masked by a task-irrelevant distractor, but are gradually recovered during the delay period; while in early visual cortex, a backward masking effect might have erased the responses to the most recent item, leaving the representation of the first item relatively intact. Figure 5. View largeDownload slide Population-level color reconstructions over delay in the Remember First and Remember Second conditions in Experiment 2. (A) Color reconstructions in the Remember First and Remember Second conditions in V1 and IPS, estimated from data averaged across 14−16 s after trial onset. Responses were shifted to a common center where 0° represented the cued color on each trial and was averaged across all participants. Channel responses are estimated BOLD responses in relative amplitude. (B) Estimated channel responses to the cued and uncued colors compared with other colors (average channel response to the remaining 6 colors), for the Remember First and Remember Second conditions in V1 and IPS. Error bars indicate ± 1 SEM. Figure 5. View largeDownload slide Population-level color reconstructions over delay in the Remember First and Remember Second conditions in Experiment 2. (A) Color reconstructions in the Remember First and Remember Second conditions in V1 and IPS, estimated from data averaged across 14−16 s after trial onset. Responses were shifted to a common center where 0° represented the cued color on each trial and was averaged across all participants. Channel responses are estimated BOLD responses in relative amplitude. (B) Estimated channel responses to the cued and uncued colors compared with other colors (average channel response to the remaining 6 colors), for the Remember First and Remember Second conditions in V1 and IPS. Error bars indicate ± 1 SEM. Because functional activation was lacking in FEF, we performed an additional analysis, in which we calculated the Recency Index in an anatomical ROI (sPCS from the Destrieux atlas) corresponding to FEF. We found that the Recency Index still approached significance (P = 0.056). This result suggested that the recency effect also applied to FEF in Experiment 2, despite a lack of elevated activity in this region. Discussion In Experiments 1 and 2, we examined population-level feature representations of sequentially presented items during a delay in occipital, parietal, and frontal cortices using IEMs. In Experiment 1, participants were shown 2 orientations in a sequence and were required to remember one of them. We showed that there were substantial differences between the feature representations in parietal and frontal cortices and those in early visual cortex. Whereas the task-relevant item was selectively maintained in early visual cortex regardless of its serial position, consistent with previous findings (Harrison and Tong 2009; Serences et al. 2009); parietal and frontal cortices preferentially maintained the second item in the 2-item sequence. These results suggest that the mnemonic representations in frontoparietal regions are strongly modulated by temporal-order-based attentional priority. We further demonstrated that this neural recency privilege could not be simply explained by stronger residual stimulus-driven activity to the more recent item, since in Experiment 2, a task-irrelevant color mask delayed, but did not erase, the emergence of the recency privilege in IPS. This instead suggests that the privileged representation was closely related to the prioritized attentional state caused by temporal recency. Topographic maps of visual spatial attention have been well established in parietal and frontal cortices (Schluppeck et al. 2005; Silver et al. 2005; Swisher et al. 2007; Silver and Kastner 2009; Jerde et al. 2012). Several alternative explanations for Experiment 1 could thus be based on these attention-driven topographic maps such as spatial attention or subtle eye movements along the oriented lines for the most recent grating, or preparative motor signals towards the target position on the probe orientation wheel. We excluded these possibilities by conducting a second experiment that involved color memory. Because color has no spatial structure, similar results observed in Experiment 2 cannot be explained by visuotopic maps in these regions. Although our results showed enhanced representation for the most recent item in the parietal and frontal cortices, these results cannot be simply accounted for by exclusive responses to the second item at the time of encoding for several reasons. First, in Experiment 1, the channel responses in IPS to the cued item were comparable to those to the uncued item and marginally higher than baseline when the first item was remembered. This result suggests the coexistence of mnemonic representation of the to-be-remembered item, and representation of the task-irrelevant but temporally prioritized item. Second, in Experiment 2, the responses to both the first and second items were decreased and remained indistinguishable from baseline in IPS when the first item was remembered, whereas robust representation was still observed when the second item was remembered. Thus, exclusive responses to the most recent item cannot explain the lack of response to the second item in the Remember First condition. It is instead more likely that the privileged mnemonic signals to the most recent item were reduced by the presence of the mask. The disappearance of the recency privilege during the early stage of the delay period, and rebound of representations for the second color during the later stage of the delay period in IPS suggest that it may take extra time for the recency privilege to recover when an attention capturing task-irrelevant mask was presented after the most recent memory item. In previous studies on the feature-specific representations in parietal and frontal cortices during VWM maintenance, mixed results were found when memory stimuli were presented in a sequence. Some were able to decode mnemonic contents from parietal (Christophel et al. 2012; Albers et al. 2013; Bettencourt and Xu 2016) and frontal (Albers et al. 2013; Lee et al. 2013) cortices, while others failed (Riggall and Postle 2012; Emrich et al. 2013). The results in the current study may help to reconcile these seemingly conflicting results. One possibility, as indicated in our study, is that each item in a sequence is represented at different strengths at the neural level during VWM maintenance. Previous studies using sequential presentations did not take temporal order into account and this may have complicated their results. It is also possible that the encoding method may be more sensitive at detecting fine-grained feature differences, especially in the higher-order parietal and frontal cortices with coarse resolution of fMRI signals (Riley and Constantinidis 2015). Although fMRI studies have demonstrated that the contents of VWM can be maintained in parietal and frontal regions (Christophel et al. 2012; Albers et al. 2013; Lee et al. 2013; Ester et al. 2015; Bettencourt and Xu 2016; Yu and Shim 2017), it remains unclear whether the neural codes in these higher-order cortical areas are comparable to feature-selective responses in early visual cortex. Some studies propose that representations maintained in parietal and frontal areas during delays are different from those in early visual cortex, since these representations cannot be generalized to response patterns evoked in a perceptual task by using decoding methods (Albers et al. 2013). Consistent with the previous study, our findings demonstrated a lack of generalizability of the population-level, feature reconstructions across tasks in parietal and frontal cortices. It suggests that parietal and frontal cortices have more flexible neural codes that may be reorganized depending on the context of each specific task (Sarma et al. 2016). Preferential encoding of the most recent item and the lack of generalizability of the neural codes in parietal and frontal cortices during VWM suggest the multidimensionality of the feature-selective responses in frontoparietal regions. This result is consistent with neurophysiological findings, which showed that the key feature of neuronal selectivity in parietal and frontal cortices is mixed selectivity and high dimensionality. The same neuron can be selective for multiple aspects of the task and the selectivity of the neurons cannot be simply explained by linear sums of the responses to each aspect of the task (Rigotti et al. 2013; Raposo et al. 2014). There has also been evidence showing that prefrontal neurons can encode both task-relevant and task-irrelevant information simultaneously (Lauwereyns et al. 2001; Donahue and Lee 2015). Given that higher-order cortical neurons are less topographically organized (Riley and Constantinidis 2015), it is likely that the complexity of population-level neural codes in parietal and frontal cortices observed in the current study may reflect a mixture of responses from different populations of neurons with distinct selectivity patterns in each voxel. Regarding the roles of early visual cortex: on one hand, our study demonstrated that early visual cortex maintains robust representations of the remembered information that closely follows task demand and that it may use a common neural code shared by different visual tasks, ranging from perception to memory, which confirms the sensory recruitment hypothesis proposed in previous studies (Awh and Jonides 2001; Pasternak and Greenlee 2005; Postle 2006; D’Esposito 2007; Harrison and Tong 2009; Serences et al. 2009; Riggall and Postle 2012; Emrich et al. 2013). On the other hand, consistent with a recent study (Bettencourt and Xu 2016), we showed that the representations in early visual cortex are susceptible to the presentation of a visual mask. However, the mnemonic representations in the retinotopic cortex can also gradually recover toward the end of delay in V1. A recent monkey neurophysiology study demonstrated the encoding of the most recent item only in the prefrontal neurons during VWM (Konecky et al. 2017), which is consistent with our current findings in LPFC. It suggests that the neural recency privilege can be observed at the single-unit level and that LPFC may act as the source of the “recency privilege” signals that impact representations in IPS and FEF. The coexistence of mnemonic and attentional signals in IPS and FEF may therefore be a consequence of the combination of downstream signals from LPFC and upstream signals from sensory cortex. One missing piece from the current work is to establish the link between the multiple roles of parietal and frontal cortices and behavior. In Experiment 1, we revealed a small but robust difference (about 2°) between memory performance for the first and second items, which is consistent with the neural recency privilege. However, in Experiment 2, the difference between conditions was no longer significant, although we could replicate the recency effect outside the scanner using a similar task. There exist substantial differences between the 2 experiments (e.g., delay estimation vs. change detection, task difficulty), and previous work suggest that the amplitude of the recency effect could be determined by multiple factors, such as memory load (Gorgoraptis et al. 2011; Kool et al. 2014) and the type of tasks (Oberauer 2003). The arbitrary angular difference (10°–15°) we used in the change detection task could also have had an influence on the detection of the behavioral recency effect. To summarize, these results supported our hypothesis that mnemonic signals in parietal and frontal cortices are modulated by attentional priority based on temporal order, whereas early visual cortex maintains robust mnemonic representations, although these representations may be susceptible to visual distractors for a brief period of time. This distinction suggests differential natures of the mnemonic representations in different brain areas such that VWM representations in higher-order frontoparietal regions are more likely to be influenced by bottom-up attentional priority signals than those in early visual cortex. This could arise from a close integration of mnemonic representations and attentional saliency map extended to the temporal domain in frontoparietal areas. It would be interesting for future research to examine how the neural codes in parietal and frontal cortices could be decomposed into different components that may relate to distinct processes in VWM. Funding Burke Award and IBS-R015-D1 to W.M.S. Authors’ Contributions Q.Y. and W.M.S. designed the experiment. Q.Y. conducted the experiment and analyzed the data. W.M.S. supervised the entire project. Q.Y. and W.M.S. wrote the article. Notes Conflict of Interest: None declared. References Albers AM , Kok P , Toni I , Dijkerman HC , de Lange FP . 2013 . Shared representations for working memory and mental imagery in early visual cortex . Curr Biol . 23 : 1427 – 1431 . Google Scholar Crossref Search ADS PubMed WorldCat Awh E , Jonides J . 2001 . Overlapping mechanisms of attention and spatial working memory . Trends Cogn Sci . 5 : 119 – 126 . Google Scholar Crossref Search ADS PubMed WorldCat Bettencourt KC , Xu Y . 2016 . Decoding the content of visual short-term memory under distraction in occipital and parietal areas . Nat Neurosci . 19 : 150 – 157 . Google Scholar Crossref Search ADS PubMed WorldCat Bisley JW , Goldberg ME . 2010 . Attention, intention, and priority in the parietal lobe . Annu Rev Neurosci . 33 : 1 – 21 . Google Scholar Crossref Search ADS PubMed WorldCat Brainard DH . 1997 . The Psychophysics Toolbox . Spat Vis . 10 : 433 – 436 . Google Scholar Crossref Search ADS PubMed WorldCat Broadbent DE , Broadbent MHP . 1981 . Recency effects in visual memory . Q J Exp Psychol A . 33 : 1 – 15 . Google Scholar Crossref Search ADS WorldCat Brouwer GJ , Heeger DJ . 2009 . Decoding and reconstructing color from responses in human visual cortex . J Neurosci . 29 : 13992 – 14003 . Google Scholar Crossref Search ADS PubMed WorldCat Brouwer GJ , Heeger DJ . 2011 . Cross-orientation suppression in human visual cortex . J Neurophysiol . 106 : 2108 – 2119 . Google Scholar Crossref Search ADS PubMed WorldCat Carpenter AF , Baud-Bovy G , Georgopoulos AP , Pellizzer G . 2018 . Encoding of serial order in working memory: neuronal activity in motor, premotor, and prefrontal cortex during a memory scanning task . J Neurosci . 38 : 4912 – 4933 . Google Scholar Crossref Search ADS PubMed WorldCat Cavanagh P , Tyler CW , Favreau OE . 1984 . Perceived velocity of moving chromatic gratings . J Opt Soc Am A . 1 : 893 – 899 . Google Scholar Crossref Search ADS PubMed WorldCat Christophel TB , Hebart MN , Haynes JD . 2012 . Decoding the contents of visual short-term memory from human visual and parietal cortex . J Neurosci . 32 : 12983 – 12989 . Google Scholar Crossref Search ADS PubMed WorldCat Colby CL , Goldberg ME . 1999 . Space and attention in parietal cortex . Annu Rev Neurosci . 22 : 319 – 349 . Google Scholar Crossref Search ADS PubMed WorldCat Cox RW . 1996 . AFNI: software for analysis and visualization of functional magnetic resonance neuroimages . Comput Biomed Res . 29 : 162 – 173 . Google Scholar Crossref Search ADS PubMed WorldCat Destrieux C , Fischl B , Dale A , Halgren E . 2010 . Automatic parcellation of human cortical gyri and sulci using standard anatomical nomenclature . Neuroimage . 53 : 1 – 15 . Google Scholar Crossref Search ADS PubMed WorldCat Donahue CH , Lee D . 2015 . Dynamic routing of task-relevant signals for decision making in dorsolateral prefrontal cortex . Nat Neurosci . 18 : 295 – 301 . Google Scholar Crossref Search ADS PubMed WorldCat D’Esposito M . 2007 . From cognitive to neural models of working memory . Philos Trans R Soc Lond B Biol Sci . 362 : 761 – 772 . Google Scholar Crossref Search ADS PubMed WorldCat D’Esposito M , Postle BR , Rypma B . 2000 . Prefrontal cortical contributions to working memory: evidence from event-related fMRI studies . Exp Brain Res . 133 : 3 – 11 . Google Scholar Crossref Search ADS PubMed WorldCat Emrich SM , Riggall AC , Larocque JJ , Postle BR . 2013 . Distributed patterns of activity in sensory cortex reflect the precision of multiple items maintained in visual short-term memory . J Neurosci . 33 : 6516 – 6523 . Google Scholar Crossref Search ADS PubMed WorldCat Engel SA , Rumelhart DE , Wandell BA , Lee AT , Glover GH , Chichilnisky EJ , Shadlen MN . 1994 . fMRI of human visual cortex . Nature . 369 : 525 . Google Scholar Crossref Search ADS PubMed WorldCat Ester EF , Anderson DE , Serences JT , Awh E . 2013 . A neural measure of precision in visual working memory . J Cogn Neurosci . 25 : 754 – 761 . Google Scholar Crossref Search ADS PubMed WorldCat Ester EF , Sprague TC , Serences JT . 2015 . Parietal and frontal cortex encode stimulus-specific mnemonic representations during visual working memory . Neuron . 87 : 893 – 905 . Google Scholar Crossref Search ADS PubMed WorldCat Fecteau JH , Munoz DP . 2006 . Salience, relevance, and firing: a priority map for target selection . Trends Cogn Sci . 10 : 382 – 390 . Google Scholar Crossref Search ADS PubMed WorldCat Fischl B , Liu A , Dale AM . 2001 . Automated manifold surgery: constructing geometrically accurate and topologically correct models of the human cerebral cortex . IEEE Trans Med Imaging . 20 : 70 – 80 . Google Scholar Crossref Search ADS PubMed WorldCat Fischl B , Sereno MI , Dale AM . 1999 . Cortical surface-based analysis. II: Inflation, flattening, and a surface-based coordinate system . Neuroimage . 9 : 195 – 207 . Google Scholar Crossref Search ADS PubMed WorldCat Fuster JM , Alexander GE . 1971 . Neuron activity related to short-term memory . Science . 173 : 652 – 654 . Google Scholar Crossref Search ADS PubMed WorldCat Gorgoraptis N , Catalao RF , Bays PM , Husain M . 2011 . Dynamic updating of working memory resources for visual objects . J Neurosci . 31 : 8502 – 8511 . Google Scholar Crossref Search ADS PubMed WorldCat Gosseries O , Yu Q , LaRocque JJ , Starrett MJ , Rose NS , Cowan N , Postle BR . 2018 . Parietal-occipital interactions underlying control- and representation-related processes in working memory for nonspatial visual features . J Neurosci . 38 : 4357 – 4366 . Google Scholar Crossref Search ADS PubMed WorldCat Harrison SA , Tong F . 2009 . Decoding reveals the contents of visual working memory in early visual areas . Nature . 458 : 632 – 635 . Google Scholar Crossref Search ADS PubMed WorldCat Jerde TA , Merriam EP , Riggall AC , Hedges JH , Curtis CE . 2012 . Prioritized maps of space in human frontoparietal cortex . J Neurosci . 32 : 17382 – 17390 . Google Scholar Crossref Search ADS PubMed WorldCat Konecky RO , Smith MA , Olson CR . 2017 . Monkey prefrontal neurons during Sternberg task performance: full contents of working memory or most recent item? J Neurophysiol . 117 : 2269 – 2281 . Google Scholar Crossref Search ADS PubMed WorldCat Kool W , Conway AR , Turk-Browne NB . 2014 . Sequential dynamics in visual short-term memory . Atten Percept Psychophys . 76 : 1885 – 1901 . Google Scholar Crossref Search ADS PubMed WorldCat Lauwereyns J , Sakagami M , Tsutsui K , Kobayashi S , Koizumi M , Hikosaka O . 2001 . Responses to task-irrelevant visual features by primate prefrontal neurons . J Neurophysiol . 86 : 2001 – 2010 . Google Scholar Crossref Search ADS PubMed WorldCat Lee SH , Kravitz DJ , Baker CI . 2013 . Goal-dependent dissociation of visual and prefrontal cortices during working memory . Nat Neurosci . 16 : 997 – 999 . Google Scholar Crossref Search ADS PubMed WorldCat Marshuetz C , Smith EE , Jonides J , DeGutis J , Chenevert TL . 2000 . Order information in working memory: fMRI evidence for parietal and prefrontal mechanisms . J Cogn Neurosci . 12 : 130 – 144 . Google Scholar Crossref Search ADS PubMed WorldCat Mazziotta J , Toga A , Evans A , Fox P , Lancaster J , Zilles K , Woods R , Paus T , Simpson G , Pike B , et al. . 2001 . A probabilistic atlas and reference system for the human brain: International Consortium for Brain Mapping (ICBM) . Philos Trans R Soc Lond B Biol Sci . 356 : 1293 – 1322 . Google Scholar Crossref Search ADS PubMed WorldCat McElree B , Dosher BA . 1989 . Serial position and set size in short-term memory: the time course of recognition . J Exp Psychol: Gen . 118 : 346 – 373 . Google Scholar Crossref Search ADS WorldCat Miller EK , Erickson CA , Desimone R . 1996 . Neural mechanisms of visual working memory in prefrontal cortex of the macaque . J Neurosci . 16 : 5154 – 5167 . Google Scholar Crossref Search ADS PubMed WorldCat Nee DE , Jonides J . 2008 . Neural correlates of access to short-term memory . Proc Natl Acad Sci USA . 105 : 14228 – 14233 . Google Scholar Crossref Search ADS PubMed WorldCat Oberauer K . 2003 . Selective attention to elements in working memory . Exp Psychol . 50 : 257 – 269 . Google Scholar Crossref Search ADS PubMed WorldCat Pasternak T , Greenlee MW . 2005 . Working memory in primate sensory systems . Nat Rev Neurosci . 6 : 97 – 107 . Google Scholar Crossref Search ADS PubMed WorldCat Pelli DG . 1997 . The VideoToolbox software for visual psychophysics: transforming numbers into movies . Spat Vis . 10 : 437 – 442 . Google Scholar Crossref Search ADS PubMed WorldCat Phillips WA , Christie DFM . 1977 . Components of visual memory . Q J Exp Psychol . 29 : 117 – 133 . Google Scholar Crossref Search ADS WorldCat Postle BR . 2006 . Working memory as an emergent property of the mind and brain . Neuroscience . 139 : 23 – 38 . Google Scholar Crossref Search ADS PubMed WorldCat Raposo D , Kaufman MT , Churchland AK . 2014 . A category-free neural population supports evolving demands during decision-making . Nat Neurosci . 17 : 1784 – 1792 . Google Scholar Crossref Search ADS PubMed WorldCat Riggall AC , Postle BR . 2012 . The relationship between working memory storage and elevated activity as measured with functional magnetic resonance imaging . J Neurosci . 32 : 12990 – 12998 . Google Scholar Crossref Search ADS PubMed WorldCat Rigotti M , Barak O , Warden MR , Wang XJ , Daw ND , Miller EK , Fusi S . 2013 . The importance of mixed selectivity in complex cognitive tasks . Nature . 497 : 585 – 590 . Google Scholar Crossref Search ADS PubMed WorldCat Riley MR , Constantinidis C . 2015 . Role of prefrontal persistent activity in working memory . Front Syst Neurosci . 9 : 181 . Google Scholar PubMed WorldCat Sarma A , Masse NY , Wang XJ , Freedman DJ . 2016 . Task-specific versus generalized mnemonic representations in parietal and prefrontal cortices . Nat Neurosci . 19 : 143 – 149 . Google Scholar Crossref Search ADS PubMed WorldCat Schluppeck D , Glimcher P , Heeger DJ . 2005 . Topographic organization for delayed saccades in human posterior parietal cortex . J Neurophysiol . 94 : 1372 – 1384 . Google Scholar Crossref Search ADS PubMed WorldCat Serences JT , Ester EF , Vogel EK , Awh E . 2009 . Stimulus-specific delay activity in human primary visual cortex . Psychol Sci . 20 : 207 – 214 . Google Scholar Crossref Search ADS PubMed WorldCat Serences JT , Yantis S . 2006 . Selective visual attention and perceptual coherence . Trends Cogn Sci . 10 : 38 – 45 . Google Scholar Crossref Search ADS PubMed WorldCat Sereno MI , Dale AM , Reppas JB , Kwong KK , Belliveau JW , Brady TJ , Rosen BR , Tootell RB . 1995 . Borders of multiple visual areas in humans revealed by functional magnetic resonance imaging . Science . 268 : 889 – 893 . Google Scholar Crossref Search ADS PubMed WorldCat Silver MA , Kastner S . 2009 . Topographic maps in human frontal and parietal cortex . Trends Cogn Sci . 13 : 488 – 495 . Google Scholar Crossref Search ADS PubMed WorldCat Silver MA , Ress D , Heeger DJ . 2005 . Topographic maps of visual spatial attention in human parietal cortex . J Neurophysiol . 94 : 1358 – 1371 . Google Scholar Crossref Search ADS PubMed WorldCat Sreenivasan KK , Curtis CE , D’Esposito M . 2014 . Revisiting the role of persistent neural activity during working memory . Trends Cogn Sci . 18 : 82 – 89 . Google Scholar Crossref Search ADS PubMed WorldCat Srimal R , Curtis CE . 2008 . Persistent neural activity during the maintenance of spatial position in working memory . Neuroimage . 39 : 455 – 468 . Google Scholar Crossref Search ADS PubMed WorldCat Swisher JD , Halko MA , Merabet LB , McMains SA , Somers DC . 2007 . Visual topography of human intraparietal sulcus . J Neurosci . 27 : 5326 – 5337 . Google Scholar Crossref Search ADS PubMed WorldCat Todd JJ , Marois R . 2004 . Capacity limit of visual short-term memory in human posterior parietal cortex . Nature . 428 : 751 – 754 . Google Scholar Crossref Search ADS PubMed WorldCat Ungerleider LG , Courtney SM , Haxby JV . 1998 . A neural system for human visual working memory . Proc Natl Acad Sci USA . 95 : 883 – 890 . Google Scholar Crossref Search ADS PubMed WorldCat Vergauwe E , Hardman KO , Rouder JN , Roemer E , McAllaster S , Cowan N . 2016 . Searching for serial refreshing in working memory: using response times to track the content of the focus of attention over time . Psychon Bull Rev . 23 : 1818 – 1824 . Google Scholar Crossref Search ADS PubMed WorldCat Wilken P , Ma WJ . 2004 . A detection theory account of change detection . J Vis . 4 : 1120 – 1135 . Google Scholar Crossref Search ADS PubMed WorldCat Yu Q , Shim WM . 2017 . Occipital, parietal, and frontal cortices selectively maintain task-relevant features of multi-feature objects in visual working memory . Neuroimage . 157 : 97 – 107 . Google Scholar Crossref Search ADS PubMed WorldCat Zanto TP , Rubens MT , Thangavel A , Gazzaley A . 2011 . Causal role of the prefrontal cortex in top-down modulation of visual processing and working memory . Nat Neurosci . 14 : 656 – 661 . Google Scholar Crossref Search ADS PubMed WorldCat Zokaei N , Gorgoraptis N , Bahrami B , Bays PM , Husain M . 2011 . Precision of working memory for visual motion sequences and transparent motion surfaces . J Vis . 11 : 11 – 18 . Google Scholar Crossref Search ADS WorldCat © The Author(s) 2018. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)
TI - Temporal-Order-Based Attentional Priority Modulates Mnemonic Representations in Parietal and Frontal Cortices
JF - Cerebral Cortex
DO - 10.1093/cercor/bhy184
DA - 2019-07-05
UR - https://www.deepdyve.com/lp/oxford-university-press/temporal-order-based-attentional-priority-modulates-mnemonic-TVdhmBMpGO
SP - 3182
VL - 29
IS - 7
DP - DeepDyve
ER -