Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 7-Day Trial for You or Your Team.

Learn More →

Task and task‐free FMRI reproducibility comparison for motor network identification

Task and task‐free FMRI reproducibility comparison for motor network identification INTRODUCTION Functional magnetic resonance imaging (fMRI) is increasingly used to localize specific brain functions in the individual patient. Localization of the primary motor cortex (M1) is a frequent clinical issue in neurosurgery [Fernandez et al., ] and neurology [Johansen‐Berg et al., ], for which typically a fMRI study is performed while the patient is performing a task with alternating instructions (task fMRI). The M1 can be well localized with this approach [Bandettini et al., ; Kim et al., ; Ramsey et al., ; Rao et al., ; van Gelderen et al., ]. However, motor deficits can severely impair task performance of the patient and introduce false positive or false negative activation. This can have a profound influence on the reliability of the resulting brain activation maps and on their clinical interpretation [Jiang et al., ; Krings et al., ; Rutten and Ramsey, ]. A possible solution to this problem is to study patients in a resting state (without the subject performing a specific task in the scanner). At rest, brain regions exhibit spontaneous fluctuations at low frequencies (0.01–0.1 Hz) in the blood oxygen level‐dependent (BOLD) signal. When different brain areas show temporally correlated signals, as has for instance been shown with bilateral primary motor areas, these areas can be regarded as forming a functional network. Several functional networks that have been identified with resting‐state (task‐free) fMRI [e.g., Damoiseaux et al., ; van de Ven et al., ] have been shown to qualitatively overlap with task fMRI activation [e.g., Biswal et al., ; Greicius et al., ; Kokkonen et al., ; Smith et al., ]. The simplest analysis technique for identifying functional connectivity networks is to extract the BOLD time course from voxels in a region of interest (seed region) and determine its temporal correlation with the time course of all other brain voxels (also called univariate analysis). Previous studies have shown that task‐free fMRI can localize M1 functionality [Biswal et al., ; Cordes et al., ; Lowe et al., ; Xiong et al., ]. A recent study suggested that functional subregions of the motor network during rest are organized in a somatotopic fashion, in that they are linked one‐on‐one to their homologue in the contralateral hemisphere [van den Heuvel and Hulshoff Pol, ]. A high test‐retest reliability of fMRI maps is a pre‐requisite for their application in clinical practice. Studies on task fMRI reliability show mixed results for individual motor functional brain maps. Repeated scans of the same subject yields similar but not identical maps [McGonigle et al., ], with overlap ranging from 20 to 62% [Havel et al., ; Ramsey et al., ; Yetkin et al., ; Zandbelt et al., ]. The reproducibility of single‐subject fMRI is affected by multiple sources of variability, including attention and practice [Ungerleider et al., ; Veltman et al., ], physiology (cardiac and respiratory artifacts) [Birn et al., ; Biswal et al., ], and equipment (MRI field inhomogeneities and image signal‐to‐noise ratio) [Raemaekers et al., ]. To date, only a few studies have investigated the reliability of task‐free fMRI [Meindl et al., ; Shehzad et al., ; Zuo et al., ]. However, these studies did not focus on the reliability of the M1 localization. The main goal of our study is to investigate the test‐retest reliability of task‐free fMRI for M1 identification and compare it to the conventional task fMRI to assess its potential for use in clinical practice. Two measures commonly used to assess reliability of functional brain maps are the intraclass correlation (ICC) [Bartko and Carpenter, ; Shrout and Fleiss, ], and the ratio of overlapping activation [Ramsey et al., ; Rombouts et al., ]. ICC represents a general reliability measure based on between‐session correlations of Z ‐ (or T‐ ) values of brain statistic images, and is independent of the choice for a particular significance threshold. The ratio of overlapping activation is a more specific measure in that it looks at common voxels surviving a particular significance threshold in both scan sessions. From a clinical point of view, the overlap measure is more informative because the neurosurgeon or neurologist will interpret fMRI results based on conventional P or Z ‐thresholding (with Z ‐values usually ranging from 3.5 to 5) [Mattay et al., ; McGonigle et al., ; Miki et al., ; Ramsey et al., ; Rombouts et al., ; Rutten et al., ; Smith et al., ; Yetkin et al., ]. Overlap results based on conventional Z ‐thresholding are not appropriate for comparing different fMRI experimental designs, as in the case of task and task‐free fMRI. Different fMRI designs differ in sensitivity and, thereby, produce different Z ‐values. Setting a conventional Z ‐threshold (e.g., at a Z ‐value of 5), while having different maximum Z ‐values between the designs, will categorize a different number of voxels as active. This affects the ratio of overlap and makes direct comparison not appropriate. A straightforward alternative is then to set a threshold that is exceeded by a fixed number of voxels [Tegeler et al., ]. This approach is also justified by the fact that the goal is to determine where activity is located, and not whether there is activity (classical null hypothesis for scientific application). Here we quantify and compare between‐session reproducibility of motor task and task‐free fMRI activity. Sixteen healthy subjects participated in this study with a test‐retest interval of on average 7 weeks. We assess reproducibility in terms of ICC values and the ratio of overlap [Raemaekers et al., ]. We first estimate the ratio of overlapping activation for different Z ‐thresholds, and then for different thresholds exceeded by a fixed number of the most active voxels [Tegeler et al., ]. Finally, we look at the percentage of voxels located in the primary motor cortex for different thresholds. MATERIALS AND METHODS Participants Sixteen healthy subjects (8 females and 8 males, mean age 39.4 years, SD 10.7) participated in the experiment after giving informed consent. The research protocol and consent forms were approved by the medical ethics committee for research in humans (METC) of the University Medical Center Utrecht (UMC Utrecht, The Netherlands) in accordance with the Declaration of Helsinki (2008). Subjects were screened for medical, neurological, and psychiatric illnesses, use of medication, and metal implants. They had no previous experience with fMRI or with the tasks performed in the scanner, and all were strongly right handed ( M = 0.85; SD = 0.15) according to the Edinburgh Handedness inventory [Oldfield, ]. Participants were requested to abstain from nicotine and caffeine for 4 h prior to the scan sessions, and from over‐the‐counter medications for 24 h. The test and retest took place with an average interval of approximately 7 weeks (mean 50.7 days, SD 22.6). Experimental Design The fMRI experimental design was the same for both scan sessions and included, in order, a motor task, and a task‐free experiment. Participants were informed about the experimental procedure, and briefly practiced the motor task with the aid of a laptop before the scanning session. We used a PC, a rear projection screen, and a video projector system for stimulus presentation. Visual stimuli were projected in green and red on a dark background. For the motor task subjects repeatedly touched the thumb once with each of the digits backward and forward. Subjects were instructed to perform the task with the right hand when a flickering (2 Hz) green circle was presented on the screen, and to rest when a flickering red circle was presented. Subjects made movements at the rate of the flickering circle. The task was presented in a block‐design with eight cycles of 29.8 sec of movement, and 29.8 sec of rest. The experimental trial always began with a control period, in that both hands were relaxed. Blocks were time locked to the fMRI scans. During the task‐free fMRI experiment, the scanner room was darkened and participants were instructed to relax with their eyes closed, to think of nothing in particular, and not to fall asleep. As verified afterwards, no participant reported to have fallen asleep or to have been close to falling asleep. Data Acquisition All images were obtained with a whole body 3.0 Tesla (3T) Philips Achieva MRI scanner (Philips Medical Systems, Best, The Netherlands). The participant's head was held in place with padding. Heartbeat was recorded using a pulse‐oximeter placed on the left index finger. Respiration was measured with a pneumatic belt positioned at the level of the abdomen [Birn et al., ]. A T1‐weighted structural image of the whole brain in saggital orientation was acquired for anatomical reference (3D fast field echo (FFE) pulse sequence; acquisition parameters: repetition time (TR) 8.4 ms, excitation time (TE) 3.8 ms; field of view (FOV) 288 × 288 × 175 mm; voxel size 1 mm isotropic; SENSE p‐reduction/s‐reduction, 2/1.3; flip‐angle, 8°; 175 slices, scan duration = 265.8 sec). For functional scans, a 3D‐PRESTO (Principles of Echo Shifting with a Train of Observations) pulse sequence [van Gelderen et al., ] with parallel imaging [Golay et al., ; Neggers et al., ] was used. This method involves a combination of echo shifting [Duyn et al., ; Moonen et al., ] and multiple gradient echoes per radio frequency excitation [Liu et al., ]. It allows whole brain coverage with very short acquisition times by already applying the next excitation before signal readout. The very short acquisition times are ideal to reduce artifacts caused by motion during scans and physiological aliasing. This is of particular importance for the motor cortex since it has been shown to be more prone to physiological contaminations than the visual and auditory cortex [Cordes et al., ]. We used the following parameters: TR 22.5 msec; effective TE, 32.4 msec (using a shifted echo [Liu et al., ]); FOV 256 × 224 × 160 mm, voxel size 4 mm isotropic; matrix 64 × 56 × 40; SENSE p‐reduction/s‐reduction, 1.8/2; flip‐angle 10°; scan time per volume (scan duration) 0.6075 sec. Task and task‐free functional images encompassed the whole brain and were acquired in saggital orientation, with a foot–head frequency encoding direction. In total, 442 task and 400 task‐free fMRI scans were acquired, but only the first 400 task fMRI scans were included in the analysis to make an exact comparison with task‐free fMRI. Between the task and task‐free fMRI, two additional functional scans were acquired: a PRESTO scan of the same volume of brain tissue and with the same parameters was acquired in 0.72 sec, but with a higher flip‐angle (27°, FA27) for the image coregistration routine (see below in Data preprocessing section); and a diffusion tensor imaging scan not reported in this study which lasted 6:02 min. Data Preprocessing All functional images of both scan sessions were realigned (2nd degree b‐spline) and resliced (4th degree b‐spline) to the high contrast functional image (Fa27) of the first scan session using SPM5 ( http://www.fil.ion.ucl.ac.uk/spm/ ). We then corrected for cardiac and respiratory effects as this is shown to be beneficial for fMRI [Shmueli et al., ; Wise et al., ], especially when the motor cortex is investigated [Birn et al., ]. For correction of cardiac and respiratory artifacts, custom Matlab scripts were used (Aztec, http://www.ni‐utrecht.nl/downloads/aztec ). First, the task and task‐free BOLD signal was corrected for the effects of the phase of the cardiac and respiratory cycle using RETROICOR [Glover et al., ]. Subsequently, the task and task‐free BOLD signal were corrected for the effects of heart rate, heart rate variability, and respiration volume per time unit [Birn et al., ] using a multiple regression approach. The method used here is described in detail in the study by van Buuren et al. [ ]. Task‐free functional images were then lowpass filtered (0.08 Hz) with a finite impulse response filter, and highpass filtered in FSL, version 5.92 (Gaussian‐weighted least squares straight‐line fitting, with sigma = 50 sec) ( http://www.fmrib.ox.ac.uk/fsl/ ) [Smith et al., ]. The Task functional images were only highpass filtered (sigma = 29.8 sec). Task and task‐free functional images were skull stripped [Smith, ] and intensity normalized (grand mean scaling) in FSL. No spatial smoothing was performed on the functional images [Machielsen et al., ; Ramsey et al., ; Rombouts et al., ; Specht et al., ], as considered inappropriate for clinical decision making [e.g., Rutten et al., ]. For the task‐free analysis, (see Data analysis section) we did not regress out the global signal as in previous fMRI studies [Fox et al., ; Fransson, ; Greicius et al., ; Kelly et al., ]. Regressing out the global signal may remove true functional signal [Birn et al., ], may introduce spurious negative activations in task fMRI analysis [Aguirre et al., ], and anti‐correlations in task‐free fMRI analysis [Murphy et al., ]. Instead, we decided to separately regress out cerebrospinal fluid (CSF) and white‐matter (WM) signal. Regressing out the CSF signal is beneficial as low frequency components in CSF have previously been reported with similar frequencies to resting‐state networks [Friese et al., ].We regress out the WM signal as resting‐state networks are shown to be confined within the gray matter [Damoiseaux et al., ; De Luca et al., ]. To regress out the CSF and WM signal, masks of the CSF and WM were made using the segmentation of SPM5. The resulting WM and CSF masks were then registered to the Fa27 of the first scan session (the reference space of the functional data) and eroded to the maximum (threshold 0.99, minimum 0, maximum 1). Finally, the seed region mask was derived from the overlap between the 10 most active motor task fMRI voxels of scan Session 1 and of scan Session 2 [Cordes et al., ]. These voxels were restricted to (a) the anatomically defined handknob [Tegeler et al., ; Yousry et al., ], and (b) fall within the precentral gyrus. This allows using exactly the same seed voxels for both scan sessions. It also minimizes on the one hand the risk of selecting false positive seed voxels, and on the other hand the number of voxels to be excluded from the reliability analysis (see Analysis of reliability section). The precentral gyrus mask is derived from the Harvard‐Oxford Cortical Structural Atlas (in FSL). The number of voxels included in the seed region masks ranged from 2 to 10 (mean 5.1, SD 2.7). Data Analysis FMRI data processing was carried out using a whole‐brain univariate General Linear Model (FEAT in FSL). Time series statistical analysis was carried out using FILM, with pre‐whitening accounting for local autocorrelation [Woolrich et al., ]. The hemodynamic response function for the motor task functional images was modeled using a boxcar convolved with a double gamma variate function and its temporal derivative. To model motor task‐free functional connectivity, three custom explanatory variables (EVs) were chosen, sampled from the functional data itself: the mean time courses of the WM, CSF, and of the seed region. The seed region EV represents our factor of interest. Featquery (part of FSL) was used for extracting the timecourse of each EV. For visualization purposes, individual task and task‐free activation maps were generated using statistical correction ( P < 0.05) for multiple comparisons based on Gaussian Random Field (GRF) [Worsley, ] (Figs. and ), and at different thresholds exceeded by a fixed number of the most active voxels [Tegeler et al., ] (Figs. and , and in Supporting Information Figs. A and B). Motor task activation of Subjects 2 and 12 for scan Session 1 (in red) and scan Session 2 (in blue), rendered on subject's anatomy, in turn resliced in MNI space. Axial slices are shown in neurological orientation (left is left) with the corresponding coordinate on top of each slice. Z ‐values for activated voxels exceeding the threshold ( P < 0.05 corrected) ranged from 5.1 to 19.3 for scan Session 1, and from 5.1 to 18.7 for scan Session 2 for Subject 2 . Z ‐values ranged from 5.1 to 16.3 for scan Session 1, and from 5.1 to 20.5 for scan Session 2 for Subject 12. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com .] Task‐free functional connectivity of Subjects 2 and 12 for scan Sessions 1 (in red) and 2 (in blue), rendered on subject's anatomy, in turn resliced in MNI space. Axial slices are shown in neurological orientation (left is left) with the corresponding coordinate on top of each slice. Z ‐values for activated voxels exceeding the threshold ( P < 0.05 corrected) ranged from 5.1 to 29.8 for scan Session 1, and from 5.1 to 31.2 for scan Session 2 for Subject 2 . Z ‐values ranged from 5.1 to 29.4 for scan Session 1, and from 5.1 to 34.3 for scan Session 2 for Subject 12. The highest Z ‐values are found near the seed region which is included in the figure. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com .] The mean functional image was registered by a two‐step procedure to the subject's structural space, and then to standard space [Jenkinson et al., ; Jenkinson and Smith, ]. These transformations were applied to the images of the contrasts of interest to put them in subjects' structural and standard space, and to the precentral gyrus mask. The precentral gyrus mask was used to investigate the percentage of voxels located in M1 for task and task‐free fMRI at different thresholds. Analysis of Reliability Reliability analysis was performed on whole brain Z ‐statistic images ( Z ‐volume). For the task‐free fMRI, the voxels in the seed region and their direct neighbors were excluded from the analysis, as including these voxels biases the results toward increased reliability of task‐free fMRI due to high correlations of these voxels with themselves and their direct neighbors (e.g., high correlations with neighboring voxels may be the result of correlations in physiological noise or due to partial voluming). We assess reliability of functional brain maps by means of the ICC [Bartko and Carpenter, ; Shrout and Fleiss, ] of contrast Z ‐values for pairs of individual activation maps (ICC) [Raemaekers et al., ], and the ratio of overlapping activation ( R overlap 12 ) [Ramsey et al., ; Rombouts et al., ]. Both measures take the location of activation into account. However, in contrast to the R overlap 12 , the ICC is not dependent on the choice for a particular significance threshold and includes all brain voxels. A two‐way random ICC (formula 2.1 from Shrout and Fleiss [ ]) for absolute agreement between the measurements was used, with the following equation: ICC within = MS between − MS error MS between + MS error + 2 ( MS column − MS error ) where MS between is the mean square of the variance in Z ‐values between voxels; MS column is the mean square of the systematic (column) differences in voxel Z ‐values between the two sessions; MS error is equal to the mean square of the within voxel variance (over sessions) after removal of the systematic session (column) variance. Fisher's Z ‐transformation was used on the individually estimated ICC before group‐wise comparisons: ICC ' = ( 1 2 ) log ( 1 + ICC within 1 − ICC within ) The relative amount of volume overlap in activation between the two sessions for the task and task free Z ‐statistic images of individual subjects was calculated by using the formula proposed by Rombouts et al. [ ]: R Overlap 12 = 2 × V overlap V 1 + V 2 where V 1 and V 2 denote the number of supra‐threshold voxels in the Z ‐volume in Session 1 and Session 2, respectively, and V overlap the number of supra‐threshold voxels in both Z ‐volumes. The R Overlap 12 can range from 0 (no overlap) to 1 (perfect overlap), and is a descriptive statistic for the ratio of the number of voxels that are active in both sessions and the total number of active voxels. We calculate the individual ratio of overlapping activation for task and task‐free fMRI for different Z ‐values present in all subjects and sessions, and for different thresholds exceeded by a fixed number of the most active voxels. We consider Z ‐thresholding as conventional when Z ‐values range from 3.5 to 5 [Mattay et al., ; McGonigle et al., ; Miki et al., ; Ramsey et al., ; Rombouts et al., ; Rutten et al., ; Smith et al., ; Yetkin et al., ]. RESULTS fMRI Results The individual functional activation patterns detected here are spatially consistent with previous studies that used a finger opposition task [Bandettini et al., ; Kim et al., ; Ramsey et al., b ; van Gelderen et al., ], and resting‐state studies that investigated the motor functional network with a univariate analysis [Biswal et al., ; Cordes et al., ; Lowe et al., ; Xiong et al., ]. Activation patterns of two individuals are shown in Figure for task fMRI, and in Figure for task‐free fMRI. These maps are corrected for multiple comparisons by thresholding at P < 0.05 significance level that corresponds to a Z‐ threshold of approximately 5 [Smith et al., ]. At this threshold ( P < 0.05), the mean number of voxels categorized as active for task fMRI was 744 (SD = 565) in scan Session 1, and 758 (SD = 546) in scan Session 2. The mean number of voxels categorized as active for task‐free fMRI was 6,291 (SD = 2,268) in scan Session 1, and 6,663 (SD = 2345) in scan Session 2. Compared to task fMRI, a higher number of voxels was categorized as active during resting state for both scan sessions (both P < 0.001). ICC Results Whole brain ICC′ for the motor task fMRI ranged from 0.20 to 0.60 ( M = 0.42; SD = 0.12) across subjects and were all significant (all P < 0.001). Whole brain ICC′ for the task‐free fMRI ranged from 0.06 to 0.69 ( M = 0.25; SD = 0.16) across subjects and were all significant (all P < 0.001) (Fig. ). A paired sample t ‐test showed higher task fMRI ICC′ values as compared to the task‐free fMRI ( t (15) = 2.77, P = 0.01). This comparison was based on task‐free analysis that excluded the voxels in the seed region. Because the exclusion of the voxels in the seed region may have biased the analysis toward poorer reproducibility of task‐free fMRI, this comparison was re‐performed after inclusion of the voxels in the seed region. When the voxels in the seed region were included in the analysis, task‐free fMRI ICC' values did not change ( M = 0.26; SD = 0.16) and were still significantly lower than the task fMRI values ( t (15) = 2.68, P = 0.02). ICC results are therefore robust and suggest a higher reliability for the task fMRI as compared to task‐free fMRI. Individual whole brain ICC within values for motor task (depicted by blue open squares) and task‐free (depicted by red open circles) fMRI. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com .] Overlap Results Overlap as a function of Z‐thresholding Task‐free fMRI maps showed higher maximum Z‐ values compared to the task maps for both scan sessions ( t (15) for Session 1 is 5.2 and for Session 2 is 6.21, both P < 0.001). For task and task‐free fMRI, to calculate the averaged ratio of overlap for different Z ‐thresholds were taken into account Z ‐values that were present in all subjects across all the task and task‐free functional datasets. The maximum Z ‐value present in all subjects and scan sessions was 9 for task, and 11.5 for task‐free datasets. Paired sample t ‐tests between task and task‐free fMRI were performed at each Z ‐threshold (from 0.5 to 9 in steps of 0.5). Task‐free fMRI overlap values are higher than those of task fMRI at Z ‐thresholds ranging from 1.5 to 2.5 ( t (15) is, respectively, 2.47, 2.49, 2.16, all P = 0.05). There is no significant difference between the two fMRI techniques at Z ‐thresholds ranging from 0.5 to 1.5, and at Z ‐thresholds ranging from 2.5 to 7 (all P > 0.05). Task fMRI overlap values are higher than those of task‐free fMRI at Z ‐thresholds ranging from 7.5 to 9 ( t (15) is, respectively, 2.13, 2.26, 2.19, 2.09, all P = 0.05). While there is no difference between the two techniques at conventional Z ‐thresholding, task fMRI overlap values are higher than task‐free fMRI at higher thresholds. Across all subjects averaged ratio of overlapping activation for task and task‐free fMRI thresholding for different Z ‐values is shown in Figure . However, at all Z‐ thresholds (0.5–9) a higher number of voxels was categorized as active during resting state as compared to task for both scan sessions ( t (15) ranged from 2.57 to 8.70, all P < 0.02). This means that when the same Z ‐value is used as a threshold for both fMRI techniques, there is always a higher number of voxels categorized as active for task‐free fMRI as compared to task fMRI. A large number of active voxels in both condition biases the formula for the overlap toward increased reliability (the size of the overlap increases approximately linearly with the percentage of voxels active in the two sessions, even when the activation patterns that are produced during the two sessions are completely uncorrelated). Therefore, the overlap comparison between the two fMRI techniques is not straightforward as it is biased toward increased reliability of task‐free fMRI. In conclusion, an alternative way of thresholding is necessary in order to compare the overlap results between the task and task‐free fMRI. Between session averaged across all subjects overlap values for task (depicted by blue open squares) and task‐free (depicted by red open circles) fMRI for different thresholds based on Z ‐values. The maximum Z ‐value present in all subjects and sessions was 9 for task, and 11.5 for task‐free fMRI. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com .] Overlap as a function of fixed number of voxels thresholding A more straightforward alternative to compare the overlap results of task and task‐free fMRI is to set a threshold that is exceeded by a fixed number of the most active voxels. For assessment of overlap, we evaluated a range from 10 to 10,000 voxels (about half of our scan volume). Paired sample t ‐tests between task and task‐free fMRI were performed at each threshold (from 10 to 100 in steps of 10, and from 100 to 10,000 in steps of 100). Task fMRI overlap values are higher than those of task‐free fMRI at thresholds of 10–5,500 voxels ( t (15) ranged from 2.2 to 8.95, all P < 0.01 from 10 to 2,800, and all P < 0.05 from 2,900 to 5,500). At this range, task fMRI shows the highest average overlap value at a threshold of 100 voxels. There was no significant difference between the two techniques at thresholds of 5,600–10,000 voxels. Across subjects averaged ratio of overlapping activation for task and task‐free fMRI at each threshold is given in Figure . Between session averaged across subjects overlap values for task (depicted by blue open squares) and task‐free (depicted by red open circles) fMRI for different supra‐threshold voxels (range 10–10,000 voxels). The x ‐axis is scaled logarithmically. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com .] The 100 most active motor task voxels of Subjects 2 and 12 for scan Sessions 1 (in red) and 2 (in blue), rendered on subject's anatomy, in turn resliced in MNI space. Axial slices are shown in neurological orientation (left is left) with the corresponding coordinate on top of each slice. Corresponding Z ‐values ranged from 10 to 19.3 for scan Session 1, and from 8.3 to 18.7 for scan Session 2 for Subject 2. Corresponding Z ‐values ranged from 9.1 to 16.3 for scan Session 1, and from 11.9 to 20.5 for scan Session 2 for Subject 12. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com .] The 100 most active task‐free functional connectivity voxels of Subjects 2 and 12 for scan Sessions 1 (in red) and 2 (in blue), rendered on subject's anatomy, in turn resliced in MNI space. Axial slices are shown in neurological orientation (left is left) with the corresponding coordinate on top of each slice. Corresponding Z ‐values ranged from 23.2 to 29.8 for scan Session 1, and from 21.8 to 31.2 for scan Session 2 for Subject 2. Corresponding Z ‐values ranged from 12 to 29.4 for scan Session 1, and from 21 to 34.3 for scan Session 2 for Subject 12. The highest Z ‐values are found near the seed region which is included in the figure. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com .] At a threshold of 10 voxels, the averaged corresponding Z ‐threshold for task fMRI is 13.6 (SD = 2.7) for scan Session 1, and 13.9 (SD = 2.7) for scan Session 2. At this threshold the averaged corresponding Z ‐threshold for task‐free fMRI is 19.6 (SD = 3.6) for scan Session 1, and 20.7 (SD = 4) for scan Session 2. At a threshold of 5,500 voxels, the averaged corresponding Z ‐threshold for task fMRI is 2 (SD = 0.7) for scan Session 1, and 1.8 (SD = 0.6) for scan Session 2. At this threshold, the averaged corresponding Z ‐threshold for task‐free fMRI is 5.1 (SD = 1.6) for scan Session 1, and 5.4 (SD = 1.9) for scan Session 2. In conclusion, task fMRI overlap values are higher than those of task‐free fMRI for a range of supra‐threshold voxels that correspond to conventional and high Z‐ thresholds. Finally, we investigated how the overlap values for both fMRI techniques were influenced by thresholding. We labeled the range from 10 to 1,000 voxels as “high threshold,” and the range from 1,000 to 5,500 voxels as “conventional threshold.” For task fMRI, across subjects averaged ratio of overlap is 0.47 (SD = 0.14) at high, and 0.44 (SD = 0.09) at conventional threshold. For task‐free fMRI, across subjects averaged ratio of overlap is 0.19 (SD = 0.11) at high, and 0.30 (SD = 0.11) at conventional threshold. Overlap values for task fMRI are higher than task‐free fMRI at high and conventional thresholds. Rigorous testing with linear models for interaction effects between the fMRI technique and thresholding is not appropriate in this case as overlap values at neighboring thresholds are highly correlated, which would create an artificial significant effect. Alternatively, overlap differences for different thresholds can be calculated. Ten thresholds were chosen: 100, and 1,000–9,000 in steps of 1,000. From the overlap value at each of the chosen thresholds was subtracted the overlap value of the next coming threshold (e.g., 100 − 1,000; 1,000 − 2,000…9,000 − 10,000). The difference in overlap between the two fMRI techniques is 1% at a supra‐threshold of 5,000 voxels, it increases to 5% at a supra‐threshold of 1,000 voxels, and increases further to 11% at a supra‐threshold of 100 voxels. The difference in overlap between the two fMRI techniques increased at higher thresholds. To investigate whether these results were biased from the exclusion of the voxels in the seed region for the task‐free analysis, the comparison between the two fMRI techniques was re‐performed after inclusion of these voxels. When the voxels of the seed region were included in the task‐free fMRI analysis, the difference between task and task‐free fMRI did not change at a supra‐threshold of 5,000 and 1,000 voxels, while it decreased from 11 to 6% at a supra‐threshold of 100 voxels. However, the difference between the two techniques persisted as task‐free fMRI overlap values were still lower than the task fMRI values, especially at higher thresholds. In conclusion, overlap improved for task fMRI from conventional to high thresholds, but it decreased for task‐free fMRI. We further investigated whether the decrease in overlap seen at high thresholds for task‐free fMRI could be related to the size of the seed region used for the task‐free analysis. For example, the proportion of random noise may be larger for the mean time series of small seed regions, thereby decreasing the reproducibility of task‐free fMRI especially at higher thresholds. We examined the correlation across subjects between the number of the voxels included in the seed region and the overlap results at a threshold of 1,000 and 100 voxels and found no significant correlation (respectively, P = 0.13 and 0.44). These results indicated that the decreased reliability of resting‐state activation at higher thresholds was not related to the size of the seed region used for the task‐free analysis. Scan Order Results We investigated whether differences in reliability between task and task‐free fMRI were due the fact that the task fMRI was always acquired prior to the task‐free fMRI [Peltier et al., ; Waites et al., ]. Because subjects had to lie in the scanner for a longer period of time they were more likely to move or to fall asleep during the task‐free scans, thereby influencing the functional connectivity in the resting state [e.g., Kiviniemi et al., ]. We first investigated whether there was a decrease in the temporal signal to noise ratio (tSNR) during the task‐free scans as a direct consequence of increased motion due to the fact that subjects may become more uncomfortable. The tSNR was computed as the standard deviation of each voxel's time series after spatial preprocessing, and then averaged across all voxels in the brain for each subject. Results showed no significant difference on the tSNR between the task and task‐free fMRI in the first (Mean tSNR = 55.43 for task and 55.49 for task‐free fMRI) or second scan session (Mean tSNR = 55.52 for both, task and task‐free fMRI). We also investigated whether there was an increase in subject motion during the task‐free fMRI by using the affine parameters that were calculated during the realignment. We computed the total displacement during scanning for each voxel and then averaged across all voxels in the brain resulting in a single value for each scan session. There was no significant differences in displacement of voxels between the task and task‐free fMRI in the first ( t (15) = 0.01, P = 0.995) or second scan session ( t (15) = 0.28, P = 0.782). We tested for indications that subjects had fallen asleep during the task‐free scans by examining the physiological parameters acquired in each of the scan sessions. Usually, if a subject would fall asleep, then the heart rate and respiration frequency will go down, while the respiration amplitude will go up [Shmueli et al., ]. The mean heart beat intervals and the change in slopes of their linear fits were compared between the two fMRI techniques in order to investigate heart rate differences and heart rate changes during each of the scan sessions. Compared to task fMRI, subjects' heart rate was lower for the task‐free fMRI session (lowest P = 0.001, t (15) = 3.97), but stable throughout the scanning session (lowest P = 0.20, t (15) = 1.33) suggesting that subjects did not fall asleep. In order to investigate respiratory changes and respiratory differences during scanning between task and task‐free fMRI, first, the frequency spectra of the respiratory signal were calculated for separate quarters of the scanning sessions (each quarter was 100 scans). From each frequency spectrum, the height and the location of the highest frequency were extracted which represented subsequently the amplitude and frequency of respiration. Linear functions were then fitted to the amplitude and frequency of respiration across the four quarters, and the change in their slopes were compared between the two fMRI techniques. The amplitude (lowest P = 0.24, t (15) = 1.24) and the frequency of respiration (lowest P = 0.17, t (15) = 1.46) were similar and stable throughout the scanning sessions for both fMRI techniques. In conclusion, analysis of tSNR, subject motion, and physiological parameters show no scan order effects on the task‐free fMRI [Fair et al., ]. Source of Reliability Difference The difference in reliability found may not be related to differences in the underlying BOLD signal but to differences in data processing. To investigate the source of the reliability difference, we ran the seed‐based functional connectivity analysis on the task fMRI data (with exclusion of seed voxels). In this way, any difference in reliability between task and task‐free reliability could be ascribed to this difference in underlying BOLD signal. Compared to the previous GLM task fMRI analysis, whole brain ICC′ for the seed‐based task fMRI maps increased ( M = 0.51; SD = 0.11) across subjects and were all significant (all P < 0.001). A paired sample t ‐test showed significantly higher ICC′ values for seed‐based task fMRI as compared to task‐free fMRI maps ( t (15) = 5.01, P < 0.001) (see Fig. A in Supporting Information). Overlap values were also higher for seed‐based task as compared to task‐free fMRI at all ranges of supra‐threshold voxels (all P < 0.001 for all paired comparisons from 10 to 10,000 voxels). Overlap results still showed a better reliability for seed‐based task fMRI when different ranges of Z ‐thresholding were used (lowest P = 0.03 for all paired comparisons for Z = 6–11.5). Seed‐based task fMRI overlap results as a function of fixed number of voxels thresholding and of Z ‐thresholding are given respectively in Figures B and C in Supporting Information. We conclude that the higher reliability found for task fMRI reflects properties of the underlying BOLD signal. M1 fMRI Results The percentage of voxels located in M1 relative to the whole brain (anatomically defined in the precentral gyrus) for task and task‐free fMRI was calculated to investigate whether the most active voxels are always located in M1. Different thresholds were used evaluating a range from 50 to 10,000 voxels. The seed region used to model motor task‐free functional connectivity was excluded from this analysis. In Table we give an overview of these results and show that: (a) there is no difference between Task and task‐free fMRI in the percentage of voxels located in M1 at different thresholds ( P > 0.05); (b) the higher the threshold, the higher the percentage of active voxels in M1 for both techniques (both P < 0.001). Activation patterns of two individuals using at a threshold of 100 voxels are shown in Figure for task fMRI, and in Figure for task‐free fMRI. Anatomically evaluated, the highest active voxels are found in the handknob for task fMRI [Tegeler et al., ; Yousry et al., ], and in bilateral M1 for task‐free fMRI [van den Heuvel and Hulshoff Pol, ]. These results support the assumption that the most active voxels of the motor network are located in “critical” brain areas for motor task performance [Beisteiner et al., ]. In Supporting Information, two individual results are shown at a threshold of 20 voxels for task fMRI (Fig. A), and 50 voxels for task‐free fMRI (Fig. B). The percentage of voxels located in M1 in each scan session for task and task‐free fMRI at different supra‐threshold voxels Task fMRI Task‐free fMRI Session 1 Session 2 Session 1 Session 2 50 Mean 64 66 63 65 SD 13 13 20 21 100 Mean 58 58 55 58 SD 9 10 19 20 500 Mean 43 44 39 41 SD 6 8 13 14 1,000 Mean 36 37 33 35 SD 6 7 10 11 5,000 Mean 21 22 21 22 SD 2 3 5 5 10,000 Mean 17 18 17 18 SD 1 1 3 2 Units: number of voxels. Units: %. DISCUSSION The present study investigates the between session reproducibility of individual functional motor brain maps identified with task and task‐free fMRI. Results show that task fMRI has better reproducibility than task‐free fMRI for motor network identification. Task fMRI ICC values are higher than those of task‐free fMRI (Fig. ), and overlap values using thresholds exceeded by a fixed number of voxels are higher for task FMRI as compared to task‐free fMRI (Fig. ). Furthermore, overlap improved for task fMRI at higher thresholds, while it decreased for task‐free fMRI. Finally, the percentage of voxels active in M1 (relative to the whole brain) was comparable for the two fMRI techniques, and showed a higher percentage of voxels in M1 at higher thresholds (Table ). Both reliability measures showed that task‐free fMRI identifies the functional motor network less reliably than task fMRI. Our results therefore suggest that caution must be taken before applying task‐free fMRI in clinical practice. These results contrast with a previous study that suggested that task‐free fMRI was more reliable than task fMRI in identifying the functional motor network [Zhang et al., ]. The difference may be associated with the fact that Zhang et al. [ ] assessed reliability qualitatively and the number of scans they acquired during resting state was greater than during task performance. The present study compares the reliability of the task‐free motor network to task fMRI quantitatively in the same individuals. Furthermore, we report similar task‐free ICC and overlap results to a previous resting‐state study that investigated the reliability of the supplementary motor area (SMA) network [Shehzad et al., ]. Though Shehzad et al. [ ] suggest that resting‐state functional connectivity is robust and reliable, we find it to be less reliable than task‐induced activity. The higher reliability found for task fMRI more likely reflects properties of the underlying BOLD signal: driving the motor system with an explicit task results in greater fluctuations in the BOLD signal that can be mapped more reliably. The lower reliability of the resting‐state functional connectivity is not due to the fact that task‐free fMRI scans were always acquired later than the task fMRI scans. Though scans acquired late in time may have a reduced tSNR and may be confounded by fatigue effects such as excessive movement or sleep during the scanning [Peltier et al., ; Waites et al., ], we found no such effects in our data as in a previous study by Fair et al. [ ]. Especially for higher statistical thresholds, task fMRI activity is more reliable than resting‐state connectivity. This result is in agreement with a previous resting‐state study that found lower intersession reliability in the SMA at higher than conventional thresholds [Shehzad et al., ]. This may be a reflection of a difference in the nature of the spatial pattern of the motor network during resting state compared to task fMRI. Although previous research shows that motor network connectivity displays a detailed topography during resting state [van den Heuvel and Hulshoff Pol, ], it may still be the case that the distinction in connectivity between neighboring voxels does not match the spatial details in the pattern of activation produced by task fMRI. In a less focal pattern (i.e., lower peaks and troughs in the activation pattern), the calculated overlap values would be relatively small even for the most active voxels. The lower overlap at higher thresholds for resting state compared to task fMRI may thus be explained by a less focal pattern of resting‐state connectivity. A less focal resting‐state connectivity pattern does not necessarily imply that its spatial distribution is too coarse for motor network identification. Just like for task fMRI, a higher percentage of voxels is found located in the primary motor cortex for task‐free fMRI at higher thresholds. This clearly indicates that task and task‐free fMRI show functional activity and connectivity in critical brain areas for motor task performance (see Supporting Information, Figs. A and B). Our results suggest that task‐free fMRI can properly identify the motor network. Furthermore, the spatial distribution of our connectivity maps is consistent with previous task‐free fMRI studies that investigated the motor functional network with a univariate analysis [Biswal et al., ; Cordes et al., ; Lowe et al., ; Xiong et al., ] (see Fig. ). Comparison of overlap results between the two fMRI techniques based on Z‐ thresholding is not straightforward. Different experimental paradigms have different sensitivity and produce (throughout the entire brain) different Z ‐values. We found higher Z ‐values for the task‐free motor network as compared to the task‐related network. Due to sensitivity differences, using a single conventional Z‐ threshold for both paradigms is not an adequate comparison because a different number of voxels is categorized as active. Such a difference could systematically bias reliability estimates. The current experiment showed similar reproducibility between the two techniques at a single conventional Z ‐threshold (e.g., Z = 5) (Fig. ). However, thresholding from 10 to 5,500 of the most active voxels clearly showed better overlap results of task fMRI as compared to task‐free fMRI. In turn, these thresholds based on the number of voxels corresponded to high and conventional Z‐ thresholding. Comparing the paradigms at a threshold where a fixed number of voxels is active, thus, provides more detailed and useful information. Importantly, this approach is attractive for clinical application, given that activity needs to be localized rather than proven to occur. For scientific purposes, statistical thresholding is indicated as the null hypothesis typically states absence of activity. An alternative option to the seed‐based analysis used in this experiment is independent component analysis (ICA) [Beckmann et al., ; Kiviniemi et al., ; van de Ven et al., ]. ICA does not rely on a priori seeding, and seems ideal to localize the motor functional networks in healthy volunteers and patients with altered brain functional topography [Kokkonen et al., ; Zhang et al., ]. However, this method has its own limitations such as an occasional failure to separate different functional networks from each other [Zuo et al., ] or from noise [Birn et al., ], and being subjective and labor intensive in selecting the functional network of interest from others [Zhang et al., ]. Nevertheless, the reliability levels obtained in the present study are very similar to test‐retest studies that have used ICA [Meindl et al., ; Zuo et al., ]. The findings about the location of the most active voxels are of particular relevance for the use of task and task‐free fMRI in clinical practice. Even for the well known functional motor network, a major challenge for the neurosurgeon during presurgical planning is to disentangle critical brain areas from non‐critical areas when relying on conventional Z ‐thresholding [Beisteiner et al., ; Rutten and Ramsey, ]. While this is complicated for task fMRI, it is even more so for task‐free fMRI considering the size of the volume categorized as active [Cordes et al., ; Xiong et al., ]. Our findings support the use of higher fMRI thresholds for clinical purposes because a high percentage of voxels are located in the primary motor cortex for both techniques. However, at higher thresholds the task fMRI voxels showed to be more reliable than the task‐free ones. Therefore, caution should be taken in using task‐free fMRI as a substitute of task fMRI for diagnostic examinations [Brannen et al., ; Fernandez et al., ], and for comparisons between healthy volunteers and patients [Manoach et al., ]. The reproducibility results reported here are dependent on the method used to analyze the functional brain maps. The analysis used has been shown to be efficient in previous test‐retest studies [Smith et al., ]. However, differences in pre‐ and postprocessing between the task and task‐free fMRI may bias the estimation of their reproducibility [Miki et al., ]. Because we use a seed region based on task fMRI for the resting‐state analysis [Cordes et al., ], the reliability of task‐free may be even overestimated compared to clinical situations where no task fMRI data are available. While we take a clinical perspective in this study and consider smoothing inappropriate [Rutten et al., ], smoothing may still be beneficial for test‐retest purposes as it gives adhesion to a less focal resting‐state connectivity pattern [Rombouts et al., ]. CONCLUSIONS A rigorous test‐retest comparison showed task fMRI better reproducible than task‐free fMRI for motor network identification in healthy subjects. Task fMRI ICC values were higher than those of task‐free fMRI. Overlap values were higher for task fMRI as compared to task‐free fMRI when thresholds exceeded by a fixed number of voxels were used. Overlap further improved for task fMRI at higher thresholds, while it decreased for task‐free fMRI. This is of relevance considering that the most active voxels of the motor network are located in critical brain areas for motor task performance. The reliability results presented here suggest that caution must be taken before applying task‐free fMRI in clinical practice as a replacement of task fMRI. However, resting‐state fMRI may be still suitable for patients with task compliance problems because, just like task fMRI, it can properly identify the motor network. As it is not clear whether these findings generalize to other brain systems, future studies should address the reliability of other resting‐state networks relevant for neurology and neurosurgery. ACKNOWLEDGMENTS Authors thank the following researchers at the UMC in Utrecht, The Netherlands: M. P. van den Heuvel and S. F. W. Neggers for helpful discussions; T. E. Gladwin, M. van Buuren, and M. Vink, for making use of Aztec ( http://www.ni‐utrecht.nl/downloads/aztec ). http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Human Brain Mapping Wiley

Task and task‐free FMRI reproducibility comparison for motor network identification

Download PDF
13 pages

Loading next page...
 
/lp/wiley/task-and-task-free-fmri-reproducibility-comparison-for-motor-network-vuv288Mmer

References (81)

Publisher
Wiley
Copyright
"Copyright © 2012 Wiley Periodicals, Inc."
ISSN
1065-9471
eISSN
1097-0193
DOI
10.1002/hbm.22180
pmid
22987751
Publisher site
See Article on Publisher Site

Abstract

INTRODUCTION Functional magnetic resonance imaging (fMRI) is increasingly used to localize specific brain functions in the individual patient. Localization of the primary motor cortex (M1) is a frequent clinical issue in neurosurgery [Fernandez et al., ] and neurology [Johansen‐Berg et al., ], for which typically a fMRI study is performed while the patient is performing a task with alternating instructions (task fMRI). The M1 can be well localized with this approach [Bandettini et al., ; Kim et al., ; Ramsey et al., ; Rao et al., ; van Gelderen et al., ]. However, motor deficits can severely impair task performance of the patient and introduce false positive or false negative activation. This can have a profound influence on the reliability of the resulting brain activation maps and on their clinical interpretation [Jiang et al., ; Krings et al., ; Rutten and Ramsey, ]. A possible solution to this problem is to study patients in a resting state (without the subject performing a specific task in the scanner). At rest, brain regions exhibit spontaneous fluctuations at low frequencies (0.01–0.1 Hz) in the blood oxygen level‐dependent (BOLD) signal. When different brain areas show temporally correlated signals, as has for instance been shown with bilateral primary motor areas, these areas can be regarded as forming a functional network. Several functional networks that have been identified with resting‐state (task‐free) fMRI [e.g., Damoiseaux et al., ; van de Ven et al., ] have been shown to qualitatively overlap with task fMRI activation [e.g., Biswal et al., ; Greicius et al., ; Kokkonen et al., ; Smith et al., ]. The simplest analysis technique for identifying functional connectivity networks is to extract the BOLD time course from voxels in a region of interest (seed region) and determine its temporal correlation with the time course of all other brain voxels (also called univariate analysis). Previous studies have shown that task‐free fMRI can localize M1 functionality [Biswal et al., ; Cordes et al., ; Lowe et al., ; Xiong et al., ]. A recent study suggested that functional subregions of the motor network during rest are organized in a somatotopic fashion, in that they are linked one‐on‐one to their homologue in the contralateral hemisphere [van den Heuvel and Hulshoff Pol, ]. A high test‐retest reliability of fMRI maps is a pre‐requisite for their application in clinical practice. Studies on task fMRI reliability show mixed results for individual motor functional brain maps. Repeated scans of the same subject yields similar but not identical maps [McGonigle et al., ], with overlap ranging from 20 to 62% [Havel et al., ; Ramsey et al., ; Yetkin et al., ; Zandbelt et al., ]. The reproducibility of single‐subject fMRI is affected by multiple sources of variability, including attention and practice [Ungerleider et al., ; Veltman et al., ], physiology (cardiac and respiratory artifacts) [Birn et al., ; Biswal et al., ], and equipment (MRI field inhomogeneities and image signal‐to‐noise ratio) [Raemaekers et al., ]. To date, only a few studies have investigated the reliability of task‐free fMRI [Meindl et al., ; Shehzad et al., ; Zuo et al., ]. However, these studies did not focus on the reliability of the M1 localization. The main goal of our study is to investigate the test‐retest reliability of task‐free fMRI for M1 identification and compare it to the conventional task fMRI to assess its potential for use in clinical practice. Two measures commonly used to assess reliability of functional brain maps are the intraclass correlation (ICC) [Bartko and Carpenter, ; Shrout and Fleiss, ], and the ratio of overlapping activation [Ramsey et al., ; Rombouts et al., ]. ICC represents a general reliability measure based on between‐session correlations of Z ‐ (or T‐ ) values of brain statistic images, and is independent of the choice for a particular significance threshold. The ratio of overlapping activation is a more specific measure in that it looks at common voxels surviving a particular significance threshold in both scan sessions. From a clinical point of view, the overlap measure is more informative because the neurosurgeon or neurologist will interpret fMRI results based on conventional P or Z ‐thresholding (with Z ‐values usually ranging from 3.5 to 5) [Mattay et al., ; McGonigle et al., ; Miki et al., ; Ramsey et al., ; Rombouts et al., ; Rutten et al., ; Smith et al., ; Yetkin et al., ]. Overlap results based on conventional Z ‐thresholding are not appropriate for comparing different fMRI experimental designs, as in the case of task and task‐free fMRI. Different fMRI designs differ in sensitivity and, thereby, produce different Z ‐values. Setting a conventional Z ‐threshold (e.g., at a Z ‐value of 5), while having different maximum Z ‐values between the designs, will categorize a different number of voxels as active. This affects the ratio of overlap and makes direct comparison not appropriate. A straightforward alternative is then to set a threshold that is exceeded by a fixed number of voxels [Tegeler et al., ]. This approach is also justified by the fact that the goal is to determine where activity is located, and not whether there is activity (classical null hypothesis for scientific application). Here we quantify and compare between‐session reproducibility of motor task and task‐free fMRI activity. Sixteen healthy subjects participated in this study with a test‐retest interval of on average 7 weeks. We assess reproducibility in terms of ICC values and the ratio of overlap [Raemaekers et al., ]. We first estimate the ratio of overlapping activation for different Z ‐thresholds, and then for different thresholds exceeded by a fixed number of the most active voxels [Tegeler et al., ]. Finally, we look at the percentage of voxels located in the primary motor cortex for different thresholds. MATERIALS AND METHODS Participants Sixteen healthy subjects (8 females and 8 males, mean age 39.4 years, SD 10.7) participated in the experiment after giving informed consent. The research protocol and consent forms were approved by the medical ethics committee for research in humans (METC) of the University Medical Center Utrecht (UMC Utrecht, The Netherlands) in accordance with the Declaration of Helsinki (2008). Subjects were screened for medical, neurological, and psychiatric illnesses, use of medication, and metal implants. They had no previous experience with fMRI or with the tasks performed in the scanner, and all were strongly right handed ( M = 0.85; SD = 0.15) according to the Edinburgh Handedness inventory [Oldfield, ]. Participants were requested to abstain from nicotine and caffeine for 4 h prior to the scan sessions, and from over‐the‐counter medications for 24 h. The test and retest took place with an average interval of approximately 7 weeks (mean 50.7 days, SD 22.6). Experimental Design The fMRI experimental design was the same for both scan sessions and included, in order, a motor task, and a task‐free experiment. Participants were informed about the experimental procedure, and briefly practiced the motor task with the aid of a laptop before the scanning session. We used a PC, a rear projection screen, and a video projector system for stimulus presentation. Visual stimuli were projected in green and red on a dark background. For the motor task subjects repeatedly touched the thumb once with each of the digits backward and forward. Subjects were instructed to perform the task with the right hand when a flickering (2 Hz) green circle was presented on the screen, and to rest when a flickering red circle was presented. Subjects made movements at the rate of the flickering circle. The task was presented in a block‐design with eight cycles of 29.8 sec of movement, and 29.8 sec of rest. The experimental trial always began with a control period, in that both hands were relaxed. Blocks were time locked to the fMRI scans. During the task‐free fMRI experiment, the scanner room was darkened and participants were instructed to relax with their eyes closed, to think of nothing in particular, and not to fall asleep. As verified afterwards, no participant reported to have fallen asleep or to have been close to falling asleep. Data Acquisition All images were obtained with a whole body 3.0 Tesla (3T) Philips Achieva MRI scanner (Philips Medical Systems, Best, The Netherlands). The participant's head was held in place with padding. Heartbeat was recorded using a pulse‐oximeter placed on the left index finger. Respiration was measured with a pneumatic belt positioned at the level of the abdomen [Birn et al., ]. A T1‐weighted structural image of the whole brain in saggital orientation was acquired for anatomical reference (3D fast field echo (FFE) pulse sequence; acquisition parameters: repetition time (TR) 8.4 ms, excitation time (TE) 3.8 ms; field of view (FOV) 288 × 288 × 175 mm; voxel size 1 mm isotropic; SENSE p‐reduction/s‐reduction, 2/1.3; flip‐angle, 8°; 175 slices, scan duration = 265.8 sec). For functional scans, a 3D‐PRESTO (Principles of Echo Shifting with a Train of Observations) pulse sequence [van Gelderen et al., ] with parallel imaging [Golay et al., ; Neggers et al., ] was used. This method involves a combination of echo shifting [Duyn et al., ; Moonen et al., ] and multiple gradient echoes per radio frequency excitation [Liu et al., ]. It allows whole brain coverage with very short acquisition times by already applying the next excitation before signal readout. The very short acquisition times are ideal to reduce artifacts caused by motion during scans and physiological aliasing. This is of particular importance for the motor cortex since it has been shown to be more prone to physiological contaminations than the visual and auditory cortex [Cordes et al., ]. We used the following parameters: TR 22.5 msec; effective TE, 32.4 msec (using a shifted echo [Liu et al., ]); FOV 256 × 224 × 160 mm, voxel size 4 mm isotropic; matrix 64 × 56 × 40; SENSE p‐reduction/s‐reduction, 1.8/2; flip‐angle 10°; scan time per volume (scan duration) 0.6075 sec. Task and task‐free functional images encompassed the whole brain and were acquired in saggital orientation, with a foot–head frequency encoding direction. In total, 442 task and 400 task‐free fMRI scans were acquired, but only the first 400 task fMRI scans were included in the analysis to make an exact comparison with task‐free fMRI. Between the task and task‐free fMRI, two additional functional scans were acquired: a PRESTO scan of the same volume of brain tissue and with the same parameters was acquired in 0.72 sec, but with a higher flip‐angle (27°, FA27) for the image coregistration routine (see below in Data preprocessing section); and a diffusion tensor imaging scan not reported in this study which lasted 6:02 min. Data Preprocessing All functional images of both scan sessions were realigned (2nd degree b‐spline) and resliced (4th degree b‐spline) to the high contrast functional image (Fa27) of the first scan session using SPM5 ( http://www.fil.ion.ucl.ac.uk/spm/ ). We then corrected for cardiac and respiratory effects as this is shown to be beneficial for fMRI [Shmueli et al., ; Wise et al., ], especially when the motor cortex is investigated [Birn et al., ]. For correction of cardiac and respiratory artifacts, custom Matlab scripts were used (Aztec, http://www.ni‐utrecht.nl/downloads/aztec ). First, the task and task‐free BOLD signal was corrected for the effects of the phase of the cardiac and respiratory cycle using RETROICOR [Glover et al., ]. Subsequently, the task and task‐free BOLD signal were corrected for the effects of heart rate, heart rate variability, and respiration volume per time unit [Birn et al., ] using a multiple regression approach. The method used here is described in detail in the study by van Buuren et al. [ ]. Task‐free functional images were then lowpass filtered (0.08 Hz) with a finite impulse response filter, and highpass filtered in FSL, version 5.92 (Gaussian‐weighted least squares straight‐line fitting, with sigma = 50 sec) ( http://www.fmrib.ox.ac.uk/fsl/ ) [Smith et al., ]. The Task functional images were only highpass filtered (sigma = 29.8 sec). Task and task‐free functional images were skull stripped [Smith, ] and intensity normalized (grand mean scaling) in FSL. No spatial smoothing was performed on the functional images [Machielsen et al., ; Ramsey et al., ; Rombouts et al., ; Specht et al., ], as considered inappropriate for clinical decision making [e.g., Rutten et al., ]. For the task‐free analysis, (see Data analysis section) we did not regress out the global signal as in previous fMRI studies [Fox et al., ; Fransson, ; Greicius et al., ; Kelly et al., ]. Regressing out the global signal may remove true functional signal [Birn et al., ], may introduce spurious negative activations in task fMRI analysis [Aguirre et al., ], and anti‐correlations in task‐free fMRI analysis [Murphy et al., ]. Instead, we decided to separately regress out cerebrospinal fluid (CSF) and white‐matter (WM) signal. Regressing out the CSF signal is beneficial as low frequency components in CSF have previously been reported with similar frequencies to resting‐state networks [Friese et al., ].We regress out the WM signal as resting‐state networks are shown to be confined within the gray matter [Damoiseaux et al., ; De Luca et al., ]. To regress out the CSF and WM signal, masks of the CSF and WM were made using the segmentation of SPM5. The resulting WM and CSF masks were then registered to the Fa27 of the first scan session (the reference space of the functional data) and eroded to the maximum (threshold 0.99, minimum 0, maximum 1). Finally, the seed region mask was derived from the overlap between the 10 most active motor task fMRI voxels of scan Session 1 and of scan Session 2 [Cordes et al., ]. These voxels were restricted to (a) the anatomically defined handknob [Tegeler et al., ; Yousry et al., ], and (b) fall within the precentral gyrus. This allows using exactly the same seed voxels for both scan sessions. It also minimizes on the one hand the risk of selecting false positive seed voxels, and on the other hand the number of voxels to be excluded from the reliability analysis (see Analysis of reliability section). The precentral gyrus mask is derived from the Harvard‐Oxford Cortical Structural Atlas (in FSL). The number of voxels included in the seed region masks ranged from 2 to 10 (mean 5.1, SD 2.7). Data Analysis FMRI data processing was carried out using a whole‐brain univariate General Linear Model (FEAT in FSL). Time series statistical analysis was carried out using FILM, with pre‐whitening accounting for local autocorrelation [Woolrich et al., ]. The hemodynamic response function for the motor task functional images was modeled using a boxcar convolved with a double gamma variate function and its temporal derivative. To model motor task‐free functional connectivity, three custom explanatory variables (EVs) were chosen, sampled from the functional data itself: the mean time courses of the WM, CSF, and of the seed region. The seed region EV represents our factor of interest. Featquery (part of FSL) was used for extracting the timecourse of each EV. For visualization purposes, individual task and task‐free activation maps were generated using statistical correction ( P < 0.05) for multiple comparisons based on Gaussian Random Field (GRF) [Worsley, ] (Figs. and ), and at different thresholds exceeded by a fixed number of the most active voxels [Tegeler et al., ] (Figs. and , and in Supporting Information Figs. A and B). Motor task activation of Subjects 2 and 12 for scan Session 1 (in red) and scan Session 2 (in blue), rendered on subject's anatomy, in turn resliced in MNI space. Axial slices are shown in neurological orientation (left is left) with the corresponding coordinate on top of each slice. Z ‐values for activated voxels exceeding the threshold ( P < 0.05 corrected) ranged from 5.1 to 19.3 for scan Session 1, and from 5.1 to 18.7 for scan Session 2 for Subject 2 . Z ‐values ranged from 5.1 to 16.3 for scan Session 1, and from 5.1 to 20.5 for scan Session 2 for Subject 12. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com .] Task‐free functional connectivity of Subjects 2 and 12 for scan Sessions 1 (in red) and 2 (in blue), rendered on subject's anatomy, in turn resliced in MNI space. Axial slices are shown in neurological orientation (left is left) with the corresponding coordinate on top of each slice. Z ‐values for activated voxels exceeding the threshold ( P < 0.05 corrected) ranged from 5.1 to 29.8 for scan Session 1, and from 5.1 to 31.2 for scan Session 2 for Subject 2 . Z ‐values ranged from 5.1 to 29.4 for scan Session 1, and from 5.1 to 34.3 for scan Session 2 for Subject 12. The highest Z ‐values are found near the seed region which is included in the figure. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com .] The mean functional image was registered by a two‐step procedure to the subject's structural space, and then to standard space [Jenkinson et al., ; Jenkinson and Smith, ]. These transformations were applied to the images of the contrasts of interest to put them in subjects' structural and standard space, and to the precentral gyrus mask. The precentral gyrus mask was used to investigate the percentage of voxels located in M1 for task and task‐free fMRI at different thresholds. Analysis of Reliability Reliability analysis was performed on whole brain Z ‐statistic images ( Z ‐volume). For the task‐free fMRI, the voxels in the seed region and their direct neighbors were excluded from the analysis, as including these voxels biases the results toward increased reliability of task‐free fMRI due to high correlations of these voxels with themselves and their direct neighbors (e.g., high correlations with neighboring voxels may be the result of correlations in physiological noise or due to partial voluming). We assess reliability of functional brain maps by means of the ICC [Bartko and Carpenter, ; Shrout and Fleiss, ] of contrast Z ‐values for pairs of individual activation maps (ICC) [Raemaekers et al., ], and the ratio of overlapping activation ( R overlap 12 ) [Ramsey et al., ; Rombouts et al., ]. Both measures take the location of activation into account. However, in contrast to the R overlap 12 , the ICC is not dependent on the choice for a particular significance threshold and includes all brain voxels. A two‐way random ICC (formula 2.1 from Shrout and Fleiss [ ]) for absolute agreement between the measurements was used, with the following equation: ICC within = MS between − MS error MS between + MS error + 2 ( MS column − MS error ) where MS between is the mean square of the variance in Z ‐values between voxels; MS column is the mean square of the systematic (column) differences in voxel Z ‐values between the two sessions; MS error is equal to the mean square of the within voxel variance (over sessions) after removal of the systematic session (column) variance. Fisher's Z ‐transformation was used on the individually estimated ICC before group‐wise comparisons: ICC ' = ( 1 2 ) log ( 1 + ICC within 1 − ICC within ) The relative amount of volume overlap in activation between the two sessions for the task and task free Z ‐statistic images of individual subjects was calculated by using the formula proposed by Rombouts et al. [ ]: R Overlap 12 = 2 × V overlap V 1 + V 2 where V 1 and V 2 denote the number of supra‐threshold voxels in the Z ‐volume in Session 1 and Session 2, respectively, and V overlap the number of supra‐threshold voxels in both Z ‐volumes. The R Overlap 12 can range from 0 (no overlap) to 1 (perfect overlap), and is a descriptive statistic for the ratio of the number of voxels that are active in both sessions and the total number of active voxels. We calculate the individual ratio of overlapping activation for task and task‐free fMRI for different Z ‐values present in all subjects and sessions, and for different thresholds exceeded by a fixed number of the most active voxels. We consider Z ‐thresholding as conventional when Z ‐values range from 3.5 to 5 [Mattay et al., ; McGonigle et al., ; Miki et al., ; Ramsey et al., ; Rombouts et al., ; Rutten et al., ; Smith et al., ; Yetkin et al., ]. RESULTS fMRI Results The individual functional activation patterns detected here are spatially consistent with previous studies that used a finger opposition task [Bandettini et al., ; Kim et al., ; Ramsey et al., b ; van Gelderen et al., ], and resting‐state studies that investigated the motor functional network with a univariate analysis [Biswal et al., ; Cordes et al., ; Lowe et al., ; Xiong et al., ]. Activation patterns of two individuals are shown in Figure for task fMRI, and in Figure for task‐free fMRI. These maps are corrected for multiple comparisons by thresholding at P < 0.05 significance level that corresponds to a Z‐ threshold of approximately 5 [Smith et al., ]. At this threshold ( P < 0.05), the mean number of voxels categorized as active for task fMRI was 744 (SD = 565) in scan Session 1, and 758 (SD = 546) in scan Session 2. The mean number of voxels categorized as active for task‐free fMRI was 6,291 (SD = 2,268) in scan Session 1, and 6,663 (SD = 2345) in scan Session 2. Compared to task fMRI, a higher number of voxels was categorized as active during resting state for both scan sessions (both P < 0.001). ICC Results Whole brain ICC′ for the motor task fMRI ranged from 0.20 to 0.60 ( M = 0.42; SD = 0.12) across subjects and were all significant (all P < 0.001). Whole brain ICC′ for the task‐free fMRI ranged from 0.06 to 0.69 ( M = 0.25; SD = 0.16) across subjects and were all significant (all P < 0.001) (Fig. ). A paired sample t ‐test showed higher task fMRI ICC′ values as compared to the task‐free fMRI ( t (15) = 2.77, P = 0.01). This comparison was based on task‐free analysis that excluded the voxels in the seed region. Because the exclusion of the voxels in the seed region may have biased the analysis toward poorer reproducibility of task‐free fMRI, this comparison was re‐performed after inclusion of the voxels in the seed region. When the voxels in the seed region were included in the analysis, task‐free fMRI ICC' values did not change ( M = 0.26; SD = 0.16) and were still significantly lower than the task fMRI values ( t (15) = 2.68, P = 0.02). ICC results are therefore robust and suggest a higher reliability for the task fMRI as compared to task‐free fMRI. Individual whole brain ICC within values for motor task (depicted by blue open squares) and task‐free (depicted by red open circles) fMRI. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com .] Overlap Results Overlap as a function of Z‐thresholding Task‐free fMRI maps showed higher maximum Z‐ values compared to the task maps for both scan sessions ( t (15) for Session 1 is 5.2 and for Session 2 is 6.21, both P < 0.001). For task and task‐free fMRI, to calculate the averaged ratio of overlap for different Z ‐thresholds were taken into account Z ‐values that were present in all subjects across all the task and task‐free functional datasets. The maximum Z ‐value present in all subjects and scan sessions was 9 for task, and 11.5 for task‐free datasets. Paired sample t ‐tests between task and task‐free fMRI were performed at each Z ‐threshold (from 0.5 to 9 in steps of 0.5). Task‐free fMRI overlap values are higher than those of task fMRI at Z ‐thresholds ranging from 1.5 to 2.5 ( t (15) is, respectively, 2.47, 2.49, 2.16, all P = 0.05). There is no significant difference between the two fMRI techniques at Z ‐thresholds ranging from 0.5 to 1.5, and at Z ‐thresholds ranging from 2.5 to 7 (all P > 0.05). Task fMRI overlap values are higher than those of task‐free fMRI at Z ‐thresholds ranging from 7.5 to 9 ( t (15) is, respectively, 2.13, 2.26, 2.19, 2.09, all P = 0.05). While there is no difference between the two techniques at conventional Z ‐thresholding, task fMRI overlap values are higher than task‐free fMRI at higher thresholds. Across all subjects averaged ratio of overlapping activation for task and task‐free fMRI thresholding for different Z ‐values is shown in Figure . However, at all Z‐ thresholds (0.5–9) a higher number of voxels was categorized as active during resting state as compared to task for both scan sessions ( t (15) ranged from 2.57 to 8.70, all P < 0.02). This means that when the same Z ‐value is used as a threshold for both fMRI techniques, there is always a higher number of voxels categorized as active for task‐free fMRI as compared to task fMRI. A large number of active voxels in both condition biases the formula for the overlap toward increased reliability (the size of the overlap increases approximately linearly with the percentage of voxels active in the two sessions, even when the activation patterns that are produced during the two sessions are completely uncorrelated). Therefore, the overlap comparison between the two fMRI techniques is not straightforward as it is biased toward increased reliability of task‐free fMRI. In conclusion, an alternative way of thresholding is necessary in order to compare the overlap results between the task and task‐free fMRI. Between session averaged across all subjects overlap values for task (depicted by blue open squares) and task‐free (depicted by red open circles) fMRI for different thresholds based on Z ‐values. The maximum Z ‐value present in all subjects and sessions was 9 for task, and 11.5 for task‐free fMRI. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com .] Overlap as a function of fixed number of voxels thresholding A more straightforward alternative to compare the overlap results of task and task‐free fMRI is to set a threshold that is exceeded by a fixed number of the most active voxels. For assessment of overlap, we evaluated a range from 10 to 10,000 voxels (about half of our scan volume). Paired sample t ‐tests between task and task‐free fMRI were performed at each threshold (from 10 to 100 in steps of 10, and from 100 to 10,000 in steps of 100). Task fMRI overlap values are higher than those of task‐free fMRI at thresholds of 10–5,500 voxels ( t (15) ranged from 2.2 to 8.95, all P < 0.01 from 10 to 2,800, and all P < 0.05 from 2,900 to 5,500). At this range, task fMRI shows the highest average overlap value at a threshold of 100 voxels. There was no significant difference between the two techniques at thresholds of 5,600–10,000 voxels. Across subjects averaged ratio of overlapping activation for task and task‐free fMRI at each threshold is given in Figure . Between session averaged across subjects overlap values for task (depicted by blue open squares) and task‐free (depicted by red open circles) fMRI for different supra‐threshold voxels (range 10–10,000 voxels). The x ‐axis is scaled logarithmically. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com .] The 100 most active motor task voxels of Subjects 2 and 12 for scan Sessions 1 (in red) and 2 (in blue), rendered on subject's anatomy, in turn resliced in MNI space. Axial slices are shown in neurological orientation (left is left) with the corresponding coordinate on top of each slice. Corresponding Z ‐values ranged from 10 to 19.3 for scan Session 1, and from 8.3 to 18.7 for scan Session 2 for Subject 2. Corresponding Z ‐values ranged from 9.1 to 16.3 for scan Session 1, and from 11.9 to 20.5 for scan Session 2 for Subject 12. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com .] The 100 most active task‐free functional connectivity voxels of Subjects 2 and 12 for scan Sessions 1 (in red) and 2 (in blue), rendered on subject's anatomy, in turn resliced in MNI space. Axial slices are shown in neurological orientation (left is left) with the corresponding coordinate on top of each slice. Corresponding Z ‐values ranged from 23.2 to 29.8 for scan Session 1, and from 21.8 to 31.2 for scan Session 2 for Subject 2. Corresponding Z ‐values ranged from 12 to 29.4 for scan Session 1, and from 21 to 34.3 for scan Session 2 for Subject 12. The highest Z ‐values are found near the seed region which is included in the figure. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com .] At a threshold of 10 voxels, the averaged corresponding Z ‐threshold for task fMRI is 13.6 (SD = 2.7) for scan Session 1, and 13.9 (SD = 2.7) for scan Session 2. At this threshold the averaged corresponding Z ‐threshold for task‐free fMRI is 19.6 (SD = 3.6) for scan Session 1, and 20.7 (SD = 4) for scan Session 2. At a threshold of 5,500 voxels, the averaged corresponding Z ‐threshold for task fMRI is 2 (SD = 0.7) for scan Session 1, and 1.8 (SD = 0.6) for scan Session 2. At this threshold, the averaged corresponding Z ‐threshold for task‐free fMRI is 5.1 (SD = 1.6) for scan Session 1, and 5.4 (SD = 1.9) for scan Session 2. In conclusion, task fMRI overlap values are higher than those of task‐free fMRI for a range of supra‐threshold voxels that correspond to conventional and high Z‐ thresholds. Finally, we investigated how the overlap values for both fMRI techniques were influenced by thresholding. We labeled the range from 10 to 1,000 voxels as “high threshold,” and the range from 1,000 to 5,500 voxels as “conventional threshold.” For task fMRI, across subjects averaged ratio of overlap is 0.47 (SD = 0.14) at high, and 0.44 (SD = 0.09) at conventional threshold. For task‐free fMRI, across subjects averaged ratio of overlap is 0.19 (SD = 0.11) at high, and 0.30 (SD = 0.11) at conventional threshold. Overlap values for task fMRI are higher than task‐free fMRI at high and conventional thresholds. Rigorous testing with linear models for interaction effects between the fMRI technique and thresholding is not appropriate in this case as overlap values at neighboring thresholds are highly correlated, which would create an artificial significant effect. Alternatively, overlap differences for different thresholds can be calculated. Ten thresholds were chosen: 100, and 1,000–9,000 in steps of 1,000. From the overlap value at each of the chosen thresholds was subtracted the overlap value of the next coming threshold (e.g., 100 − 1,000; 1,000 − 2,000…9,000 − 10,000). The difference in overlap between the two fMRI techniques is 1% at a supra‐threshold of 5,000 voxels, it increases to 5% at a supra‐threshold of 1,000 voxels, and increases further to 11% at a supra‐threshold of 100 voxels. The difference in overlap between the two fMRI techniques increased at higher thresholds. To investigate whether these results were biased from the exclusion of the voxels in the seed region for the task‐free analysis, the comparison between the two fMRI techniques was re‐performed after inclusion of these voxels. When the voxels of the seed region were included in the task‐free fMRI analysis, the difference between task and task‐free fMRI did not change at a supra‐threshold of 5,000 and 1,000 voxels, while it decreased from 11 to 6% at a supra‐threshold of 100 voxels. However, the difference between the two techniques persisted as task‐free fMRI overlap values were still lower than the task fMRI values, especially at higher thresholds. In conclusion, overlap improved for task fMRI from conventional to high thresholds, but it decreased for task‐free fMRI. We further investigated whether the decrease in overlap seen at high thresholds for task‐free fMRI could be related to the size of the seed region used for the task‐free analysis. For example, the proportion of random noise may be larger for the mean time series of small seed regions, thereby decreasing the reproducibility of task‐free fMRI especially at higher thresholds. We examined the correlation across subjects between the number of the voxels included in the seed region and the overlap results at a threshold of 1,000 and 100 voxels and found no significant correlation (respectively, P = 0.13 and 0.44). These results indicated that the decreased reliability of resting‐state activation at higher thresholds was not related to the size of the seed region used for the task‐free analysis. Scan Order Results We investigated whether differences in reliability between task and task‐free fMRI were due the fact that the task fMRI was always acquired prior to the task‐free fMRI [Peltier et al., ; Waites et al., ]. Because subjects had to lie in the scanner for a longer period of time they were more likely to move or to fall asleep during the task‐free scans, thereby influencing the functional connectivity in the resting state [e.g., Kiviniemi et al., ]. We first investigated whether there was a decrease in the temporal signal to noise ratio (tSNR) during the task‐free scans as a direct consequence of increased motion due to the fact that subjects may become more uncomfortable. The tSNR was computed as the standard deviation of each voxel's time series after spatial preprocessing, and then averaged across all voxels in the brain for each subject. Results showed no significant difference on the tSNR between the task and task‐free fMRI in the first (Mean tSNR = 55.43 for task and 55.49 for task‐free fMRI) or second scan session (Mean tSNR = 55.52 for both, task and task‐free fMRI). We also investigated whether there was an increase in subject motion during the task‐free fMRI by using the affine parameters that were calculated during the realignment. We computed the total displacement during scanning for each voxel and then averaged across all voxels in the brain resulting in a single value for each scan session. There was no significant differences in displacement of voxels between the task and task‐free fMRI in the first ( t (15) = 0.01, P = 0.995) or second scan session ( t (15) = 0.28, P = 0.782). We tested for indications that subjects had fallen asleep during the task‐free scans by examining the physiological parameters acquired in each of the scan sessions. Usually, if a subject would fall asleep, then the heart rate and respiration frequency will go down, while the respiration amplitude will go up [Shmueli et al., ]. The mean heart beat intervals and the change in slopes of their linear fits were compared between the two fMRI techniques in order to investigate heart rate differences and heart rate changes during each of the scan sessions. Compared to task fMRI, subjects' heart rate was lower for the task‐free fMRI session (lowest P = 0.001, t (15) = 3.97), but stable throughout the scanning session (lowest P = 0.20, t (15) = 1.33) suggesting that subjects did not fall asleep. In order to investigate respiratory changes and respiratory differences during scanning between task and task‐free fMRI, first, the frequency spectra of the respiratory signal were calculated for separate quarters of the scanning sessions (each quarter was 100 scans). From each frequency spectrum, the height and the location of the highest frequency were extracted which represented subsequently the amplitude and frequency of respiration. Linear functions were then fitted to the amplitude and frequency of respiration across the four quarters, and the change in their slopes were compared between the two fMRI techniques. The amplitude (lowest P = 0.24, t (15) = 1.24) and the frequency of respiration (lowest P = 0.17, t (15) = 1.46) were similar and stable throughout the scanning sessions for both fMRI techniques. In conclusion, analysis of tSNR, subject motion, and physiological parameters show no scan order effects on the task‐free fMRI [Fair et al., ]. Source of Reliability Difference The difference in reliability found may not be related to differences in the underlying BOLD signal but to differences in data processing. To investigate the source of the reliability difference, we ran the seed‐based functional connectivity analysis on the task fMRI data (with exclusion of seed voxels). In this way, any difference in reliability between task and task‐free reliability could be ascribed to this difference in underlying BOLD signal. Compared to the previous GLM task fMRI analysis, whole brain ICC′ for the seed‐based task fMRI maps increased ( M = 0.51; SD = 0.11) across subjects and were all significant (all P < 0.001). A paired sample t ‐test showed significantly higher ICC′ values for seed‐based task fMRI as compared to task‐free fMRI maps ( t (15) = 5.01, P < 0.001) (see Fig. A in Supporting Information). Overlap values were also higher for seed‐based task as compared to task‐free fMRI at all ranges of supra‐threshold voxels (all P < 0.001 for all paired comparisons from 10 to 10,000 voxels). Overlap results still showed a better reliability for seed‐based task fMRI when different ranges of Z ‐thresholding were used (lowest P = 0.03 for all paired comparisons for Z = 6–11.5). Seed‐based task fMRI overlap results as a function of fixed number of voxels thresholding and of Z ‐thresholding are given respectively in Figures B and C in Supporting Information. We conclude that the higher reliability found for task fMRI reflects properties of the underlying BOLD signal. M1 fMRI Results The percentage of voxels located in M1 relative to the whole brain (anatomically defined in the precentral gyrus) for task and task‐free fMRI was calculated to investigate whether the most active voxels are always located in M1. Different thresholds were used evaluating a range from 50 to 10,000 voxels. The seed region used to model motor task‐free functional connectivity was excluded from this analysis. In Table we give an overview of these results and show that: (a) there is no difference between Task and task‐free fMRI in the percentage of voxels located in M1 at different thresholds ( P > 0.05); (b) the higher the threshold, the higher the percentage of active voxels in M1 for both techniques (both P < 0.001). Activation patterns of two individuals using at a threshold of 100 voxels are shown in Figure for task fMRI, and in Figure for task‐free fMRI. Anatomically evaluated, the highest active voxels are found in the handknob for task fMRI [Tegeler et al., ; Yousry et al., ], and in bilateral M1 for task‐free fMRI [van den Heuvel and Hulshoff Pol, ]. These results support the assumption that the most active voxels of the motor network are located in “critical” brain areas for motor task performance [Beisteiner et al., ]. In Supporting Information, two individual results are shown at a threshold of 20 voxels for task fMRI (Fig. A), and 50 voxels for task‐free fMRI (Fig. B). The percentage of voxels located in M1 in each scan session for task and task‐free fMRI at different supra‐threshold voxels Task fMRI Task‐free fMRI Session 1 Session 2 Session 1 Session 2 50 Mean 64 66 63 65 SD 13 13 20 21 100 Mean 58 58 55 58 SD 9 10 19 20 500 Mean 43 44 39 41 SD 6 8 13 14 1,000 Mean 36 37 33 35 SD 6 7 10 11 5,000 Mean 21 22 21 22 SD 2 3 5 5 10,000 Mean 17 18 17 18 SD 1 1 3 2 Units: number of voxels. Units: %. DISCUSSION The present study investigates the between session reproducibility of individual functional motor brain maps identified with task and task‐free fMRI. Results show that task fMRI has better reproducibility than task‐free fMRI for motor network identification. Task fMRI ICC values are higher than those of task‐free fMRI (Fig. ), and overlap values using thresholds exceeded by a fixed number of voxels are higher for task FMRI as compared to task‐free fMRI (Fig. ). Furthermore, overlap improved for task fMRI at higher thresholds, while it decreased for task‐free fMRI. Finally, the percentage of voxels active in M1 (relative to the whole brain) was comparable for the two fMRI techniques, and showed a higher percentage of voxels in M1 at higher thresholds (Table ). Both reliability measures showed that task‐free fMRI identifies the functional motor network less reliably than task fMRI. Our results therefore suggest that caution must be taken before applying task‐free fMRI in clinical practice. These results contrast with a previous study that suggested that task‐free fMRI was more reliable than task fMRI in identifying the functional motor network [Zhang et al., ]. The difference may be associated with the fact that Zhang et al. [ ] assessed reliability qualitatively and the number of scans they acquired during resting state was greater than during task performance. The present study compares the reliability of the task‐free motor network to task fMRI quantitatively in the same individuals. Furthermore, we report similar task‐free ICC and overlap results to a previous resting‐state study that investigated the reliability of the supplementary motor area (SMA) network [Shehzad et al., ]. Though Shehzad et al. [ ] suggest that resting‐state functional connectivity is robust and reliable, we find it to be less reliable than task‐induced activity. The higher reliability found for task fMRI more likely reflects properties of the underlying BOLD signal: driving the motor system with an explicit task results in greater fluctuations in the BOLD signal that can be mapped more reliably. The lower reliability of the resting‐state functional connectivity is not due to the fact that task‐free fMRI scans were always acquired later than the task fMRI scans. Though scans acquired late in time may have a reduced tSNR and may be confounded by fatigue effects such as excessive movement or sleep during the scanning [Peltier et al., ; Waites et al., ], we found no such effects in our data as in a previous study by Fair et al. [ ]. Especially for higher statistical thresholds, task fMRI activity is more reliable than resting‐state connectivity. This result is in agreement with a previous resting‐state study that found lower intersession reliability in the SMA at higher than conventional thresholds [Shehzad et al., ]. This may be a reflection of a difference in the nature of the spatial pattern of the motor network during resting state compared to task fMRI. Although previous research shows that motor network connectivity displays a detailed topography during resting state [van den Heuvel and Hulshoff Pol, ], it may still be the case that the distinction in connectivity between neighboring voxels does not match the spatial details in the pattern of activation produced by task fMRI. In a less focal pattern (i.e., lower peaks and troughs in the activation pattern), the calculated overlap values would be relatively small even for the most active voxels. The lower overlap at higher thresholds for resting state compared to task fMRI may thus be explained by a less focal pattern of resting‐state connectivity. A less focal resting‐state connectivity pattern does not necessarily imply that its spatial distribution is too coarse for motor network identification. Just like for task fMRI, a higher percentage of voxels is found located in the primary motor cortex for task‐free fMRI at higher thresholds. This clearly indicates that task and task‐free fMRI show functional activity and connectivity in critical brain areas for motor task performance (see Supporting Information, Figs. A and B). Our results suggest that task‐free fMRI can properly identify the motor network. Furthermore, the spatial distribution of our connectivity maps is consistent with previous task‐free fMRI studies that investigated the motor functional network with a univariate analysis [Biswal et al., ; Cordes et al., ; Lowe et al., ; Xiong et al., ] (see Fig. ). Comparison of overlap results between the two fMRI techniques based on Z‐ thresholding is not straightforward. Different experimental paradigms have different sensitivity and produce (throughout the entire brain) different Z ‐values. We found higher Z ‐values for the task‐free motor network as compared to the task‐related network. Due to sensitivity differences, using a single conventional Z‐ threshold for both paradigms is not an adequate comparison because a different number of voxels is categorized as active. Such a difference could systematically bias reliability estimates. The current experiment showed similar reproducibility between the two techniques at a single conventional Z ‐threshold (e.g., Z = 5) (Fig. ). However, thresholding from 10 to 5,500 of the most active voxels clearly showed better overlap results of task fMRI as compared to task‐free fMRI. In turn, these thresholds based on the number of voxels corresponded to high and conventional Z‐ thresholding. Comparing the paradigms at a threshold where a fixed number of voxels is active, thus, provides more detailed and useful information. Importantly, this approach is attractive for clinical application, given that activity needs to be localized rather than proven to occur. For scientific purposes, statistical thresholding is indicated as the null hypothesis typically states absence of activity. An alternative option to the seed‐based analysis used in this experiment is independent component analysis (ICA) [Beckmann et al., ; Kiviniemi et al., ; van de Ven et al., ]. ICA does not rely on a priori seeding, and seems ideal to localize the motor functional networks in healthy volunteers and patients with altered brain functional topography [Kokkonen et al., ; Zhang et al., ]. However, this method has its own limitations such as an occasional failure to separate different functional networks from each other [Zuo et al., ] or from noise [Birn et al., ], and being subjective and labor intensive in selecting the functional network of interest from others [Zhang et al., ]. Nevertheless, the reliability levels obtained in the present study are very similar to test‐retest studies that have used ICA [Meindl et al., ; Zuo et al., ]. The findings about the location of the most active voxels are of particular relevance for the use of task and task‐free fMRI in clinical practice. Even for the well known functional motor network, a major challenge for the neurosurgeon during presurgical planning is to disentangle critical brain areas from non‐critical areas when relying on conventional Z ‐thresholding [Beisteiner et al., ; Rutten and Ramsey, ]. While this is complicated for task fMRI, it is even more so for task‐free fMRI considering the size of the volume categorized as active [Cordes et al., ; Xiong et al., ]. Our findings support the use of higher fMRI thresholds for clinical purposes because a high percentage of voxels are located in the primary motor cortex for both techniques. However, at higher thresholds the task fMRI voxels showed to be more reliable than the task‐free ones. Therefore, caution should be taken in using task‐free fMRI as a substitute of task fMRI for diagnostic examinations [Brannen et al., ; Fernandez et al., ], and for comparisons between healthy volunteers and patients [Manoach et al., ]. The reproducibility results reported here are dependent on the method used to analyze the functional brain maps. The analysis used has been shown to be efficient in previous test‐retest studies [Smith et al., ]. However, differences in pre‐ and postprocessing between the task and task‐free fMRI may bias the estimation of their reproducibility [Miki et al., ]. Because we use a seed region based on task fMRI for the resting‐state analysis [Cordes et al., ], the reliability of task‐free may be even overestimated compared to clinical situations where no task fMRI data are available. While we take a clinical perspective in this study and consider smoothing inappropriate [Rutten et al., ], smoothing may still be beneficial for test‐retest purposes as it gives adhesion to a less focal resting‐state connectivity pattern [Rombouts et al., ]. CONCLUSIONS A rigorous test‐retest comparison showed task fMRI better reproducible than task‐free fMRI for motor network identification in healthy subjects. Task fMRI ICC values were higher than those of task‐free fMRI. Overlap values were higher for task fMRI as compared to task‐free fMRI when thresholds exceeded by a fixed number of voxels were used. Overlap further improved for task fMRI at higher thresholds, while it decreased for task‐free fMRI. This is of relevance considering that the most active voxels of the motor network are located in critical brain areas for motor task performance. The reliability results presented here suggest that caution must be taken before applying task‐free fMRI in clinical practice as a replacement of task fMRI. However, resting‐state fMRI may be still suitable for patients with task compliance problems because, just like task fMRI, it can properly identify the motor network. As it is not clear whether these findings generalize to other brain systems, future studies should address the reliability of other resting‐state networks relevant for neurology and neurosurgery. ACKNOWLEDGMENTS Authors thank the following researchers at the UMC in Utrecht, The Netherlands: M. P. van den Heuvel and S. F. W. Neggers for helpful discussions; T. E. Gladwin, M. van Buuren, and M. Vink, for making use of Aztec ( http://www.ni‐utrecht.nl/downloads/aztec ).

Journal

Human Brain Mapping – Wiley

Published: Jan 1, 2014

Keywords: ; ; ; ; ; ;

There are no references for this article.