The authors have declared that no competing interests exist.
How do we choose a particular action among equally valid alternatives? Nonhuman primate findings have shown that decision-making implicates modulations in unit firing rates and local field potentials (LFPs) across frontal and parietal cortices. Yet the electrophysiological brain mechanisms that underlie free choice in humans remain ill defined. Here, we address this question using rare intracerebral electroencephalography (EEG) recordings in surgical epilepsy patients performing a delayed oculomotor decision task. We find that the temporal dynamics of high-gamma (HG, 60–140 Hz) neural activity in distinct frontal and parietal brain areas robustly discriminate free choice from instructed saccade planning at the level of single trials. Classification analysis was applied to the LFP signals to isolate decision-related activity from sensory and motor planning processes. Compared with instructed saccades, free-choice trials exhibited delayed and longer-lasting HG activity during the delay period. The temporal dynamics of the decision-specific sustained HG activity indexed the unfolding of a deliberation process, rather than memory maintenance. Taken together, these findings provide the first direct electrophysiological evidence in humans for the role of sustained high-frequency neural activation in frontoparietal cortex in mediating the intrinsically driven process of freely choosing among competing behavioral alternatives.
How do we choose a particular action among equally valid alternatives? Intracranial recordings in humans performing a delayed oculomotor decision task reveal that deliberating between competing alternatives during free choice is indexed by sustained high frequency neural activity in a network of parieto-frontal brain areas.
Deciding where to look to explore the visual world, i.e., picking one out of many alternative targets is a crucial aspect of our daily interactions with the environment. Exploring the neural mechanisms underlying eye-movement control provides a promising approach for learning about sensorimotor and cognitive aspects of voluntary action selection and planning [1]. Studies in nonhuman primates have extensively described the temporal dynamics of spiking activity and local field potentials (LFPs) in frontoparietal areas when animals perform delayed oculomotor response tasks. Planning an instructed saccade involves the same effector specific circuits that execute eye movements, namely, the frontal eye fields (FEFs) [2,3] and the lateral intraparietal area (LIP), [4–8], as well as a characteristic sustained neuronal activity in frontal areas, representing information about stimuli maintained in working memory during the delay period [9–11]. On the other hand, although monkey studies have demonstrated that free-choice decision processes (e.g., choosing between 2 equal reward options) appear to be represented in an effector specific frontoparietal network [12–21], the temporal dynamics of the neural correlates of free-choice decisions compared with those underlying instructed planning are poorly understood.
In humans, behavioral and neural signatures of voluntary, free choice have been studied extensively (see reference [22] for review). Converging evidence from neuroimaging studies suggests that the neural processes that mediate saccade decisions, planning, and execution arise across large-scale brain networks that involve parietal, frontal, and motor cortices [1,23–25]. A parietal oculomotor field (PEF), located in the posterior part of the parietal cortex (which is thought to correspond to LIP in monkeys) [26], seems to be principally implicated in triggering reflex saccades. By contrast, the FEF is thought to play a central role in preparation of the saccades by coding both the motor preparation and the intention to make a saccade [27–33]. Lastly, functional magnetic resonance imaging (fMRI) studies have shown that internally driven decisions were associated with greater activation of a neural system involving premotor (and particularly, the caudal presupplementary motor area [preSMA]) and prefrontal areas such as the medial and dorsolateral prefrontal cortex [34–37]. In a study by Rowe and colleagues, the authors showed that the prefrontal cortex was involved in both response selection and maintenance within working memory during a ‘‘free-selection” task [38]. Importantly, 2 fMRI studies using the same delayed saccade task used in this article have specifically shown that voluntary saccades were preceded by activation in the dorsolateral prefrontal cortex (DLPFC) and in the FEF, suggesting the involvement of these areas in the process of choosing where to look when facing 2 possible visual targets [39,40]. However, fMRI unfortunately cannot resolve the precise temporal dynamics of activity in these brain areas, nor can it probe the role of rhythmic brain activity. To address these questions, electrophysiological investigations are required.
Noninvasive electrophysiological studies have demonstrated the involvement of high-frequency neuronal oscillations in several areas (“eye or oculomotor fields”) of the cerebral cortex during saccade planning and execution using techniques such as magnetoencephalography (MEG) and electropencephalography (EEG) [41–45]. Of note, 2 MEG studies showed increased medial frontal gamma power during the response competition in delayed motor tasks [41,46], and one EEG study found differences in the P300 event-related component during action selection when comparing free choice and instructed planning [47]. Despite being extremely insightful, noninvasive techniques have several limitations in terms of signal quality, spatial resolution, and sensitivity to artifacts. Fortunately, it is possible in some rare cases to access invasive electrophysiological recordings in humans (e.g., surgical epilepsy patients) and thus, probe task-based changes via direct LFP recordings. The latter reflect the synchronized postsynaptic potentials of local populations of neurons [48,49] and allow for direct comparisons between invasive recordings of population-level activity in human and nonhuman primates. A handful of studies have benefited from direct recordings of neural activity (e.g., in human FEF and DLPFC) to probe neural activation in the frontal eye fields during saccade execution (peri-saccade activity) in humans using intracranial EEG [50–52]. Importantly, Lachaux and colleagues [50] found that the preparation and the generation of saccades were subserved by focal and transient increases in high-gamma (HG) activity (above 60 Hz) in the FEF. Yet, to our knowledge, no study has so far investigated the neural correlates of oculomotor decisions (i.e., free-choice saccades) using direct intracranial recordings in humans.
Taken together, previous findings from oculomotor and decision-making studies in human and nonhuman primates provide converging evidence for the central role of high-frequency LFP components in eye-movement selection and execution. Although, some evidence from noninvasive studies partly support these observations in humans, direct electrophysiological measurements are necessary to bridge the gap between human and nonhuman primate literature on oculomotor decision-making. In the present study, we probe for the first time the temporal, spectral, and spatial characteristics of human cortical networks engaged in the selection, planning, and execution of saccades with unprecedented resolution thanks to multisite intracerebral EEG (iEEG) recordings. In particular, we set out to test the predictions that (1) the temporal dynamics of delay-period LFP would differ between instructed and self-chosen saccade trials and that (2) the most prominent differences will be visible in high-frequency LFP components in key frontal and parietal areas.
In brief, we found that the intrinsically driven process of selecting among competing behavioral alternatives during free-choice decisions is associated with sustained increases of broadband HG (60–140 Hz) activity in distinct frontal and parietal areas that might reflect the process of deliberation between competing alternatives. By contrast, instructed saccade trials were associated with short-lived transient HG increases. The unique intracerebral recordings reported here provide important insights into the spatiotemporal characteristics of the neural patterns underlying free choice and help bridge the gap with previous animal electrophysiology and noninvasive studies in humans.
Six participants (4 females, mean age 30.3 ± 9.6, see Material and methods and Fig 1B and 1C) performed a delayed saccade task (Fig 1A) while electrophysiological data were recorded from multilead EEG depth electrodes. In each trial, participants were instructed to perform horizontal saccades toward 1 of 2 targets but only after a variable delay period. The information about saccade direction was indicated by a visually presented central cue (Cue 1), followed by a saccade execution Go signal (Cue 2). The task consisted of 3 interleaved experimental conditions (Fig 1A): In the Free condition, a diamond at Cue 1 prompted the participants to freely choose the direction of the forthcoming saccade. In the Instructed condition, an arrow pointing left or right indicated to the participants the direction of the saccade they were to prepare. After a variable delay (3.5–7.75 seconds) during which the participants prepared the adequate saccade while fixating the central fixation point, a Go signal (Cue 2) prompted the participants to immediately execute the saccade. In the Control condition, participants were presented with a square at Cue 1, indicating that they would need to wait for the Go signal (Cue 2) to find out the required saccade direction and execute it immediately. Behavioral saccade onset latency data were collected, and spectral power features were extracted from the iEEG data across multiple time windows and all electrode sites. Power features were computed in 5 standard frequency bands: theta (θ) [4–8 Hz], alpha (α) [8–15 Hz], beta (β) [16–30 Hz], low-gamma (low γ) [30–60 Hz] and high gamma (high γ, HG) [60–140 Hz]. A supervised machine learning framework was implemented to decode (through space, time, and frequency) the experimental conditions (free, instructed, and control) and thereby identify the most discriminant neural patterns that distinguish between free-choice and instructed actions during saccade planning and execution (see Material and methods for details).
Experimental design and distribution of intracranial electrode contacts across participants.
A. Experimental design of the delayed motor task. For each trial, participants were instructed to perform horizontal saccades toward one of 2 targets after a delay of 3,750 milliseconds, 5,750 milliseconds or 7,750 milliseconds, depending on a visually presented central cue appearing briefly for 250 milliseconds. B. Top, left, and right views of the number of recording sites that contribute to each vertex (i.e., spatial density) projected on a standard 3D MNI brain. Electrodes contribute to a location when they are within 10 mm of a given site on the brain surface. In all brain images, right side of the image is the right side of the brain. C. Top, left, and right view of the depth-electrode recording sites, projected on a standard 3D MNI brain. Each color represents a participant. Left: Rostral is up; Right: Medial views. D. Barplot of mean reaction times for the 3 conditions across all participants (Control, Instructed, Free). Each triangle represents the mean reaction times for 1 participant. The data underlying this panel D can be found in S1 Data. MNI, Montreal Neurological Institute.
We computed the mean reaction times (RTs, i.e., saccade onset latency, see Material and methods) for each experimental condition across all participants and found that mean RTs were significantly longer for the Control condition (mean RT = 466 ± 66 milliseconds) compared with both Free (mean RT = 334 ± 36 milliseconds; t(6) = 2.75, p = 0.0403) and Instructed (mean RT = 321 ± 33 milliseconds; t(6) = 2.68, p = 0.0435) conditions (see Fig 1D). No significant differences were found between Free and Instructed conditions (t(6) = 1.31, p = 0.24). These results were also observed at the single participant level in 5 out of 6 participants (see Material and methods section). These results are consistent with the fact that the availability of saccade target information (whether self-generated or instructed) during the delay period allowed the participants to plan the upcoming saccades and hence, execute them faster upon the Go signal compared with the Control condition, in which no directional information was available during the delay period. Mean saccade duration, saccade speed, mean latency, and the number of saccades executed per condition by each participant are reported in S2 Table.
To assess modulations of the neural activity across the 3 delayed saccade trial types (i.e., Free, Instructed, and Control) over space, frequency, and time, we computed time-frequency representations (locked either on stimulus onset, i.e., Cue 1, or saccade execution cue, i.e., Cue 2), as well as single-trial spectral amplitude envelopes in multiple frequency bands (theta (θ) [4–8 Hz], alpha (α) [8–15 Hz], beta (β) [16–30 Hz], low-gamma (low γ) [30–60 Hz] and high gamma (high γ, HG) [60–140 Hz]). Fig 2 illustrates these feature computations and the high quality of the intracranial data by showing time-frequency maps derived from electrodes in FEF and intraparietal sulcus (IPS) in participant 2, as well as single-trial HG activity, aligned to stimulus presentation and to saccade Go signal (ordered by saccade onset latencies). In addition, we used linear discriminant analysis (LDA) to probe the ability of these spectral features to decode experimental conditions from single-trial data. Importantly, we applied this machine learning framework individually to data from each recording site and in a time-resolved manner over the course of the task (see Material and methods section for details).
Illustrative time-frequency maps and single-trial HG activity in FEF and IPS.
Time-frequency maps (left) and single-trial HG plots (right) from 2 recording sites in an illustrative participant (P2). Data are shown for the 3 experimental conditions (Control, Instructed, and Free), during planning (Cue 1, stimulus onset), and execution (Cue 2, go signal). Trials in the single-trial gamma plots are sorted according to saccade onset latencies. FEF, frontal eye field; HG, high-gamma; IPS, intraparietal sulcus; Modul., modulations; Rel., relative.
To identify the neural patterns related to making autonomous choices, we first compared the delay-period neural responses observed during free-choice saccade trials to those recorded during instructed saccade trials. This was conducted by applying LDA to classify Free versus Instructed saccade trials based on spectral amplitude estimated during the delay interval in all 5 frequency bands (Fig 3, see also S4 Fig). Panels A-C of Fig 3 show that, among all frequency bands, HG activity was the neural feature that provided the highest decoding accuracy (DA) and largest number of significantly decoding sites when classifying Free versus Instructed trials, during the delay period ([0; 3,000 milliseconds] after Cue 1). The HG activity led to statistically significant classification in 61 sites (4 out of 6 participants) and yielded a maximum DA of 92.9% and a mean DA of 79% (Fig 3A–3C). Interestingly, out of the 2 participants that did not yield any significantly decoding sites, P5 was the only participant that showed similar RTs across the 3 conditions (see Fig 1D). This might reflect the fact that this participant did not make use of Cue 1, which would explain why HG activity could not significantly decode Free versus Instructed conditions during the delay period in this participant.
Single-trial classification of Free versus Instructed trials based on delay-period HG activity.
A. Summary of all significant electrodes by participant across frequencies showing that the largest clusters were found in the HG frequency band. B. Mean and C. Maximum decoding accuracies across participants and significant electrodes for each frequency band for Free versus Instructed classification (error bars represent SEM). D. Time course of baseline corrected (−500 to−100 milliseconds) HG activity aligned on Cue 1, for all electrodes that significantly classify Free versus Instructed conditions and H. its associated mean decoding accuracy across significant electrodes. E. Maximum decoding accuracies across participants and significant electrodes for each frequency bands for Free versus Instructed multielectrode classification. F. Relative mean HG peak activity (in %) and G. latency (in milliseconds) for electrodes significantly decoding Free versus Instructed conditions during the delay period (from 0 to 3,000 milliseconds after Cue 1). I. Decoding Free versus Instructed conditions with HG activity in 5 successive time windows during the delay period (0 to 500 milliseconds; 500 to 1,000 milliseconds; 1,000 to 1,500 milliseconds; 1,500 to 2,000 milliseconds; 2,000 to 3,500 milliseconds after Cue 1, and −2,000 to 0 milliseconds before Cue 2). Only sites with significant decoding accuracies are shown (p < 0.01, with max stats correction across electrodes, time, and frequency bands). J. Percent relative power change ([Free − Instructed]/Instructed) for all significant sites shown in panel I. The data underlying this Figure can be found in S1 Data. DA, decoding accuracy; elec., electrodes; Freq., frequency; HG, high-gamma; Inst., instructed; Nb, number; Rel., relative.
We then used a multifeature classification approach (Free versus Instructed trials) in which observations across all electrode sites were now included simultaneously in the decoding feature space (repeated for each frequency band). We assessed the statistical significance of time-resolved DA using permutation tests, corrected for multiple comparisons across participants (electrodes, frequencies, and time points). As shown in Fig 3E, the multisite DA was highest for HG activity, reaching 86.8%. Given that both single and multisite classification results (Fig 3A–3C and 3E) indicate that HG amplitude is the most prominent predictor of target class (Free versus Instructed), the next sections of the results focus on the characterization of the fine-grained temporal and spatial profiles of HG neural decoding.
Averaging the HG data across all trials from all significantly decoding sites illustrates the temporal dynamics of delay HG activity that distinguish between Free and Instructed saccade trials (Fig 3D). The associated time-resolved mean DA is shown in Fig 3H. Because the analysis is based on averaging across all sites, panels D and H only provide a schematic representation of the temporal dynamics, without statistical assessment. Thus, we conducted standard paired t tests to further quantify the difference in HG peak amplitudes and latencies between Instructed (mean peak HG amplitude = +39% ± 0.74; mean peak latency = 475 milliseconds ± 14) and Free (mean peak HG amplitude = +24% ± 1.18; mean peak latency = 812 milliseconds ± 97) conditions from all 61 significantly decoding sites (in 4 out of 6 participants). The results revealed significant differences (peak amplitude: t(4) = 8.34, p < 0.003, Fig 3F; peak latency: t(4) = 21, p < 0.0002, Fig 3G) and were also confirmed in single-trial analyses performed individually in each of the 4 participants (p < 0.05, see Material and methods). All the statistical results within and across participants are listed in S3 Table.
Additionally, Fig 3I and 3J represent HG decoding dynamics resolved across both space and time: In the early part of the delay period, significant decoding electrodes are associated with stronger HG power in the Instructed than in the Free condition. But over time, HG power then becomes higher for the Free saccade planning than for Instructed saccade planning during later stages of the delay period in frontoparietal brain areas. More specifically, we show that all significant electrodes in the (0, 500 milliseconds) time-window after Cue 1 during the delay period are associated with higher HG activity in the Instructed condition (Fig 3I and 3J, [0, 500 milliseconds]). On the other hand, as time goes by, significant electrodes begin to become more associated with higher HG activity in the Free compared with the Instructed condition. From 1,000 to 2,000 milliseconds after Cue 1, we see that all significant electrodes are now associated with higher power in the Free condition (Fig 3I and 3J, [1,000, 1,500 milliseconds], [1,500, 2,000 milliseconds]). Additionally, we find that from 1,500 milliseconds to 2,000 milliseconds after Cue 1, the only electrodes that still significantly decode Free from Instructed conditions are located in frontal regions (Fig 3I and 3J; [1,500, 2,000 milliseconds]). Interestingly, we also found sites for which HG activity was still significantly stronger in Free than Instructed, at the end of the delay period, from −2,000 to 0 milliseconds before Cue 2 (Fig 3I and 3J). This suggests that some electrodes may display persistent activity lasting throughout the whole delay period and led to subsequent analyses described in more detail in the following sections.
In order to further characterize the temporal HG dynamics specific to Free and to Instructed saccades (beyond the strict difference between the two) while also taking into account low-level stimulus-related processes, we replicated the same decoding framework but now with the goal of distinguishing each of the main 2 conditions from the Control condition (i.e., Instructed versus Control, and Free versus Control, see Fig 4). First, we found that over the first 3,000 milliseconds after Cue 1, there was an overlap of 56 electrodes between the electrodes that significantly classify Instructed versus Control trials as well as Free versus Control trials. In other words, 82.4% of the sites that significantly discriminate Free versus Control conditions also significantly discriminate Instructed versus Control conditions (Fig 4A). Furthermore, we found that when participants freely chose the saccade direction, the delay HG activity lasted, on average, 618 milliseconds ± 57, whereas the instructed saccade condition displayed mean HG durations of only 368 milliseconds ± 60. The difference was statistically significant (length of time points above the significance threshold in Free versus Control compared with Instructed versus Control classifications, t(4) = 3.52, p < 0.04 across 4 participants and confirmed in intraparticipant analysis in 1 individual with p < 0.05, see Fig 4B). We also found that HG activity during Instructed saccade planning (mean onset = 152 milliseconds ± 39) reached significant classification earlier than HG activity in the free-choice condition (mean onset = 465 milliseconds ± 49) when compared with the control (latency of first significant decoding accuracies in Free versus Control compared with those associated with Instructed versus Control classifications, t(4) = 7.09, p < 0.006 across 4 participants, see Fig 4C). This difference was confirmed with intraparticipant analyses in 3 out of 4 participants with p < 0.05. Lastly, we show that DA peaked significantly earlier in the Instructed condition (mean onset = 527 milliseconds ± 61) than in the Free condition (mean onset = 822 milliseconds ± 70) when compared with the Control condition (Peak DA latencies in Free versus Control compared with Instructed versus Control classifications, t(5) = 3.39, p < 0.03 across 5 participants, confirmed with intraparticipant analyses in 4 out of 5 participants with p < 0.05, see Fig 4D). All the statistical results within and across participants are listed in S3 Table.
Temporal dynamics of HG activity during Free and Instructed (versus Control) conditions during the delay period.
A. Location of electrode sites where HG activity discriminates Free versus Control and/or Instructed versus Control mapped on transparent 3D brain images for all participants (p < 0.01, corrected). Left: electrodes colored in green, blue, and yellow, respectively, indicate sites that discriminate Free versus Control trials only, Instructed versus Control only, or both Free versus Control and Instructed versus Control during the delay period (0 to 3,000 milliseconds after Cue 1). Right: colors indicate different participants. B. Duration (length of time points) above the significance threshold C. Decoding onset (i.e., latency of first significant decoding accuracies) D. Latency of the peak decoding accuracies (in milliseconds) for sites significantly decode Free versus Control (in green) and Instructed versus Control (in blue) across participants. E, F. Time course of baseline corrected (−500 to −100 milliseconds) HG activity aligned on Cue 1, for all electrodes that significantly classify Instructed versus Control (E) and Free versus Control (F) conditions, and G, H. Their associated mean decoding accuracy across significant electrodes in time, respectively. I. Temporal generalization of trial-type decoding using HG activity across significant sites derived from the previous analyses (Free versus Control and Instructed versus Control) during the delay period (0 to 3,000 milliseconds after Cue 1) for 4 participants. Generalization matrices show decoding performance plotted as a function of training time (vertical axis) and testing time (horizontal axis). Decoding of Instructed versus Control (left column) trials illustrates the expected profile for transient coding, while decoding of Free versus Control (right column) trials leads to smoother and extended decoding patterns, typical of a single process that is sustained over time. The data underlying this Figure can be found in S1 Data. DA, decoding accuracy; HG, high-gamma; Inst., Instructed; Nb, Number; Rel., Relative.
In order to further characterize the fine temporal organization of information-processing during the delay period in the Instructed and Free-choice conditions, we probed cross-temporal generalization [11,53] of decoding Instructed versus Control and Free versus Control conditions using HG activity (Fig 4I). In brief, temporal generalization consists in training a classifier with data from one time point, t1, and testing it on data from a different time point, t2. In principle, cross-temporal generalization indicates that the neural code identified at t1 also occurs at t2 (see Material and methods). More specifically, we used temporal generalization to better characterize short-lived (transient) and longer-lasting (sustained) HG activity processes underlying Free and Instructed planning. Our findings show that HG activity during instructed saccade planning yields a generalization pattern typical of transient coding (see the first column of Fig 4I), whereas free choice is characterized by a HG decoding process that is more sustained in time (second column of Fig 4I). Taken together, the observed cross-temporal decoding patterns and their accuracies are consistent with the view that decision-related discriminant HG activity during the delay period is more sustained and starts later in Free-choice trials compared with Instructed saccade trials. In contrast, when no choice is involved, task-related information reflected in HG activity is more transient and most relevant shortly after stimulus onset. Importantly, the cross-temporal generalization results also highlight that although HG decoding in the Free choice is more sustained, it does not last systematically the entire duration of the delay period until the GO signal (Cue 2). This is consistent with our earlier observations in Fig 3I and 3J, that over the course of the delay period, fewer sites discriminate between free and instructed trials. This may suggest that although several sites display sustained HG activity for free-choice saccade trials, only a few actually display persistent increases up until saccade execution. This important distinction is probed in the next section.
To specifically isolate the brain areas where HG increases index neural processing specific to free saccade decisions, we used a conjunction analysis (Free > Control Ո Free > Instructed) applied to all electrode sites with significant classification of Free versus control and Free versus Instructed trials. Importantly, we conducted this conjunction analysis in 2 distinct time windows during the delay period: An “early” window was defined as the first 2,000 milliseconds after Cue1, and a “late” window from −2,000 to 0 milliseconds before Cue 2. Fig 5A depicts for both time windows the sites with significant decoding accuracies for all participants in which the increase in HG activity was stronger during free choice compared both with control (Free > Control) and instructed (Free > Instructed) trials (see also S5 Fig for more details). Fig 5B combines the results for early and late into a common representation (35 electrodes, 4 participants) indicating, thereby, for each free-choice specific site whether it showed HG increases only in the early (0 to 2,000 milliseconds after Cue 1), only in late (−2,000 to 0 milliseconds before Cue 2) or in both early and late intervals. We found that most free-choice-specific sites exhibited enhanced HG activity only during the early part of the delay period (29 electrodes, 3/4 participants) in a network of regions including superior frontal gyrus (SFG), middle frontal gyrus (MFG), SMA, IPS, and FEF (see Fig 5C and 5D, first 2 rows for illustrative early electrodes in IPS and SFG). However, for 5 sites (in 2/4 participants) located in SFG (2 electrodes), MFG (2 electrodes), and FEF (1 electrode), significant decoding accuracies were found both in the early and late parts of the delay period, indicating HG activity spanning the entire duration of the delay period (see Fig 5C and 5D, last row for an illustrative early + late electrode in SFG). The localization details of all 35 electrodes depicted in Fig 5B are provided in S4 Table. Note that, among all participants, the electrode site with maximum HG DA for both Free versus Control (86.1%) and Free versus Instructed (91.4%) was located in the right IPS (P2, electrode derivation p9-p8, see Fig 5, last row of C, D).
Early and late free-choice-specific HG activity.
A, Electrode sites with significant decoding accuracies (p < 0.01, corrected) for all participants mapped on transparent 3D brain images when HG activity is significantly stronger in the Free condition than in the Control condition (first row) and when HG activity is significantly stronger in the Free condition than in the Instructed condition (second row) during the delay period, from 0 to 2,000 milliseconds after Cue 1 (first column, early) and from −2,000 milliseconds before Cue 2 (second column, late). B. Electrode sites where HG is higher in Free compared with Instructed and Control, determined by a conjunction analysis (Free > Control U Free > Instructed). Free-choice-specific sites are colored in blue if significant decoding was observed in the early part of the delay; in yellow if significant decoding was found in the late part; and in green for sites that survived the conjunction analysis both in early and late phases of the delay period. For 3 individual electrodes, we plotted HG activity over time (C, The data underlying this panel can be found in S1 Data), single-trial plots (D, upper row) and time-frequency-maps (D, lower row) for Free, Instructed, and Control conditions. DA, decoding accuracy; Freq., frequency; HG, high-gamma; IPS, intraparietal sulcus; MFG, middle frontal gyrus; modul., modulations; Rel., relative; SFG, superior frontal gyrus.
To further appreciate individual participant contributions to the global findings, we also analyzed all electrode sites that survived the conjunction analysis in Fig 5B, grouping the data either by regions of interest (ROIs) or delay-period window (early or late). The results (Fig 6) largely speak to the similarity of temporal HG dynamics across regions and conditions. Data from P2 indicate that the Control condition elicits HG responses in IPS but not in MFG and that the strongest and longer-lasting HG responses in MFG comes from the Free-choice trials (Fig 6A and 6B). This is consistent with an involvement of parietal regions—among other things—in low-level sensory processing and a prominent role of frontal HG activity in deliberation. The distinction between merely “longer-lasting” HG activity (early) and “persistent” HG activity throughout the delay period (early and late) is shown in Fig 6C. We then conducted an analysis to evaluate trial history effects during the free-choice condition. We used an unpaired t test to evaluate statistical differences between n-1 conditional probabilities and random (history-free) probabilities for each participant and found no obvious across-trial dependences in choice behavior during the free condition for 5/6 subjects (see S1 Fig). For one participant (P2), we found a significant trial history effect that was driven by a tendency to alternate behavior (e.g., left, right, left, right…). Interestingly, this participant showed the same delayed and sustained HG free-choice effects observed in the other participants, with the notable difference that the HG response was shorter-lived compared with other participants (Fig 6B). This result seems to be consistent with our interpretation that the sustained HG activity observed during the Free condition is indeed related to deliberation between competing alternatives in frontoparietal brain areas. Specifically, this analysis suggests that the tendency of P2 to alternate between left and right saccade choices was associated with shorter deliberation during the delay (indexed by shorter sustained HG responses). Note that although significant, the observed alternating behavior in P2 was not systematic (see conditional probabilities reported in S1 Fig). Lastly, to verify whether our findings could not be confounded by spatial tuning effects in the delay period, we replicated the classification analyses separately for left and right saccade trials. However, no significantly spatially tuned delay-period decoding was found in the Free, Instructed, or Control conditions (see S6 Fig). More specifically, the 3 illustrative sites shown in Fig 5C and 5D did not show statistically significant differences when trials for left and right saccades were investigated separately (see S6B–S6D Fig). We also ruled out the possibility that our findings were confounded by involuntary saccades made in response to the presentation of Cue 1 by contrasting mean EOG traces for left and right trials and for Free and Instructed conditions (see S3 Fig).
Mean HG activity time courses for free-choice-specific sites grouped here by (A) ROIs, (B) subjects, and (C) early/late. Mean time course of baseline corrected (−500 to −100 milliseconds) HG activity for Free, Instructed, and Control conditions aligned on Cue 1 (first column) and Cue 2 (second columns) in electrodes that have enhanced HG in the free-choice condition compared with both the control and instructed saccade conditions (i.e., determined by a conjunction analysis (see Fig 5B). The data underlying this figure can be found in S1 Data. HG, high-gamma; FEF, frontal eye field; IPS, intraparietal sulcus; MFG, middle frontal gyrus; Rel., relative; ROI, region of interest; SMA, supplementary motor area.
Previous reports using intracranial EEG in humans have shown that saccade execution in response to a Go signal is associated with distributed increases in HG power [50]. Yet it has been so far hard to determine whether such gamma activity reflects target selection, motor planning, actual oculomotor commands, or a combination thereof. Analyzing the execution component (cue 2) of the delayed saccade paradigm in the present study provides an opportunity to address some of these questions. Therefore, in the final analysis, we set out to compare HG responses induced by the Go signal in conditions in which target selection already occurred in the delay period (i.e., Free and Instructed) to the Control condition, in which participants were given no information on saccade target before the Go signal. To achieve this, we conducted a supervised classification analysis on Free, Instructed, and Control conditions, but this time using data collected during saccade execution (0 to 2,000 milliseconds after Cue 2). We found that, during saccade execution, the HG power modulations were similar whether the saccade was instructed or self-chosen (i.e., no significant classification between Free and Instructed). However, HG activity associated with saccade execution in the Control condition was significantly different from both Free and Instructed saccades (Fig 7A). These results are consistent with the fact that no significant difference in reaction time (saccade onset latency) were found between Free and Instructed conditions, whereas mean reaction time across participants was significantly longer for the Control condition compared with both Free and Instructed trials (see Fig 1D).
Single-trial HG activity decoding during saccade execution.
A. Electrodes with significant decoding accuracies (p < 0.01, corrected) for all participants are mapped on transparent 3D brain images when HG activity is significantly stronger in the Control condition than in the Free condition (first row) and when HG activity is significantly stronger in the Control condition than in the Instructed condition (second row) in the interval from 0 to 2,000 milliseconds after Cue 2. B. using a conjunction analysis (Control > Free Ո Control > Instructed), we show sites in which HG is stronger in the Control condition than in the Free and Instructed conditions. Colored sites correspond to 3 individual electrodes, for which we plotted HG activity over time (C, The data underlying this panel can be found in S1 Data), single-trial HG plots (sorted according to RTs) (D, upper row) and time-frequency-maps (D, lower row) for Free, Instructed, and Control conditions. DA, decoding accuracy; HG, high-gamma; FEF, frontal eye field; Freq., frequency; IPS, intraparietal sulcus; MFG, middle frontal gyrus; modul., modulations; Rel., relative; ROI, region of interest; SMA, supplementary motor area.
Next, we examined the trial-by-trial relationship between saccade onset latency and neural activity specifically in areas that exhibit these significant HG differences between the Control condition and the Free and Instructed conditions. To this end, we first used a conjunction analysis to identify the sites of interest defined as sites for which significant classification was mediated by HG power in the Control condition being higher than in the other 2 conditions (Fig 7B). This identified 28 electrodes in parietal and frontal regions (2/6 participants at p < 0.01). Interestingly, in 43% of these significant sites, the reverse pattern was true during the delay period: HG activity was significantly stronger in the Free and the Instructed conditions compared with the Control condition during the delay. Three representative examples of this task-specific HG pattern inversion between delay and execution windows are shown in Fig 7C (first column). This suggests the involvement of these HG responses in action selection processes: When such processes are engaged during the delay period during Free and Instructed conditions, they do not need to be repeated during the execution period. By contrast, in the Control condition, no action selection processes were possible during the delay period (hence, the weaker HG activity), but they were recruited at execution.
By plotting the mean time courses of HG power (Fig 7C), as well as mean time-frequency representations and single-trial HG power plots, sorted according to RT (Fig 7D), it becomes clear that the temporal dynamics of HG power differ quite substantially depending on electrode location and trial type. In order to probe the various relationships between saccade onset and HG activity in these areas in a quantitative manner, we computed Pearson's rank correlation coefficients between saccade onset latency and HG onset latency across trials in each of the 3 experimental conditions (see Material and methods). Significant correlations were observed in a limited number of sites, and the results need to be interpreted with caution. This said, we observed 3 correlation patterns that we considered to be of interest: For some sites, the onset of HG activity after the Go signal did not correlate with saccade onset in any of the 3 conditions (pattern 1, execution independent). Other sites exhibited correlations between HG onset and saccade onsets across all conditions (pattern 2, oculomotor execution). Finally, in the third pattern, correlations between HG onset and saccade onset latencies were only observed in Control trials (pattern 3, oculomotor planning). The recording sites that displayed these patterns came from distinct brain areas; we found evidence for pattern 1 (i.e., no correlation with saccade onsets in any condition) for the electrodes located in the IPS (4 sites, 1 participant) (e.g., Fig 7C and 7D, first row and S7 Fig) (e.g., Electrode p9-p8; Control [p = 0.62, r = −0.06], Free [p = 0.69, r = −0.06], and Instructed [p = 0.43, r = −0.12]). Next, pattern 2 (i.e., saccade and HG onset latencies significantly correlated in all 3 conditions) was observed in SMA (1 out of 4 electrodes) (e.g., Fig 7C and 7D second row and S7 Fig; Electrode b3-b2; Control [p = 0.001, r = 0.32], Free [p = 0.003, r = 0.29], and Instructed [p = 0.041, r = 0.2]). The fact that these correlations were present in the 3 conditions is consistent with the involvement of SMA in oculomotor execution. Thirdly, in the middle frontal gyri (1 out of 3 electrode), we observed the third pattern 3, namely that HG onsets were correlated with saccade onset latencies only in the Control condition (e.g., Fig 7C and 7D, last row and S7 Fig: Electrode m13-m12; Control [p = 0.04, r = 0.26], Free [p = 0.13, r = 0.26], and Instructed [p = 0.67, r = −0.07]). Despite being of interest, this observation is not surprising given that the relevant sites did not have significant HG response after cue 2 in the Free and Instructed conditions (e.g., third row of Fig 7C and 7D). The detailed list of HG and saccade onset correlations across the 3 conditions are available in S5 Table.
The present paper provides, to the best of our knowledge, the first investigation of the neural dynamics underlying oculomotor decision-making in the human brain using intracranial EEG. Our results confirm the hypothesis that oculomotor decision processes (free-choice saccades) in humans are associated with sustained enhances in the high-frequency component of population-level neuronal activity across a parieto-frontal circuit. Single-trial classification allowed for a fine-grained investigation of the spatial, temporal, and spectral properties of the neural dynamics underlying free-choice saccades. In particular, compared with lower-frequency components, broadband high-frequency (60–140 Hz) activity in parietal and frontal brain areas yielded the highest classification between free-choice and instructed saccade trials. Importantly, although the brain regions associated with both conditions were largely shared, the temporal dynamics of the HG activity during the delay period were distinct. The free-choice trials were associated with a delayed and more sustained induced HG response, whereas instructed saccade trials were characterized by an earlier transient HG response. Critically, the longer-lasting HG response in free-choice trials did not systematically persist throughout the duration of the delay period. This stands in stark contrast to the well-established dynamics of persistent neural activity typically associated with working memory during delayed motor tasks. As will be discussed in more detail, these findings expand previous noninvasive human studies and bridge the gap with animal investigations of free-choice tasks.
Compared with planning an instructed saccade, we found that freely deciding where to look is associated with a longer-lasting augmentation of broadband HG activity in the delay period. The enhanced HG activity induced in the free-choice trials spans a decision circuit that consists primarily of frontal and parietal brain areas. The large frequency bandwidth of the observed HG responses reported here (60–140 Hz) distinguishes the present results from investigations of narrow-band gamma oscillation. Broadband HG activity has been suggested to reflect a global enhancement of the local neuronal firing in the underlying cortical tissue and has been occasionally described as a “spike bleedthrough” in the power spectrum of the LFP signals [54–59]. Consistent with previous work, the HG activity reported here most likely reflects the cumulative discharge of local neuronal networks in superficial cortical layers [55,56,58–60]. Given previous reports showing tight correlations between neuronal firing and high-frequency components of the LFP, it is tempting to consider the observed HG activity in our study to be a marker of local neural activation or a correlate of neuronal firing. As such, the sustained HG activity specifically observed during the free-choice trials probably indexes an extended active deliberation process.
Both the trial-by-trial classification framework as well as standard trial-by-trial statistical tests indicate that what we refer to as longer-lasting or sustained HG activity (in free-choice trials) is not maintained up to saccade execution. In fact, in most of the probed parietal and frontal areas, the elevated HG activity ceases to be significant around 2,000 milliseconds after Cue 1, irrespective of the total duration of the delay period (3,750, 5,750, or 7,750 milliseconds). As revealed by our analysis contrasting early versus late delay-period activity, out of 34 sites that exhibited decision-specific HG enhancements within the first 2 seconds following stimulus onset, only 5 sites still showed significant HG increases in the last 2 seconds leading up to the Go cue. We argue that the sustained HG activity reported here differs from the more persistent patterns of neuronal responses reported in parietal and prefrontal areas in humans and monkeys performing working memory tasks, which generally remain elevated until movement execution [9–11]. Rather than reflecting information maintenance or successive changes of mind, we believe that the extended HG activity—observed specifically in free-choice trials—reflects the unfolding of internally driven action selection. This ongoing process of deliberation between competing alternatives extends over time until commitment to a saccade choice. Importantly, cross-temporal generalization decoding confirmed that HG activity during free-choice trials reflects a single, sustained process, rather than a dynamic coding phenomenon [11]. In other words, the off-diagonal cross-temporal decoding results argue against the view that the reported sustained HG activity actually is indicative of multiple, distinct successive processes [53]. More generally, we propose that the sustained HG response reported here is related to an ongoing free-choice process rather than to working memory. Indeed, participants had to maintain information about the direction of the upcoming saccade during the delay period whether it was self-determined (Free condition) or cued (Instructed condition). Therefore, working memory requirements are similar across both conditions and do not account for the observed increase in HG activity in free choice compared with instructed saccade trials. This is in line with the increasingly accepted view that persistent neuronal firing is a ubiquitous phenomenon observed broadly across many cortical and subcortical areas and that it reflects not only working memory maintenance but also a variety of other cognitive processes, including decision-making [61–64]. In the light of the aforementioned findings and existing theories of action selection [65–71], we argue that the observed sustained HG activity in frontal and parietal areas during free saccade trials in the present task may reflect sustained neuronal firing that helps maintain enhanced competition between various potential movement plans with equal rewarding outcomes compared with a single movement plan in instructed saccades [cf. 72] until commitment to a particular choice. This interpretation is consistent with the involvement of the frontoparietal network in implementing action selection (free choice) when maximally competing alternatives are present both in humans [73,74] and in monkeys [5,16].
The present study provides the first account of the neural dynamics underlying free-choice saccades using invasive recordings in humans. Although some of our findings provide critical confirmation of previous noninvasive research in this field, other observations extend or stand at odds with the noninvasive literature. First of all, an fMRI study based on the exact same paradigm used here found that the free-choice condition specifically activated the DLPFC, alongside supplementary eye fields (SEF), FEF, and IPS [75,76]. This spatial correspondence between these blood-oxygen-level-dependent (BOLD) responses and our iEEG HG activity in an identical task is expected and consistent with the view that broadband HG activity band modulations largely co-localize with BOLD variations in humans [77–83]. In addition to corroborating BOLD localization results through electrophysiological evidence, our findings expand the fMRI account by incorporating the temporal and frequency dynamics in key parietal and frontal nodes of the decision circuit. Our findings are also consistent with an event-related fMRI study by Curtis and colleagues [64] in which the authors found sustained BOLD responses in parietal and frontal areas during a delayed pro and antisaccade task. Importantly, the deferred saccade task they used eliminated the memory maintenance component inherent in memory-guided saccade tasks by keeping the visual cue present throughout the preparation interval. Their interpretation of the parieto-frontal BOLD responses as reflecting spatial selection and preparation of saccades, rather than working memory, is in line with our interpretations. However, the fact that their task did not include a free-choice saccade condition and the sluggish and delayed nature of the hemodynamic response (which peaks seconds after the neuronal response) limit direct comparisons between the temporal dynamics reported in their study using fMRI and the ones reported here using iEEG.
Investigating the electrophysiological correlates of oculomotor behavior with noninvasive methods is challenging because eye movements generate artifacts in EEG and MEG signals. However, by focusing on the delay activity preceding execution, a few MEG studies have reported compelling evidence for enhanced parieto-frontal gamma oscillations during saccade planning
Compared with guided saccades, autonomously choosing between competing alternatives has been shown with MEG to yield stronger sustained gamma increases that persist until movement execution [41]. Our findings are, in part, consistent with these MEG results. However, a notable difference with the latter findings is that their delay period was fixed and short (1 second), whereas our delay period varied from 3.75 to 7.75 seconds. This discrepancy probably explains why the gamma increase they report lasted the entire duration of the delay period, whereas we found that it drops around 2 to 2.5 seconds after stimulus onset. In fact, these important differences emphasize the importance of using long and variable delay periods to disentangle interpretations based on motor preparation or memory maintenance [9–11,53,84] from processes directly involved in choosing between alternatives. Further differences between previous MEG investigations and the present results can be due to the differences in number of alternative saccade targets. Arguably, with only left or right saccade choices, our experimental design places less emphasis on memory maintenance during the delay than tasks that involve, for example, 16 targets [41]. Moreover, the absence of a pure sensory control condition in other studies may also be a limiting factor that we were able to overcome in this study. The Control condition used here allowed us to disentangle low-level stimulus processing from internally generated plans. Overall, compared with previous noninvasive human studies, the spatial and temporal resolution of iEEG and its high signal-to-noise ratio allowed for a fine-grained investigation of the temporal, spectral, and spatial dynamics of the brain responses at play. Our ability to conduct these analyses up to frequencies of 140 Hz and at the single-trial level, is an important advantage when it comes to bridging the gap with animal research.
Comparisons of HG activity across trial types and correlation analyses between saccade onset and HG onset latencies revealed region-specific temporal patterns of activity during the execution period (Fig 7). We found an increase in HG activity specific to Control trials (i.e., the only condition during which participants did not plan a decision during the delay period) when compared with both Free and Instructed trials during the execution period. This finding implies the involvement of HG in oculomotor planning processes (including action selection and oculomotor preparation). Furthermore, we probed potential correlations between single-trial HG response onsets and saccade onsets. Significant correlations were scarce, and the results, although interesting, need to be considered with caution; in all IPS sites, HG power onset did not correlate with saccade onset latency in any of the 3 experimental conditions. These analyses and the single-trial maps (sorted by RT) suggest that HG in IPS was not locked to oculomotor execution but was rather aligned to the Go stimulus onset. This is consistent the role of the intraparietal in preparing and redirecting movements and movement intentions (i.e., motor attention, see 85). By contrast, in SMA (BA6), a significant correlation between HG onset and saccade onsets was observed in all 3 conditions. This may reflect SMA involvement in eye-movement execution [86], irrespective of whether target information was present prior to Cue 2. Interestingly we also found evidence for correlations between HG onset and saccade onsets that only occurred for Control trials (i.e., when no action selection processes were engaged during the delay period) but not for the Free or Instructed trials. This occurred for instance in right MFG and thereby suggests that this area is involved in saccade execution only when the participants could not plan the direction of their saccades during the delay period. This view is consistent with previous findings in humans that suggest that the DLPFC is necessary for the executive control of saccades [87].
Participants in our study were neurosurgical patients with drug-resistant epilepsy. To minimize the effect of epilepsy-related alterations and artifacts, we followed strict data exclusion procedures in line with our previous intracranial EEG work [88–91]. These consist primarily of systematic inspection of the data and exclusion of signals showing typical epileptic waveforms (e.g., epileptic spikes). In addition, we excluded data from any electrode subsequently identified by the clinical staff as being part of the resection area. Moreover, it should also be noted that the advantages of depth stereotactic electroencephalography (SEEG) recordings (including high spatial and temporal resolution, high signal-to-noise ratio across a wide range of frequencies up to 140 Hz, 68), come at the cost of heterogenous spatial sampling among participants. This limitation is inherent to all iEEG studies. The electrode implantation across the 6 participants (see Fig 1B and 1C) yielded a reasonable coverage (a total of 778 intracerebral sites) of frontal and central areas, but the posterior parietal cortex was under-represented, and none of our participants were implanted in the occipital cortex for instance. Furthermore, we and others have shown that unwanted eye movements can potentially lead to artifacts in iEEG due to saccadic spike potentials in extraocular muscles. These eye-movement artifacts occur in SEEG electrodes, especially in the high-gamma range and are most prominent in anterior and medial temporal lobe [92,93]. The localization and time course of HG activity reported in the present study suggest that it is not attributable to ocular artifacts. Moreover, we also verified that brain signals during the delay period were not affected by unwanted eye movements; averaging EOG signals with respect to stimulus onset did not reveal any systematic EOG activity in left-versus-right instructed saccade trials. Lastly, based on nonhuman primate studies [16,87,94], we expected to see differences between right and left saccades during the delay period. However, no differences were found when we trained an LDA algorithm to classify left and right saccades in Free and Instructed conditions (see S6 Fig). This means that the iEEG signals recorded in the present study do not seem to carry spatial information. This could in part be attributed to the fact that saccade directions were cued symbolically and foveally or because of larger fields captured in bipolar SEEG recordings compared with electrophysiological recordings in macaques. It may also very well be possible that other signal features not considered in our analyses may be able to successfully predict saccade direction from delay-period neural activity. Future analyses are needed to address such questions.
To conclude, the present study provides the first direct electrophysiological investigation of delayed eye-movement decisions using depth recordings in humans. Compared with instructed saccades, we found free-choice saccades to be associated with a more sustained HG activity in a parieto-frontal network. In a few prefrontal sites this HG enhancement persisted throughout the duration of the delay period; however, in most of the decision-related sites, HG activity modulations were present only in the early part of the delay period (i.e., first 2 seconds). We interpret the sustained HG activity as reflecting deliberation processes lasting until commitment to a choice. These results bridge the gap between findings in human and nonhuman primates and expands our understanding of the brain’s spatial, temporal, and spectral dynamics underlying human decision-making.
All requests for further information and resources should be directed to and will be fulfilled by the Lead Contact, Thomas Thiery (thomas.thiery@umontreal.ca).
All participants provided written informed consent, and the experimental procedures were approved by the local Ethical Committee (CPP Sud-Est V n° 09-CHU-12), which was carried out according to the Declaration of Helsinki.
Six patients with drug-resistant epilepsy participated in this study (6 females, mean age 30.3 ± 9.6). The participants were stereotactically implanted with multisite EEG depth electrodes at the Epilepsy Department of the Grenoble Neurological Hospital (Grenoble, France). In collaboration with the medical staff, and based on visual inspection, electrodes presenting pathological waveforms were discarded from the present study. All participants had normal vision without corrective glasses. All participants provided written informed consent, and the experimental procedures were approved by the local Ethical Committee (CPP Sud-Est V n° 09-CHU-12). Patient-specific clinical details can be found in S1 Table.
Each participant was implanted with SEEG electrodes (diameter of 0.8 mm). Depending on the implanted structure, electrodes were composed of 10 to 15 contacts that were 2-mm wide and 1.5-mm apart (DIXI Medical Instrument, Besançon, France). Intracranial EEG signals were recorded from a total of 778 intracerebral sites across all participants (between 128 and 133 sites per participant). At the time of acquisition, a white-matter electrode was used as reference, and data were sampled at 1,024 Hz and bandpass filtered between 0.1 and 250 Hz. Electrode locations were determined in each individual participant using the stereotactic implantation scheme. The coordinates of each electrode contact were given following these references: origin (anterior commissure), anteroposterior axis (anterior commissure–posterior commissure), and vertical axis (interhemispheric plane). The electrodes were then localized in each individual participant using Talairach coordinates, which were then transformed to MNI coordinate system using standard procedures (i.e., tal2mni.m MATLAB function) (Fig 1C). We then automatically assigned electrodes to brain regions based on 3 distinct atlases: Brodmann areas, the Automated Anatomical Labeling (AAL; 95), and the Multiresolution Intrinsic Segmentation Template (MIST; 96). The mapping from coordinates to brain areas (using this atlas) is publicly available in several toolboxes, including the one we used here (Visbrain, http://visbrain.org/, see [97]).
At the beginning of each trial, participants were asked to fixate a central fixation point that appeared at the center of the screen, along with 2 lateral points for 500 milliseconds. Lateral points are always visible and were located within a 14° visual angle (−7° and +7° around the central point). Participants were then instructed to perform horizontal saccades toward one of the 2 targets, depending on a visually presented central cue appearing briefly for 250 milliseconds. In the FREE condition, the cue (outline diamond-shaped) indicated the participants were free to decide the direction (Right or Left) of the saccade they would execute at the upcoming go signal (Cue 2). In this condition (FREE), no specific instruction was given concerning the timing of decisions. In the INSTRUCTED condition, participants prepared a saccade toward the target indicated by the cue (empty arrow). As soon as the central cue disappeared a variable delay period began (3,750, 5,750, or 7,750 milliseconds, selected with equal probability for each trial, i.e., 33.3%) during which the participants prepared the (chosen or instructed) saccade while fixating a central fixation point. The overall luminance and stimulus area of the first cue were matched across all 3 trial types to exclude differential visual effects. Next, a GO signal (a central filled double-arrow in the FREE condition or an arrow pointing to one of the 2 targets in the INSTRUCTED condition) indicated that the participants could execute the saccade (Execution period). In the CONTROL condition, an empty central rectangle indicated that the participants should continue central fixation without preparing any saccade. A variable delay (3,750, 5,750, or 7,750 milliseconds) was then followed by a Go signal indicating the direction of the saccade to be executed immediately, i.e., without prior preparation. After every saccade execution, no visual feedback about the trial performance was given, and participants had to fixate the central fixation point for 500 milliseconds. Next, everything disappeared from the screen during 1,500 milliseconds until the start of the next trial (intertrial interval). Each trial type (FREE, CONTROL, INSTRUCTED) was presented with the same (33.3%) probability, and trials were pseudorandomly interleaved. In all conditions, participants were asked to execute the saccade as soon as possible after the Go signal. We excluded trials in which participants took longer than 750 milliseconds to execute the saccade after the Go cue, and the EOG signal was visually inspected to exclude trials that contained abnormal eye movements and/or spontaneous saccades made during fixation or during the delay period (see S3 Fig). Across all participants, approximately 75% of the trials were retained for further analysis.
Based on the EOG traces (see S2 Fig), we computed saccade onset latencies for each trial and for all participants in the Control, Instructed, and Free conditions. Saccade onset latencies were identified from EOG traces recorded during each experimental conditions using a routine semiautomatic saccade detection procedure: A custom MATLAB (The Mathworks, Inc.) program provided an initial automatic detection of saccade onsets and allowed for interactive manual adjustments of the marker latencies (see S2 Table). In order to test whether RTs differed significantly across conditions, we used a 2-tailed paired Student t test and compared mean RTs for Control versus Instructed, Control versus Free, and Instructed versus Free conditions. Standard errors of the mean together with means, t-statistics and p-values were reported. To confirm these results in individual participant data, we used a 2-tailed unpaired Student t test to compare RTs between condition on a trial-by-trial basis within each participant (Control versus Instructed, Control versus Free, and Instructed versus Free). To account for differences in numbers of trials per condition, we used a bootstrap procedure (n = 100) each time randomly selecting values to match the condition with the least number of trials. Lastly, we conducted an analysis to evaluate trial history effects during the free-choice condition. We computed n-1 conditional probabilities to determine whether the saccade direction (Left, L or Right, R) of trial n-1 influenced trial n in the free-choice condition. The statistical significance of the obtained conditional probabilities was evaluated by computing statistical thresholds using permutation tests (n = 1,000, p < 0.001). A null distribution was generated by repeatedly (n = 1,000) computing conditional probabilities for each participant obtained after randomly permuting class labels (Left and Right).
Oculomotor performance was followed online using horizontal and vertical electro-oculograms (EOG), allowing to measure the amplitude and the speed of saccades (see S2 Fig), as well as the errors made by each participant. Four electrodes placed around the eyes to measure horizontal and vertical eye movements with a sampling frequency of 1,024 Hz. Saccade onsets were identified from EOG traces recorded during each experimental condition using a routine semiautomatic saccade detection procedure. A custom MATLAB script identified saccade onsets by detecting the onset of the characteristic slope (by computing the derivative of the EOG signal). The automatic detection of saccade onset was then fine-tuned through interactive manual adjustments of the marker latencies. To make sure our results cannot be attributed to saccades made after the presentation of Cue 1 during the delay period, we used a 2-tailed paired Student t test at each moment in time to compare mean right and left EOG responses, as well as mean responses during Free and Instructed conditions across all participants and found no significant differences.
SEEG data preprocessing was conducted according to our routine procedures [88,89,98–100]. These included signal bipolarization, in which each electrode site was re-referenced to its direct neighbor. Bipolar re-referencing can increase sensitivity and reduce artifacts by canceling out distant signals that are picked up by adjacent electrode contacts (e.g., mains power). The spatial resolution of bipolar SEEG electrodes was approximately 3 mm [89,101]. Next, using visual inspection and time-frequency explorations of the signal, we excluded electrodes containing pathological epileptic activity. The preprocessing led to a total of 543 bipolar derivations across all participants (see Fig 1B).
We conducted power analyses in several standard-frequency bands defined as follows: theta (θ) [4–8 Hz], alpha (α) [8–15 Hz], beta (β) [16–30 Hz], low gamma (low γ) [30–60 Hz], and HG [60–140 Hz]. This was achieved by first filtering the raw EEG signals using a finite impulse response filtering (FIR1, order = 3) and then computing the Hilbert transform over 400-millisecond time windows with an overlap of 50 milliseconds. The power features used for classification were computed as mean power over 400-millisecond time windows with an overlap of 50 milliseconds during the delay period (–500 to 2,500 milliseconds), where t = 0 milliseconds corresponds to the onset of Cue 1, and execution (–500 to 1,500 milliseconds), where t = 0 milliseconds corresponds to the onset of Cue 2. The classification was applied on non-normalized power. For single-trial representations, the same methods were used but using 60-millisecond time windows with an overlap of 10 milliseconds. Whenever present, baseline normalization was only used for visualization purposes (time-frequency maps and single-trial representation). Baseline normalization was achieved for each frequency band by subtracting then dividing by the mean of a 400-millisecond baseline window, i.e., the prestimulus rest period ([–500 milliseconds, –100 milliseconds]).
We set out to explore the feasibility of using multisite human LFP data (543 bipolar electrode sites) to perform classifications during motor planning and execution. To this end, we implemented a machine learning framework for trial-by-trial classification using spectral power. Several classification techniques were initially tested for the single-feature classification procedure, including LDA, k-nearest-neighbor (KNN), and support vector machine (SVM). The classification accuracy results were very similar across the 3 methods. The LDA algorithm [102] was the fastest and was therefore chosen for this study given the computationally demanding permutation tests used to evaluate classifier performance. In brief, for a 2-dimensional problem, the LDA algorithm tries to find a hyperplane that maximizes the mean distance between the mean of the 2 classes while minimizing interclass variance.
Single-trial classification performance was evaluated in each participant separately. We used a standard stratified 10-fold cross-validation approach with Scikit-learn, a Python 3 package dedicated to machine learning analyses [103]. First, the data set was pseudorandomly split into 10 equally sized observations: 9 segments were used for training the classifier, and the last one as the test set. This procedure was repeated 10 times, such that every observation in the data was used exactly once for testing, and at least once for training, but never at the same time. This strict separation of training and testing ensures the test data was naive and did not violate basic classification principles (e.g., 104). The use of stratification seeks to ensure that the relative proportion of labels (or classes) in the whole data set is reasonably preserved within each of the segments after the split. Next, the performance of the achieved decoding was calculated using the DA metric, which was computed as the mean correct classification across all folds. The statistical significance of the obtained decoding accuracies was evaluated by computing statistical thresholds using permutation tests (n = 100, p < 0.01). In other words, a null distribution is generated by repeatedly (n = 100) computing the classification accuracy obtained after randomly permuting class labels [105]. In all our decoding analyses, we used maximum statistics to correct across electrodes, frequency bands and time with a statistical threshold at p < 0.01.
We explored the temporal dynamics of HG activity during the delay period for Free versus Control and Instructed versus Control conditions by probing cross-temporal generalization [11,53]. In principle, we employed the same LDA decoding approach as described in the Signal classification section, except that the classifiers trained at a given time point were now tested at every other point in time, resulting in 2D cross-temporal decoding matrices (see[11,53]). Once t LDA classifiers have been fitted (where t is the duration of a trial expressed in time samples), each classifier is tested on its decoding generalization at any time t′. This method thus leads to a temporal generalization matrix of training time × generalization time (see Fig 4). In each cell of the matrix, decoding performance is summarized by the DA. Classifiers trained and tested at the same time point correspond to the diagonal of this matrix and are thus referred to as “diagonal” decoding. The decoding performance obtained when t′ differ from t is referred to as “off-diagonal” decoding. To identify where HG activity was significantly different between conditions in the temporal generalization matrices, we used the binomial cumulative distribution to derive statistical significance thresholds (see [105]. Finally, we compared our results with the repertoire of canonical dynamical patterns in temporal generalization matrices established by previous studies [11,53].
To perform the multifeature analysis, we use the Exhaustive Feature Selection (EFS) method from mlxtend [106] applied for each frequency band for each participant. The EFS algorithm will test all the possible combinations of the frequency bands and will select which feature or set of features allows for better decoding of our 2 conditions (Free versus Instructed, see Fig 3E). The feature selection is scored on a stratified validation data set consisting of one third of the data. The EFS is repeated with all possibilities of validation set and the best selected features are counted for each electrode.
To statistically compare the peak of HG activity across the Instructed and Free conditions from all relevant sites during the delay period, we first extracted the peak of HG activity from all electrodes that exhibited significant decoding of Free versus Instructed conditions. We then compared the peak HG activity for all electrodes within participants using unpaired t tests (Instructed–Free). This gave us t-values and p-values for each individual participant (e.g., for HG activity, Instructed–Free, 4/6 participants). We then averaged the peak HG activity across electrodes and used a paired t test to assess whether the effect was statistically significant across participants at the group level.
To assess whether activity was decoded earlier, peaked earlier, and whether it was more sustained in the Free condition than in the Instructed condition, we first conducted 2 separate classifications (Free versus Control and Instructed versus Control). We then computed the timing at which each significant electrode (1) started to decode (based on first time bin of significant decoding) and (2) maximally decoded (based on the timing of the maximum accuracy) Free versus Control and Instructed versus Control conditions. We compared these timings for all electrodes within participants with an unpaired Student t test. This gave us t-values and p-values for each participant. We then averaged the first significant timings across electrodes and used a paired Student t test to assess whether the effect was statistically significant across participants at the group level. The same analysis was used to determine whether HG activity was more sustained in time in the Free condition compared with the Instructed condition. To this end, we counted the total number of time points with significant decoding between Free versus Control and Instructed versus Control and ran both within and across participant comparisons correlations between RTs and HG activity.
In addition, we computed Pearson’s rank correlation coefficients between RTs and the onset of HG activity for each trial in the Free, Instructed, and Control conditions during saccade execution. The onset of HG activity was determined in each trial by detecting the time point after which HG activity was greater than 2 standard deviations for at least 2 consecutive time bins. The statistical significance of correlations was established by using a 2-sided test whose null hypothesis is that 2 sets of data are uncorrelated. The p-value thus indicates the probability of an uncorrelated system producing datasets that have a Pearson rank correlation at least as extreme as the one computed from these datasets.
To facilitate the interpretation of the results, all significant task-based feature modulations and decoding results were remapped from the intracranial electrode sites onto a standard cortical representation. To achieve this, all electrode coordinates were transformed from individual Talairach space to standard MNI space using Visbrain [97], an open-source Python 3 package dedicated to brain signal visualization, to map the data from iEEG sites onto 3D images of transparent brains. This cortical representation technique is in line with methods used in previous iEEG studies [88,89,107] and allowed for brain-wide visualization of significant features and decoding performances.
Electrophysiological data were analyzed using Python 3, in conjunction with toolboxes including Visbrain [97] for data visualization and mlxtend [106] as well as Scikit-learn [103] for machine learning analyses. Data and custom Python analysis scripts are available upon reasonable request from Thomas Thiery (thomas.thiery@umontreal.ca).
The numerical data used in all figures are included in S1 Data.
We wish to thank P. Cisek and A. Green for valuable discussions and comments.
| BOLD | blood-oxygen-level-dependent |
| DA | decoding accuracy |
| DLPFC | dorsolateral prefrontal cortex |
| EEG | electroencephalography |
| FEF | frontal eye field |
| fMRI | functional magnetic resonance imaging |
| HG | high-gamma |
| iEEG | intracerebral EEG |
| IPS | intraparietal sulcus |
| LDA | linear discriminant analysis |
| LFP | local field potential |
| LIP | lateral intraparietal area |
| MEG | magnetoencephalography |
| MFG | middle frontal gyrus |
| PEF | parietal oculomotor field |
| preSMA | presupplementary motor area |
| ROI | region of interest |
| RT | reaction time |
| SEF | supplementary eye field |
| SFG | superior frontal gyrus |
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
Decoding the neural dynamics of free choice in humans
Facebook
Twitter
Linkedin
Whatsapp
