PLoS Computational Biology

Calmon, Lucille;Colosi, Elisabetta;Bassignana, Giulia;Barrat, Alain;Colizza, Vittoria

doi: 10.1371/journal.pcbi.1012661pmid: 39652593

Introduction Face-to-face contacts between individuals represent crucial transmission paths for respiratory viruses, such as SARS-CoV-2, influenza or RSV (respiratory syncytial virus) [1]. These contacts occur throughout day-to-day life, e.g., in public transport, among children in schools, coworkers in workplaces, household members, as well as in the wider community, creating opportunities for the spread of diseases. The global contact network among individuals in a population, which thus approximates possibilities for transmission, remains challenging to measure [2–4]. Nonetheless, an increasing number of research groups have developed ways to automatically record high-resolution empirical contact networks in a variety of settings, particularly in various types of schools [5–10], but also in a university [11], in an office building [12], in a conference [13], in hospital wards [14–17], in households in rural villages [18] or on a cruise-ship [19]. These empirical contact networks in turn can provide valuable input to inform models of infectious disease spread and to design or evaluate intervention strategies [5, 15, 17, 20–25]. Several characteristics of the network of contacts impact the patterns of a spread unfolding on that network. For instance, clustering (the tendency of the contacts of a person to be in contacts themselves) [26, 27], heterogeneity in the number of contacts per individual [28], contact duration [9, 29], and the repetitions of contacts in successive periods (e.g., days) [13, 30–33] all influence the disease spread. The role of contact repetition [26, 27] in particular has been recently considered [33, 34]. When generating contact networks from contact matrices [35, 36] at a daily temporal scale to inform disease spreading models, not taking into account the fact that a non-negligible fraction of contacts are repeated from one day to the next (up to 35% of reported contacts according to some surveys [31]) can lead to overestimated attack rates [33]. Additionally, contact repetition (over different days) and retention (the tendency of people to remain in contact over time with an individual) can also affect the identification of superspreaders and superspreading events [34]. When building synthetic data on contacts between individuals to feed e.g. agent-based models of disease spread [13, 37, 38], correctly taking into account the presence of repeated contacts is therefore crucial. Despite the increased availability of high-resolution data on contact patterns [3, 5–14, 16, 19], each data set remains limited to a given specific context and data collection period, and most often to a relatively short data collection time window of a few days. On the other hand, agent-based models need to be simulated on long time scales and their results should not depend on the specificity of a data set collection time [25, 29]. These models thus need to be fed by realistic synthetic data covering arbitrarily long time scales. Such inputs are obtained by longitudinally extending recorded data, often by simply repeating the empirical data over and over [13, 21, 22, 24, 39], which means that each and every contact or contact pattern is repeated periodically in such synthetic data, with no variation. So far, few works have explored the potential of leveraging data on the actual amount of repetition of contacts between different periods (e.g., days) to extend longitudinally existing data sets in a way that respects the balance between repeated contacts and more casual, randomly occurring ones. This is particularly relevant for contacts recorded in educational settings where (1) various respiratory viral infections circulate among students [40] that densely mix in an indoor context [6, 7] and (2) strong social ties due for instance to friendships and associated with longer contacts [41] drive behaviour, inducing repeated encounters with correlated characteristics that co-exist with casual interactions [7, 8]. Here, we tackle this issue and present and illustrate methods to use empirical contact data recorded during a few days to generate long-term synthetic contact data with realistic statistical properties and on arbitrarily long timescales, in the context of a secondary school [8]. We specifically design and compare two mechanisms for this purpose: In the “Friendship-based approach”, we generate synthetic contacts that take into account data on the repetition of contacts across different days, which we interpret as a sign of probable friendship between the corresponding individuals; in the “Class-mixing-based approach” on the other hand, we preserve the mixing patterns between classes in each day of synthetic data, but do not consider any other memory effects between days (i.e., we do not take into account the data on contact repetition). As a baseline, we also consider a simple procedure of looping the empirical contact data. We then use the three types of obtained synthetic data sets to separately inform a realistic agent-based model of the spread of SARS-CoV-2 in a school [24]. In order to evaluate the impact of taking into account friendship relations in this context, we compare the outcomes of the numerical simulations fed by the various types of synthetic contact data, focusing on the one hand on the distributions of outbreak sizes, and on the other hand on the infection networks and infection trees [42, 43], which summarize the preferential paths along which the disease progresses through the population. Methods Empirical contact networks We consider empirical contact data collected among students in the second grade of “Classes Préparatoires” [8]. These classes are offered by some high-schools in France during two additional years to prepare students for entrance examinations to higher-education establishments in France. In the “Lycée Thiers” (Marseille, France) where the data was gathered, the second grade of these classes is organised in 9 classes of 3 specialties (mathematics: 3 “MP” classes, biology: 3 “BIO” classes, and physics-chemistry: 3 “PC” classes). Close face-to-face proximity events were recorded between students in these 9 classes during school hours [7], as described in [8, 44]: such contacts were recorded between students wearing RFID (Radio Frequency Identification) sensors that exchanged low-power data packets when in close proximity (up to 1–1.5 meters). Data was collected over four and a half days (Monday the 2nd to Friday the 6th of December 2013), with a high participation rate: out of the 379 students in the 9 second grade classes, 327 participated to the study (86.3% participation rate). The resulting data sets are temporal networks were nodes represent students and temporal edges represent recorded proximity events with a temporal resolution of 20 seconds [44, 45]. Such data can also be expressed as lists of timestamps tij for each pair (i, j) of individuals who have been in contact. We moreover build, from the data and without loss of information, daily contact networks as follows: for each day d of the deployment, such a network Gd includes students as nodes and encodes the occurrence of contacts between them during that day as contact links, denoted (i, j), that carry both a weight and a contact timeline . The weight is given by the cumulative time recorded in contact between students i and j during day d, and the “activity timeline” is given by the list {tij}d of all contact timestamps between nodes i and j during day d (note that is thus simply proportional to the length of the list {tij}d). Using the daily contact networks, we identify the pairs of students who have been in contact in at least two different days of the data collection. Interestingly, the data also includes information on friendship relations between students, as gathered from a survey [8]. We use the combination of contact data and friendship data to show in the S1 Text that the repetition of contacts across distinct days is an indication of the likely existence of a friendship relationship between the students. We will thus refer in the following to the pairs of students that are in contact during at least two days of the deployment as “friendship links”. Note that the links of each daily network Gd will thus be divided into (1) a set Fd of friendship links (i, j) that are observed in at least one other daily network Gd′ (with d′ ≠ d) and (2) a set Cd of “casual” links between individuals who have been observed in contact only in day d. Synthetic extension of the contact data We propose here two methods to leverage the empirical daily contact networks and create synthetic ones among the same students, with high-resolution timelines and preserving several important statistical properties observed in the empirical data. For each day, each method takes as input a daily contact network and can output an arbitrary number of similar, statistically plausible, synthetic daily contact networks. Both methods preserve in particular the class-mixing matrix of the day considered, defined as the number of contact links NAB between each pair of classes A, B and within each class A = B during that day. In the first method, which we call the “friendship-based approach”, friendship links of the day considered are preferentially retained in the synthetic version of that day. This approach hence preserves the local structure of the students friendships, while additional random contact links mimic stochasticity in contact behaviour (the “casual” links in the data). In a second approach, the “class-mixing-based approach”, the distinction between friendship and casual links is not taken into account to create the synthetic data, which only preserve the class-mixing matrix and the overall timeline of when interactions occur within a class or between two classes (to respect e.g. the fact that interactions between classes may take place only when the school schedule allows it). Friendship-based approach. In this approach, we take into account that, as discussed above, the contact patterns among students are not fully random but bear similarities over different days. Contacts between a given pair of students may reoccur in different days [6, 34, 46], in particular between friends, and the contacts total duration (link weights) of friendship links also tend to be larger [7]. We thus propose the algorithm below (summarized in Fig 1) to generate synthetic daily contact networks. The algorithm takes as input (1) one day of data collection, that we call the base day and (2) the separation between friendship links and casual links of that day. It can thus be ran independently for each day of data collection once the friendship links have been extracted from the comparison of the various empirical daily contact networks. The algorithm generates contact links class by class, and pair of classes by pair of classes until all classes (and pairs of classes) have been considered. Combining all the contact events generated provides synthetic contacts inheriting the properties of the daily contact network of the base day considered. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Schematic representation of the friendship-based approach to synthetic contacts generation. (A) Daily high-resolution empirical contact networks are schematically represented. They form the starting point (input) of the method. Friendship links are highlighted in orange, while blue links correspond to empirical links occurring only on a single day. (B) Different lists required for the algorithm are shown with examples of entries. Note that the timelines (lists of timestamps) are expressed in seconds from midnight on the initial day of the deployment, and the weights (“w”) are expressed in seconds. These lists are specific for each class and pair of classes, and each base day. (C) Description of the different steps of the algorithm. When assigning randomly a timeline and weight to contact links (step 3 of (C)), an entry of the third table is drawn. The algorithm operates class by class, and pair of classes by pair of classes, to generate synthetic contact networks. (D) Schematic representation of the generated synthetic contact networks. These networks inherit properties from the empirical contact networks, such as the number of links per class and between each pair of classes, a fraction of the friendship links (depicted in orange) and of the non-repeated links (depicted in blue). These links are complemented by random links (depicted in green) that were not necessarily observed in the base day. https://doi.org/10.1371/journal.pcbi.1012661.g001 For a given base day d, the algorithm proceeds as follows to build synthetic contact networks between the students of each pair of classes A and B (and among the students of each class A = B). 1. Inputs from data: the algorithm uses as inputs the number of contact links between classes A and B on the base day, the lists of contact links between them (i.e., all contact links such that i is in class A and j is in class B or vice-versa), of their weights (cumulative contact durations) and associated timelines . The inputs include as well the friendship links on the base day between individuals of classes A and B, which are put in a dictionary of the form , where {w}ij is the list of weights of (i, j), and {tl}ij the list of timelines of (i, j) on all days in which the friendship link is observed in the data. 2. Taking friendships into account: Each of the friendship links in is included as a contact link in the synthetic network with probability f. For each such link (i, j) added in this step, a synthetic weight and timeline are generated from {w}ij and {tl}ij (see S1 Text for details). 3. Remaining links: Additional links are added one at a time. Each such link is either extracted from the list of empirical contact links , with probability p, or created randomly (with probability 1 − p) by choosing at random two students respectively in classes A and B. In both cases, with probability ptr we add an extra step to preserve transitivity in the network [47] (the fraction of closed triangles, i.e. of structures {(i, j), (j, k), (k, i)} among all possible connected triads, i.e., such that at least {(i, j), (j, k)} exist). Specifically, if the transitivity of the current synthetic network between classes A and B is lower than the empirical one, the additional link is chosen (either in or randomly) such that it closes a triangle; conversely, if the current transitivity is too high, the additional link is chosen in order to create an open triangle. With probability 1 − ptr, the choice of the additional link is made independently from the transitivity of the network. This procedure of adding links is iterated until the synthetic contact network between classes A and B (or within class A = B) contains contact links. Each added contact link is then associated a timeline and matching weight selected randomly from and . 4. Weight correction: Once the synthetic contact network between classes A and B (or within class A = B) includes the correct number of contact links, timestamps can be added or removed from activity timelines in the network (longer timelines are preferably modified) to ensure that the sum of all link weights in the synthetic contact network between classes A and B (or within class A = B) is within a 10% tolerance of the corresponding quantity in the empirical contact network of the base day—see S1 Text for details. The above steps are repeated for every pair of classes A and B and for each class (A = B). The resulting synthetic contact networks are then merged into daily synthetic contact networks involving all individuals present in the base day. By construction, these synthetic contact networks preserve the number of contact links and the total time in contact within and between each class (steps 3 and 4). In addition, as the timelines of the contact links are taken from the lists of empirical ones, the timetable of the base day is also preserved. In the algorithm, the parameter f controls the average fraction of friendship links that are preserved in the synthetic data. The parameter 1 − p controls the amount of stochasticity, by allowing links in the synthetic network between students who have actually not been observed in contact during the base day. In the following, we describe how we tune these parameters (f and p), as well as ptr, to reproduce some features of the empirical networks. Parameters optimisation. We tune the parameters of the model in order to take into account the day-to-day similarity between daily contact networks observed empirically. To quantify this similarity, we consider the local cosine similarity [48], which measures the similarity in the contact links of an individual i in two daily contact networks (on days d1, d2). Mathematically, it is given by (1) where (resp. ) refers to the set of nodes j in contact with node i on day d1 (resp. d2) with contact link weights wij(d1) (resp. wij(d2)). The sums in the denominators run over (resp. ) which is the set of nodes in (resp. ) that are also present in the daily network of day d2 (resp. d1). This local-node-based similarity measure ranges from 0 to 1. When i has contact links with disjoint sets of individuals on days d1 and d2, LCS(i, d1, d2) = 0. Partially overlapping contact links with different cumulative duration (encoded in the weights wij(d1) and wij(d2)) instead lead to 0 < LCS(i, d1, d2)<1, reaching 1 only for an individual i with exclusively the same contact links and proportional weights in both days. The distribution of values of the local similarity between any pairs of data collection days for the participating students [8] exhibits relative maxima at 0 and 1 values but is spread over all possible similarity values, showing a strong heterogeneity of similarity patterns (see Results Section) [30]. We thus tune the algorithm parameters f, p and ptr to reproduce this specific shape. Specifically, we perform a grid-exploration for these three parameters to minimise the Jensen-Shannon distance [49, 50] between the synthetic and empirical distributions of local cosine similarities. Details on the optimisation process, the definition of the Jensen-Shannon distance as well as a sensitivity analysis on the distance measure used can be found in the S1 Text. In the following, contact networks generated with this method, using optimised parameters, are called friendship-based (daily) contact networks. These synthetic daily contact networks can also be expressed as lists of temporal contacts between individuals (with the same temporal resolution as the empirical data), which we call friendship-based (synthetic) contacts. Class-mixing-based approach. In the “class-mixing-based approach”, the synthetic contacts are generated in order to reflect the empirical mixing patterns between classes observed during the base day considered. However, the algorithm does not take into account the distinction between friendship and casual links, nor the transitivity (see S1 Text for details on the procedure). It is thus approximately recovered instead by the previous algorithm with parameters f = 0, p = 0 and ptr = 0. In particular, the number of contact links within each class and among each pair of classes is preserved, as in the friendship-based approach. However, the contact links between students are renewed each day independently of any previous occurrence. Overall, the synthetic contacts obtained in this approach preserve the class structure of the school, as well as the timetable features, but not the individual friendships. Creating synthetic contact data of arbitrary length. The algorithms described above create synthetic daily contact networks from empirical base days. As the school timetable has typically a weekly periodicity, a natural procedure to create synthetic data covering e.g. n weeks, starting from empirical data collected during one week, is to create n instances of each empirical base day, and to create synthetic weeks using a synthetic instance of each weekday (note that the students are considered isolated during the weekends [21, 24]). In the present case, as, on the Monday, the data was collected only during half a day (Supplementary figures of [8]), we use the Tuesday data as base day to create the Mondays synthetic contact networks. The synthetic contact sequences built using the friendship-based approach applied in this way to the four base days of data (Tuesday, Wednesday, Thursday, Friday) are called “Friendship 4d” in the following, while synthetic contact sequences obtained with the class-mixing-based approach are called “Class Mixing 4d”. As several data collection efforts have been carried out over even shorter timescales [3, 44], we also mimic here such cases, by artificially restricting the number of base days available. For the friendship-based algorithm, the minimal number of base days (in order to separate friendship from casual links) is two, while for the class-mixing-based approach we can even restrict the data to one day. We then create synthetic weekly sequences by combining the synthetic instances obtained from the various base days as detailed in Table 1. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Contact sequences used in the analysis. https://doi.org/10.1371/journal.pcbi.1012661.t001 In addition, we considered as a baseline synthetic contact sequences built exclusively by repeating empirical ones, i.e., by looping over empirical contact data as done in many previous works. We call the resulting data looped contacts, where the repeated empirical data can include from one to four of the available daily contact networks. Contact link weights of all sequences are uniformly rescaled to match the total daily interaction time in the “Looped 4d” contacts. This ensures the comparability of epidemiological outputs on contact sequences generated from different sequences of base days. We denote in the following the various synthetic contact sequences by ctx (where x is one of the sequence names of Table 1). Transmission model In the following, we leverage on the contact sequences from Table 1 to feed the stochastic agent-based model from Ref. [24] and simulate numerically the spread of SARS-CoV-2 in the school population. In this model, when a transmission event occurs, the exposed individual (E) becomes infectious and pre-symptomatic (Ip) after a time τE. The pre-symptomatic phase lasts τp, after which the individual enters either a sub-clinical (Isc) or clinical (Ic) infectious state, which lasts τI before recovery (R). The durations of each stage (τE, τp, τI) are drawn at random for each individual from Gamma distributions parameterised following the literature [24]. The probability of developing a clinical (versus a sub-clinical infection) is instead a fixed parameter. The per unit-time transmission probability when a susceptible individual is in contact with an infectious one depends on the transmission rate β and on the relative infectiousness and susceptibility of the individuals in contact, determined from infectious status. We model a vaccinated student population (homogeneous 50% coverage) by reducing the relative infectiousness of vaccinated individuals by 20% and their relative susceptibility to infection by 50%. Moreover, we model partial immunity from previous infection by SARS-CoV-2 pre-Omicron variants in 40% of the population. The susceptibility of immune individuals to infection is reduced by 81%. As time progresses, contacts encoded in the contact sequence considered are replayed in the numerical simulation. To increase computational efficiency, high-resolution 20 seconds contacts are aggregated in 15 minutes steps: each resulting contact is weighted by the fraction of time actually spent in contact over the 15 minutes for each pair. Each such contact event in which an infectious individual is interacting with a susceptible one represents an opportunity of transmission for the simulated disease. We consider a simple spreading scenario in the absence of control measures initiated by a single infectious individual (the seed) among the students. The transmission rate β is tuned as in [24], to achieve an effective reproductive number in the population with partial vaccination and immunity R = 1.5 (see S1 Text for details on the calibration). Extraction and comparison of infection pathways For each contact sequence (ctx, in Table 1) and for each initial seed s, we build an infection network [42, 43], denoted , by aggregating transmission chains obtained from different simulations (Fig 2A and 2B). These infection networks are directed, in order to encode transmission events from i to j and from j to i separately. Both occurrences indeed correspond to different transmission chains. Each may occur with a different probability (depending on vaccination statuses of the pair, disease stages during which the contact occurs). Each directed edge ℓ = (i, j) ≠ (j, i) from student i to j is weighted by the occurrence probability of a transmission event from i to j (denoted pℓ(s, ctx)), estimated by the fraction of simulations in which such an event is observed [43]. This probability depends not only on the per unit time probability of transmission (see S1 Text), but also on the relative positions of i and j in the contact network. The latter impacts in particular whether i is infected before or after j. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Construction of infection pathways. (A) Examples of simulated transmission chains between individuals are shown for a given seed. (B) An infection network built from 135 realisations of the model initialised with the same seed is shown. (C) Maximum spanning tree extracted from the infection network of (B). All results are obtained with ctx = “Friendship 4d” contacts. Darker edges in (A) correspond to transmission events between different classes. Edge widths in (B) and (C) are proportional to their probability of occurrence pℓ(s, ctx). Edges with probability of occurrence <0.01 are omitted for readability in (B). Nodes of the same class share the same color, and the seed is highlighted in black. Visualisations generated with Gephi [51]. https://doi.org/10.1371/journal.pcbi.1012661.g002 Infection networks form an aggregated representation of the outbreak, in which a large fraction (or even all) of the contact links between individuals in the population are represented. We thus also consider summarized versions, the infection trees (Fig 2C). Each infection tree, denoted , is given by the (directed) maximum spanning tree of the corresponding infection network, as described in [42]. It approximates the “most likely” infection pathway from the seed s to each node, by retaining edges ℓ that form a directed tree, such that their summed pℓ(s, ctx) is maximum. The infection pathways defined in this way (networks and trees) describe the progression of the outbreak between students, for each given seed s. To adopt a more coarse-grained view, we also build pathways using aggregated transmission chains between classes, in order to represent the pathways of importation of cases into previously unexposed classes. We refer to the S1 Text for additional details on the construction of the four types of infection pathways introduced. In order to investigate the impact of the type of synthetic data used in the simulations, we compare the infection networks obtained with the same given seed s for two different contact sequences (denoted by cta and ctb) using the global cosine similarity defined by: (2) where (resp. ) denotes the set of edges of (resp. ). Note that we can also compute the global cosine similarity between pairs of infection trees. This global cosine similarity takes values between 0 and 1 and provides a measure of overlap between two infection networks (or trees). It is equal to 0 if and only if the networks (or trees) compared have no edges in common, and reaches 1 if and only if the two networks or trees are identical in their edges and respective weights. Empirical contact networks We consider empirical contact data collected among students in the second grade of “Classes Préparatoires” [8]. These classes are offered by some high-schools in France during two additional years to prepare students for entrance examinations to higher-education establishments in France. In the “Lycée Thiers” (Marseille, France) where the data was gathered, the second grade of these classes is organised in 9 classes of 3 specialties (mathematics: 3 “MP” classes, biology: 3 “BIO” classes, and physics-chemistry: 3 “PC” classes). Close face-to-face proximity events were recorded between students in these 9 classes during school hours [7], as described in [8, 44]: such contacts were recorded between students wearing RFID (Radio Frequency Identification) sensors that exchanged low-power data packets when in close proximity (up to 1–1.5 meters). Data was collected over four and a half days (Monday the 2nd to Friday the 6th of December 2013), with a high participation rate: out of the 379 students in the 9 second grade classes, 327 participated to the study (86.3% participation rate). The resulting data sets are temporal networks were nodes represent students and temporal edges represent recorded proximity events with a temporal resolution of 20 seconds [44, 45]. Such data can also be expressed as lists of timestamps tij for each pair (i, j) of individuals who have been in contact. We moreover build, from the data and without loss of information, daily contact networks as follows: for each day d of the deployment, such a network Gd includes students as nodes and encodes the occurrence of contacts between them during that day as contact links, denoted (i, j), that carry both a weight and a contact timeline . The weight is given by the cumulative time recorded in contact between students i and j during day d, and the “activity timeline” is given by the list {tij}d of all contact timestamps between nodes i and j during day d (note that is thus simply proportional to the length of the list {tij}d). Using the daily contact networks, we identify the pairs of students who have been in contact in at least two different days of the data collection. Interestingly, the data also includes information on friendship relations between students, as gathered from a survey [8]. We use the combination of contact data and friendship data to show in the S1 Text that the repetition of contacts across distinct days is an indication of the likely existence of a friendship relationship between the students. We will thus refer in the following to the pairs of students that are in contact during at least two days of the deployment as “friendship links”. Note that the links of each daily network Gd will thus be divided into (1) a set Fd of friendship links (i, j) that are observed in at least one other daily network Gd′ (with d′ ≠ d) and (2) a set Cd of “casual” links between individuals who have been observed in contact only in day d. Synthetic extension of the contact data We propose here two methods to leverage the empirical daily contact networks and create synthetic ones among the same students, with high-resolution timelines and preserving several important statistical properties observed in the empirical data. For each day, each method takes as input a daily contact network and can output an arbitrary number of similar, statistically plausible, synthetic daily contact networks. Both methods preserve in particular the class-mixing matrix of the day considered, defined as the number of contact links NAB between each pair of classes A, B and within each class A = B during that day. In the first method, which we call the “friendship-based approach”, friendship links of the day considered are preferentially retained in the synthetic version of that day. This approach hence preserves the local structure of the students friendships, while additional random contact links mimic stochasticity in contact behaviour (the “casual” links in the data). In a second approach, the “class-mixing-based approach”, the distinction between friendship and casual links is not taken into account to create the synthetic data, which only preserve the class-mixing matrix and the overall timeline of when interactions occur within a class or between two classes (to respect e.g. the fact that interactions between classes may take place only when the school schedule allows it). Friendship-based approach. In this approach, we take into account that, as discussed above, the contact patterns among students are not fully random but bear similarities over different days. Contacts between a given pair of students may reoccur in different days [6, 34, 46], in particular between friends, and the contacts total duration (link weights) of friendship links also tend to be larger [7]. We thus propose the algorithm below (summarized in Fig 1) to generate synthetic daily contact networks. The algorithm takes as input (1) one day of data collection, that we call the base day and (2) the separation between friendship links and casual links of that day. It can thus be ran independently for each day of data collection once the friendship links have been extracted from the comparison of the various empirical daily contact networks. The algorithm generates contact links class by class, and pair of classes by pair of classes until all classes (and pairs of classes) have been considered. Combining all the contact events generated provides synthetic contacts inheriting the properties of the daily contact network of the base day considered. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Schematic representation of the friendship-based approach to synthetic contacts generation. (A) Daily high-resolution empirical contact networks are schematically represented. They form the starting point (input) of the method. Friendship links are highlighted in orange, while blue links correspond to empirical links occurring only on a single day. (B) Different lists required for the algorithm are shown with examples of entries. Note that the timelines (lists of timestamps) are expressed in seconds from midnight on the initial day of the deployment, and the weights (“w”) are expressed in seconds. These lists are specific for each class and pair of classes, and each base day. (C) Description of the different steps of the algorithm. When assigning randomly a timeline and weight to contact links (step 3 of (C)), an entry of the third table is drawn. The algorithm operates class by class, and pair of classes by pair of classes, to generate synthetic contact networks. (D) Schematic representation of the generated synthetic contact networks. These networks inherit properties from the empirical contact networks, such as the number of links per class and between each pair of classes, a fraction of the friendship links (depicted in orange) and of the non-repeated links (depicted in blue). These links are complemented by random links (depicted in green) that were not necessarily observed in the base day. https://doi.org/10.1371/journal.pcbi.1012661.g001 For a given base day d, the algorithm proceeds as follows to build synthetic contact networks between the students of each pair of classes A and B (and among the students of each class A = B). 1. Inputs from data: the algorithm uses as inputs the number of contact links between classes A and B on the base day, the lists of contact links between them (i.e., all contact links such that i is in class A and j is in class B or vice-versa), of their weights (cumulative contact durations) and associated timelines . The inputs include as well the friendship links on the base day between individuals of classes A and B, which are put in a dictionary of the form , where {w}ij is the list of weights of (i, j), and {tl}ij the list of timelines of (i, j) on all days in which the friendship link is observed in the data. 2. Taking friendships into account: Each of the friendship links in is included as a contact link in the synthetic network with probability f. For each such link (i, j) added in this step, a synthetic weight and timeline are generated from {w}ij and {tl}ij (see S1 Text for details). 3. Remaining links: Additional links are added one at a time. Each such link is either extracted from the list of empirical contact links , with probability p, or created randomly (with probability 1 − p) by choosing at random two students respectively in classes A and B. In both cases, with probability ptr we add an extra step to preserve transitivity in the network [47] (the fraction of closed triangles, i.e. of structures {(i, j), (j, k), (k, i)} among all possible connected triads, i.e., such that at least {(i, j), (j, k)} exist). Specifically, if the transitivity of the current synthetic network between classes A and B is lower than the empirical one, the additional link is chosen (either in or randomly) such that it closes a triangle; conversely, if the current transitivity is too high, the additional link is chosen in order to create an open triangle. With probability 1 − ptr, the choice of the additional link is made independently from the transitivity of the network. This procedure of adding links is iterated until the synthetic contact network between classes A and B (or within class A = B) contains contact links. Each added contact link is then associated a timeline and matching weight selected randomly from and . 4. Weight correction: Once the synthetic contact network between classes A and B (or within class A = B) includes the correct number of contact links, timestamps can be added or removed from activity timelines in the network (longer timelines are preferably modified) to ensure that the sum of all link weights in the synthetic contact network between classes A and B (or within class A = B) is within a 10% tolerance of the corresponding quantity in the empirical contact network of the base day—see S1 Text for details. The above steps are repeated for every pair of classes A and B and for each class (A = B). The resulting synthetic contact networks are then merged into daily synthetic contact networks involving all individuals present in the base day. By construction, these synthetic contact networks preserve the number of contact links and the total time in contact within and between each class (steps 3 and 4). In addition, as the timelines of the contact links are taken from the lists of empirical ones, the timetable of the base day is also preserved. In the algorithm, the parameter f controls the average fraction of friendship links that are preserved in the synthetic data. The parameter 1 − p controls the amount of stochasticity, by allowing links in the synthetic network between students who have actually not been observed in contact during the base day. In the following, we describe how we tune these parameters (f and p), as well as ptr, to reproduce some features of the empirical networks. Parameters optimisation. We tune the parameters of the model in order to take into account the day-to-day similarity between daily contact networks observed empirically. To quantify this similarity, we consider the local cosine similarity [48], which measures the similarity in the contact links of an individual i in two daily contact networks (on days d1, d2). Mathematically, it is given by (1) where (resp. ) refers to the set of nodes j in contact with node i on day d1 (resp. d2) with contact link weights wij(d1) (resp. wij(d2)). The sums in the denominators run over (resp. ) which is the set of nodes in (resp. ) that are also present in the daily network of day d2 (resp. d1). This local-node-based similarity measure ranges from 0 to 1. When i has contact links with disjoint sets of individuals on days d1 and d2, LCS(i, d1, d2) = 0. Partially overlapping contact links with different cumulative duration (encoded in the weights wij(d1) and wij(d2)) instead lead to 0 < LCS(i, d1, d2)<1, reaching 1 only for an individual i with exclusively the same contact links and proportional weights in both days. The distribution of values of the local similarity between any pairs of data collection days for the participating students [8] exhibits relative maxima at 0 and 1 values but is spread over all possible similarity values, showing a strong heterogeneity of similarity patterns (see Results Section) [30]. We thus tune the algorithm parameters f, p and ptr to reproduce this specific shape. Specifically, we perform a grid-exploration for these three parameters to minimise the Jensen-Shannon distance [49, 50] between the synthetic and empirical distributions of local cosine similarities. Details on the optimisation process, the definition of the Jensen-Shannon distance as well as a sensitivity analysis on the distance measure used can be found in the S1 Text. In the following, contact networks generated with this method, using optimised parameters, are called friendship-based (daily) contact networks. These synthetic daily contact networks can also be expressed as lists of temporal contacts between individuals (with the same temporal resolution as the empirical data), which we call friendship-based (synthetic) contacts. Class-mixing-based approach. In the “class-mixing-based approach”, the synthetic contacts are generated in order to reflect the empirical mixing patterns between classes observed during the base day considered. However, the algorithm does not take into account the distinction between friendship and casual links, nor the transitivity (see S1 Text for details on the procedure). It is thus approximately recovered instead by the previous algorithm with parameters f = 0, p = 0 and ptr = 0. In particular, the number of contact links within each class and among each pair of classes is preserved, as in the friendship-based approach. However, the contact links between students are renewed each day independently of any previous occurrence. Overall, the synthetic contacts obtained in this approach preserve the class structure of the school, as well as the timetable features, but not the individual friendships. Creating synthetic contact data of arbitrary length. The algorithms described above create synthetic daily contact networks from empirical base days. As the school timetable has typically a weekly periodicity, a natural procedure to create synthetic data covering e.g. n weeks, starting from empirical data collected during one week, is to create n instances of each empirical base day, and to create synthetic weeks using a synthetic instance of each weekday (note that the students are considered isolated during the weekends [21, 24]). In the present case, as, on the Monday, the data was collected only during half a day (Supplementary figures of [8]), we use the Tuesday data as base day to create the Mondays synthetic contact networks. The synthetic contact sequences built using the friendship-based approach applied in this way to the four base days of data (Tuesday, Wednesday, Thursday, Friday) are called “Friendship 4d” in the following, while synthetic contact sequences obtained with the class-mixing-based approach are called “Class Mixing 4d”. As several data collection efforts have been carried out over even shorter timescales [3, 44], we also mimic here such cases, by artificially restricting the number of base days available. For the friendship-based algorithm, the minimal number of base days (in order to separate friendship from casual links) is two, while for the class-mixing-based approach we can even restrict the data to one day. We then create synthetic weekly sequences by combining the synthetic instances obtained from the various base days as detailed in Table 1. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Contact sequences used in the analysis. https://doi.org/10.1371/journal.pcbi.1012661.t001 In addition, we considered as a baseline synthetic contact sequences built exclusively by repeating empirical ones, i.e., by looping over empirical contact data as done in many previous works. We call the resulting data looped contacts, where the repeated empirical data can include from one to four of the available daily contact networks. Contact link weights of all sequences are uniformly rescaled to match the total daily interaction time in the “Looped 4d” contacts. This ensures the comparability of epidemiological outputs on contact sequences generated from different sequences of base days. We denote in the following the various synthetic contact sequences by ctx (where x is one of the sequence names of Table 1). Friendship-based approach. In this approach, we take into account that, as discussed above, the contact patterns among students are not fully random but bear similarities over different days. Contacts between a given pair of students may reoccur in different days [6, 34, 46], in particular between friends, and the contacts total duration (link weights) of friendship links also tend to be larger [7]. We thus propose the algorithm below (summarized in Fig 1) to generate synthetic daily contact networks. The algorithm takes as input (1) one day of data collection, that we call the base day and (2) the separation between friendship links and casual links of that day. It can thus be ran independently for each day of data collection once the friendship links have been extracted from the comparison of the various empirical daily contact networks. The algorithm generates contact links class by class, and pair of classes by pair of classes until all classes (and pairs of classes) have been considered. Combining all the contact events generated provides synthetic contacts inheriting the properties of the daily contact network of the base day considered. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Schematic representation of the friendship-based approach to synthetic contacts generation. (A) Daily high-resolution empirical contact networks are schematically represented. They form the starting point (input) of the method. Friendship links are highlighted in orange, while blue links correspond to empirical links occurring only on a single day. (B) Different lists required for the algorithm are shown with examples of entries. Note that the timelines (lists of timestamps) are expressed in seconds from midnight on the initial day of the deployment, and the weights (“w”) are expressed in seconds. These lists are specific for each class and pair of classes, and each base day. (C) Description of the different steps of the algorithm. When assigning randomly a timeline and weight to contact links (step 3 of (C)), an entry of the third table is drawn. The algorithm operates class by class, and pair of classes by pair of classes, to generate synthetic contact networks. (D) Schematic representation of the generated synthetic contact networks. These networks inherit properties from the empirical contact networks, such as the number of links per class and between each pair of classes, a fraction of the friendship links (depicted in orange) and of the non-repeated links (depicted in blue). These links are complemented by random links (depicted in green) that were not necessarily observed in the base day. https://doi.org/10.1371/journal.pcbi.1012661.g001 For a given base day d, the algorithm proceeds as follows to build synthetic contact networks between the students of each pair of classes A and B (and among the students of each class A = B). 1. Inputs from data: the algorithm uses as inputs the number of contact links between classes A and B on the base day, the lists of contact links between them (i.e., all contact links such that i is in class A and j is in class B or vice-versa), of their weights (cumulative contact durations) and associated timelines . The inputs include as well the friendship links on the base day between individuals of classes A and B, which are put in a dictionary of the form , where {w}ij is the list of weights of (i, j), and {tl}ij the list of timelines of (i, j) on all days in which the friendship link is observed in the data. 2. Taking friendships into account: Each of the friendship links in is included as a contact link in the synthetic network with probability f. For each such link (i, j) added in this step, a synthetic weight and timeline are generated from {w}ij and {tl}ij (see S1 Text for details). 3. Remaining links: Additional links are added one at a time. Each such link is either extracted from the list of empirical contact links , with probability p, or created randomly (with probability 1 − p) by choosing at random two students respectively in classes A and B. In both cases, with probability ptr we add an extra step to preserve transitivity in the network [47] (the fraction of closed triangles, i.e. of structures {(i, j), (j, k), (k, i)} among all possible connected triads, i.e., such that at least {(i, j), (j, k)} exist). Specifically, if the transitivity of the current synthetic network between classes A and B is lower than the empirical one, the additional link is chosen (either in or randomly) such that it closes a triangle; conversely, if the current transitivity is too high, the additional link is chosen in order to create an open triangle. With probability 1 − ptr, the choice of the additional link is made independently from the transitivity of the network. This procedure of adding links is iterated until the synthetic contact network between classes A and B (or within class A = B) contains contact links. Each added contact link is then associated a timeline and matching weight selected randomly from and . 4. Weight correction: Once the synthetic contact network between classes A and B (or within class A = B) includes the correct number of contact links, timestamps can be added or removed from activity timelines in the network (longer timelines are preferably modified) to ensure that the sum of all link weights in the synthetic contact network between classes A and B (or within class A = B) is within a 10% tolerance of the corresponding quantity in the empirical contact network of the base day—see S1 Text for details. The above steps are repeated for every pair of classes A and B and for each class (A = B). The resulting synthetic contact networks are then merged into daily synthetic contact networks involving all individuals present in the base day. By construction, these synthetic contact networks preserve the number of contact links and the total time in contact within and between each class (steps 3 and 4). In addition, as the timelines of the contact links are taken from the lists of empirical ones, the timetable of the base day is also preserved. In the algorithm, the parameter f controls the average fraction of friendship links that are preserved in the synthetic data. The parameter 1 − p controls the amount of stochasticity, by allowing links in the synthetic network between students who have actually not been observed in contact during the base day. In the following, we describe how we tune these parameters (f and p), as well as ptr, to reproduce some features of the empirical networks. Parameters optimisation. We tune the parameters of the model in order to take into account the day-to-day similarity between daily contact networks observed empirically. To quantify this similarity, we consider the local cosine similarity [48], which measures the similarity in the contact links of an individual i in two daily contact networks (on days d1, d2). Mathematically, it is given by (1) where (resp. ) refers to the set of nodes j in contact with node i on day d1 (resp. d2) with contact link weights wij(d1) (resp. wij(d2)). The sums in the denominators run over (resp. ) which is the set of nodes in (resp. ) that are also present in the daily network of day d2 (resp. d1). This local-node-based similarity measure ranges from 0 to 1. When i has contact links with disjoint sets of individuals on days d1 and d2, LCS(i, d1, d2) = 0. Partially overlapping contact links with different cumulative duration (encoded in the weights wij(d1) and wij(d2)) instead lead to 0 < LCS(i, d1, d2)<1, reaching 1 only for an individual i with exclusively the same contact links and proportional weights in both days. The distribution of values of the local similarity between any pairs of data collection days for the participating students [8] exhibits relative maxima at 0 and 1 values but is spread over all possible similarity values, showing a strong heterogeneity of similarity patterns (see Results Section) [30]. We thus tune the algorithm parameters f, p and ptr to reproduce this specific shape. Specifically, we perform a grid-exploration for these three parameters to minimise the Jensen-Shannon distance [49, 50] between the synthetic and empirical distributions of local cosine similarities. Details on the optimisation process, the definition of the Jensen-Shannon distance as well as a sensitivity analysis on the distance measure used can be found in the S1 Text. In the following, contact networks generated with this method, using optimised parameters, are called friendship-based (daily) contact networks. These synthetic daily contact networks can also be expressed as lists of temporal contacts between individuals (with the same temporal resolution as the empirical data), which we call friendship-based (synthetic) contacts. Class-mixing-based approach. In the “class-mixing-based approach”, the synthetic contacts are generated in order to reflect the empirical mixing patterns between classes observed during the base day considered. However, the algorithm does not take into account the distinction between friendship and casual links, nor the transitivity (see S1 Text for details on the procedure). It is thus approximately recovered instead by the previous algorithm with parameters f = 0, p = 0 and ptr = 0. In particular, the number of contact links within each class and among each pair of classes is preserved, as in the friendship-based approach. However, the contact links between students are renewed each day independently of any previous occurrence. Overall, the synthetic contacts obtained in this approach preserve the class structure of the school, as well as the timetable features, but not the individual friendships. Creating synthetic contact data of arbitrary length. The algorithms described above create synthetic daily contact networks from empirical base days. As the school timetable has typically a weekly periodicity, a natural procedure to create synthetic data covering e.g. n weeks, starting from empirical data collected during one week, is to create n instances of each empirical base day, and to create synthetic weeks using a synthetic instance of each weekday (note that the students are considered isolated during the weekends [21, 24]). In the present case, as, on the Monday, the data was collected only during half a day (Supplementary figures of [8]), we use the Tuesday data as base day to create the Mondays synthetic contact networks. The synthetic contact sequences built using the friendship-based approach applied in this way to the four base days of data (Tuesday, Wednesday, Thursday, Friday) are called “Friendship 4d” in the following, while synthetic contact sequences obtained with the class-mixing-based approach are called “Class Mixing 4d”. As several data collection efforts have been carried out over even shorter timescales [3, 44], we also mimic here such cases, by artificially restricting the number of base days available. For the friendship-based algorithm, the minimal number of base days (in order to separate friendship from casual links) is two, while for the class-mixing-based approach we can even restrict the data to one day. We then create synthetic weekly sequences by combining the synthetic instances obtained from the various base days as detailed in Table 1. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Contact sequences used in the analysis. https://doi.org/10.1371/journal.pcbi.1012661.t001 In addition, we considered as a baseline synthetic contact sequences built exclusively by repeating empirical ones, i.e., by looping over empirical contact data as done in many previous works. We call the resulting data looped contacts, where the repeated empirical data can include from one to four of the available daily contact networks. Contact link weights of all sequences are uniformly rescaled to match the total daily interaction time in the “Looped 4d” contacts. This ensures the comparability of epidemiological outputs on contact sequences generated from different sequences of base days. We denote in the following the various synthetic contact sequences by ctx (where x is one of the sequence names of Table 1). Transmission model In the following, we leverage on the contact sequences from Table 1 to feed the stochastic agent-based model from Ref. [24] and simulate numerically the spread of SARS-CoV-2 in the school population. In this model, when a transmission event occurs, the exposed individual (E) becomes infectious and pre-symptomatic (Ip) after a time τE. The pre-symptomatic phase lasts τp, after which the individual enters either a sub-clinical (Isc) or clinical (Ic) infectious state, which lasts τI before recovery (R). The durations of each stage (τE, τp, τI) are drawn at random for each individual from Gamma distributions parameterised following the literature [24]. The probability of developing a clinical (versus a sub-clinical infection) is instead a fixed parameter. The per unit-time transmission probability when a susceptible individual is in contact with an infectious one depends on the transmission rate β and on the relative infectiousness and susceptibility of the individuals in contact, determined from infectious status. We model a vaccinated student population (homogeneous 50% coverage) by reducing the relative infectiousness of vaccinated individuals by 20% and their relative susceptibility to infection by 50%. Moreover, we model partial immunity from previous infection by SARS-CoV-2 pre-Omicron variants in 40% of the population. The susceptibility of immune individuals to infection is reduced by 81%. As time progresses, contacts encoded in the contact sequence considered are replayed in the numerical simulation. To increase computational efficiency, high-resolution 20 seconds contacts are aggregated in 15 minutes steps: each resulting contact is weighted by the fraction of time actually spent in contact over the 15 minutes for each pair. Each such contact event in which an infectious individual is interacting with a susceptible one represents an opportunity of transmission for the simulated disease. We consider a simple spreading scenario in the absence of control measures initiated by a single infectious individual (the seed) among the students. The transmission rate β is tuned as in [24], to achieve an effective reproductive number in the population with partial vaccination and immunity R = 1.5 (see S1 Text for details on the calibration). Extraction and comparison of infection pathways For each contact sequence (ctx, in Table 1) and for each initial seed s, we build an infection network [42, 43], denoted , by aggregating transmission chains obtained from different simulations (Fig 2A and 2B). These infection networks are directed, in order to encode transmission events from i to j and from j to i separately. Both occurrences indeed correspond to different transmission chains. Each may occur with a different probability (depending on vaccination statuses of the pair, disease stages during which the contact occurs). Each directed edge ℓ = (i, j) ≠ (j, i) from student i to j is weighted by the occurrence probability of a transmission event from i to j (denoted pℓ(s, ctx)), estimated by the fraction of simulations in which such an event is observed [43]. This probability depends not only on the per unit time probability of transmission (see S1 Text), but also on the relative positions of i and j in the contact network. The latter impacts in particular whether i is infected before or after j. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Construction of infection pathways. (A) Examples of simulated transmission chains between individuals are shown for a given seed. (B) An infection network built from 135 realisations of the model initialised with the same seed is shown. (C) Maximum spanning tree extracted from the infection network of (B). All results are obtained with ctx = “Friendship 4d” contacts. Darker edges in (A) correspond to transmission events between different classes. Edge widths in (B) and (C) are proportional to their probability of occurrence pℓ(s, ctx). Edges with probability of occurrence <0.01 are omitted for readability in (B). Nodes of the same class share the same color, and the seed is highlighted in black. Visualisations generated with Gephi [51]. https://doi.org/10.1371/journal.pcbi.1012661.g002 Infection networks form an aggregated representation of the outbreak, in which a large fraction (or even all) of the contact links between individuals in the population are represented. We thus also consider summarized versions, the infection trees (Fig 2C). Each infection tree, denoted , is given by the (directed) maximum spanning tree of the corresponding infection network, as described in [42]. It approximates the “most likely” infection pathway from the seed s to each node, by retaining edges ℓ that form a directed tree, such that their summed pℓ(s, ctx) is maximum. The infection pathways defined in this way (networks and trees) describe the progression of the outbreak between students, for each given seed s. To adopt a more coarse-grained view, we also build pathways using aggregated transmission chains between classes, in order to represent the pathways of importation of cases into previously unexposed classes. We refer to the S1 Text for additional details on the construction of the four types of infection pathways introduced. In order to investigate the impact of the type of synthetic data used in the simulations, we compare the infection networks obtained with the same given seed s for two different contact sequences (denoted by cta and ctb) using the global cosine similarity defined by: (2) where (resp. ) denotes the set of edges of (resp. ). Note that we can also compute the global cosine similarity between pairs of infection trees. This global cosine similarity takes values between 0 and 1 and provides a measure of overlap between two infection networks (or trees). It is equal to 0 if and only if the networks (or trees) compared have no edges in common, and reaches 1 if and only if the two networks or trees are identical in their edges and respective weights. Results As already described in [7], the total daily interaction time within each class is larger than between pairs of classes (see Fig 3A for day 2). More interaction time is also observed between classes of the same specialty than between classes of different disciplines. This block diagonal structure is correctly reproduced by the synthetic contact networks (Fig 3B and 3C). The global school activity timeline (total interaction time per time window of 15 minutes) is also well reproduced by both types of synthetic contact data (Fig 3D). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Comparison between the different contact sequences. (A) Daily total time measured in contact within and between classes for the recorded contacts on day 2. (B) Same as (A) for the corresponding friendship-based contacts. (C) Same as (A) for the corresponding class-mixing-based contacts. (D) Total time measured in contact between all individuals in the school on successive 15 minutes time steps on days 2 and 3 for the three types of contacts (empirical and two types of synthetic data). (E) Distribution of students’ local cosine similarities for each pair of days observed in the empirical contacts (black), together with the same distribution obtained with the friendship-based algorithm with optimised parameters, averaged over 10 realisations. (F) Same as (E) obtained instead from 10 realisations of the class-mixing-based approach. (G) Global similarities between the daily contact networks of consecutive days (computed by applying Eq 2 to the contact networks), for contact sequences obtained with different versions of the algorithm (each color corresponds to one single iteration of the contact sequence). https://doi.org/10.1371/journal.pcbi.1012661.g003 As discussed above, the parameters of the friendship-based approach are tuned to reproduce the observed distribution of local cosine similarities in the contact links of each student in different days. We obtain as optimal parameters f = 0.8 (average fraction of friendship links included), 1 − p = 0.6 (probability of drawing remaining links randomly) and ptr = 0.75 (probability of adjusting for transitivity). We show in Fig 3E that the resulting distribution of similarities mimics indeed well the empirical one, and Fig 3F shows that the class-mixing-based approach leads instead to a qualitatively different picture with only small values of the similarities between the contact links of each individual in different days. This picture is confirmed by Fig 3G, which displays the global cosine similarity (calculated by applying Eq 2 between the daily contact networks) of consecutive days of the complete contact sequences: the values obtained with contact sequences built using the friendship-based approach are close to the empirical ones, while the class-mixing-based approach leads to very small values. We refer to the S1 Text for figures showing how the two approaches reproduce other properties of the empirical contact networks. For instance, friendship-based contact sequences closely preserve the fraction of repeated contacts, while class-mixing-based contacts underestimate it significantly (see S1 Text, Fig M). Friendship-based contacts additionally perform better than class-mixing-based contacts in reproducing the number of triangles and transitivity, within class and between classes (see S1 Text, Fig N). They also reproduce more closely than class-mixing-based contacts the degree distribution within class (number of contact links within the class of each individual, see S1 Text, Fig O) observed in empirical contacts. Finally, class-mixing-based and friendship-based contact sequences preserve equally well the distributions of node strengths (see S1 Text Fig O), link weights and number of distinct contact events per pair and day (see S1 Text Fig P), as well as characteristic features of the contacts such as the total daily time spent interacting, number of nodes and number of links (see S1 Text Fig N). Let us now discuss and compare the outcomes of the epidemic spreading simulations performed on the various contact sequences of Table 1. Fig 4 first focuses on the resulting distributions of epidemic sizes, by showing them as violin plots in Fig 4A, 4B and 4C. The distributions obtained with the three friendship-based contact sequences (Fig 4A) are visually very similar. This is confirmed quantitatively by the low Jensen-Shannon distances [49, 50] between these three distributions (Fig 4D). Looped contacts instead lead to epidemic size distributions with a different shape (Fig 4C), as confirmed from the Jensen-Shannon distances (Fig 4D). The distance with the epidemic size distributions obtained with the friendship-based contact sequences is particularly large when only one day is looped over. As the comparison of Fig 4A and 4C indicates, this is due to an overall shift of the distribution to lower values. This shift is due to the restricted number of propagation pathways when the same contact patterns are repeated every day (“Looped 1d” case). When two days or more are included in the looped data, the range of epidemic size values reached by the simulations becomes similar to the one obtained with the friendship-based data. However, the shape of the distribution remains different: large outbreaks are in fact more probable, and outbreaks of small and intermediate sizes are much less probable. Finally, friendship-based and class-mixing-based contact sequences lead to visually rather similar distributions of epidemic sizes, with close values of the medians. Differences however emerge for large epidemic sizes, which are more probable when using contact sequences that preserve the balance between friendships and casual contacts. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Epidemic size distributions. (A) Distributions (Gaussian kernel density estimations) of the final epidemic sizes obtained with friendship-based contact sequences. (B) Same as (A) for class-mixing-based contact sequences. (C) Same as (A) for looped contact sequences. The distributions are computed over simulations leading to a fraction of infected individuals larger than 20% (over 120 days) in order to better highlight differences between the distributions. Results including all simulations are shown in the S1 Text, Fig S and T. The first and third quartiles (25% and 75%) are indicated with dotted lines while the median is shown with a dashed line. (D) Jensen-Shannon distance between all pairs of distributions. For each contact sequence, 150 simulations are conducted for each of the 325 students as seed (48, 750 simulations for each contact sequence). https://doi.org/10.1371/journal.pcbi.1012661.g004 To go beyond these distributions, we also characterise and summarise the propagation paths in the population, for each seed s and contact sequence ctx considered, by the infection networks and trees (computed between students on the one hand and between classes on the other hand), as described in the Methods section. We present here the analysis of the infection networks (Fig 5) and refer to the S1 Text for the same analysis concerning the infection trees. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Pairwise comparisons of simulated infection networks between students and classes. (A) Distributions over all seeds of the global cosine similarities are shown for infection networks obtained from pairs of contact sequences cta and ctb in “Friendship 2d”, “Friendship 3d” and “Friendship 4d” for infection networks between students. (B) Distributions of over all seeds s for different sequences ctb (class-mixing-based and looped contacts) with cta fixed to “Friendship 2d” for infection networks between students. (C) Same as (B) with cta fixed to “Friendship 4d”. (D) Same as (A) for infection networks between classes. (E) Same as (B) for infection networks between classes. (F) Same as (C) for infection networks between classes. For each contact sequence, infection networks are obtained from 150 simulations for each seed, and each of the 325 students are successively considered as seed s. https://doi.org/10.1371/journal.pcbi.1012661.g005 Fig 5A shows that the infection networks between students resulting from simulations performed on friendship-based contact sequences generated from two, three or four days of data and with the same seed are highly similar for almost all seeds (Fig 5A shows the distribution of GCS(s, cta, ctb) for all seeds s and different pairs of contact sequences cta, ctb): the number of days used when building the contact sequences does not impact strongly the propagation patterns in the population. Contact sequences built from the class-mixing-based approach lead to infection networks having significantly lower similarity with the infection networks of the friendship-based synthetic data (Fig 5B and 5C). Note that the distributions of GCS(s, cta, ctb) when cta and ctb are both class-mixing-based contact sequences, but built using different numbers of days, peak at approximately 0.4 (see S1 Text Fig X): a quite high diversity of infection networks is thus also obtained within the class-mixing-based approach. These results suggest significant differences in the detailed spreading patterns observed with the two types of synthetic data. On the other hand, simulations on looped contact data are again rather similar to the ones of friendship-based contact sequences (Note also that, for cta and ctb both looped contact sequences –built with different numbers of base days– the infection networks are highly similar, with distributions of GCS(s, cta, ctb) peaking between 0.75 and 0.9, see S1 Text Fig Y). The results concerning the similarities between infection networks might be put in relation with the distributions of similarities of the individual contact links in different days: they are indeed close for looped and friendship-based synthetic data, while they are very different for class-mixing-based data (Fig 3E, 3F and 3G). This is consistent with the fact that the number of days looped over has an additional impact on the similarity between the resulting infection networks and the ones of the friendship-based data. In particular, infection networks obtained on “Looped 1d” synthetic contact sequences are significantly more dissimilar than for the rest of the looped contact sequences, due to the lack of diversity of these contact sequences. These contact sequences indeed inherit only the contact links and properties of a single day of measurements and lack a realistic diversity in contact behaviour (the local cosine similarities between days are all trivially equal to 1). Looping over as many days (or more) than used to produce the friendship-based contact data leads to infection networks more similar to the ones obtained with the friendship-based data (e.g., Fig 5C indicates an increasing similarity with the infection networks of friendship-based contact sequences built from four base days as the number of days looped over increases from one to four). When considering the seed wise infection networks between classes, we obtain high values of the global cosine similarities (Fig 5D, 5E and 5F) regardless of the types of contact sequences used. These epidemiological outputs are thus not affected by the distinction between friendship and casual links. Only a slight shift to lower similarity values can be observed with the “Looped 1d” contacts. This can be caused by pairs of classes that do not interact at all on the day looped over, causing slight differences in spreading patterns. Interactions between these pairs of classes are instead observed as soon as two days of empirical data are taken into account. Note that class-mixing-based contact sequences generated from a single base day of data are less affected by this issue, because the stochastic character of the algorithm creates many different pathways between the other classes, and overall produces enough pathways between these non-interacting classes through a third class. The presence of outliers in these distributions (as evidenced by the tails of the distribution) reveals that the propagation paths of the disease between classes remain sensitive to the type of contact sequences considered for a minority of the seeds. We refer to the S1 Text for additional results teasing out the individual impact of the friendship-preserving parameter f and transitivity preserving parameter ptr on epidemic size distributions and infection pathways. Discussion We have proposed and explored two approaches to generate synthetic contact sequences with realistic statistical properties between students in a school. Both preserve the class-mixing matrices (i.e., the amount of interactions within each class and between pairs of classes) observed daily during the data collection, as well as the global timeline of activity driven by the underlying timetable. The class-mixing-based approach creates contact links independently for each simulated day. On the other hand, the friendship-based approach preserves the balance between contacts that reoccur over different days (indicative of friendship links between students) and casual contacts that were observed only on one of the days of the data collection. This balance is preserved both at the global level and in terms of its heterogeneity between students. We have then performed numerical simulations of a model describing the spread of an infectious disease in the population, informed by synthetic contact sequences built using the two approaches and relying on a varying amount of empirical data. Specifically, we have considered the spreading dynamics of SARS-CoV-2, as an example of recent important concern, in particular in school settings. We have compared the outcomes of the simulations at the level both of the distributions of final epidemic sizes and of the most probable infection pathways between students and across classes. The friendship-based approach successfully reproduces the main features of empirically observed contact patterns in a school population, while the class-mixing-based approach fails to capture the heterogeneity of similarity levels in the contact patterns of individuals on different days. Both types of synthetic contacts accurately represent the global features of the school where the base data was collected (class-mixing matrix, activity timeline). By preserving as well the balance between friendship relations that lead to repeated contacts and casual encounters, synthetic contact sequences generated with the friendship-based approach provide realistic contact data that can be used for modelling purposes. We note that our work notably differs from several other mechanisms that have been proposed to go beyond the simple looping procedure to extend contact network data while preserving some properties of the data, either in a general context [39] or for epidemic modelling purposes [13, 24]. In particular, the approach of [39] does not preserve class-mixing patterns and does not take into account the existence of repeated contacts (friendships), leading to more random interaction patterns than actually observed. On the other hand, references [13] and [24] focused only on reproducing the global fraction of repeated contacts (whatever their durations) between daily networks (in [13], by a reshuffling of nodes that led to an identical underlying structure in successive days, in [24] by a non-optimized ad-hoc procedure), without dealing with the population heterogeneity. Here, by considering the whole distribution of local cosine similarities between the neighborhood of individuals in different days, and by performing an optimization of the parameters of the algorithm, we can create synthetic contact sequences that faithfully reproduce the heterogeneity of contact behaviours in the population. Finally, several mechanisms to generate realistic synthetic contact data have also been devised to correct for incompleteness of data due to diverse types of sampling [52–55]. These methods however are designed to reconstruct missing contacts occurring alongside empirically recorded contacts, covering thus the data collection window. The friendship-based and class-mixing-based approaches that we have presented fill a different purpose, and instead longitudinally extend limited existing data. When feeding synthetic data to numerical simulations of spreading models, taking into account the presence of friendships, which lead to repeated contacts with correlated durations in successive days, impacted mostly the transmission patterns at the level of students. Indeed, infection pathways between students are consistently dissimilar when feeding the model with either friendship-based or class-mixing-based contact sequences. Preserving the balance between friendships and casual encounters is therefore essential to accurately characterise disease spreading patterns between students. Previous works have already noted that contact repetition must also be captured when building daily contact networks [33] from survey data [35, 36]. In that context, ignoring the reported repetition of contacts [31] overestimates the weekly number of distinct contacts per individual. This in turn provokes transmission opportunities that would not occur otherwise, resulting in simulated higher attack rates and lower extinction rates [33]. In the present study, which considers the specific context of interactions in a school, we do not observe the effects reported in Ref. [33], although friendship-based and class-mixing-based approaches indeed lead to different distinct numbers of contact per individual over a set period (e.g., one week). This can be explained by the rather high density of contacts among individuals in the school: the repetition of contacts in this context does not hinder the spread of the disease as a sufficiently high number of distinct contacts between students is present in all cases (see S1 Text Fig Q). Finally, it has been discussed that contact repetition must be considered to accurately identify superspreaders and super-spreading events from contact data [34]. While our study does not explicitly investigate this aspect, we expect the friendship-based contact data to faithfully capture super-spreading dynamics thanks to the fact that it preserves the correlations between contact patterns in different days. Changing scale from the individual to the classes, we find that preserving class-mixing patterns and global activity levels throughout the day is sufficient to predict the spread of the disease at the scale of classes. Consequently, while taking into account friendship links in synthetic contact data is necessary to evaluate localised and fine grained mitigation strategies targeted to students (e.g. reactive testing of contacts), larger scale protocols targeting classes (such as reactive class closure), grades (closure of all classes of a grade) or the entire school (school closure) can be evaluated using synthetic contact data generated with the class-mixing-based approach. This is in line with the negligible impact of randomising contacts per class on average transmissions found in Ref. [9], and our analysis confirms this in the case of class-based infection pathways. Our study also highlights consequences of looping over the data, an approach commonly used to inform models with high-resolution data. We note two opposite effects at play. Firstly, looping over recorded data limits the pool of susceptible individuals an infectious individual can pass the infection to (see S1 Text Fig Q). This results in a general shift of epidemic size distributions to lower sizes. This effect is strongest when the number of distinct contacts an individual has over their infectious period is low. In the looped contacts considered, the contact networks are sufficiently dense that this effect is insignificant as soon as two days are looped over. A second effect instead causes the inflated outbreaks observed. The periodic recurrence of all contacts can reinforce transmission routes as identical contacts reoccur during the infectious period. This effect counteracts the depletion of susceptible individuals. In our simulations, we find that it dominates as soon as two days of data are looped over, yielding inflated epidemic sizes. Larger epidemic sizes are known to occur when repeated contacts are not included [33], when contact duration are averaged over the contacts [9, 13] and where the network of contacts is discarded in favor of an all-mixing approximation [29]. The larger outbreaks observed with the looped contacts are instead attributed here to the reinforcement of contacts looped over, in a context where sufficient contacts allow unhindered propagation. Finally, our results contribute to the design of efficient data collection windows. Both infection pathways and epidemic size distributions obtained from friendship-based contacts remain highly similar to each other as two, three or four base days are used to generate the synthetic contact sequences. Additionally, infection pathways built from friendship-based contacts obtained with two days of data remain similar to pathways built from looping over two, three or four days of data. Friendship-based contact sequences useful for modelling purposes may therefore be obtained from two days of data. This is in line with previous results showing that two days of recordings are sufficient to accurately predict the epidemic threshold in a temporal network of contacts measured over three days in a workplace context [56]. We note that, as contacts in schools or workplaces are dictated by timetables and shifts, one day of data may already provide representative global features of the contacts (classes, or groups interacting more tightly, structure of a typical day). A minimum of a second day is then necessary to distinguish stable relationships from casual contacts. Our work presents several limitations worth discussing. First, the friendship-based approach associates to each synthetic contact link a total daily duration and timeline observed in the class, or pair of classes of the contact. This pool of weights and timelines may be too limited when few interactions occur, leading to an unrealistic lack of diversity in the timelines. This does not impact spreading dynamics for diseases progressing slower than the timeline time step [13], as is the case here. In the case of faster disease spread, it may become necessary to generate synthetic timelines from observed timestamps [52] in order to avoid unrealistic repetitions. Second, the numbers of contacts are exactly preserved for each class and pair of classes. Pairs of classes not interacting at all during the data collection period are thus also not interacting in the synthetic data, which may not be realistic over longer periods. In our specific case, this does not affect the friendship-based approach as all classes have interacted by the end of the second day. In other settings, care should be taken to ensure enough contacts are observed to populate the class-mixing matrix. Moreover, the number of days a contact must reoccur over to be considered a friendship was fixed to two, regardless of the time spent interacting on either days. Further tuning of this criteria may be necessary, e.g. for a longer data set in which even casual contacts might be repeated. Finally, further validation of our algorithm is limited by the absence of longer recordings gathered in the school setting. Acquiring such data sets would be highly valuable for future research. We finally note that the friendship and class-mixing-based approaches extend existing short recordings in the school context to inform models with realistic contact inputs. Generalising these to other contexts might require to capture additional contact characteristics. For example, synthetic contacts in the healthcare setting should capture the turnover of patients due to intakes and discharges of patients, as well as staff shifts. Our approach would therefore need to be modified to capture these elements in order to extend contact data measured e.g. in healthcare settings [16]. Conclusion The friendship-based and class-mixing-based approaches provide modellers with generalisable methods to generate synthetic contact sequences over long time-scales from existing data. The friendship-based approach captures repeatability in contacts inherent to social behaviour, a feature crucial for the prediction of infection pathways between students. This contributes to a growing toolkit allowing modellers to inform agent-based models with data of increasing realism without the need for further expensive data collection. Supporting information S1 Text. A separate pdf file is provided containing complementary technical details and analyses. https://doi.org/10.1371/journal.pcbi.1012661.s001 (PDF) Acknowledgments We thank Diego Andrès Contreras for helpful discussions.

Guet-McCreight, Alexandre;Mazza, Frank;Prevot, Thomas D.;Sibille, Etienne;Hay, Etay

doi: 10.1371/journal.pcbi.1012693pmid: 39729407

Introduction Major depressive disorder (depression) is a leading cause of disability, but the efficacy of current treatment methods is often partial or patients are treatment-resistant [1,2], indicating disease mechanisms that the current methods do not address. In recent years, reduced cortical inhibition has been implicated as a mechanism in depression [3–8], and new pharmacology that provided positive allosteric modulation of the α5 subunit of GABAA (α5-GABAA) receptors (α5-PAM) in chronically stressed mice [9,10] had antidepressant, anxiolytic, and pro-cognitive effects. However, testing the effect of the new drugs on human brain activity is currently limited. We have recently overcome these limitations by characterizing the effects of α5-PAM on cortical function and electroencephalography (EEG) measures of efficacy in silico using detailed models of human depression microcircuits [11]. This computational approach can further offer dose prediction tools via systematic characterization of simulated EEG effects across varying levels of depression severity and applied dose. Due to their subunit selectivity, α5-PAM are optimized to selectively boost inhibition generated by somatostatin-expressing (SST) interneurons, which provide synaptic and extrasynaptic (tonic) inhibition to pyramidal (Pyr) neuron apical dendrites via α5-GABAA receptors [12–15]. A loss of SST interneuron inhibition in depression is implicated by reduced SST expression in human patients postmortem [16,17], increased anxiety- and depression-like symptoms in rodents with brain-wide SST interneuron silencing [18], and SST interneuron transcriptome deregulations compared to other cell types following chronic stress in rodents [19]. This is further supported by the joint inhibitory effects and co-release of SST and GABA [20,21]. SST interneurons mediate lateral inhibition through disynaptic loops [22,23] and maintain low Pyr neuron spike rates at baseline [24–26]. A reduced SST interneuron inhibition in depression would thus increase baseline cortical activity (noise) and impair signal-to-noise ratio (SNR) of cortical processing [8,27,28]. Detailed biophysical models can capture key properties of EEG [29–31], and thus provide powerful tools for linking cellular and circuit mechanisms to brain activity and clinically-relevant signals. We previously showed in silico that α5-PAM would recover cortical activity, function and EEG spectral profile in detailed models of human depression microcircuits with reduced SST interneuron inhibition, and we highlighted EEG biomarkers that can monitor α5-PAM efficacy [11]. However, the study utilized an average reduced SST interneuron inhibition as a model of depression microcircuits [28] and EEG effects [30], and did not consider varying depression severity level. A similar approach was used by detailed modeling studies that identified the effects of schizophrenia-related gene variants on neuronal cellular mechanisms and EEG features [32]. Other studies linked cellular and circuit mechanisms to features of the EEG response during task processing in health [33] and in relation to schizophrenia biomarkers [34]. EEG biomarkers have been used broadly in depression classification and treatment prediction. Particularly, elevated theta and alpha power are characteristic features in the frontal and parietal electrodes of depression patients [35–39] as well as increased aperiodic activity [40]. We previously showed mechanistically, using detailed computational models, that increases in theta, alpha, and aperiodic power would result from reduced SST interneuron inhibition [30]. EEG power in different frequency bands also served to predict treatment response in depression [41]. In particular, theta and alpha power in depression patients have been used as indicators of treatment response [42–46], and similarly served as biomarkers in our previous simulated cortical microcircuit response to α5-PAM [11]. In this study we simulated individual microcircuits using our previous detailed models of human microcircuits in health and depression [28], but with varying levels of depression severity in terms of reduced SST interneuron inhibition. For each individual microcircuit we simulated α5-PAM dose response using our previous models of α5-PAM effects [11], and developed machine learning models to predict optimal dose from EEG biomarkers for recovering cortical microcircuit activity, function and EEG profile. Results We simulated α5-PAM dose effects on detailed models of prototypical human cortical microcircuits with different levels of depression severity, in terms of reduced inhibition from SST interneurons (Fig 1A). The models included four key neuron types (Pyr neurons and SST, PV and VIP interneurons) with type-specific synaptic properties and connection probabilities. Depression severity ranged from 0 to 40% reduced SST-mediated synaptic and tonic inhibition onto all cell types in the microcircuit. We simulated α5-PAM dose effects on the apical dendrites of Pyr neurons, ranging from 0 to 150% relative to the effect of a reference 3 μM dose of α5-PAM (ligand GL-II-73) on human cortical Pyr neurons, which we had characterized experimentally and modeled previously (see Methods). We simulated baseline (resting-state) activity in the microcircuits, together with dipole moments from which we calculated EEG signals. We thus simulated dose responses in 100 “individual microcircuits” (microcircuits with randomized connectivity and activity) with different levels of depression severity (Fig 1B). EEG power increased with severity of SST interneuron inhibition (Fig 2A), in particular in the θ frequency band (4–8 Hz; Pearson correlation, r = 0.69, p = 1.52e-15; Fig 2B) and α frequency band (8–12 Hz; r = 0.75, p = 1.90e-19; Fig 2C), as well as in the broadband (3–30 Hz) range of the 1/f component (r = 0.71, p = 1.83e-16; Fig 2D). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Simulating EEG of inhibition loss severity in depression and treatment response. A. Simulated neuronal spiking and EEG in human cortical L2/3 microcircuits in health, depression and under application of α5-PAM. Microcircuits were comprised of 1000 detailed neuron models of four types: Pyr (black), SST (red), PV (green), and VIP (yellow). The connectivity schematic (top left) highlights the cell-specific connectivity, the mechanisms of depression (MDD; loss of SST tonic and synaptic inhibition onto all cell types) and α5-PAM doses (boosted SST tonic and synaptic inhibition to Pyr neurons). B. We simulate five levels of SST inhibition loss severity (0%, 10%… 40%) across 20 different microcircuits each, representing a total of 100 different individual microcircuits. For each microcircuit we simulated a dose-response of α5-PAM (0%, 25%, 50% … 150% relative to the reference dose) and identified ground-truth optimal dose and range. https://doi.org/10.1371/journal.pcbi.1012693.g001 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Simulated EEG biomarkers of depression severity. A. PSD of simulated EEG from each severity level of SST interneuron inhibition loss (bootstrapped mean and 95% confidence intervals across microcircuits). Inset–PSD plotted in log scale. B-D. PSD power in theta band (4–8 Hz, B), the alpha band (8–12 Hz, C), and broadband 1/f (3–30 Hz, D) for each level of SST interneuron inhibition loss (grey–healthy standard deviation; dashed line–healthy mean; error bars = mean and standard deviation). All asterisks denote significant paired t-tests (p < 0.05) with effect sizes greater than 1 when compared to healthy. https://doi.org/10.1371/journal.pcbi.1012693.g002 For each individual microcircuit, we computed the PSD following each α5-PAM dose administration (Fig 3A and 3B) and extracted PSD metrics (1/f, α, and θ power). We fitted a dose-response function (relating dose and PSD metrics: Dose = m1∙1/f + m2∙θ + m3∙α + b) and used it to identify the optimal dose and optimal dose ranges that would bring the microcircuit’s EEG features to the healthy mean and the healthy ranges, respectively (Fig 3C). We then used the optimal doses to fit a dose prediction model, using multivariate linear regression with the depression PSD metric values as input features (Fig 3D). Prediction accuracy for the test subset of individual depression microcircuits was high (90% ± 5%, n = 50 permutations of fit/test microcircuit sets) and better than using univariate linear regression models with single PSD feature (1/f: 77% ± 6%, p = 3.25e-18, Cohen’s d = -2.1; θ: 84% ± 7%, p = 3.72e-6, Cohen’s d = -1.0; α: 79% ± 7%, p = 4.25e-15, Cohen’s d = -1.9; Fig 3E). For the cases where the dose prediction models were incorrect, they did not tend to underestimate or overestimate the correct dose (under-estimation: 4.7% ± 4.4%, over-estimation: 5.3% ± 2.8%, p > 0.05, Cohen’s d = 0.1). We selected the dose prediction model with the highest accuracy (predicted dose = 0.31∙1/f + 0.77∙θ + 1.18∙α - 0.64) and used it for the rest of the analysis. The predicted doses recovered EEG features for the test subset of individual depression microcircuits to the healthy range (normalized 1/f: 0.46 ± 0.11 vs 0.48 ± 0.13, p > 0.05, Cohen’s d = -0.2; normalized θ: 0.20 ± 0.10 vs 0.25 ± 0.10, p > 0.05, Cohen’s d = 0.5; normalized α: 0.39 ± 0.10 vs 0.34 ± 0.07, p > 0.05, Cohen’s d = -0.5; Fig 3F). Inclusion of periodic features did not improve accuracy compared to using raw α and θ power and aperiodic 1/f. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. EEG biomarkers of depression severity predict α5-PAM dose accurately. A. Schematic illustrating the approach—we found optimal α5-PAM dose for each individual microcircuit based on power spectral biomarkers of their simulated EEG, and used the optimal doses to develop dose prediction models for restoring the EEG metrics back to healthy ranges across microcircuits. B. Example PSD profiles for one individual microcircuit (with 30% reduced SST interneuron inhibition, magenta), and under application of 100% of the reference α5-PAM dose (blue). Healthy mean and full ranges are shown in grey. C. Dose-response for the same individual microcircuit as in B, with 30% reduced SST interneuron inhibition (circle outlined in magenta), plotted across three EEG features (1/f, θ, α). A fit of the response was used to obtain the optimal dose (diamond color) and range with respect to the healthy EEG mean (diamond position) and ranges (grey cube), respectively. D. Predicted doses for each individual microcircuit (including healthy) as a function of its EEG features at baseline (before α5-PAM application). E. Percent of correct dose prediction for test sets of individual microcircuits, for dose prediction models using either multivariate (MV) or single EEG biomarkers (50 permutations; blue = under-estimated errors; red = over-estimated errors). F. EEG metrics of all individual microcircuits before (magenta) and after applying the predicted optimal α5-PAM dose (blue). https://doi.org/10.1371/journal.pcbi.1012693.g003 The model’s predicted dose based on EEG biomarkers also recovered microcircuit spiking and function in terms of failed and false detection rates, as measured from pre- and post- stimulus Pyr neuron spike rate distributions below and above the signal detection threshold, respectively. We compared the simulated baseline spiking activity for each individual depression microcircuit in the test subset and each dose to spiking activity following a brief stimulus (Fig 4A) and calculated the proportion of failed and false detection errors based on microcircuit baseline and response spike rates (Fig 4B). We then estimated the functional recovery due to the predicted dose from the dose-response curves for Pyr neuron spike rate (Fig 4C), failed detections (Fig 4D) and false detections (Fig 4E), for each individual depression microcircuit in the test subset. We used a linear fit for the spike rate curves, and exponential fits for the detection rates curves. The predicted doses recovered the spike rates to the healthy ranges (0.76 ± 0.02 Hz vs 0.74 ± 0.07 Hz, p > 0.05, Cohen’s d = -0.4, Fig 4F), as well as failed detection rates (1.50 ± 0.25% vs 1.32 ± 0.59%, p > 0.05, Cohen’s d = -0.4) and false detection rates (1.59 ± 0.31% vs 1.42 ± 0.74%, p > 0.05, Cohen’s d = -0.3). For some individual microcircuits with 10–30% SST interneuron inhibition loss, the predicted doses based on EEG biomarkers were slightly over-estimated, as measured by functional metrics, and for some individual microcircuits with 40% reduced SST interneuron inhibition the predicted doses were slightly under-estimated, although they still brought the functional metrics much closer to the healthy range compared to before α5-PAM application. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Predicted α5-PAM dose using EEG biomarkers recovers microcircuit spiking and function. A. Example raster plot of simulated baseline spiking and response to a brief stimulus. Dashed line indicates stimulus time. Cell type color code is the same as in Fig 1A. B. Distributions of pre-stimulus firing rates from an example microcircuit with 40% SST inhibition reduction before (magenta) and after (blue) application of predicted dose. Average post-stimulus firing rates were similar across conditions, and are shown in black solid line. The overlaps between pre- and post- stimulus curves indicate failed and false signal detection errors. C—E. Dose-response curves in terms of Pyr neuron spike rate (C), failed detection rates (D) and false detection rates (E) for an example individual microcircuit with severity 40% SST inhibition reduction. The predicted dose based on EEG and the corresponding functional metrics is shown by the blue dot. F. Mean and standard deviation of spike rate (left), failed detection rates (middle), and false detection rates (right) in simulated depression (MDD) microcircuits before and after applying the predicted optimal dose. Grey area shows the healthy range. https://doi.org/10.1371/journal.pcbi.1012693.g004 Dose prediction models using an artificial neural network (ANN) or support vector machine (SVM) with multivariate EEG biomarker inputs had comparable dose prediction accuracy as using linear regression (ANN: 90% ± 3.5%, SVM: 92% ± 5.3%). Predicted doses using the ANN recovered EEG features (normalized 1/f: 0.49 ± 0.11, p > 0.05, Cohen’s d = 0.3; normalized θ: 0.25 ± 0.11, p > 0.05, Cohen’s d = 0.4; normalized α: 0.35 ± 0.10, p > 0.05, Cohen’s d = -0.3), spike rates (0.78 ± 0.09 Hz, p > 0.05, Cohen’s d = 0.3), as well as failed detection rates (1.68 ± 0.84%, p > 0.05, Cohen’s d = 0.3) and false detection rates (1.85 ± 1.06%, p > 0.05, Cohen’s d = 0.3). Predicted doses using SVM also recovered EEG features (normalized 1/f: 0.45 ± 0.13, p > 0.05, Cohen’s d = -0.09; normalized θ: 0.21 ± 0.10, p > 0.05, Cohen’s d = 0.2; normalized α: 0.34 ± 0.10, p > 0.05, Cohen’s d = -0.4), spike rates (0.73 ± 0.10 Hz, p > 0.05, Cohen’s d = -0.4), as well as failed detection rates (1.28 ± 0.74%, p > 0.05, Cohen’s d = -0.4) and false detection rates (1.36 ± 0.88%, p > 0.05, Cohen’s d = -0.3). To assess consistency in explainability between our prediction models we compared the mean SHAP feature importance values in each. In all cases, the SHAP values were largest for α, followed by θ, and then 1/f (linear regression: α = 0.14 ± 0.03, θ = 0.12 ± 0.02, 1/f = 0.05 ± 0.02; ANN: α = 0.11 ± 0.03, θ = 0.10 ± 0.03, 1/f = 0.09 ± 0.03; SVM: α = 0.15 ± 0.03, θ = 0.13 ± 0.03, 1/f = 0.04 ± 0.02). When generating dose prediction models that used Pyr neuron spike rates (Dose = 2.66∙rate—2.11) instead of EEG to predict doses, recovery of functional metrics was improved (Fig 5A). Pyr neuron spike rates recovered to the healthy ranges for all individual depression microcircuits in the test dataset (0.76 ± 0.01 Hz, p > 0.05, Cohen’s d = -0.06), as were the failed detection rates (1.40 ± 0.15%, p > 0.05, Cohen’s d = -0.5) and false detection rates (1.49 ± 0.18%, p > 0.05, Cohen’s d = -0.4). Dose prediction based on Pyr neuron spike rates also recovered the EEG features back within healthy ranges in 96% (22 of 23) of test microcircuits (Fig 5B), with group statistics not significantly different from healthy for 1/f (0.49 ± 0.09, p > 0.05, Cohen’s d = 0.3) or α (0.37 ± 0.06, p > 0.05, Cohen’s d = -0.1), although exhibiting a slightly larger θ power (0.26 ± 0.09, p = 0.049, Cohen’s d = 0.6) due to the one test microcircuit that did not recover fully within healthy ranges. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Dose prediction and recovery using microcircuit spike rates. A. Mean and SD of spike rates (left), failed detection rates (middle), and false detection rates (right) in individual depression microcircuits (MDD) and after applying the predicted doses. Grey area shows the healthy range. B. EEG metrics of all individual microcircuits before (magenta) and after applying the predicted α5-PAM dose based on spike rate (blue). C. EEG metric (left: 1/f, middle: α, right: θ) correlations with Pyr neuron spike rate across all individual microcircuits (black: healthy; color indicates % reduced SST interneuron inhibition). R values are shown in the top right. D. Variance across individual microcircuits was larger than the variance due to state (10 healthy microcircuits across 10 activity states). Error bars show SD. Different microcircuits are denoted by different colors. E. Average SD of spike rate across microcircuits or states. https://doi.org/10.1371/journal.pcbi.1012693.g005 Microcircuit spike rates therefore served better than EEG as biomarkers of SST interneuron inhibition loss, and although EEG metrics were correlated with microcircuit spike rates, some of the variance was independent (1/f: r = 0.67, p = 2.55e-14; α: r = 0.75, p = 4.56e-19; θ: r = 0.68, p = 9.94e-15, Fig 5C). Interestingly, baseline spike rate variability was driven largely by differences between individual microcircuits (microcircuit synaptic connectivity) rather than between activity states (the timing of background inputs to the microcircuits) within microcircuits (variance across healthy individual microcircuits: 0.033 ± 0.001 Hz, variance across states: 0.009 ± 0.002 Hz, p = 2.14e-17, Cohen’s d: -14.4; Fig 5D–5E). Therefore, differences in microcircuit connectivity between individual microcircuits resulted in some microcircuits tending to have hypo-active baseline firing and others hyper-active firing, which consequently influenced the required dose. Individual microcircuits were also distinguished as being either hypo- or hyper- active in terms of their EEG metrics, although there was no significant difference in the variance across individual microcircuits compared to activity states (1/f: p > 0.05, Cohen’s d: -0.4; α: p > 0.05, Cohen’s d: 0.02; θ: p > 0.05, Cohen’s d: -0.2). Discussion In this work, we showed in silico that α5-PAM therapeutic doses, which effectively recover cortical microcircuit EEG, spiking activity and function, can be predicted from simulated EEG biomarkers of depression severity in terms of reduced SST interneuron inhibition. We used detailed simulations of human microcircuits to mechanistically link severity of SST interneuron inhibition loss, EEG biomarkers and effective α5-PAM doses. Our machine learning predictors and candidate EEG biomarkers could serve to facilitate translation efforts of α5-PAM pharmacological treatment by stratifying depression patients that may benefit from α5-PAM, and by predicting treatment outcome and monitoring α5-PAM efficacy with non-invasive brain signals. We modelled depression severity in terms of SST interneuron inhibition reduction, and thus the tools we developed would be relevant to a subset of depression patients, which could amount to 50% of the patients as indicated postmortem [16]. Simulated α5-PAM recovered power in lower EEG frequency bands (θ, α) in particular but also involved broadband effects on the aperiodic component, as in our previous study which also showed a recovery of EEG event amplitudes across all frequencies [11]. Some of the relevant EEG biomarkers we found, such as increased power in the theta and alpha frequency bands, have also played a role in previous machine learning methods that stratified depression patients by EEG features [35–37,41], though the specifics vary depending on the electrode location and analytical methods. For example, in one study, an increase in θ and α power was mainly seen in parietal and occipital regions in people with depression using an independent components approach [37]. However, the “black box” machine learning methods are generally blind to any underlying mechanisms and are therefore limited in terms of accurately stratifying a particular mechanistic subtype [41]. Most previous machine learning studies also focused on classifying depression patients from healthy controls [41,47] rather than estimating severity. In addition, there have been only a few studies that found correlations between EEG features and severity of depression symptoms, which were limited to higher (beta and gamma) frequency bands [48,49] or theta cordance following antidepressant treatment [50]. More work is therefore needed to validate the severity biomarkers we have characterized. We constrained the models with the effect of 3 μM of the α5-PAM compound, GL-II-73 (i.e., the reference 100% dose) as previously [11], which is in the range for selectively targeting α5 subunit receptors without substantially activating other subunits [9,10]. We note that this reference dose of GL-II-73 (3 μM) roughly equates to a 1.16 mg kg-1 dose. In rodents, this dose was also within the range that yielded anxiolytic, antidepressant, and pro-cognitive effects without sedation [10]. Though these experiments support our chosen reference dose, differences between rodents and humans may yield different brain concentrations, and further experiments will be required to determine the effective dose in human brain tissue. To maintain the interpretability of our models in only selectively targeting of α5-GABAA receptors, we limited the upper range of doses we simulated to 150% of the reference dose, since higher doses are expected to have broader effects of boosting inhibition non-selectively [9,10,51], such as inhibition onto Pyr basal dendrites, as well as PV, SST, and VIP interneurons. Higher doses, for example, could thus elicit peak frequency shifts, in line with the effects of non-selective benzodiazepines, such as diazepam [51]. In some individual microcircuits with severe SST reductions the upper-range dose was necessary for recovery, but for most individual microcircuits the required doses were lower, supporting the dose range we examined. As well, while we focused on modelling the peak steady-state effect of α5-PAM, we note that the effects of α5-PAM follow a time course with rise and decline phases [51] and can also result in longer-lasting changes to the microcircuit [52]. We used our previous detailed models of human cortical microcircuits that simulated the effects of depression mechanisms on cortical microcircuit activity and function [11,28]. These models reproduced key aspects of human resting-state EEG [30], at the level of single EEG channels, due to the use of realistic human neuronal morphologies, human synaptic properties, and inclusion of key neuron types and connections that are shared by many cortical areas and thus form a prototypical cortical microcircuit. Future studies could refine our results and methods by including deeper layer circuits [53–55], layer 1 interneurons [56,57], and additional mechanisms of depression that are rescued by α5-PAM such as spine density loss [9,58]. Potential future applications of this work could be dose prediction in the context of personalized individual microcircuit models, for example, using microcircuits fitted to capture individual depression patient EEG/MEG. These models could also inform single nodes in personalized virtual brain models [59] to improve simulated whole-brain signals and treatment prediction. The machine learning models for dose prediction we have developed required only a few key EEG features and we selected these features based on whether they exhibited strong correlations with reduction in SST interneuron inhibition. Future implementations could consider alternative and more systematic feature selection methods for linear regression or SVM models, and alternative network architectures and feature sets for the ANN models. Nevertheless, the different machine learning models predicted dose with high accuracy and performed comparably well, indicating that the features we chose were sufficient. EEG feature importance was also consistent across the different prediction models. While multiple mechanisms could lead to similar EEG biomarkers [60–62], in previous work we demonstrated that the effects of SST interneuron inhibition loss on EEG are distinct from the effects of PV interneuron inhibition loss [30]. Other mechanisms of depression such as spine loss may also overlap with the effects of SST interneuron inhibition loss on baseline spiking [63] and thus on resting-state EEG, however we showed previously that SST interneuron inhibition loss has a more predominant effect than spine loss on baseline activity [63] and thus on EEG. SST interneuron cell loss may also lead to similar outputs as SST interneuron inhibition loss, but we note that only a loss of SST expression has been observed in depression and not a loss of cells [16]. Additional EEG features (e.g. periodic component [64] and oscillatory event metrics [65]) could help differentiate the effects of different mechanisms, but we note that many of these features are correlated with each other and thus may be of limited use. While there may be more informative brain data features for predicting treatment than EEG, for example spike rates as we showed in this study, our main aim was to utilize metrics that are clinically relevant, such as the non-invasive EEG signals. In this work we provide the first demonstration of α5-PAM dose prediction using simulated EEG biomarkers of reduced SST interneuron inhibition severity in detailed depression microcircuit models with α5-PAM effects. Our study overcomes limitations of doing the above in living humans, and thus our tools could serve to better stratify depression patients that may benefit from α5-PAM treatment and enable EEG-based treatment outcome prediction and monitoring efficacy. Methods Models of human cortical microcircuits in health and depression We used our previous morphologically- and biophysically-detailed models of human L2/3 cortical microcircuits in health and depression [11,28]. The models were comprised of 1000 neurons (80% Pyr, 5% SST, 7% PV, and 8% VIP) distributed across a 500x500x950 μm3 volume, and reproduced neuronal firing and synaptic properties as measured in human neurons. We included key connection types fitted to human data, including Pyr → Pyr [66], SST → Pyr apical dendrites [22] and PV → Pyr basal dendrites [67], cell type proportions estimated from human RNA-seq data [68], and tonic inhibition fitted to human electrophysiology data [11,69]. All NMDA/AMPA excitatory and GABAA inhibitory synapses were modelled using presynaptic short-term plasticity parameters for vesicle-usage, facilitation, and depression, and separate rise and decay parameters for the AMPA and NMDA components of excitatory synapses (τrise,NMDA = 2 ms; τdecay,NMDA = 65 ms; τrise,AMPA = 0.3 ms; τdecay,AMPA = 3 ms; τrise,GABA = 1 ms; τdecay,GABA = 10 ms) [32,70,71]. For tonic inhibition we used a previous model of outward rectifying tonic inhibition [72]. For complete list of data provenance in our models, please refer to supplemental tables in our previous work [28]. The models were simulated using NEURON 7.7 [73] and LFPy 2.0.2 (Python 3.7.6) [74] on SciNet parallel computing [75]. We modelled individual healthy and depression microcircuits with different severity (0, 10, … 40%, n = 20 per group, n = 100 in total) in terms of reduced SST interneuron synaptic and tonic inhibition conductance onto all cell types [28]. Individual microcircuits differed in synaptic connectivity, background inputs, and spatial placement of neurons in the L2/3 volume. Simulating α5-PAM application As in previous work [11], we simulated the effect of α5-PAM on tonic and synaptic inhibition in Pyr apical dendrites. We simulated a reference dose of α5-PAM application corresponding to experimentally-measured effects of a 3 μM dose of GL-II-73 on human Pyr neuron tonic inhibition current, which we captured using a 60% boost of tonic inhibition conductance to Pyr neuron apical dendrites [11]. We then applied the same boost proportionally to SST → Pyr synaptic conductance [11]. This model thus captured the degree of selectivity of the α5-PAM in targeting Pyr neuron apical dendrites [12], where α5 is almost exclusively expressed in human neocortex [76]. We simulated the application of different doses of α5-PAM relative to this reference dose (25% to 150%, where 100% is the reference dose). Our models of α5-PAM effects aimed to capture the peak steady-state effect, which lasts for tens of minutes [51]. Microcircuit baseline spiking activity We simulated baseline activity in the microcircuit as described previously [11], driving the microcircuit with background excitatory inputs of Ornstein Uhlenbeck (OU) point processes [77]. Independent excitatory OU point processes were placed at the midway points along the length of each dendritic arbor, and for Pyr neuron models, we placed 5 additional OU processes along the apical dendrites (at 10%, 30%, 50%, 70%, 90% of the apical length). The mean and standard deviation of each OU conductance were scaled up exponentially with relative distance from soma (ranging from 0 to 1) to normalize their effect. α5-PAM effects on firing rates We previously showed that a loss of SST interneuron inhibition leads to increased baseline spiking and thus a decrease in signal-to-noise ratio and signal processing, which could be recovered with α5-PAM [11,28]. We therefore used recovery of baseline spike rates to healthy ranges as an important measure of α5-PAM efficacy more directly on neuronal activity and function in addition to EEG. Simulated microcircuit EEG and power spectral analysis Along with the baseline activity, we simulated resting-state EEG from the microcircuit models in LFPy 2.0.2 (Python 3.7.6) using a four-sphere volume conductor model (representing grey matter, cerebrospinal fluid, skull, and scalp with radii of 79 mm, 80 mm, 85 mm, and 90 mm, respectively) that assumed homogeneous, isotropic, and linear (frequency-independent) conductivity [11,30]. The conductivity for each sphere was 0.047 S m−1, 1.71 S m−1, 0.02 S m−1, and 0.41 S m−1, respectively [11,30,78]. We computed EEG power spectral density (PSD) using Welch’s method [79] from the Scipy python module with 2s time windows. We also decomposed the EEG power spectra (in the 3–30 Hz range) into periodic and aperiodic components using the FOOOF toolbox [64]. The aperiodic component was a 1/f function parameterized by vertical offset and exponent parameters. We fitted the periodic oscillatory component with up to 3 Gaussian peaks defined by center frequency, bandwidth (min: 2 Hz, max: 6 Hz), and power magnitude (relative peak threshold: 2, minimum peak height: 0) [11,30]. We extracted EEG features including the area under the curve of power in different frequency bands (θ = 4–8 Hz, α = 8–12 Hz) and the 1/f aperiodic component (3–30 Hz range). Feature values were normalized by transforming to log10 space, z-scored relative to all conditions, and scaled to values ranging from -1 to 1. Optimal dose and optimal dose range estimations For each individual microcircuit, we fitted a dose-response function (relating dose and EEG features), and identified the optimal dose and optimal dose ranges for the microcircuit by finding the dose for which the function intersected with the EEG feature values of the healthy mean and the healthy ranges, respectively. Dose predictor models We used the optimal dose and EEG features (1/f, θ and α power) of 70% of the microcircuits to fit a multivariate linear prediction model of the form: We tested the predictor performance using the remaining 30% of microcircuits, in terms of the proportion of correct dose predictions. A dose was correct if it fell within the microcircuit’s optimal dose range. We generated multiple models using 50 train/test permutations, and selected the best performing prediction model. As an alternative type of model, we generated predictor models similarly but instead used a linear support vector machine (SVM) in python [80], with a linear kernel, a regularization parameter of 10, a tolerance value of 0.001, and an epsilon value of 0.1. As another type of predictor model, we trained an artificial neural network (ANN) using the tensorflow python package [81]. The ANN comprised of 3 input nodes (corresponding to the EEG features above), 9 hidden layer nodes with ReLU activation, and 1 output layer node with linear activation. For learning, we used an Adam optimizer with a learning rate of 0.01, mean absolute error as the loss and training accuracy functions, and initialized weights from a normal distribution centered at zero. In both SVM and ANN cases, we fit the models using 70% of the data and tested using 30%. For the SVM we ran 50 train/test permutations, and for the ANN we ran 10 train/test permutations (for computational efficiency), from which we selected the best performing prediction model. We computed SHapley Additive exPlanations (SHAP) feature importance values for each of the prediction models using the SHAP module in python [82]. Failed/false detection metrics We calculated failed and false detection errors in the different conditions using the distribution of Pyr neuron firing rates at baseline (n = 23,950 windows of 50 ms for each microcircuit) and a reference distribution of simulated firing rates in response to brief stimulus (calculated across 200 stimulus presentation, in the 5–55 ms period post-stimulus), since we have previously shown that response rate was not impacted by SST interneuron inhibition loss alone or application of α5-PAM [11,28]. The intersection point of the two distributions defined the stimulus detection threshold. We computed probability of false detections as the integral of the pre-stimulus distribution above the detection threshold divided by the integral of the entire pre-stimulus distribution. Similarly, we computed the probability of failed detections as the integral of the post-stimulus distribution under the detection threshold divided by the integral of the entire post-stimulus distribution. Intra- versus inter- microcircuit variability analysis To analyze the variance in spike rates due to state randomizations versus microcircuit randomizations, we simulated 10 randomized states across 10 randomized microcircuits (i.e., individual microcircuits). Randomizing the microcircuit comprised of resampling synaptic connections and neuron positions in space, whereas randomizing the state comprised of resampling background OU noise inputs. Statistics For linear correlations, we used two-sided Pearson correlations. For group comparisons we used two-sided paired- and independent-sample t-tests, where indicated. For independent-sample t-test cases where variance between groups were significantly different from each other (using the Levene test for equal variances) we performed a Welch’s t-test. Cohen’s d was calculated as follows: Models of human cortical microcircuits in health and depression We used our previous morphologically- and biophysically-detailed models of human L2/3 cortical microcircuits in health and depression [11,28]. The models were comprised of 1000 neurons (80% Pyr, 5% SST, 7% PV, and 8% VIP) distributed across a 500x500x950 μm3 volume, and reproduced neuronal firing and synaptic properties as measured in human neurons. We included key connection types fitted to human data, including Pyr → Pyr [66], SST → Pyr apical dendrites [22] and PV → Pyr basal dendrites [67], cell type proportions estimated from human RNA-seq data [68], and tonic inhibition fitted to human electrophysiology data [11,69]. All NMDA/AMPA excitatory and GABAA inhibitory synapses were modelled using presynaptic short-term plasticity parameters for vesicle-usage, facilitation, and depression, and separate rise and decay parameters for the AMPA and NMDA components of excitatory synapses (τrise,NMDA = 2 ms; τdecay,NMDA = 65 ms; τrise,AMPA = 0.3 ms; τdecay,AMPA = 3 ms; τrise,GABA = 1 ms; τdecay,GABA = 10 ms) [32,70,71]. For tonic inhibition we used a previous model of outward rectifying tonic inhibition [72]. For complete list of data provenance in our models, please refer to supplemental tables in our previous work [28]. The models were simulated using NEURON 7.7 [73] and LFPy 2.0.2 (Python 3.7.6) [74] on SciNet parallel computing [75]. We modelled individual healthy and depression microcircuits with different severity (0, 10, … 40%, n = 20 per group, n = 100 in total) in terms of reduced SST interneuron synaptic and tonic inhibition conductance onto all cell types [28]. Individual microcircuits differed in synaptic connectivity, background inputs, and spatial placement of neurons in the L2/3 volume. Simulating α5-PAM application As in previous work [11], we simulated the effect of α5-PAM on tonic and synaptic inhibition in Pyr apical dendrites. We simulated a reference dose of α5-PAM application corresponding to experimentally-measured effects of a 3 μM dose of GL-II-73 on human Pyr neuron tonic inhibition current, which we captured using a 60% boost of tonic inhibition conductance to Pyr neuron apical dendrites [11]. We then applied the same boost proportionally to SST → Pyr synaptic conductance [11]. This model thus captured the degree of selectivity of the α5-PAM in targeting Pyr neuron apical dendrites [12], where α5 is almost exclusively expressed in human neocortex [76]. We simulated the application of different doses of α5-PAM relative to this reference dose (25% to 150%, where 100% is the reference dose). Our models of α5-PAM effects aimed to capture the peak steady-state effect, which lasts for tens of minutes [51]. Microcircuit baseline spiking activity We simulated baseline activity in the microcircuit as described previously [11], driving the microcircuit with background excitatory inputs of Ornstein Uhlenbeck (OU) point processes [77]. Independent excitatory OU point processes were placed at the midway points along the length of each dendritic arbor, and for Pyr neuron models, we placed 5 additional OU processes along the apical dendrites (at 10%, 30%, 50%, 70%, 90% of the apical length). The mean and standard deviation of each OU conductance were scaled up exponentially with relative distance from soma (ranging from 0 to 1) to normalize their effect. α5-PAM effects on firing rates We previously showed that a loss of SST interneuron inhibition leads to increased baseline spiking and thus a decrease in signal-to-noise ratio and signal processing, which could be recovered with α5-PAM [11,28]. We therefore used recovery of baseline spike rates to healthy ranges as an important measure of α5-PAM efficacy more directly on neuronal activity and function in addition to EEG. Simulated microcircuit EEG and power spectral analysis Along with the baseline activity, we simulated resting-state EEG from the microcircuit models in LFPy 2.0.2 (Python 3.7.6) using a four-sphere volume conductor model (representing grey matter, cerebrospinal fluid, skull, and scalp with radii of 79 mm, 80 mm, 85 mm, and 90 mm, respectively) that assumed homogeneous, isotropic, and linear (frequency-independent) conductivity [11,30]. The conductivity for each sphere was 0.047 S m−1, 1.71 S m−1, 0.02 S m−1, and 0.41 S m−1, respectively [11,30,78]. We computed EEG power spectral density (PSD) using Welch’s method [79] from the Scipy python module with 2s time windows. We also decomposed the EEG power spectra (in the 3–30 Hz range) into periodic and aperiodic components using the FOOOF toolbox [64]. The aperiodic component was a 1/f function parameterized by vertical offset and exponent parameters. We fitted the periodic oscillatory component with up to 3 Gaussian peaks defined by center frequency, bandwidth (min: 2 Hz, max: 6 Hz), and power magnitude (relative peak threshold: 2, minimum peak height: 0) [11,30]. We extracted EEG features including the area under the curve of power in different frequency bands (θ = 4–8 Hz, α = 8–12 Hz) and the 1/f aperiodic component (3–30 Hz range). Feature values were normalized by transforming to log10 space, z-scored relative to all conditions, and scaled to values ranging from -1 to 1. Optimal dose and optimal dose range estimations For each individual microcircuit, we fitted a dose-response function (relating dose and EEG features), and identified the optimal dose and optimal dose ranges for the microcircuit by finding the dose for which the function intersected with the EEG feature values of the healthy mean and the healthy ranges, respectively. Dose predictor models We used the optimal dose and EEG features (1/f, θ and α power) of 70% of the microcircuits to fit a multivariate linear prediction model of the form: We tested the predictor performance using the remaining 30% of microcircuits, in terms of the proportion of correct dose predictions. A dose was correct if it fell within the microcircuit’s optimal dose range. We generated multiple models using 50 train/test permutations, and selected the best performing prediction model. As an alternative type of model, we generated predictor models similarly but instead used a linear support vector machine (SVM) in python [80], with a linear kernel, a regularization parameter of 10, a tolerance value of 0.001, and an epsilon value of 0.1. As another type of predictor model, we trained an artificial neural network (ANN) using the tensorflow python package [81]. The ANN comprised of 3 input nodes (corresponding to the EEG features above), 9 hidden layer nodes with ReLU activation, and 1 output layer node with linear activation. For learning, we used an Adam optimizer with a learning rate of 0.01, mean absolute error as the loss and training accuracy functions, and initialized weights from a normal distribution centered at zero. In both SVM and ANN cases, we fit the models using 70% of the data and tested using 30%. For the SVM we ran 50 train/test permutations, and for the ANN we ran 10 train/test permutations (for computational efficiency), from which we selected the best performing prediction model. We computed SHapley Additive exPlanations (SHAP) feature importance values for each of the prediction models using the SHAP module in python [82]. Failed/false detection metrics We calculated failed and false detection errors in the different conditions using the distribution of Pyr neuron firing rates at baseline (n = 23,950 windows of 50 ms for each microcircuit) and a reference distribution of simulated firing rates in response to brief stimulus (calculated across 200 stimulus presentation, in the 5–55 ms period post-stimulus), since we have previously shown that response rate was not impacted by SST interneuron inhibition loss alone or application of α5-PAM [11,28]. The intersection point of the two distributions defined the stimulus detection threshold. We computed probability of false detections as the integral of the pre-stimulus distribution above the detection threshold divided by the integral of the entire pre-stimulus distribution. Similarly, we computed the probability of failed detections as the integral of the post-stimulus distribution under the detection threshold divided by the integral of the entire post-stimulus distribution. Intra- versus inter- microcircuit variability analysis To analyze the variance in spike rates due to state randomizations versus microcircuit randomizations, we simulated 10 randomized states across 10 randomized microcircuits (i.e., individual microcircuits). Randomizing the microcircuit comprised of resampling synaptic connections and neuron positions in space, whereas randomizing the state comprised of resampling background OU noise inputs. Statistics For linear correlations, we used two-sided Pearson correlations. For group comparisons we used two-sided paired- and independent-sample t-tests, where indicated. For independent-sample t-test cases where variance between groups were significantly different from each other (using the Levene test for equal variances) we performed a Welch’s t-test. Cohen’s d was calculated as follows: Acknowledgments AGM and EH thank the Krembil Foundation for their generous funding support. AGM thanks the Labatt Family Network for Research on the Biology of Depression for funding support. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Special thanks to A. Sherrington for contributing the face drawing in Figs 1B and 3A.

Jamieson, Nyall;Charalambous, Christiana;Schultz, David M.;Hall, Ian

doi: 10.1371/journal.pcbi.1012041pmid: 39729413

Introduction In epidemiology, the temporal relationship between key events in an individual’s infection history is important to understand. For example, a disease that has a long delay from infection to onset of infectiousness may be amenable to contact tracing, and the relationship between these two events can be important for disease control [1, 2]. Often these events are a simplification of a continuous process (i.e., infectivity may not start or end at specific times but instead increase and then decrease over time). For diseases such as Legionnaires’ disease, which spread via airborne dispersion from environmental sources (rather than person-to-person contact), characterisation of the incubation period is critical for source identification (or reverse epidemiology). Here, we consider the time from infection to symptom onset. The relationship between viral or bacterial load in one’s body and onset of symptoms can be difficult to describe. In brief, the presence of a virus or bacteria within an individual results in an inflammatory immune response that leads to an observable response of symptoms. An exact mathematical model accurately describing the infection process is not feasible to develop due to the large number of different cytokines and cell interactions involved in the immune response, as well as a lack of a clear understanding of how the pro-inflammatory cytokines relate to the appearance of symptoms and a lack of data to parameterise each specific process in the immune response. Previous models for the incubation period provide parsimonious simplifications of the infection process and include in-host models (often assuming symptom onset is proportional to bacterial load [3]) through to simpler probability models (justified on model parsimony or computational capacity). In the latter case, popular distributions include the gamma, log-normal and Weibull distributions [4–6]. Application of these common distributions is primarily based on heuristic justification. These common distributions share similarities in that they are right-skewed, defined on a positive support and flexible so that they can model a wide range of incubation-period datasets. More specific arguments for common distributions can be described as follows. The gamma distribution is a generalisation of the Erlang distribution with non-integer shape, n. In which case, the Erlang distribution is the sum of n exponentially distributed random events, and so fitting to data can help inform the structure of compartmental models [7]. The log-normal distribution is a skewed distribution often applied to biological processes in which the process mean time is relatively low, but its variance is large and results from taking the exponential of a series of normally distributed events. Finally, the Weibull distribution is a classic reliability-theory distribution where the hazard of an event occurring is strictly monotonic over time. To illustrate the heuristic justification of distributions, we consider Legionnaires’ disease and the statistical analysis that has been conducted in the literature for studying the incubation period. In this case, several papers have used a range of days (2–10) prior to symptom onset and considered all days in this period as a potential infection date [8–14]. Alternatively, others have assumed a median incubation period of either five days [15] or seven days [16], with infection dates obtained by subtracting the median from the symptom onset date. Another common approach is to consider a gamma-distributed incubation period [17]. All papers that take this approach have followed the ideas and method proposed in [4] using a gamma distribution to describe an outbreak in Melbourne [18]. One issue arising is that incubation-period data are given as an integer number of days, implying that each case becomes infected at the same moment from the exposure, and that symptoms develop in an integer amount of days. To illustrate this issue, take two cases in which symptom onset occurs the day after infection. The individual could have been infected at 11:59pm and became symptomatic at 00:01am the next day, or alternatively they could have been infected at 00:01am and became symptomatic at 11:59pm the next day. These two scenarios are 2 minutes and 1 day, 23 hours, 58 minutes long, respectively, but they both correspond to one integer day in the dataset. These simplifications give a lower resolution of the time delay between these events due to lack of knowledge of the exact infection and symptom onset times. Essentially, continuous distributions are being fitted to discretized versions of continuous data, and the result is interval data with censored start and end times. This type of discretized data are commonly used for analysis without consideration for the censoring issue. Using standard probability distributions, as well as censored incubation-period data in statistical analysis, is likely to produce biased inference. In reality, the individual was likely not infected at the beginning of their infection date. Similarly, symptoms likely did not appear at the end of their symptom onset date. Therefore, recorded incubation-periods are likely to be inflated with a positive bias. Using incubation-period data expressed as an integer number of days will likely lead to a false understanding of delays between key events for specific diseases, such as the incubation period, and produce incorrect conclusions. A model describing the incubation period of Legionnaires’ disease has been built with this type of data [4], but the model is flawed and can be improved upon by accounting for the issues mentioned above. There are various ways to handle the censoring issue, which we discuss in the next section. In this paper, a new model for incubation periods is derived with potentially stronger justification for its validity than methods currently used in the literature. We apply our new model to a variety of diseases and provide statistically significant changes in the mean incubation period, specifically for Legionnaires’ disease, compared to results obtained from using currently accepted and used models. We also apply techniques that remove the bias from fitting models to censored data and allow for reliable model-fitting, providing a new understanding of the incubation periods of various diseases. We apply these methods to anthrax, salmonellosis and campylobacteriosis, as well as taking a specific focus on Legionnaires’ disease to illustrate the typical kind of improvement achievable with these methods. For the successful models, we develop some distribution theory, calculating their moments and quantile functions, which can be found in S1 Appendix in the Supplementary Material. Materials and methods In this section, we develop methods for handling both of the problems discussed in the introduction. First, we adapt the methods developed in [19] for use on incubation-period data in order to account for its censored nature. Second, we consider a probabilistic approach to develop a new model for incubation periods of diseases. We assume exponential growth of bacteria early after infection, as well as a further assumption of the probability of symptom onset being proportional to the bacterial load within an individual until saturating once some load has been reached. Third, we discuss the methods for analysing our fitted models and how we determine which model performs better, so that we can conclude whether or not our developed model offers more reliable results than using methods currently developed in the literature. Finally, we introduce the data used for incubation-period analysis and discuss the reasons why this data are considered censored. Doubly interval-censored modelling Methods for handling censored data in epidemiological studies have been proposed in the literature to develop discrete analogues of continuous distributions that preserve properties of their continuous counterparts [20]. However, most of these methods either focus on preserving one property, do not result in valid probability mass functions, or assume that infection occurs exactly at midnight. These discrete analogues are not designed to account for the nature in which the continuous-time incubation-period process is recorded as discrete data. The exact time at which symptoms occur in an individual cannot be determined based on when they reported their illness to authorities. Similarly, the exact time that an individual becomes infected is also difficult to ascertain. We need a method for handling the fact that these times are unknown (i.e., to account for the uncertainty within a model), so that analysis of any subsequent models is reliable. To consider doubly censored data, a natural approach is to forget the assumption that the exact infection and symptom onset times are known and introduce a time period in which these two events may occur, with a probability distribution for the occurrence within this period [19]. The method proposed in [19], which considers doubly interval-censored (DI) data, is described as follows. Define T and S to be the time of infection and symptom onset respectively (with t and s being realisations of these random variables respectively), and Z = S − T as the incubation period of the infection. Consider two intervals where T and S could lie within because the exact times of T and S are not known. In other words, let T ∈ (TL, TR) and S ∈ (SL, SR). The incubation period Z is given as a random variable with p.d.f. f(s − t) (Fig 1). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Diagram visualising the doubly interval-censoring method [19], highlighting the data typically observed, but accounting for the fact that infection and symptom onset times are not observed exactly and intervals of possible times must be considered. https://doi.org/10.1371/journal.pcbi.1012041.g001 The p.d.f. of T is defined as fT(t) and the p.d.f. of S is defined as fS(s). The time at which a person becomes infected and the time taken from infection to symptom onset are independent, which leads to fS(s | t) = f(s − t | t) = f(s − t). Finally, define the joint p.d.f. of T and S as From this, the likelihood for a doubly interval-censored observation x is derived. To implement methods found in [19] to incubation-period data, the following approach is taken. Because the data are rounded to the nearest day, a natural assumption is that TL = 0 and TR = 1, so infection occurs at any point on the infection date. Defining x to be the number of days from exposure to symptom onset, set SR = x and SL = x − 1, so that the symptoms develop at some point on the stated date of symptom onset. There is not much evidence to indicate what distribution fT(t) might be, so a reasonable assumption would be to let fT(t) be uniform (i.e., fT(t) = 1 on t ∈ (0, 1), 0 otherwise). Other options could be to permit a lower chance during nighttime or a higher chance when people are outdoors, but these will depend on specific release scenarios and are not likely particularly identifiable in data. As f(s − t) is the p.d.f. of the incubation period, the log-likelihood is calculated as follows: (1) In the next section, we develop various distributions to describe the incubation period, and later fit the doubly interval-censored model to these distributions to determine which one provides the most optimal fit. Derivation of the incubation period model Incubation period data describes the cases who become symptomatic. Given the knowledge that all individuals in the data will become symptomatic, this section discusses different mathematical models for the occurrence of symptoms onset within a population. We explore how the results for these different methods link, and we develop a new model for incubation periods, by starting from a probabilistic approach of symptom-onset occurrence. A probability-based approach. A continuous-time mathematical model can be built considering the hazard rate of symptom-onset occurrence. We first consider the option of using an exponential survival model with time-varying hazard. Define N(t) as the population of individuals who are infected, but are not yet symptomatic at time t, and Q(t) as the population of individuals who are symptomatic at time t with N(0) = N0 and Q(0) = 0 and Q(t) + N(t) = N0, . Next, assume that a hazard rate function λ(t) describes the risk that a not-yet-symptomatic individual will start to experience symptoms at a point in time t, given that they have not already succumb to symptoms by time t. Then 1 − δtλ(t) will be the probability that the individual will remain asymptomatic within a small interval (t, t + δt), where in this context δt represents a small time increment in time. Hence (1 − δtλ(t))N(t) is the probability that nobody who is not-yet-symptomatic will start experiencing symptoms within a small increment δt from t, and (1 − δtλ(t))N(t)δt is the probability that nobody new will experience symptoms in a small increment δt from t. Following this, define δQ(t) = 1 − (1 − δtλ(t))N(t)δt to be the probability that there is at least one individual who starts to experience new symptoms in a small increment δt from t. By writing μ(t) = −log(1 − δtλ(t)), the probability of any new symptom onset appearance can be written as δQ(t) = 1 − e−μ(t)N(t)δt. Using a Taylor expansion on the exponential term, dividing by δt, and taking the limit δt → 0 changes this probability to a rate as follows: (2) This approach leads to a separable ordinary differential equation analogous to the cumulative distribution of the exponential distribution with a time-varying rate parameter. It can be deduced that and that is the accumulated hazard. Hence the rate of symptom onset, μ(t), is the hazard function of an individual becoming symptomatic. Therefore, the hazard of an individual becoming symptomatic at a point in time is equal to the rate of symptom onset at that time. The scenario discussed here can be considered from an inhomogeneous Poisson-process perspective, and the results of the hazard are identical to the inhomogeneous exponentially distributed model. It can be noted here that if μ(t) is constant that this would lead to the exponential distribution and if μ(t) ∝ ta for some constant a this would suggest the incubation period is a Weibull distributed random variable. The Erlang distribution arises by assuming the incubation period is the sum of a number of stages of constant length μ. However, various studies have shown that the bacterial load within an individual is positively correlated with the probability of symptom onset [3, 21]. The relationship between bacterial load and probability of symptom onset is complex and varies from bacteria-to-bacteria [21]. Additionally, a positive correlation between load and probability of symptom onset has been observed for viral infections [22]. For parsimony, we assume that symptom onset is likely proportional to bacterial (or viral) load at low loads (i.e., the early stages of infection) before saturating at large loads. The bacterial population early after infection will be approximately some exponential function of time [3, 23, 24]. Therefore, the left tail of the c.d.f. of the incubation-period distribution is given by some function , whilst in the later stage, the c.d.f. should tend to 1 exponentially given by a function G2(t), as is the case of the hazard function above. Mathematically, with a median T, and considering the case where G(t) = G1(t) = G2(t), an equation for the c.d.f. that satisfies these conditions is given as follows: (3) where for some function g(s). The ODE that arises in (3) defines the Burr family of distributions and is discussed in further detail in the next section. Burr distribution. A Burr distribution is a distribution whose c.d.f., F(t), (4) is the solution of (3). Theoretically, there are no constraints on G(t) in (4). Twelve main distributions within the Burr family have been characterized [25], named as Burr type I, Burr type II, up-to Burr type XII, but we only consider Burr distributions defined over a domain of (0, ∞). Some delay distributions arising in epidemiology do permit negative values. For example, the time from symptom onset in infector to symptom onset in infectee could be negative. In this paper, we limit consideration to strictly positive cases. A negative incubation period is not possible, nor is a fixed upper-limit constraint expected. The only biologically feasible distributions are types III, X and XII. The type III distribution could be derived from the flexible generalized gamma distribution with the scale parameter following an inverse Weibull distribution [26]. Similarly, the type XII distribution could be derived from the Weibull distribution where the scale parameter follows an inverse generalized gamma distribution [26]. The Burr distributions and the gamma distribution have parameters that share the same symbols for notational simplicity, although they have different interpretations and their fitted estimates cannot be directly compared. To avoid confusion, we provide a subscript for each parameter to clarify which distribution this parameter corresponds to (i.e., αIII for the α parameter in the type III Burr model) in the text but drop this in tables and figures for brevity. Further, the type III, X and XII distributions used in this research are a generalization of types III, X and XII Burr distributions used in the literature [25], where the time variable is scaled by an additional parameter. Type X is defined with two variables that provide models as parsimonious as the three distributions previously trialled: gamma, log-normal, and Weibull. Further, both type III and XII distributions have two shape parameters αIII,XII and βIII,XII. Finally, the type III distribution has a scale parameter γIII,XII and the type XII distribution has a location parameter, the median T. General derived Burr distribution. In (3), g(t) has a physical interpretation; the function tends to the rate of symptom onset μ(t) in individuals at a time t as t increases. Given F(t) = (1 + e−G(t))−1, in general, then G(t) → t/βD (or g(t) → 1/βD) for some constant βD as t → ∞ on the basis that the hazard rate approaches constant over time for relatively long incubation periods. In principle, F(0) = 0, so G(t) → −∞ for t → 0 (or G(0) is very large if not actually infinite). Taking the above into account, we propose g(t) = 1/βD + αD/t, and as such G(t) = t/βD + αD log(t) + C, where C is a constant of integration. We define TD as the median, which satisfies G(TD) = 0. Hence, C = −TD/βD − αD log(TD) and thus Equations for the c.d.f. and p.d.f. for the derived Burr distribution, as well as the gamma and other Burr distributions, are given in Table 1. As discussed, TD is the median of the distribution. The reciprocal of βD is the eventual constant rate of symptom onset in individuals for t ≫ TD. Additionally, there are two details worth noting when analysing the physical interpretation of αD. First, αD is an exponent of t that controls the increase in probability density, as for t ≪ TD. Second, the general derived Burr distribution approaches the exponential distribution for t ≫ TD. The rate at which the derived Burr distribution approaches a constant hazard (as for an exponential distribution) increases for decreasing αD. Finally, all parameters must be strictly greater than zero. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. The Burr distributions valid over (0, ∞) and previously trialled distributions [4] with their corresponding p.d.f and c.d.f. https://doi.org/10.1371/journal.pcbi.1012041.t001 Model comparison We fit each type of Burr distribution to the data, and assess all the models in terms of their goodness of fit in comparison to the more widely used gamma distribution. The most commonly used methods for model selection are the Akaike information criterion (AIC) and Bayesian information criterion (BIC) [27]. Generally, AIC puts more emphasis on good model prediction, whereas BIC favours model parsimony [27]. Because our goal is good model prediction, the AIC will be used in deciding desirable model fits. Additionally, we calculate the Akaike weights ω for each model fit, which can be used for further model comparison [28]. Akaike weights are used to compare the validity of a Burr distributed model over the gamma distributed model once fit to data. The ratio wi/wG, where wi is the weight for the ith model and wG is the weight of the gamma distributed model. This ratio may be interpreted as how much more likely model i is a better fitting model than the gamma model. Alternatively, we also derive the normalized probability that the ith model is preferable to the gamma model, given by wi/(wi + wG). The final method of comparison considered is the Bayes factor [29]. The maximum likelihood estimates that we obtain can be considered maximum a posteriori estimates with a uniform prior and are used in this context for conducting the Bayes Factor calculations. Larger Bayes factor values indicate stronger evidence to support one model over another. Incubation-period data To test these models, we employ incubation-period data from an outbreak of Legionnaires’ disease in Melbourne in April 2000 [18]. The data for the Melbourne outbreak contains the number of days taken for each Legionnaires’ disease case to develop symptoms from their exposure date, and several potential distributions for fitting the data have been compared [4]. The case data only contains individuals that visited the known source once within the weeks before the outbreak and have a known date of symptom onset. Therefore, the timing of infection and symptom onset events are known correctly to a single day. The results indicated that the gamma distribution provided the best fit [4] out of their proposed models. Further, we gather incubation-period data for anthrax, campylobacteriosis and salmonellosis for analysis. The anthrax outbreak in 1979 contains data for the known incubation-periods of patients [30]. Investigations into the outbreak, climate conditions and human/animal presence highlighted a single exposure on 2 April 1979. The date of symptom onset was provided as known in most cases. However, for the few remaining cases, an estimated symptom onset date was obtained by subtracting 3 days (the mean delay between symptom onset and death) from the date of death. A literature review has been conducted analysing different salmonellosis studies that contain full data of the incubation periods [31]. Awofisayo-Okuyelu et al. [31] noticed that the incubation periods varied between studies. They grouped studies into subsets using a clustering process, in which the grouped studies did not have any statistically significant difference in their incubation-period data. Similarly, Awofisayo-Okuyelu et al. [32] conducted a review for campylobacteriosis in which the incubation periods varied between studies, and they combined datasets which were not statistically significantly different using a clustering process similar to [31]. For both the salmonellosis and campylobacteriosis datasets, a quality assessment was carried out during their literature review [31, 32]. Incubation period data obtained was assessed based on whether cases were linked to a clearly defined exposure and accuracy of the reported symptom onset time, with the lowest level of resolution in the symptom onset being a period of 24 hours [31, 32]. We provide an Excel sheet of the incubation-period data for these other diseases in S1 Data in the Supplementary Material. The data gathered for these diseases share a similarity with the Legionnaires’ disease data, in that the data contains the integer number of days taken for each case to develop symptoms. The fact the data for all of these diseases contains integer days implies that each case takes an exact multiple of 24 hours from infection to the appearance of symptoms, which is not realistic. If we assume that the dates of infection and symptom onset are accurate, then we know the date of these events, but the specific times on the given days are unknown. We are dealing with doubly censored data. Doubly interval-censored modelling Methods for handling censored data in epidemiological studies have been proposed in the literature to develop discrete analogues of continuous distributions that preserve properties of their continuous counterparts [20]. However, most of these methods either focus on preserving one property, do not result in valid probability mass functions, or assume that infection occurs exactly at midnight. These discrete analogues are not designed to account for the nature in which the continuous-time incubation-period process is recorded as discrete data. The exact time at which symptoms occur in an individual cannot be determined based on when they reported their illness to authorities. Similarly, the exact time that an individual becomes infected is also difficult to ascertain. We need a method for handling the fact that these times are unknown (i.e., to account for the uncertainty within a model), so that analysis of any subsequent models is reliable. To consider doubly censored data, a natural approach is to forget the assumption that the exact infection and symptom onset times are known and introduce a time period in which these two events may occur, with a probability distribution for the occurrence within this period [19]. The method proposed in [19], which considers doubly interval-censored (DI) data, is described as follows. Define T and S to be the time of infection and symptom onset respectively (with t and s being realisations of these random variables respectively), and Z = S − T as the incubation period of the infection. Consider two intervals where T and S could lie within because the exact times of T and S are not known. In other words, let T ∈ (TL, TR) and S ∈ (SL, SR). The incubation period Z is given as a random variable with p.d.f. f(s − t) (Fig 1). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Diagram visualising the doubly interval-censoring method [19], highlighting the data typically observed, but accounting for the fact that infection and symptom onset times are not observed exactly and intervals of possible times must be considered. https://doi.org/10.1371/journal.pcbi.1012041.g001 The p.d.f. of T is defined as fT(t) and the p.d.f. of S is defined as fS(s). The time at which a person becomes infected and the time taken from infection to symptom onset are independent, which leads to fS(s | t) = f(s − t | t) = f(s − t). Finally, define the joint p.d.f. of T and S as From this, the likelihood for a doubly interval-censored observation x is derived. To implement methods found in [19] to incubation-period data, the following approach is taken. Because the data are rounded to the nearest day, a natural assumption is that TL = 0 and TR = 1, so infection occurs at any point on the infection date. Defining x to be the number of days from exposure to symptom onset, set SR = x and SL = x − 1, so that the symptoms develop at some point on the stated date of symptom onset. There is not much evidence to indicate what distribution fT(t) might be, so a reasonable assumption would be to let fT(t) be uniform (i.e., fT(t) = 1 on t ∈ (0, 1), 0 otherwise). Other options could be to permit a lower chance during nighttime or a higher chance when people are outdoors, but these will depend on specific release scenarios and are not likely particularly identifiable in data. As f(s − t) is the p.d.f. of the incubation period, the log-likelihood is calculated as follows: (1) In the next section, we develop various distributions to describe the incubation period, and later fit the doubly interval-censored model to these distributions to determine which one provides the most optimal fit. Derivation of the incubation period model Incubation period data describes the cases who become symptomatic. Given the knowledge that all individuals in the data will become symptomatic, this section discusses different mathematical models for the occurrence of symptoms onset within a population. We explore how the results for these different methods link, and we develop a new model for incubation periods, by starting from a probabilistic approach of symptom-onset occurrence. A probability-based approach. A continuous-time mathematical model can be built considering the hazard rate of symptom-onset occurrence. We first consider the option of using an exponential survival model with time-varying hazard. Define N(t) as the population of individuals who are infected, but are not yet symptomatic at time t, and Q(t) as the population of individuals who are symptomatic at time t with N(0) = N0 and Q(0) = 0 and Q(t) + N(t) = N0, . Next, assume that a hazard rate function λ(t) describes the risk that a not-yet-symptomatic individual will start to experience symptoms at a point in time t, given that they have not already succumb to symptoms by time t. Then 1 − δtλ(t) will be the probability that the individual will remain asymptomatic within a small interval (t, t + δt), where in this context δt represents a small time increment in time. Hence (1 − δtλ(t))N(t) is the probability that nobody who is not-yet-symptomatic will start experiencing symptoms within a small increment δt from t, and (1 − δtλ(t))N(t)δt is the probability that nobody new will experience symptoms in a small increment δt from t. Following this, define δQ(t) = 1 − (1 − δtλ(t))N(t)δt to be the probability that there is at least one individual who starts to experience new symptoms in a small increment δt from t. By writing μ(t) = −log(1 − δtλ(t)), the probability of any new symptom onset appearance can be written as δQ(t) = 1 − e−μ(t)N(t)δt. Using a Taylor expansion on the exponential term, dividing by δt, and taking the limit δt → 0 changes this probability to a rate as follows: (2) This approach leads to a separable ordinary differential equation analogous to the cumulative distribution of the exponential distribution with a time-varying rate parameter. It can be deduced that and that is the accumulated hazard. Hence the rate of symptom onset, μ(t), is the hazard function of an individual becoming symptomatic. Therefore, the hazard of an individual becoming symptomatic at a point in time is equal to the rate of symptom onset at that time. The scenario discussed here can be considered from an inhomogeneous Poisson-process perspective, and the results of the hazard are identical to the inhomogeneous exponentially distributed model. It can be noted here that if μ(t) is constant that this would lead to the exponential distribution and if μ(t) ∝ ta for some constant a this would suggest the incubation period is a Weibull distributed random variable. The Erlang distribution arises by assuming the incubation period is the sum of a number of stages of constant length μ. However, various studies have shown that the bacterial load within an individual is positively correlated with the probability of symptom onset [3, 21]. The relationship between bacterial load and probability of symptom onset is complex and varies from bacteria-to-bacteria [21]. Additionally, a positive correlation between load and probability of symptom onset has been observed for viral infections [22]. For parsimony, we assume that symptom onset is likely proportional to bacterial (or viral) load at low loads (i.e., the early stages of infection) before saturating at large loads. The bacterial population early after infection will be approximately some exponential function of time [3, 23, 24]. Therefore, the left tail of the c.d.f. of the incubation-period distribution is given by some function , whilst in the later stage, the c.d.f. should tend to 1 exponentially given by a function G2(t), as is the case of the hazard function above. Mathematically, with a median T, and considering the case where G(t) = G1(t) = G2(t), an equation for the c.d.f. that satisfies these conditions is given as follows: (3) where for some function g(s). The ODE that arises in (3) defines the Burr family of distributions and is discussed in further detail in the next section. Burr distribution. A Burr distribution is a distribution whose c.d.f., F(t), (4) is the solution of (3). Theoretically, there are no constraints on G(t) in (4). Twelve main distributions within the Burr family have been characterized [25], named as Burr type I, Burr type II, up-to Burr type XII, but we only consider Burr distributions defined over a domain of (0, ∞). Some delay distributions arising in epidemiology do permit negative values. For example, the time from symptom onset in infector to symptom onset in infectee could be negative. In this paper, we limit consideration to strictly positive cases. A negative incubation period is not possible, nor is a fixed upper-limit constraint expected. The only biologically feasible distributions are types III, X and XII. The type III distribution could be derived from the flexible generalized gamma distribution with the scale parameter following an inverse Weibull distribution [26]. Similarly, the type XII distribution could be derived from the Weibull distribution where the scale parameter follows an inverse generalized gamma distribution [26]. The Burr distributions and the gamma distribution have parameters that share the same symbols for notational simplicity, although they have different interpretations and their fitted estimates cannot be directly compared. To avoid confusion, we provide a subscript for each parameter to clarify which distribution this parameter corresponds to (i.e., αIII for the α parameter in the type III Burr model) in the text but drop this in tables and figures for brevity. Further, the type III, X and XII distributions used in this research are a generalization of types III, X and XII Burr distributions used in the literature [25], where the time variable is scaled by an additional parameter. Type X is defined with two variables that provide models as parsimonious as the three distributions previously trialled: gamma, log-normal, and Weibull. Further, both type III and XII distributions have two shape parameters αIII,XII and βIII,XII. Finally, the type III distribution has a scale parameter γIII,XII and the type XII distribution has a location parameter, the median T. General derived Burr distribution. In (3), g(t) has a physical interpretation; the function tends to the rate of symptom onset μ(t) in individuals at a time t as t increases. Given F(t) = (1 + e−G(t))−1, in general, then G(t) → t/βD (or g(t) → 1/βD) for some constant βD as t → ∞ on the basis that the hazard rate approaches constant over time for relatively long incubation periods. In principle, F(0) = 0, so G(t) → −∞ for t → 0 (or G(0) is very large if not actually infinite). Taking the above into account, we propose g(t) = 1/βD + αD/t, and as such G(t) = t/βD + αD log(t) + C, where C is a constant of integration. We define TD as the median, which satisfies G(TD) = 0. Hence, C = −TD/βD − αD log(TD) and thus Equations for the c.d.f. and p.d.f. for the derived Burr distribution, as well as the gamma and other Burr distributions, are given in Table 1. As discussed, TD is the median of the distribution. The reciprocal of βD is the eventual constant rate of symptom onset in individuals for t ≫ TD. Additionally, there are two details worth noting when analysing the physical interpretation of αD. First, αD is an exponent of t that controls the increase in probability density, as for t ≪ TD. Second, the general derived Burr distribution approaches the exponential distribution for t ≫ TD. The rate at which the derived Burr distribution approaches a constant hazard (as for an exponential distribution) increases for decreasing αD. Finally, all parameters must be strictly greater than zero. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. The Burr distributions valid over (0, ∞) and previously trialled distributions [4] with their corresponding p.d.f and c.d.f. https://doi.org/10.1371/journal.pcbi.1012041.t001 A probability-based approach. A continuous-time mathematical model can be built considering the hazard rate of symptom-onset occurrence. We first consider the option of using an exponential survival model with time-varying hazard. Define N(t) as the population of individuals who are infected, but are not yet symptomatic at time t, and Q(t) as the population of individuals who are symptomatic at time t with N(0) = N0 and Q(0) = 0 and Q(t) + N(t) = N0, . Next, assume that a hazard rate function λ(t) describes the risk that a not-yet-symptomatic individual will start to experience symptoms at a point in time t, given that they have not already succumb to symptoms by time t. Then 1 − δtλ(t) will be the probability that the individual will remain asymptomatic within a small interval (t, t + δt), where in this context δt represents a small time increment in time. Hence (1 − δtλ(t))N(t) is the probability that nobody who is not-yet-symptomatic will start experiencing symptoms within a small increment δt from t, and (1 − δtλ(t))N(t)δt is the probability that nobody new will experience symptoms in a small increment δt from t. Following this, define δQ(t) = 1 − (1 − δtλ(t))N(t)δt to be the probability that there is at least one individual who starts to experience new symptoms in a small increment δt from t. By writing μ(t) = −log(1 − δtλ(t)), the probability of any new symptom onset appearance can be written as δQ(t) = 1 − e−μ(t)N(t)δt. Using a Taylor expansion on the exponential term, dividing by δt, and taking the limit δt → 0 changes this probability to a rate as follows: (2) This approach leads to a separable ordinary differential equation analogous to the cumulative distribution of the exponential distribution with a time-varying rate parameter. It can be deduced that and that is the accumulated hazard. Hence the rate of symptom onset, μ(t), is the hazard function of an individual becoming symptomatic. Therefore, the hazard of an individual becoming symptomatic at a point in time is equal to the rate of symptom onset at that time. The scenario discussed here can be considered from an inhomogeneous Poisson-process perspective, and the results of the hazard are identical to the inhomogeneous exponentially distributed model. It can be noted here that if μ(t) is constant that this would lead to the exponential distribution and if μ(t) ∝ ta for some constant a this would suggest the incubation period is a Weibull distributed random variable. The Erlang distribution arises by assuming the incubation period is the sum of a number of stages of constant length μ. However, various studies have shown that the bacterial load within an individual is positively correlated with the probability of symptom onset [3, 21]. The relationship between bacterial load and probability of symptom onset is complex and varies from bacteria-to-bacteria [21]. Additionally, a positive correlation between load and probability of symptom onset has been observed for viral infections [22]. For parsimony, we assume that symptom onset is likely proportional to bacterial (or viral) load at low loads (i.e., the early stages of infection) before saturating at large loads. The bacterial population early after infection will be approximately some exponential function of time [3, 23, 24]. Therefore, the left tail of the c.d.f. of the incubation-period distribution is given by some function , whilst in the later stage, the c.d.f. should tend to 1 exponentially given by a function G2(t), as is the case of the hazard function above. Mathematically, with a median T, and considering the case where G(t) = G1(t) = G2(t), an equation for the c.d.f. that satisfies these conditions is given as follows: (3) where for some function g(s). The ODE that arises in (3) defines the Burr family of distributions and is discussed in further detail in the next section. Burr distribution. A Burr distribution is a distribution whose c.d.f., F(t), (4) is the solution of (3). Theoretically, there are no constraints on G(t) in (4). Twelve main distributions within the Burr family have been characterized [25], named as Burr type I, Burr type II, up-to Burr type XII, but we only consider Burr distributions defined over a domain of (0, ∞). Some delay distributions arising in epidemiology do permit negative values. For example, the time from symptom onset in infector to symptom onset in infectee could be negative. In this paper, we limit consideration to strictly positive cases. A negative incubation period is not possible, nor is a fixed upper-limit constraint expected. The only biologically feasible distributions are types III, X and XII. The type III distribution could be derived from the flexible generalized gamma distribution with the scale parameter following an inverse Weibull distribution [26]. Similarly, the type XII distribution could be derived from the Weibull distribution where the scale parameter follows an inverse generalized gamma distribution [26]. The Burr distributions and the gamma distribution have parameters that share the same symbols for notational simplicity, although they have different interpretations and their fitted estimates cannot be directly compared. To avoid confusion, we provide a subscript for each parameter to clarify which distribution this parameter corresponds to (i.e., αIII for the α parameter in the type III Burr model) in the text but drop this in tables and figures for brevity. Further, the type III, X and XII distributions used in this research are a generalization of types III, X and XII Burr distributions used in the literature [25], where the time variable is scaled by an additional parameter. Type X is defined with two variables that provide models as parsimonious as the three distributions previously trialled: gamma, log-normal, and Weibull. Further, both type III and XII distributions have two shape parameters αIII,XII and βIII,XII. Finally, the type III distribution has a scale parameter γIII,XII and the type XII distribution has a location parameter, the median T. General derived Burr distribution. In (3), g(t) has a physical interpretation; the function tends to the rate of symptom onset μ(t) in individuals at a time t as t increases. Given F(t) = (1 + e−G(t))−1, in general, then G(t) → t/βD (or g(t) → 1/βD) for some constant βD as t → ∞ on the basis that the hazard rate approaches constant over time for relatively long incubation periods. In principle, F(0) = 0, so G(t) → −∞ for t → 0 (or G(0) is very large if not actually infinite). Taking the above into account, we propose g(t) = 1/βD + αD/t, and as such G(t) = t/βD + αD log(t) + C, where C is a constant of integration. We define TD as the median, which satisfies G(TD) = 0. Hence, C = −TD/βD − αD log(TD) and thus Equations for the c.d.f. and p.d.f. for the derived Burr distribution, as well as the gamma and other Burr distributions, are given in Table 1. As discussed, TD is the median of the distribution. The reciprocal of βD is the eventual constant rate of symptom onset in individuals for t ≫ TD. Additionally, there are two details worth noting when analysing the physical interpretation of αD. First, αD is an exponent of t that controls the increase in probability density, as for t ≪ TD. Second, the general derived Burr distribution approaches the exponential distribution for t ≫ TD. The rate at which the derived Burr distribution approaches a constant hazard (as for an exponential distribution) increases for decreasing αD. Finally, all parameters must be strictly greater than zero. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. The Burr distributions valid over (0, ∞) and previously trialled distributions [4] with their corresponding p.d.f and c.d.f. https://doi.org/10.1371/journal.pcbi.1012041.t001 Model comparison We fit each type of Burr distribution to the data, and assess all the models in terms of their goodness of fit in comparison to the more widely used gamma distribution. The most commonly used methods for model selection are the Akaike information criterion (AIC) and Bayesian information criterion (BIC) [27]. Generally, AIC puts more emphasis on good model prediction, whereas BIC favours model parsimony [27]. Because our goal is good model prediction, the AIC will be used in deciding desirable model fits. Additionally, we calculate the Akaike weights ω for each model fit, which can be used for further model comparison [28]. Akaike weights are used to compare the validity of a Burr distributed model over the gamma distributed model once fit to data. The ratio wi/wG, where wi is the weight for the ith model and wG is the weight of the gamma distributed model. This ratio may be interpreted as how much more likely model i is a better fitting model than the gamma model. Alternatively, we also derive the normalized probability that the ith model is preferable to the gamma model, given by wi/(wi + wG). The final method of comparison considered is the Bayes factor [29]. The maximum likelihood estimates that we obtain can be considered maximum a posteriori estimates with a uniform prior and are used in this context for conducting the Bayes Factor calculations. Larger Bayes factor values indicate stronger evidence to support one model over another. Incubation-period data To test these models, we employ incubation-period data from an outbreak of Legionnaires’ disease in Melbourne in April 2000 [18]. The data for the Melbourne outbreak contains the number of days taken for each Legionnaires’ disease case to develop symptoms from their exposure date, and several potential distributions for fitting the data have been compared [4]. The case data only contains individuals that visited the known source once within the weeks before the outbreak and have a known date of symptom onset. Therefore, the timing of infection and symptom onset events are known correctly to a single day. The results indicated that the gamma distribution provided the best fit [4] out of their proposed models. Further, we gather incubation-period data for anthrax, campylobacteriosis and salmonellosis for analysis. The anthrax outbreak in 1979 contains data for the known incubation-periods of patients [30]. Investigations into the outbreak, climate conditions and human/animal presence highlighted a single exposure on 2 April 1979. The date of symptom onset was provided as known in most cases. However, for the few remaining cases, an estimated symptom onset date was obtained by subtracting 3 days (the mean delay between symptom onset and death) from the date of death. A literature review has been conducted analysing different salmonellosis studies that contain full data of the incubation periods [31]. Awofisayo-Okuyelu et al. [31] noticed that the incubation periods varied between studies. They grouped studies into subsets using a clustering process, in which the grouped studies did not have any statistically significant difference in their incubation-period data. Similarly, Awofisayo-Okuyelu et al. [32] conducted a review for campylobacteriosis in which the incubation periods varied between studies, and they combined datasets which were not statistically significantly different using a clustering process similar to [31]. For both the salmonellosis and campylobacteriosis datasets, a quality assessment was carried out during their literature review [31, 32]. Incubation period data obtained was assessed based on whether cases were linked to a clearly defined exposure and accuracy of the reported symptom onset time, with the lowest level of resolution in the symptom onset being a period of 24 hours [31, 32]. We provide an Excel sheet of the incubation-period data for these other diseases in S1 Data in the Supplementary Material. The data gathered for these diseases share a similarity with the Legionnaires’ disease data, in that the data contains the integer number of days taken for each case to develop symptoms. The fact the data for all of these diseases contains integer days implies that each case takes an exact multiple of 24 hours from infection to the appearance of symptoms, which is not realistic. If we assume that the dates of infection and symptom onset are accurate, then we know the date of these events, but the specific times on the given days are unknown. We are dealing with doubly censored data. Results Now that we have developed the Burr distribution as an incubation-period model based upon biological justifications, the next step is to fit these models to the incubation-period data of various diseases. We begin by fitting the incubation-period models to the Legionnaires’ disease data, to draw comparisons between the models’ performance. Next, we conduct the same analysis on other diseases such as anthrax, campylobacteriosis and salmonellosis. Finally, we conduct two simulations in which incubation-period data are fabricated. First, we compare the results from fitting the incubation-period models to fabricated data, as we compare the parameter estimates obtained from fitting the gamma and derived Burr distributions to this data in an attempt to assess the relationship between these parameters. Second, we fabricate doubly-censored data and fit the derived Burr distribution using the DI likelihood fitting method to this data. We aim to assess bias in the parameter estimates and the appropriate coverage of 95% confidence intervals of the parameter estimates. Analysis of the Melbourne data The gamma distribution is currently most frequently used to model Legionnaires’ disease incubation periods [4]. Therefore, we produce models using a gamma-distributed incubation period to allow for comparison between models. Models are fitted using both the standard and doubly interval-censored maximum likelihood fitting methods to offer comparison between the two methods. We begin this section by providing the results from fitting the incubation-period models to the data (Table 2). We compare the incubation-period models, as well as model-fitting approaches, and the effect that they have on our understanding of Legionnaires’ disease incubation periods. We provide analysis of the moments of these Legionnaires’ disease incubation-period models in S1 Appendix in the Supplementary Material. Further, in this appendix, we provide visual comparison of the accumulated hazard of these models for large time, to examine their ability to accurately display a Markovian property of long incubation periods. The analysis and production of plots was conducted on R, with the code provided in S1 Code in the Supplementary Material. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 2. Results from fitting the gamma and four Burr distribution models to the Melbourne incubation-period data using both the standard and DI likelihood fitting methods. https://doi.org/10.1371/journal.pcbi.1012041.t002 When fitting using the standard maximum likelihood method, type III, X, XII distributions and the derived Burr distribution perform better than the gamma distribution regardless of which scoring criterion is used. Because the type X distribution is a two-parameter distribution, the fact that its maximized log-likelihood is higher than gamma’s automatically means that its minimized AIC will be lower. Types III, XII and the derived Burr distributions perform better than the gamma distribution depending on how harshly they are penalized for their extra parameter. Based on AIC, our ideal information criterion for model selection, these perform better than the gamma distribution. On the whole, all Burr distributions perform better than the gamma distribution. From considering the Akaike weights ratio w/wG, the derived Burr, type III, and type X distributions are at least two times as likely to be a better-performing model than the gamma distributed model. Additionally, each Burr model provides at least a 62% chance of being a better fitting model than the gamma-distributed model, with the derived Burr model being 70% more likely to be better than the gamma model. Looking at the Bayes factor, there is no substantial evidence to favour the type X distribution over the gamma distribution. However, this criterion gives substantial evidence that both type III, XII distributions as well as the derived Burr distribution are all favourable over the gamma distribution. Next, when fitting using doubly interval-censoring methods, the type X distribution again outperforms the gamma distribution. Types III, XII and the derived Burr distributions perform better than the gamma model, based on AIC, even with one extra parameter. When considering the Akaike weights, all the Burr distributed models perform much better than the gamma distribution, with the derived Burr distribution being over 13 times more likely to be the better-fitting model. Additionally, when considering w/(w + wG), all Burr models are more likely to be perform better than the gamma distribution, with the derived Burr distribution being 93% likely. Finally, the Bayes factor for the types X and XII distributions both show substantial evidence of a better fit than the gamma distribution. Further, the Bayes factor for type III and the derived Burr distributions both show strong evidence of a better fit than the gamma model. The same conclusions are drawn regardless of maximum likelihood fitting method; all the distributions provide a better fit than the gamma distribution. Results using the DI method agree with the standard likelihood method in that βXII in the Burr type XII model has large standard errors. For this distribution, T centers the curve about the median and α scales the curve. The large standard errors for β indicate that β is not as important in the model fitting procedure. When fitted using standard maximum likelihood methods, all Burr distributions considered offer a similar curve when plotted, as expected, but do vary slightly as to the model value or the value of the p.d.f. at the mode (Fig 2). The Weibull distribution provides a similar modal value for the incubation period, but is more variable than the Burr models. The gamma distribution provides a slightly lower modal value than the Burr models. The log-normal model provides a noticeably different curve to the Burr models and provides a much lower modal incubation period, with a lighter left tail and heavier right tail than all the other distributions. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. A plot of the Melbourne case data with the four fitted Burr distributions included, which offer a visual representation of the incubation-period distributions trialled. https://doi.org/10.1371/journal.pcbi.1012041.g002 The maximised log-likelihood decreases when switching to the DI method for some distributions, such as the log-normal and gamma. On the other hand, the maximised log-likelihood increases for other distributions, such as the derived Burr distribution. For distributions fitted using the standard maximum likelihood fitting method that have a lower modal value lower than the modal value of the data, changing to the DI method reduces the modal value further. This larger difference results in a distribution further from the data. Therefore, lower maximised log-likelihood values are typically obtained using the DI method when the mode of the distribution is lower than the mode of the data. However, this reasoning does not hold true for the type III distribution. The mean of each fitted distribution along with a bootstrapped 95% confidence interval is calculated under both the standard method and the doubly interval-censored method to identify any differences across distributions and across methods, and is provided in S1 Appendix in the Supplementary Material. A common theme exists, which is that, for each distribution, the mean for the doubly interval-censored model is approximately a day less than the standard model (5.3 days compared to 6.3 days), with the confidence intervals for each model having no overlap across all the distributions. We bootstrap from the distributions and apply a two-sample t-test to assess investigate whether, for each distribution, the mean incubation period obtained from the doubly interval-censored method is statistically-significantly lower than the mean incubation period obtained from the standard method. These calculations and the associated p-values are provided in S1 Appendix in the Supplementary Material. These results are statistically significant and provide support for using a doubly interval-censored model to more accurately represent the incubation period of Legionnaires’ disease. For all of the distributions, the density under the doubly interval-censored approach is shifted more towards the left, indicating that the incubation period is shorter than when just taking the incubation period as exact integer days (Fig 3). Indeed, the doubly interval-censored methods account for a potential delay between the start of the infection day and the time during the infection day that infection occurs as well as a delay between the time during the symptom-onset day that symptoms appear and the end of the symptom-onset day, whereas the standard model does not account for either delay, resulting in longer times for the incubation periods. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Plots of the Melbourne data with the standard model fits in red and the doubly interval-censored model fit as a step function in yellow. Each step of the function is a horizontal line from t ∈ (a, b] where a = ⌊t⌋ and b = ⌈t⌉. https://doi.org/10.1371/journal.pcbi.1012041.g003 Application to other diseases To further check the validity of the Burr distribution, we fit the doubly interval-censored models to data of the incubation periods for different diseases: anthrax [30], campylobacteriosis [32] and salmonellosis [31]. Figures of resulting model fits provided in S1 Fig, along with the obtained parameter estimates and standard errors of these estimates contained in S1 Fig in the Supplementary Material. We use both the standard and the doubly interval-censored methods to fit the gamma and the Burr distributions, to compare which model provides a better fit (Table 3). Download: PPT PowerPoint slide PNG larger image TIFF original image Table 3. Comparing Burr, log-normal and Weibull models with the gamma model on anthrax, salmonellosis and campylobacteriosis datasets. For brevity, we define the datasets as A, S or C to represent anthrax, salmonellosis and campylobacteriosis datasets respectively. The numbers in the dataset column indicate which dataset for a given disease is being referred to, as we have multiple incubation-period datasets for Salmonella and Campylobacter. The value provided is the difference between the recorded AIC between a given model and the gamma distribution. Negative values indicate lower AIC, which is preferable. Similarly, positive values indicate higher AIC, which implies a worse model fit. https://doi.org/10.1371/journal.pcbi.1012041.t003 For Burr types III, XII and the derived Burr distribution, a difference in AIC between 0 and +2 indicates that the Burr model provides a preferable fit based on maximum likelihood estimation, but the extra parameter results in a higher AIC. Burr type III and X distributions offer mixed results across datasets and do not consistently outperform the gamma distribution. Based on maximum likelihood, the derived Burr distribution outperforms the gamma distribution for every dataset other than the third campylobacteriosis dataset. However, based on AIC, the gamma distribution becomes preferable for the anthrax dataset and the first salmonellosis dataset regardless of maximum likelihood fitting method. Additionally, based on AIC, the gamma distribution becomes preferable to the derived Burr distribution when fitting to the second and fifth campylobacteriosis datasets with the doubly interval-censored and standard maximum likelihood methods respectively. Instances in which the gamma distribution provides preferable results based on AIC is typically due to the penalty from the derived Burr distribution’s extra parameter. Therefore, these datasets result in relatively close model fits between the gamma and derived Burr distributions. These conclusions hold, to a lesser extent, for the type XII distribution. No clear pattern exists between any of the fitted αD and βD parameter estimates and the performance of the derived Burr distribution. Additionally, there is no clear pattern from the anthrax, campylobacteriosis and salmonellosis datasets as to whether the estimate of the median TD relates to the performance of the derived Burr model. However, the lack of sensitivity for TD is logical as TD solely scales the distribution about the median, and the ability of the derived Burr distribution to fit well to incubation period data will depend more on the tails in the curve and around the median, as opposed to the median itself. We can draw conclusions on which scenarios the derived Burr distribution will outperform the gamma distribution based on plots provided in S1 Fig of the Supplementary Material. The third campylobacteriosis dataset was the only dataset in which the derived Burr distribution did not outperform the gamma distribution based on either maximized likelihood or on AIC. This dataset is unique in that the incubation period ranges from one to five days. As a result, the effect of the censoring bias will be much larger, due to the fact that this incubation period is much shorter. Therefore, this is not an ideal dataset to use to assess model performance. Next, we consider the datasets in which the derived Burr distribution outperformed the gamma distribution based on maximized likelihood but not on AIC, regardless of model fitting procedure. The anthrax dataset has a high density after the mode and does not tail off, and the probability distribution of the first salmonellosis dataset does not have a clearly defined mode and is negatively skewed. The derived Burr distribution offers close results to the gamma distribution when it comes to modelling incubation periods without a clear mode or tail off in probability of illness, but is a better-performing distribution when this structure is clearer defined. Finally, fitting to the second and fifth campylobacteriosis datasets resulted in the derived Burr distribution outperforming the gamma distribution on maximized likelihood but not on AIC. The incubation period for these datasets is relatively small, meaning that the bias from the censoring issue is large when fitting models to these datasets. The campylobacteriosis datasets that resulted in the derived Burr distribution outperforming the gamma distribution were the ones in which the modal time was clearly defined and not a wide range of times at the peak of the distribution. These results further supports the hypothesis that the derived Burr distribution becomes more preferable when either the mode is more apparent, or the range of incubation periods in the datasets is not too short that the censoring becomes a larger issue. Results of model-fitting to simulated data We now further assess the validity of the Burr distributions by comparing their fits, along with those of the gamma distribution, to fabricated data. Specifically, we aim to analyse how the parameter estimates of the gamma distribution relate to the parameter estimates of the derived Burr distribution for different datasets, to gain a further understanding of how the derived Burr distribution’s parameters can be interpreted. Initially, we generate a sample of size 1000 from a gamma distribution with given shape αΓ and mean μΓ (scale βΓ = μΓ/αΓ). Then, the derived Burr distribution parameter estimates are obtained from fitting to this dataset by standard maximum likelihood, so that analysis can be conducted on the effect that varying αΓ ∈ (0.5, 5) or μΓ ∈ (1, 20) has on these estimates. First, this simulation focuses on analysing the relationship between the parameters of the gamma and derived Burr distributions. Therefore, we fit by standard maximum likelihood as opposed to the doubly interval-censored methods. A heatmap is produced to visualise this effect (Fig 4). Additionally, we repeat the same simulation with doubly-censored data to compare the parameter relationships under more realistic conditions. We simulate doubly-censored data by sampling an infection time from the uniform distribution on the infection day and adding this to a sample from a gamma distribution with αΓ ∈ (0.5, 5) and μΓ ∈ (1, 20). We take the ceiling of this sum to produce the doubly-censored incubation period. Repeating this for a sample of size 1000, we fit the derived Burr distribution using the DI method to obtain the corresponding parameter estimates. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Heatmaps of the results from the first simulation. The sub-figures (a), (c) and (e) (left column panels) represent the results from the standard likelihood fitting simulation, whereas (b), (d) and (f) (right column panels) represent the results from the DI likelihood fitting simulation. The sub-figures (a) and (b) represent the αD estimates obtained. Next, (c) and (d) represent the βD estimates obtained. Finally, (e) and (f) represent the TD estimates obtained. https://doi.org/10.1371/journal.pcbi.1012041.g004 The parameter estimates for βD and TD are invariant under the maximum likelihood fitting procedure. Fitting the derived Burr distribution using the DI methods to doubly-censored data generated from the gamma distribution results in similar parameter estimates to those obtained from fitting the derived Burr distribution using standard maximum likelihood fitting methods to non-censored data generated from the gamma distribution. However, a discrepancy exists for estimates of αD between methods. Fitting to doubly-censored gamma-distributed data with the DI method results in larger estimates of αD than fitting to gamma-distributed data with the standard method. No clear pattern exists for αD estimates for small μΓ ≈ 1. Because μΓ ≈ 1, the gamma-distributed incubation-period data are small in value, which means that the uniform infection time and ceiling function have a larger effect on the data than the distribution that generates the true incubation period. Therefore, in this circumstance the noise introduced in these two windows affects the estimation process for αD with the DI method. The shape parameter αD of the derived Burr distribution is more sensitive to small changes in the doubly-censored data when the mean incubation periods are relatively small. In general, parallels exist between the interpretations of αΓ and αD. Increasing αΓ results in a larger discrepancy between the gamma distribution and the exponential distribution. Thus, larger αΓ values result in a longer period of time required for the distribution to become Markovian. Therefore, a positive correlation between αΓ and αD is expected (Fig 4a and 4b). The results indicate that μΓ does not have an effect on the rate at which the gamma distribution becomes Markovian. Similarly, parallels exist between the interpretations of βΓ and βD. The hazard rate for the gamma distribution tends to 1/βΓ as t → ∞. Hence, 1/βΓ is the eventual rate of symptom onset for the gamma distribution. Thus, a positive correlation between βΓ and βD is logical (Fig 4c and 4d). Therefore, the effect that varying either μΓ or αΓ in μΓ = αΓβΓ has on βΓ is likely to inform the effect that varying either μΓ or αΓ has on βD. Finally, a positive correlation between μΓ and TD is expected, as they both represent a form of average. For large αΓ, the gamma distribution becomes symmetric, hence TD → μΓ. However, the correlation becomes less linear as αΓ decreases. In this case, μΓ − TD and equivalently the skewness (defined by for the gamma distribution) increases (Fig 4e and 4f). Following this simulation, we provide a second simulation in which we further assess the performance of the doubly interval-censored maximum likelihood fitting methods with the derived Burr distribution. Doubly-censored data are generated in the same way as the previous simulation, with the derived Burr distribution used instead of the gamma distribution for the true incubation period. We fit the derived Burr distribution to this fabricated data using the DI method to gain parameter estimates for this dataset. We record the bias of the parameter estimates and appropriate coverage for the 95% confidence intervals of the parameter estimates to assess the performance of the doubly interval-censored maximum likelihood fitting procedure. We opt to vary one parameter and keep the other two fixed for this simulation due to the computational demand of varying two parameters and producing heatmaps as done in the first simulation (Fig 5). The two fixed parameters are fixed at the estimates obtained from fitting to the Legionnaires’ disease dataset. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Bias and appropriate coverage of 95% confidence intervals obtained from fitting the derived Burr distribution using DI methods to doubly-censored incubation periods generated from the derived Burr distribution. The sub-figures (a), (b) and (c) represent the bias from the αD, βD and TD-varying simulations respectively. Further, the sub-figures (d), (e) and (f) represent the appropriate coverage from the αD, βD and TD-varying simulations respectively. https://doi.org/10.1371/journal.pcbi.1012041.g005 For the αD-varying simulation, the bias for αD and βD estimates increase as the true αD increases. Increasing the true αD results in a more spiked true incubation-period distribution. Further, taking the convolution of the spiked incubation-period distribution with a uniform infection window distribution and taking the ceiling results in a less-spiked doubly-censored distribution. The DI maximum likelihood fitting method fails to fully capture the original spike and estimates a flatter distribution, which results in positive bias in the estimates of αD and βD. Further, for the βD-varying simulation, the bias for βD increases, whereas the bias for αD decreases as the true βD increases. Increasing the true βD results in a less spiked true incubation period and in which case the DI method is able to extract estimates of αD close to the true value. However, in this case, the relative bias of βD remains constant as βD varies. Finally, for the TD-varying simulation, the bias in αD and βD remains approximately constant, with some variability due to the random sampling when generating datasets. For each parameter-varying simulation, we obtain almost unbiased estimates for TD, which indicates that regardless of which parameter is varied in the data-generating process, the DI method is successful at locating the median of the distribution. For the appropriate coverage, we repeat 100 iterations of the simulation to record a proportion of times in which the true parameter is contained within the 95% confidence interval. For the βD and TD-varying simulations, the appropriate coverage for each parameter centers roughly around 95%. For the αD-varying simulation, the appropriate coverage for TD centers around 95%. However, as the true αD increases, the coverage for αD increases and the coverage for βD decreases. Analysis of the Melbourne data The gamma distribution is currently most frequently used to model Legionnaires’ disease incubation periods [4]. Therefore, we produce models using a gamma-distributed incubation period to allow for comparison between models. Models are fitted using both the standard and doubly interval-censored maximum likelihood fitting methods to offer comparison between the two methods. We begin this section by providing the results from fitting the incubation-period models to the data (Table 2). We compare the incubation-period models, as well as model-fitting approaches, and the effect that they have on our understanding of Legionnaires’ disease incubation periods. We provide analysis of the moments of these Legionnaires’ disease incubation-period models in S1 Appendix in the Supplementary Material. Further, in this appendix, we provide visual comparison of the accumulated hazard of these models for large time, to examine their ability to accurately display a Markovian property of long incubation periods. The analysis and production of plots was conducted on R, with the code provided in S1 Code in the Supplementary Material. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 2. Results from fitting the gamma and four Burr distribution models to the Melbourne incubation-period data using both the standard and DI likelihood fitting methods. https://doi.org/10.1371/journal.pcbi.1012041.t002 When fitting using the standard maximum likelihood method, type III, X, XII distributions and the derived Burr distribution perform better than the gamma distribution regardless of which scoring criterion is used. Because the type X distribution is a two-parameter distribution, the fact that its maximized log-likelihood is higher than gamma’s automatically means that its minimized AIC will be lower. Types III, XII and the derived Burr distributions perform better than the gamma distribution depending on how harshly they are penalized for their extra parameter. Based on AIC, our ideal information criterion for model selection, these perform better than the gamma distribution. On the whole, all Burr distributions perform better than the gamma distribution. From considering the Akaike weights ratio w/wG, the derived Burr, type III, and type X distributions are at least two times as likely to be a better-performing model than the gamma distributed model. Additionally, each Burr model provides at least a 62% chance of being a better fitting model than the gamma-distributed model, with the derived Burr model being 70% more likely to be better than the gamma model. Looking at the Bayes factor, there is no substantial evidence to favour the type X distribution over the gamma distribution. However, this criterion gives substantial evidence that both type III, XII distributions as well as the derived Burr distribution are all favourable over the gamma distribution. Next, when fitting using doubly interval-censoring methods, the type X distribution again outperforms the gamma distribution. Types III, XII and the derived Burr distributions perform better than the gamma model, based on AIC, even with one extra parameter. When considering the Akaike weights, all the Burr distributed models perform much better than the gamma distribution, with the derived Burr distribution being over 13 times more likely to be the better-fitting model. Additionally, when considering w/(w + wG), all Burr models are more likely to be perform better than the gamma distribution, with the derived Burr distribution being 93% likely. Finally, the Bayes factor for the types X and XII distributions both show substantial evidence of a better fit than the gamma distribution. Further, the Bayes factor for type III and the derived Burr distributions both show strong evidence of a better fit than the gamma model. The same conclusions are drawn regardless of maximum likelihood fitting method; all the distributions provide a better fit than the gamma distribution. Results using the DI method agree with the standard likelihood method in that βXII in the Burr type XII model has large standard errors. For this distribution, T centers the curve about the median and α scales the curve. The large standard errors for β indicate that β is not as important in the model fitting procedure. When fitted using standard maximum likelihood methods, all Burr distributions considered offer a similar curve when plotted, as expected, but do vary slightly as to the model value or the value of the p.d.f. at the mode (Fig 2). The Weibull distribution provides a similar modal value for the incubation period, but is more variable than the Burr models. The gamma distribution provides a slightly lower modal value than the Burr models. The log-normal model provides a noticeably different curve to the Burr models and provides a much lower modal incubation period, with a lighter left tail and heavier right tail than all the other distributions. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. A plot of the Melbourne case data with the four fitted Burr distributions included, which offer a visual representation of the incubation-period distributions trialled. https://doi.org/10.1371/journal.pcbi.1012041.g002 The maximised log-likelihood decreases when switching to the DI method for some distributions, such as the log-normal and gamma. On the other hand, the maximised log-likelihood increases for other distributions, such as the derived Burr distribution. For distributions fitted using the standard maximum likelihood fitting method that have a lower modal value lower than the modal value of the data, changing to the DI method reduces the modal value further. This larger difference results in a distribution further from the data. Therefore, lower maximised log-likelihood values are typically obtained using the DI method when the mode of the distribution is lower than the mode of the data. However, this reasoning does not hold true for the type III distribution. The mean of each fitted distribution along with a bootstrapped 95% confidence interval is calculated under both the standard method and the doubly interval-censored method to identify any differences across distributions and across methods, and is provided in S1 Appendix in the Supplementary Material. A common theme exists, which is that, for each distribution, the mean for the doubly interval-censored model is approximately a day less than the standard model (5.3 days compared to 6.3 days), with the confidence intervals for each model having no overlap across all the distributions. We bootstrap from the distributions and apply a two-sample t-test to assess investigate whether, for each distribution, the mean incubation period obtained from the doubly interval-censored method is statistically-significantly lower than the mean incubation period obtained from the standard method. These calculations and the associated p-values are provided in S1 Appendix in the Supplementary Material. These results are statistically significant and provide support for using a doubly interval-censored model to more accurately represent the incubation period of Legionnaires’ disease. For all of the distributions, the density under the doubly interval-censored approach is shifted more towards the left, indicating that the incubation period is shorter than when just taking the incubation period as exact integer days (Fig 3). Indeed, the doubly interval-censored methods account for a potential delay between the start of the infection day and the time during the infection day that infection occurs as well as a delay between the time during the symptom-onset day that symptoms appear and the end of the symptom-onset day, whereas the standard model does not account for either delay, resulting in longer times for the incubation periods. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Plots of the Melbourne data with the standard model fits in red and the doubly interval-censored model fit as a step function in yellow. Each step of the function is a horizontal line from t ∈ (a, b] where a = ⌊t⌋ and b = ⌈t⌉. https://doi.org/10.1371/journal.pcbi.1012041.g003 Application to other diseases To further check the validity of the Burr distribution, we fit the doubly interval-censored models to data of the incubation periods for different diseases: anthrax [30], campylobacteriosis [32] and salmonellosis [31]. Figures of resulting model fits provided in S1 Fig, along with the obtained parameter estimates and standard errors of these estimates contained in S1 Fig in the Supplementary Material. We use both the standard and the doubly interval-censored methods to fit the gamma and the Burr distributions, to compare which model provides a better fit (Table 3). Download: PPT PowerPoint slide PNG larger image TIFF original image Table 3. Comparing Burr, log-normal and Weibull models with the gamma model on anthrax, salmonellosis and campylobacteriosis datasets. For brevity, we define the datasets as A, S or C to represent anthrax, salmonellosis and campylobacteriosis datasets respectively. The numbers in the dataset column indicate which dataset for a given disease is being referred to, as we have multiple incubation-period datasets for Salmonella and Campylobacter. The value provided is the difference between the recorded AIC between a given model and the gamma distribution. Negative values indicate lower AIC, which is preferable. Similarly, positive values indicate higher AIC, which implies a worse model fit. https://doi.org/10.1371/journal.pcbi.1012041.t003 For Burr types III, XII and the derived Burr distribution, a difference in AIC between 0 and +2 indicates that the Burr model provides a preferable fit based on maximum likelihood estimation, but the extra parameter results in a higher AIC. Burr type III and X distributions offer mixed results across datasets and do not consistently outperform the gamma distribution. Based on maximum likelihood, the derived Burr distribution outperforms the gamma distribution for every dataset other than the third campylobacteriosis dataset. However, based on AIC, the gamma distribution becomes preferable for the anthrax dataset and the first salmonellosis dataset regardless of maximum likelihood fitting method. Additionally, based on AIC, the gamma distribution becomes preferable to the derived Burr distribution when fitting to the second and fifth campylobacteriosis datasets with the doubly interval-censored and standard maximum likelihood methods respectively. Instances in which the gamma distribution provides preferable results based on AIC is typically due to the penalty from the derived Burr distribution’s extra parameter. Therefore, these datasets result in relatively close model fits between the gamma and derived Burr distributions. These conclusions hold, to a lesser extent, for the type XII distribution. No clear pattern exists between any of the fitted αD and βD parameter estimates and the performance of the derived Burr distribution. Additionally, there is no clear pattern from the anthrax, campylobacteriosis and salmonellosis datasets as to whether the estimate of the median TD relates to the performance of the derived Burr model. However, the lack of sensitivity for TD is logical as TD solely scales the distribution about the median, and the ability of the derived Burr distribution to fit well to incubation period data will depend more on the tails in the curve and around the median, as opposed to the median itself. We can draw conclusions on which scenarios the derived Burr distribution will outperform the gamma distribution based on plots provided in S1 Fig of the Supplementary Material. The third campylobacteriosis dataset was the only dataset in which the derived Burr distribution did not outperform the gamma distribution based on either maximized likelihood or on AIC. This dataset is unique in that the incubation period ranges from one to five days. As a result, the effect of the censoring bias will be much larger, due to the fact that this incubation period is much shorter. Therefore, this is not an ideal dataset to use to assess model performance. Next, we consider the datasets in which the derived Burr distribution outperformed the gamma distribution based on maximized likelihood but not on AIC, regardless of model fitting procedure. The anthrax dataset has a high density after the mode and does not tail off, and the probability distribution of the first salmonellosis dataset does not have a clearly defined mode and is negatively skewed. The derived Burr distribution offers close results to the gamma distribution when it comes to modelling incubation periods without a clear mode or tail off in probability of illness, but is a better-performing distribution when this structure is clearer defined. Finally, fitting to the second and fifth campylobacteriosis datasets resulted in the derived Burr distribution outperforming the gamma distribution on maximized likelihood but not on AIC. The incubation period for these datasets is relatively small, meaning that the bias from the censoring issue is large when fitting models to these datasets. The campylobacteriosis datasets that resulted in the derived Burr distribution outperforming the gamma distribution were the ones in which the modal time was clearly defined and not a wide range of times at the peak of the distribution. These results further supports the hypothesis that the derived Burr distribution becomes more preferable when either the mode is more apparent, or the range of incubation periods in the datasets is not too short that the censoring becomes a larger issue. Results of model-fitting to simulated data We now further assess the validity of the Burr distributions by comparing their fits, along with those of the gamma distribution, to fabricated data. Specifically, we aim to analyse how the parameter estimates of the gamma distribution relate to the parameter estimates of the derived Burr distribution for different datasets, to gain a further understanding of how the derived Burr distribution’s parameters can be interpreted. Initially, we generate a sample of size 1000 from a gamma distribution with given shape αΓ and mean μΓ (scale βΓ = μΓ/αΓ). Then, the derived Burr distribution parameter estimates are obtained from fitting to this dataset by standard maximum likelihood, so that analysis can be conducted on the effect that varying αΓ ∈ (0.5, 5) or μΓ ∈ (1, 20) has on these estimates. First, this simulation focuses on analysing the relationship between the parameters of the gamma and derived Burr distributions. Therefore, we fit by standard maximum likelihood as opposed to the doubly interval-censored methods. A heatmap is produced to visualise this effect (Fig 4). Additionally, we repeat the same simulation with doubly-censored data to compare the parameter relationships under more realistic conditions. We simulate doubly-censored data by sampling an infection time from the uniform distribution on the infection day and adding this to a sample from a gamma distribution with αΓ ∈ (0.5, 5) and μΓ ∈ (1, 20). We take the ceiling of this sum to produce the doubly-censored incubation period. Repeating this for a sample of size 1000, we fit the derived Burr distribution using the DI method to obtain the corresponding parameter estimates. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Heatmaps of the results from the first simulation. The sub-figures (a), (c) and (e) (left column panels) represent the results from the standard likelihood fitting simulation, whereas (b), (d) and (f) (right column panels) represent the results from the DI likelihood fitting simulation. The sub-figures (a) and (b) represent the αD estimates obtained. Next, (c) and (d) represent the βD estimates obtained. Finally, (e) and (f) represent the TD estimates obtained. https://doi.org/10.1371/journal.pcbi.1012041.g004 The parameter estimates for βD and TD are invariant under the maximum likelihood fitting procedure. Fitting the derived Burr distribution using the DI methods to doubly-censored data generated from the gamma distribution results in similar parameter estimates to those obtained from fitting the derived Burr distribution using standard maximum likelihood fitting methods to non-censored data generated from the gamma distribution. However, a discrepancy exists for estimates of αD between methods. Fitting to doubly-censored gamma-distributed data with the DI method results in larger estimates of αD than fitting to gamma-distributed data with the standard method. No clear pattern exists for αD estimates for small μΓ ≈ 1. Because μΓ ≈ 1, the gamma-distributed incubation-period data are small in value, which means that the uniform infection time and ceiling function have a larger effect on the data than the distribution that generates the true incubation period. Therefore, in this circumstance the noise introduced in these two windows affects the estimation process for αD with the DI method. The shape parameter αD of the derived Burr distribution is more sensitive to small changes in the doubly-censored data when the mean incubation periods are relatively small. In general, parallels exist between the interpretations of αΓ and αD. Increasing αΓ results in a larger discrepancy between the gamma distribution and the exponential distribution. Thus, larger αΓ values result in a longer period of time required for the distribution to become Markovian. Therefore, a positive correlation between αΓ and αD is expected (Fig 4a and 4b). The results indicate that μΓ does not have an effect on the rate at which the gamma distribution becomes Markovian. Similarly, parallels exist between the interpretations of βΓ and βD. The hazard rate for the gamma distribution tends to 1/βΓ as t → ∞. Hence, 1/βΓ is the eventual rate of symptom onset for the gamma distribution. Thus, a positive correlation between βΓ and βD is logical (Fig 4c and 4d). Therefore, the effect that varying either μΓ or αΓ in μΓ = αΓβΓ has on βΓ is likely to inform the effect that varying either μΓ or αΓ has on βD. Finally, a positive correlation between μΓ and TD is expected, as they both represent a form of average. For large αΓ, the gamma distribution becomes symmetric, hence TD → μΓ. However, the correlation becomes less linear as αΓ decreases. In this case, μΓ − TD and equivalently the skewness (defined by for the gamma distribution) increases (Fig 4e and 4f). Following this simulation, we provide a second simulation in which we further assess the performance of the doubly interval-censored maximum likelihood fitting methods with the derived Burr distribution. Doubly-censored data are generated in the same way as the previous simulation, with the derived Burr distribution used instead of the gamma distribution for the true incubation period. We fit the derived Burr distribution to this fabricated data using the DI method to gain parameter estimates for this dataset. We record the bias of the parameter estimates and appropriate coverage for the 95% confidence intervals of the parameter estimates to assess the performance of the doubly interval-censored maximum likelihood fitting procedure. We opt to vary one parameter and keep the other two fixed for this simulation due to the computational demand of varying two parameters and producing heatmaps as done in the first simulation (Fig 5). The two fixed parameters are fixed at the estimates obtained from fitting to the Legionnaires’ disease dataset. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Bias and appropriate coverage of 95% confidence intervals obtained from fitting the derived Burr distribution using DI methods to doubly-censored incubation periods generated from the derived Burr distribution. The sub-figures (a), (b) and (c) represent the bias from the αD, βD and TD-varying simulations respectively. Further, the sub-figures (d), (e) and (f) represent the appropriate coverage from the αD, βD and TD-varying simulations respectively. https://doi.org/10.1371/journal.pcbi.1012041.g005 For the αD-varying simulation, the bias for αD and βD estimates increase as the true αD increases. Increasing the true αD results in a more spiked true incubation-period distribution. Further, taking the convolution of the spiked incubation-period distribution with a uniform infection window distribution and taking the ceiling results in a less-spiked doubly-censored distribution. The DI maximum likelihood fitting method fails to fully capture the original spike and estimates a flatter distribution, which results in positive bias in the estimates of αD and βD. Further, for the βD-varying simulation, the bias for βD increases, whereas the bias for αD decreases as the true βD increases. Increasing the true βD results in a less spiked true incubation period and in which case the DI method is able to extract estimates of αD close to the true value. However, in this case, the relative bias of βD remains constant as βD varies. Finally, for the TD-varying simulation, the bias in αD and βD remains approximately constant, with some variability due to the random sampling when generating datasets. For each parameter-varying simulation, we obtain almost unbiased estimates for TD, which indicates that regardless of which parameter is varied in the data-generating process, the DI method is successful at locating the median of the distribution. For the appropriate coverage, we repeat 100 iterations of the simulation to record a proportion of times in which the true parameter is contained within the 95% confidence interval. For the βD and TD-varying simulations, the appropriate coverage for each parameter centers roughly around 95%. For the αD-varying simulation, the appropriate coverage for TD centers around 95%. However, as the true αD increases, the coverage for αD increases and the coverage for βD decreases. Discussion This paper has brought attention to and provides solutions to two distinct issues involved in modelling incubation periods of diseases. First, we derived a new model for delays between key events in an individual’s infection history, specifically the incubation period, that has justifiable mechanistic reasons for its validity. Second, we adapted methods for using incubation-period data, that is given as an integer number of days and has issues with bias, to fit models. We considered the probability of an individual changing from the not-yet-symptomatic population to symptomatic for deriving our mathematical model. This approach led to obtaining a differential equation equivalent to the equation defining the exponential c.d.f. with a time-varying rate parameter. We then extended the model with further assumptions to further develop the differential equation describing the incubation period. We considered the assumption that the probability of symptom onset after infection is proportional to the bacterial load before saturating at some large load, as well as considering that bacterial population is expected to grow exponentially. Further, we derived a specific distribution within the Burr family that satisfies a Markovian property of long incubation periods. Other trial functions for G(t) may offer results at least as good as this new model, and some in-host dynamics which affect the rate of symptom onset in populations could be considered for specific diseases to provide even more optimal forms of the Burr model. Further, by considering models that account for unknown infection and symptom onset times (doubly interval-censored models), we have obtained expected incubation periods for Legionnaires’ disease that are statistically significantly less than previously thought (by a whole day) using standard statistical distributions with incubation-period data. The mathematical derivation of the new model and implementation of this model with doubly interval-censored methods address both these problems, as we arrive at a mechanistic model for incubation periods. Our model has few restrictions on which diseases it can be applied to. Additionally, our research highlights the need to account for the censored nature of the data, since we observe a statistically significant difference in the mean incubation period of Legionnaires’ disease when incorporating the DI methods into the model. Our mathematical derivation leading to the Burr family of distributions provides a valid incubation-period model. This model does not consider factors such as an individual’s age, levels of immune response, susceptibility, doses received or the disease-specific in-host dynamics at play that determine if and when an individual becomes ill with an infection. For example, frailty may mean faster onset of symptoms, as may higher doses. These modelling choices mean that the exact disease-specific in-host dynamics are not considered. To derive a model considering the biological processes at play with a given disease, a different model would have to be derived based on the details of those dynamics. Additionally, our model was derived from the assumption of proportionality between bacterial (or viral) load and probability of symptom onset. Our assumption likely oversimplifies this relationship, and alternate models may be developed to assume different functional forms. However, the exact relationship between bacterial (or viral) load and probability of symptom onset varies between diseases and is not clearly understood [21]. If further research was conducted with consistent observations across diseases for this relationship, one could change the proportionality assumption to derive an alternate incubation-period model with further justification than the Burr distribution that we derived here. Use of the double interval-censored maximum likelihood fitting methods relies on the assumption that the individual was infected during a single known exposure on the date given in the data. In reality, some outbreak datasets may not have a clear date of infection for individuals. In this case, an individual may have been exposed several times over various days or they may have been subject to a continuous exposure over time. In the former scenario, concern must be placed on whether the low-dose exposure boosts the immune response or has other effects on the individual. In the latter scenario, one may conduct a sensitivity analysis in which different dates within the continuous exposure window are trialled to assess which date provides the best statistical results. One may use the DI framework with an exposure interval wider than one day. However, a preferable approach would be to investigate these two other exposure scenarios and develop a method that accounts specifically for these different assumptions. In S1 Appendix in the Supplementary Material, we noticed that all Burr distributions valid over (0, ∞), apart from the type X distribution, exhibited a Markovian property for long incubation periods. Consequently, we compared the results of using this model to the other Burr distributions to judge the validity of the Markovian assumption. The type X distribution provides successful results outperforming the gamma distribution in nearly all of the analysis (we obtain mixed results when fitting to other diseases). However, when compared to all of the other Burr distributions, the type X distribution performed the worst when fitting to the original Legionnaires’ disease dataset, the original Legionnaires’ disease dataset with doubly interval-censored methods and the other diseases with doubly interval-censored methods. Further, the type X distribution visually fits the worst to the Legionnaires’ disease data (Fig 2). These consistent results support our Markovian assumption for long incubation periods. Although non-Markovian Burr distributions provide better-performing models to the widely used gamma model under certain circumstances, the Markovian Burr models provide a further improvement in terms of distributional modelling. Our proposed model can be applied in a number of ways in epidemiology and infectious disease modelling. For example, a common area of research is to study person-to-person transmissible diseases, such as COVID-19. In this case, researchers usually develop compartmental and time-since-infection models where the infectivity of inflicted individuals infecting susceptible individuals in a population is modelled. Typically, an exponential (or Erlang) distribution from the point in time at which they are infected is used for modelling. This use of ‘Gamma’ related distributions remains necessary for ODE based compartmental models and is an appeal for modelling with the gamma distribution. In this work, we have limited to time delay distributions with range of times that are strictly positive, as must be the case with the incubation period. Some epidemiological distributions, such as generation time, are not bound by this constraint and so care would be needed in application. Furthermore, we may consider diseases that do not have a person-to-person transmissible property such as Legionnaires’ disease, which has been the focus of this research. Researchers typically track backwards from symptom onset date to predict source location of the infection for elimination and public safety. A more reliable model such as the model developed here can provide more accurate results when predicting locations or causes of Legionnaires’ disease cases, which will result in reduction of bacterial hot-spots and consequently cases of this disease. This paper provides a flexible model that can reliably fit incubation-period data to a level that is not currently in the literature and is valid for a wide range of diseases. The results of fitting the Burr distribution to the diseases considered in this paper indicate that using the Burr family of distributions as a model for incubation periods performs better than currently accepted models [4] when the mode is clearly defined or when the distribution tapers off. Supporting information S1 Appendix. Moments calculations for derived Burr and scaled type XII distributions. Further Legionnaires’ disease modelling analysis of mean incubation period and cumulative hazards. https://doi.org/10.1371/journal.pcbi.1012041.s001 (PDF) S1 Fig. Figures and parameter estimates of anthrax, campylobacteriosis and salmonellosis datasets with model fits for gamma, burr types III, X, XII and the derived Burr based on the original and doubly interval-censored methods. https://doi.org/10.1371/journal.pcbi.1012041.s002 (PDF) S1 Code. R code for conducting analysis and producing plots in this research. https://github.com/NyallJamieson/Burr-Incubation-Period. https://doi.org/10.1371/journal.pcbi.1012041.s003 (R) S1 Data. Incubation period data for the diseases analysed in this research. https://doi.org/10.1371/journal.pcbi.1012041.s004 (XLSX) Acknowledgments Disclaimer The views expressed are those of the author(s) and not necessarily those of the Department of Health or UKHSA. Disclaimer The views expressed are those of the author(s) and not necessarily those of the Department of Health or UKHSA.

Bu, Yongqi;Liang, Jiaxuan;Li, Zhen;Wang, Jianbo;Wang, Jun;Yu, Guoxian

doi: 10.1371/journal.pcbi.1012710pmid: 39724112

Introduction Cancer, a complex disease stemming from diverse origins, stands as the leading cause of premature death worldwide, significantly impeding further extension of life expectancy [1]. Cancer is generally regarded as a cellular disease [2], where genetic mutations, epigenetic changes, cellular biological backgrounds, individual patient-specific characteristics, and environmental influences may all contribute to its initiation and proliferation [3]. The high heterogeneity and complex molecular mechanisms inherent in cancer underscore the subdivision of each single-tissue cancer type into multiple molecular subtypes [4]. Patients with different subtypes typically manifest distinct clinical phenotypes, therapeutic strategy, and prognoses [5–7]. Therefore, accurate subtype diagnosis holds immense potential for propelling advancements in personalized medicine treatments, reducing mortality rates, and prolonging patient survival. However, clinical diagnosis of these molecular subtypes is often costly, time-consuming, and reliant on specialized expertise. Given that, the imperative arises to develop accurate and trustworthy computational solutions for cancer subtype diagnosis. Early researches [8–10] primarily depended on single-omics data to identify cancer subtypes and proved the feasibility of computationally diagnosing subtypes. With the rapid development of high-throughput sequencing, a wealth of multi-omics data has emerged, providing a comprehensive insight into organisms and revealing intricate mechanistic underpinnings of biological systems from diverse perspectives [11]. Consequently, more recent developments [12–14] have shifted their focus towards integrating multiple omics data. However, these methods canonically assume the availability of abundant well-annotated cancer samples characterized by completely-paired multi-omics data. Unfortunately, collecting such samples is challenging, predominantly due to limitations in inspection equipment, high testing costs, and considerations of legal and ethical aspects. In practice, only limited samples with incomplete multi-omics data are available, which significantly limits the applicability and effectiveness of the aforementioned methods. One naive solution for incomplete multi-omics data is to remove corresponding samples directly [14–16], which often results in the loss of valuable samples [17]. To mitigate such information loss, various strategies [18–21] have been explored to recover the missing omics. These methods mostly involve meticulously designed imputation processes, and some even apply distinct treatments for different scenarios, thus imposing specific requirements on the quantity of training data. However, the acquisition of well-annotated samples remains a formidable challenge in biomedical domains [22]. In such cases, it is challenging to train an accurate and reliable diagnostic model using limited samples from the in-house dataset. To address the challenge of limited training data, few-shot (or few-sample) learning has become a prevalent paradigm in the biomedical field. A typical solution is to leverage external datasets with abundant and relevant samples to support the optimization of the model applied to the target dataset [23–25]. However, these existing methods may overlook disparities among different datasets (i.e., sample distributions), which give rise to negative transfer [26]. In such cases, abundant samples from external datasets fail to further improve the model for downstream tasks on the target dataset but even make the optimization process more challenging. Here, we proposed CancerSD (Fig 1) to integrate incomplete multi-omics data from limited clinical samples for accurate Cancer Subtype Diagnosis. CancerSD designs Contrastive Learning tasks and Masking-and-Reconstruction tasks to effectively impute missing omics using available ones. Subsequently, it fuses both available and imputed data to make accurate subtype diagnoses. To alleviate the negative impact that arose from limited training samples in in-house dataset, CancerSD introduces category-level contrastive loss and extends the meta-learning framework, facilitating further optimization of the diagnostic model. Experimental results on multiple cancer datasets of typical complex cancers demonstrate the effectiveness of CancerSD. It delivers superior performance for cancer subtype diagnosis (e.g., gastric, lung, and breast cancer), offering higher authenticity and better interpretability. Furthermore, extensive analyses of the gastric cancer dataset (TCGA-STAD) indicate that CancerSD can identify discriminative molecules in different subtypes, which have associations with the stemness features of gastric cancer cells. Its scoring for subtypes serves as a valuable prognostic predictor. These results confirm the potential application of CancerSD in assisting clinical decision-making. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. The overview of CancerSD. (a) The incomplete data imputation module uses contrastive learning to extract cross-omics consistent features from available patient data. Then, it feeds these features into the generator, facilitating the imputation of missing omics in samples. (b) The cancer subtype diagnosis module leverages available and imputed omics of samples to diagnose cancer subtypes. (c) The knowledge transfer module follows the meta-learning paradigm, which develops a meta learner and a category-level contrastive loss to mine domain-specific knowledge from external datasets and to initialize a backbone network composed with the representation and diagnosis modules. (d) A series of comparison experiments and downstream analyses are conducted to evaluate the performance and application value of CancerSD. https://doi.org/10.1371/journal.pcbi.1012710.g001 Results In this study, we testified our CancerSD in various scenarios. To validate its effectiveness in cancer subtype diagnosis, we first compared CancerSD with representative and competitive algorithms on multiple cancer datasets with incomplete data, including gastric cancer (GC), lung cancer, and breast cancer (see Table A in S1 Text). More specifically, we evaluated its performance in both standard supervised learning settings and few-sample scenarios. We then investigated its adaptability to different types of omics data and its capability to impute missing omics data, thereby providing insights into the performance of CancerSD in cancer subtype diagnosis based on incomplete multi-omics data. Finally, we delved into the authenticity and application value of CancerSD for clinical diagnostics. Among the cancer datasets involved in the above experiments, lung cancer and breast cancer have been well studied in terms of multi-omics integration [18, 27, 28], with a wealth of analyses of mechanisms underlying these two cancers. In contrast, there has been relatively little similar exploration of GC datasets, despite GC remaining a significant challenge in global health care [29]. Moreover, GC exhibits distinct molecular and histopathological characteristics, such as significant changes in the tumor microenvironment and a high incidence of mixed histological subtypes [30–32]. These characteristics complicate the diagnosis and treatment for GC, making it an ideal case for evaluating the adaptability and performance of computational models. Therefore, we focus more on evaluating the performance of CancerSD on GC datasets and attempt to explore the pathology underlying GC. We primarily report and analyse the results on GC datasets, while the relevant results for lung and breast cancer are presented in S1 and S2 Text. Methodology overview of CancerSD Our CancerSD has the advantage to accurately diagnose cancer subtypes by leveraging incomplete multi-omics data of patients and to reduce its dependence on the quantity of in-house cancer samples for training by absorbing knowledge from external datasets, as depicted in Fig 1. The CancerSD pipeline comprises four components dedicated to accomplishing reliable and flexible cancer subtype diagnosis in scenarios characterized by incomplete data and scarce samples. (i) It firstly establishes the patient feature encoder, a tensor-based fusion network, to efficiently integrate multi-omics data from samples. (ii) Then, it constructs the missing omics imputation network to reliably impute missing omics of samples, which consists of an encoder, a projector, and multiple omics-specific generators. After that, it defines Contrastive Learning tasks alongside Masking-And-Reconstruction (MAR) tasks to optimize this imputation network. The former explores the consistent patient representations across different augmented views, while the latter utilizes such representations to impute the missing omics data. (iii) Next, it introduces the cancer subtype diagnosis network that fuses available and imputed omics data to calculate the probability of each patient suffered from a particular subtype. (iv) To enable model optimization on the scarce in-house clinical samples, CancerSD further proposes a knowledge transfer network to extract meta-knowledge from external datasets. We wanted to remark that the first three networks are collectively referred as CancerSD backbone or the base learner (CancerSDb), while the last network is designated as the meta learner (CancerSDm). The detailed description of the CancerSD framework is presented in S1 Fig. For clarity, we illustrated the operational workflow of CancerSD using the GC dataset as a paradigmatic example. It is important to emphasize that our CancerSD framework can be readily extended to subtype diagnosis for various cancers. The framework takes incomplete multi-omics from GC patients as input and finally outputs the probability of them being diagnosed with a certain subtype. Here, incomplete multi-omics implies that certain omics data for some patients are absent due to loss or lack of measurement. M denotes the number of omics types, and is the data matrix for the m-th omics with N patients (or samples) and dm-dimensional features. Specifically, CancerSD starts by constructing a shared encoder, which encodes the multi-omics features of each patient into an integrated representation. The encoder captures distinctive features of different omics through the omics-specific encoding networks and explores inter-omics relationships using tensor outer products. Afterward, CancerSD extracts cross-omics underlying information from available omics data of GC patients to reliably impute their missing omics. To this end, CancerSD first generates diverse augmented views by masking certain omics in samples with completely-paired multi-omics data. Then, it employs contrastive learning to guide the patient feature encoder, thereby discerning consistency across different views of the same GC patient. This acquired information is subsequently fed into generators to reconstruct the masked omics and recover the missing omics. Finally, CancerSD inputs the imputed multi-omics data along with available ones into its diagnostic network to discriminate the GC subtypes of patients. To address the practical challenge of collecting a sufficient number of well-annotated GC samples, CancerSD extends the meta-learning framework to facilitate the circulation of knowledge from external public datasets to the in-house GC samples. It introduces a category-level contrastive loss to minimize differences between samples of the same subtype across different datasets at the distribution level, which aims to selectively learn knowledge from external datasets. Finally, CancerSD utilizes the assimilated knowledge to initialize the backbone, enabling rapid adaptation to the target dataset with limited samples and consequently improving the diagnosis performance. CancerSD outperforms existing comparison cancer subtype diagnosis methods in the standard supervised learning setting To assess the effectiveness of CancerSD, we first evaluated and compared it against representative cancer subtype diagnosis methods. Considering that the existing algorithms are based on conventional supervised learning models, we adjusted the training pipeline of CancerSD to adapt to this scenario for a fair comparison. Specifically, we designated the training set as an external dataset for training and fine-tuning, while the testing set serves as the target dataset for evaluating diagnostic performance. Since the experiments conducted here do not involve cross-dataset knowledge transfer, they validated the effectiveness of CancerSD backbone. Overall, the comparison methods can be broadly categorized into three groups: Traditional Machine Learning (TML), Multi-omics Integration (MI)-based methods, and incomplete Multi-omics Integration (iMI)-based methods. A detailed description of these methods is provided in Section A in S2 Text. The experimental results presented in Fig 2a and Tables B-C in S1 Text demonstrate that CancerSD consistently makes top performance in subtype diagnosis. Then, we utilized t-SNE [33] to reduce dimensionality and visualized the embedding spaces of better-performing methods (MOGONET, DCP, and CancerSD). As depicted in Fig 2b, the challenge posed by incomplete data manifests in the distortion of sample distributions, leading to a noticeable division into two distinct clusters. This distortion greatly heightens the difficulty of subtype diagnosis. Despite these challenges, CancerSD still exhibits the best clustering result, with samples of the same subtype more concentrated in the same cluster. These observations underscore the capability of CancerSD in representation learning and its superiority in cancer subtype diagnosis using incomplete multi-omics data. In Section B in S2 Text, we conducted more analyses of the above results, thus gaining a more comprehensive and detailed perspective on the performance differences among different methods. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Diagnostic performance on the GC dataset in the standard supervised learning setting. (a) The cancer subtype diagnosis performance of CancerSD vs. comparison methods, including kNN, RFC, AE-XGBoost, MOMA, MOGONET, DCP, and APADC on the STAD dataset.(b) Sample clustering using original data and embedded representations given by CancerSD and other methods.(c) The diagnosis performance of the tested methods under different degrees of omics missingness.(d) The diagnostic accuracy of CancerSD for different GC subtypes.(e) The diagnostic probability of CancerSD for different GC subtypes. https://doi.org/10.1371/journal.pcbi.1012710.g002 Next, we evaluated the diagnostic performance of these methods under diverse degrees of multi-omics missingness. We intentionally simulated omics missing scenarios by randomly masking specific omics in samples with completely-paired multi-omics data. Specifically, the simulated missing rates are set at 0%, 25%, 50%, and 75% in sequence, indicating the specified proportion of samples selected for missing certain omics. Fig 2c and Table D in S1 Text demonstrate that CancerSD consistently makes the top performance across all the missing rates. In line with the previous analyses, the performance of the tested methods at different rates manifests a step-like distribution. iMI-based methods generally outshine their MI-based counterparts, while traditional machine learning methods display relatively poor performance. As the simulated missing rate increases, there is a gradual performance decline in all methods, with the most significant decrease witnessed in methods that involve sample similarity calculations, such as kNN and MOGONET. Moreover, it is noteworthy that when multi-omics data for samples is either complete or involves minor incomplete issues, iMI-based methods also marginally outperform MI-based ones, primarily due to the auxiliary tasks constructed for recovering the missing omics. As the degree of omics missing intensifies, a substantial performance gap becomes evident between them. Besides the simulated scenarios of multi-omics data with random missing, we also recognize that the missingness of omics data may not be entirely random in the clinical practice. Therefore, we have conducted experiments focusing on the specific omics absence, displayed the results in Tables E-G in S1 Text, and provided further analysis in Section C in S2 Text. Finally, we investigated the identification preferences of CancerSD for different GC subtypes, including Epstein-Barr virus (EBV), microsatellite instability (MSI), genomically stable (GS), and chromosomal instability (CIN) categorized by the Cancer Genome Atlas (TCGA) Research Network [34]. As shown in Fig 2d, CancerSD can easily and accurately diagnose the EBV and CIN subtypes while maintaining high accuracy for the MSI subtype. However, CancerSD struggles with the identification of the GS subtype. In Fig 2e, it is evident that the accuracy of CancerSD in diagnosing the GS subtype is significantly lower than others. Based on Fig 2b, we noted that the clusters containing samples of GS and CIN subtypes are consistently close to each other, even overlapping. We further analyzed samples where diagnostic errors occurred in the experiments and find that compared to other subtype pairs, these two subtypes are more likely to be misdiagnosed as each other (see S2 Fig). In fact, Lee et al. [35] observed cases of subtype transition between these two subtypes after metastasis (transitions from CIN to GS and vice versa), while the subtype of metastatic tumors is generally the same as the primary tumor. These findings suggest a potential similarity between the GS and CIN subtypes, which gives rise to the confusion of CancerSD toward these two subtypes. As mentioned above, the experiments conducted under standard supervised learning settings actually highlight the capabilities of CancerSDb. Furthermore, we studied more details for CancerSDb, including evaluation its robustness (see S3 Fig and Section D in S2 Text), exploring its optimal architecture (see Table H in S1 Text and Section E in S2 Text), investigating the impact of different data augmentation operations (see S4 Fig and Section F in S2 Text), and examining its sensitivity to changes in hyper-parameters (see S5 Fig and Section G in S2 Text). These experiments and analyses provide a more in-depth and comprehensive perspective on why CancerSD can make superior subtype diagnosis performance. CancerSD demonstrates superior diagnostic performance in the few-sample scenario Cancer subtype diagnosis is a classical few-sample scenario, where in-house datasets often contain only a limited number of samples, posing challenges in optimizing an accurate diagnostic model. Moreover, variations in sample collection sources and biases in sample selection contribute to significant differences among different datasets (i.e., sample distribution disparities, as illustrated in S6 Fig). Disregarding this situation and directly transferring knowledge from external datasets to in-house ones can lead to negative transfer, which potentially undermines the performance of the diagnostic model. To assess the effectiveness of CancerSD in addressing these issues, we constructed knowledge transfer tasks across different datasets. The detailed experimental setups and description of comparison approaches are presented in Section Material and methods. First, we evaluated the performance of certain cancer subtype diagnosis methods on the GSE62254 dataset under the conventional supervised learning settings, which serves as the baseline performance. In this scenario, all methods only use the data from GSE62254, with no knowledge learned from external datasets. As shown in Fig 3a, it is observed that various methods exhibit similar diagnostic performance. Particularly, there is minimal difference among the performance of iMI-based methods, including CancerSD, DCP, and APADC. This phenomenon mainly arises from the fact that GSE62254 contains mRNA expression profiles as the sole omics data, leading all methods to degrade into simple classifiers. Nonetheless, owing to auxiliary tasks such as data reconstruction, iMI-based methods still perform slightly better than others. Additionally, we observed a significant performance decline when training CancerSD with only a small amount of samples (4-way 10-shot, ten samples for each subtype). In fact, by referring to Fig 3a and Table I in S1 Text, we noted that all methods perform poorly under this scenario, which could be attributed to two main reasons. On the one hand, since only mRNA data is accessible in GSE62254 dataset, MI-based and iMI-based methods essentially degrade into single-model classifier, losing their advantage in modeling multi-omics interactions. On the other hand, all compared methods can only use a limited amount of data from GSE62254, preventing sufficient optimization and causing underfitting. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Diagnostic performance on the GC datasets in the few-sample scenario. (a) The cancer subtype diagnosis performance of CancerSD vs. comparison methods on the GSE62254 dataset under the standard supervised learning setting.(b) The cancer subtype diagnosis performance (Accuracy and F1 Score) of different methods under the multi2mRNA (upper figure) and mRNA2mRNA (lower figure) settings. Here, the red dot dash line represents the performance obtained by optimizing CancerSD with the entire training set, while the gray dotted line indicates that obtained by optimizing CancerSD with a 4-way 10-shot set.(c) The similarity of representations for samples from different datasets. https://doi.org/10.1371/journal.pcbi.1012710.g003 Next, we attempted to transfer knowledge from the STAD dataset, supporting the optimization of diagnostic models on GSE62254. Specifically, we initialized and trained diagnostic models on STAD and subsequently select a small number of samples (4-way 10-shot) from the training set of GSE62254 to fine-tune these models. To include traditional machine learning methods into the experiment, we combined STAD and the data used for fine-tune from GSE62254 into a unified training set, which was then provided to these methods. Here, we explored knowledge transfer under two strategies: from multi-omics data of STAD to the mRNA expression profile of GSE62254 (multi2mRNA) and from the mRNA expression profile of STAD to that of GSE62254 (mRNA2mRNA). As illustrated in Fig 3b, and Tables J-K in S1 Text, We observed that even if TML approaches are granted access to external datasets, they struggle to make reasonable use of these data, due to the lack of effective knowledge extraction strategies. This demonstrates the difficulty of TML methods in adapting to the few-sample scenario, compared to few-shot learning-based approaches. We also found that meta-learning-based methods generally exhibit superior transfer performance compared to pretraining-based ones. This is likely due to the emphasis of meta-learning on the cross-dataset generalization ability of the models, while pretraining strategies focus on models’ performance on the current dataset. Notably, with only a 4-way 10-shot set sampled from the training set, our CancerSD achieves performance comparable to using the entire training set. This observation emphasizes the capability of CancerSDm in cross-dataset knowledge transfer. Moreover, the performance of tested methods under the mRNA2mRNA strategy is superior (or comparable) to that under the multi2mRNA. This could be attributed to the fact that embedding spaces generated from mRNA data in different datasets are more similar in distribution than those generated separately from multi-omics and mRNA data. Consequently, under the mRNA2mRNA strategy, the models can more easily absorb knowledge from external datasets. Taking results under mRNA2mRNA as example, we further analyzed the effectiveness of CancerSDm and the importance of category-level contrastive loss. In the knowledge transfer task, CancerSD achieves an Accuracy of 67.5%, AUROC of 83.7%, Precision of 73.3%, and F1 Score of 67.6%, outperforming its comparison methods and variations across almost all metrics. The superior performance can be attributed to its powerful backbone and domain-specific knowledge transfer capabilities. Specifically, CancerSD outperforms its variants CancerSD-MOMA and CancerSD-DCP, whose backbone networks are replaced with MOMA and DCP, respectively. This superiority underscores the capability of CancerSDb to effectively impute missing omics data and integrate multiple omics for accurate and reliable cancer subtype diagnosis. On the other hand, the significant performance decline of CancerSD-w/oCLC highlights the importance of the distribution-based category-level contrastive loss. By leveraging this loss, CancerSD can alleviate the sample distribution discrepancy across different datasets and focus on extracting knowledge relevant to the assigned downstream diagnosis tasks from external datasets. Fig 3c supports this perspective by showing that representations obtained by CancerSD for samples of the same subtype from different datasets exhibit higher similarity. This observation indicates that CancerSD effectively captures the consistency of cancer subtypes across different datasets and integrates the consistency into sample representations, thereby improving the similarity among samples of the same subtype. The comparison results also demonstrate that the similarity-based category-level contrastive loss fails to acquire knowledge from datasets with more samples effectively and may even hamper the generalization ability of CancerSDb. This is because CancerSD-SIM utilizes the instance-level similarity to cluster samples of the same subtype, which potentially leads to severe overfitting problems and is susceptible to noise and outliers. In contrast, CancerSD attempts to cluster samples of the same subtype at the distribution level, thereby alleviating such issues. The above analyses provide insights into why CancerSDm can effectively extract knowledge from other datasets. In summary, CancerSDm adopts a meta-learning strategy to mine and transfer meta-knowledge from external datasets and utilizes the category-level contrastive loss to maximize the agreement of distributions between samples with the same subtype across different datasets, thereby improving the diagnostic performance of the model on target dataset. Diagnostic performance of CancerSD under different omics data types Although CancerSD fuses three types of omics data (DNA methylation profiles, miRNA expression profiles, and mRNA expression profiles) for cancer subtype diagnosis in the above experiments, it can readily adapt to different numbers of omics data types. To verify the importance of multi-omics integration in improving the diagnosis performance and to assess the capability of CancerSD in multi-omics integration, we evaluate CancerSD using various combinations of omics data. Here, we only consider samples with completely-paired multi-omics data. Fig 4a shows that the diagnostic performance of CancerSD is improved continuously by integrating more omics data. In concrete terms, CancerSD trained with all three omics outperforms the model using a combination of two types of omics. The performance of CancerSD trained with two omics is also superior to that of only single omics. These results highlight the advantages of integrating multiple omics data for more accurate subtype diagnosis. Moreover, it is worth noting that CancerSD trained with mRNA expression data performs best when employing only single omics for training. This finding suggests that mRNA features contain information conducive to distinguishing GC subtypes, potentially harboring valuable biomarkers. In contrast, the performance of CancerSD trained using miRNA data is the poorest. This may be attributed to the lower dimensionality of its original data compared to the other two types of omics (702 vs. 3278 and 4089), which provide less discriminative information for subtyping. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Diagnostic performance under different omics data types. (a) Performance comparison for subtype diagnosis using different types of omics. Among them, methylation, miRNA, and mRNA refer to make diagnosis via CancerSD using DNA methylation data, miRNA expression data, and mRNA expression data, respectively; meth+miRNA, miRNA+mRNA, and meth+mRNA refer to diagnosis with two types of omics; meth+miRNA+mRNA refers to diagnosis with three types of omics.(b) Sample similarity heatmaps obtained from representation at different levels.(c) Diagnostic performance of CancerSD using only a single type of omics under different training strategies.(d) Sample clustering based on omics and fusion representations output by CancerSD under multi-omics joint training strategy.(e) Sample clustering based on omics and fusion representations output by CancerSD under single-omics independent training strategy. https://doi.org/10.1371/journal.pcbi.1012710.g004 Expanding upon the results mentioned above, we delve deeper into the effectiveness of CancerSD in integrating multi-omics data. First, we focus on the sample similarity pattern as an illustrative example to elucidate the capability of CancerSD in extracting cross-omics consistency information. As depicted in Fig 4b, after mining subtype-related discriminative information in the corresponding omics data using omics-specific feature extractors, samples of the same subtype exhibit relatively high similarity across different omics. CancerSD adeptly identifies and captures this cross-omics consistent pattern of sample similarity, incorporating these patterns into the multi-omics fusion representation. Meanwhile, CancerSD takes into account discrepant patterns across multiple omics to mitigate the impact of erroneously high similarity (i.e., the globally high similarity from miRNA embedding). Consequently, the fusion representation learned by CancerSD more accurately reflects the correct similarity relationships between samples. Next, we conduct extensive experiments to assess the capability of CancerSD in exploring cooperation between diverse omics data. To this end, we compare the subtype diagnosis performance of CancerSD in two distinct training strategies: (i) multi-omics joint learning, which uses multiple omics data simultaneously to optimize the model; (ii) single-omics independent learning, which uses only a single type of omics to optimize the model each time. As shown in Fig 4c and Table L in S1 Text, it is evident that under the identical condition of diagnosing, CancerSD optimized by multi-omics joint learning more accurately identifies patient subtypes compared to the model optimized by single-omics independent learning strategy. To gain further insights, we investigate the omics embeddings obtained under different training strategies and evaluate the diagnostic potential of multi-omics fusion representations derived from these embeddings. Specifically, we separately construct and optimize multi-omics fusion networks (see Experimental Section) to integrate multiple omics embeddings obtained under joint learning and independent learning strategies. Then, we utilize the resulting fusion representations to perform cancer subtype diagnosis tasks. The results presented in Table L in S1 Text suggest that the fusion representations integrated from omics embeddings learned by joint learning are more conducive to identifying subtypes. Furthermore, we visualize sample clustering for different training strategies. From Fig 4d and 4e, we can find that, in comparison with the output obtained by initially performing single-omics independent learning and then integrating the resulting omics embeddings, the fusion representations obtained through multi-omics joint learning can generate a more compact clustering structure, with more apparent margins between clusters. These observations prove that CancerSD has effectively learned cooperation between different omics during the multi-omics joint optimization process, thereby enhancing the performance of multi-omics fusion. In summary, multi-omics data can offer more prosperous and more comprehensive patient features. CancerSD effectively integrates these data by extracting cross-omics consistency and cooperation information, significantly improving the performance of cancer subtype diagnosis. To further highlight the superiority of our CancerSD, we also evaluated the diagnostic performance of several comparison methods across different omics types (see Table M in S1 Text), with a concise analysis provided in Section H in S2 Text. In addition to integrating multi-omics, CancerSD also possesses a notable capability in handling missing omics data. Even in cases of extensive omics data missingness (see Table N in S1 Text), CancerSD effectively recovers biologically meaningful expression values, affirming its authenticity and effectiveness in imputing missing omics. The details regarding this aspect are presented in S7–S13 Figs and are discussed more extensively in Section I in S2 Text. CancerSD identifies important molecules related to gastric cancer Identifying important biomarkers is crucial for understanding the underlying mechanisms of GC and interpreting the corresponding diagnostic decision made by CancerSD. To this end, we investigated the importance of each molecular characteristic on the diagnostic outcomes to find potential biomarkers. Specifically, we systematically shuffled the values of each molecular characteristic across all samples in the testing set and then evaluate the diagnostic performance of CancerSD using these modified features. After that, we compared the performance with results obtained when using all features, allowing us to discern the contribution of each molecule to diagnosis tasks, where the diagnosis loss (Eq 10) serves as a quantitative indicator for multi-classification tasks. The more loss increases, the more important the currently permutated molecule is. For a more robust result, we conduct ten random experiments and take the average performance degradation as the final result. Within each type of omics, we selected the top-ranked molecular characteristics for further analysis and validation. First, we presented importance scores of the top ten ranked molecules from each omics. As shown in Fig 5a and 5b, it is evident that there are significant importance differences of molecules across various omics. Among them, mRNA features obtain the highest importance scores, while miRNA features have the lowest, indicating that CancerSD relies more on mRNA expression profiles in the diagnostic decision-making process. This perspective is further highlighted in Fig 5c, where clusters from mRNA embeddings are closer to clusters from fusion representations than from other omics embeddings for patients. We speculated that the prominence of mRNA features may be mainly attributed to two reasons. On the one hand, owing to the higher dimensionality of the raw data, mRNA features can provide richer discriminative information for subtype diagnosis. On the other hand, mRNA expression data more directly reflects the activity of genes and cellular functions. Meanwhile, mRNA expression is influenced by multiple regulatory layers, including DNA methylation and miRNA regulation, among others, potentially more comprehensively reflecting the integrated effects of gene expression regulation. The division of cancer subtypes is often associated with changes in genes. Therefore, mRNA features play a more crucial role in the diagnostic process. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Important molecules identified by CancerSD. (a) Importance scores of the top-ranked molecules identified by CancerSD in various omics.(b) Differences in the importance scores of molecules across various omics.(c) Clustering for samples of different subtypes using omics embeddings and fusion representation output by CancerSD, respectively.(d) The methylation levels of the top 100 CpG sites ranked by importance, where the CpG sites are secondary sorting based on the average values across all samples.(e) The expression levels of mRNA characteristics across different GC subtypes.(f) The expression levels of mRNA and miRNA characteristics across different GC subtypes, where the expression values subjected to log2 transformation and normalization. Wilcoxon rank-sum test is employed to evaluate the differences in the expression levels of specific molecules among patients of distinct subtypes.(g) Gene co-expression analysis result for EBV subtype.(h) KEGG Pathway Enrichment results for module-2 (ME-2, top) and module-4 (ME-4, bottom), respectively. https://doi.org/10.1371/journal.pcbi.1012710.g005 Then, we visualized the expression of molecules identified by CancerSD in each omics type to preliminarily showcase the authenticity of CancerSD in making diagnostic decisions. The molecules selected in this step are detailed in Table 1 (genes and miRNAs are listed) and Table O in S1 Text (CpG sites are listed). For the DNA methylation profile, we explored differences in methylation patterns at CpG sites among patients of distinct subtypes, focusing on the top 100 ranked features. As depicted in Fig 5d, differences in methylation patterns emerge across GC subtypes. Specifically, EBV patients exhibit elevated methylation levels at most CpG sites in comparison to other subtypes, with MSI patients ranking second. Patients with the other two subtypes show a relatively similar DNA methylation pattern. These observations are consistent with the previous study [34]. For miRNA and mRNA expression profiles, we investigated the differences in the expression levels of the top 10 important molecules among various subtypes. Based on Fig 5e–5f and S14 Fig, we could observe that most top-ranked characteristics exhibit significantly different expression levels across various subtypes. These observations indicate that CancerSD primarily relies on molecules that exhibit distinction across different subtypes in the diagnostic process. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Important molecules identified by CancerSD. https://doi.org/10.1371/journal.pcbi.1012710.t001 Next, we explored the relationship between top-10 important molecules in each omics (see Table 1) and GC. Among them, characteristics derived from DNA methylation and mRNA expression profiles are inferred as their corresponding genes. Notably, nearly half of these molecules have been extensively studied. For instance, Yasumoto et al. [36] discovered that the CXCR4/CXCL12 axis plays a role in the development of peritoneal carcinomatosis, which is an incurable complication during the advanced stages of GC. Subsequent research by Hashimoto et al. [37] revealed that blocking the CXCR4/mTOR signaling pathway may contribute to the treatment of this complication. Moreover, Xiang et al. [38] demonstrated that CXCR4 can cross-activate with CXCR2, promoting the epithelial-mesenchymal transition, metastasis, and invasion in GC. Simultaneous inhibition of these two genes has been shown to reduce the metastasis of GC effectively. These studies highlight the close association between CXCR4 and GC. Besides important genes, the miRNAs identified by CancerSD have also been confirmed to have profound associations with GC. Taking hsa-mir-488 as an example, several studies [39, 40] demonstrated that its overexpression can delay the malignant progression of GC, suggesting that hsa-mir-488 holds promise as a valuable biomarker for the diagnosis and treatment for GC. Alongside the molecules mentioned above, existing studies (see Table P in S1 Text) have reported that CD9, KLK6, HLA-B, MUC1, MET, hsa-mir-7–2, hsa-mir-944, hsa-mir-942, hsa-mir-219a-2, and hsa-mir-1305 also play an important role in the occurrence, progression, or diagnosis and treatment of GC. While the relationships between certain molecules in Table 1 and GC remain unclear, some are closely associated with other digestive tract cancers. For example, Fujita et al. [41] found that the overexpression of ENC1 may suppress the differentiation of colon cells, potentially leading to the development of colorectal cancer. This process could be achieved through JAK2/STAT5/AKT axis-mediated epithelial-mesenchymal transition and stemness. [42] In addition, Than et al. [43] indicated that CFTR is a tumor suppressor gene in intestinal cancer. Similarity mechanisms might exist in GC, and further exploration of these genes could contribute to a more profound understanding of GC, as well as elucidating connections and distinctions among various digestive tract cancers. Finally, we conducted a detailed analysis of important genes identified by CancerSD for each GC subtype, including genes derived from DNA methylation and mRNA expression profiles. During the calculation of diagnosis loss, we individually assessed the impact of feature shuffling on the diagnosis for each subtype. These processes involve partitioning the testing set based on subtypes and separately calculating diagnosis loss for each subtype. By quantifying the increase of the loss, we assign importance to specific characteristics. For the top-10% ranked important genes (619 / 6186) identified for each subtype, we utilized the WGCNA R package [44] and OmicVerse [45] to conduct gene co-expression analysis and select certain co-expression modules for subsequent analysis. Taking the EBV subtype as an example, the selected genes are clustered into six co-expression modules (as illustrated in Fig 5g). Among these modules, genes in module-2, 3, 4, 5, and 6 exhibit significant co-expression relationships, with 75, 54, 50, 33, and 32 genes within each module. We further conducted KEGG enrichment analysis on these five modules, and the results are presented in Fig 5h and Table Q in S1 Text. In module-2 (ME-2), pathways such as ECM-receptor interaction (ko04512) and cell adhesion molecules (ko04514) are prevalent in cancer and play crucial roles in GC [46, 47]. Some pathways enriched in ME-2 are also highly associated with EBV. Liang et al. [48] reported that focal adhesion (ko04510) signal pathways are often dysregulated due to EBV-associated genomic and epigenomic alterations, which may play a crucial role in the development of EBV-associated GC. Other three myocarditis-related pathways (ko05412, ko05410, and ko05414) are associated with a rare but severe complication of EBV infection [49]. In module-4 (ME-4), numerous immune-related pathways are significantly enriched, such as PD-L1 expression and PD-1 checkpoint pathway in cancer (ko05235), TNF signaling pathway (ko04668), antigen processing and presentation (ko04612). Among them, NF-κB signaling pathway (ko04064) exhibits higher positivity in EBV-positive GC than EBV-negative one [50]. It promotes the proliferation of GC cells infected with EBV, which could be attributed to the regulation of the EBV-encoded BARF1 [51]. Similarly, the overexpression of PD-L1 has been reported as a typical characteristic of the EBV subtype [34], and PD-1 inhibition is an effective treatment for patients of this subtype [52]. These two mechanisms are closely related to ko05235 pathway. Cytokine-cytokine receptor interaction (ko04060) is also one of the core pathways dysregulated in EBV-associated GC [48]. Collectively, the aforementioned pathways may play important roles in the development of EBV-associated GC. Therefore, we hypothesized that key genes within these pathways could serve as potential biomarkers or therapeutic targets for the EBV subtype. For example, CXCL10 and CXCL11, both small-molecule cytokines in the CXC chemokine family, are significantly overexpressed in the EBV subtype compared to other subtypes (see S15 Fig). These two genes regulate the migration, differentiation, and activation of immune cells through the CXCL9/10/11/CXCR3 axis, which is also directly involved in the proliferation and metastasis of cancer cells. [53] Given their roles in guiding immune cells such as T cells and leukocytes to move towards inflammatory or infected sites, [54]CXCL10/11 may contribute to better immunotherapeutic effects in EBV-positive GC patients. In more detail, CXCL10/11 are regulated by EBV-related miRNAs, with the former being regulated by ebv-miR-BART1–3p [55] and the latter being regulated by ebv-mir-BHRF1–3. [56] It is possible that EBV promotes the occurrence and development of GC through these pathways, implying the potential of CXCL10/11 as diagnostic factors for the EBV subtype. Moreover, for other GC subtypes (CIN, GS, MSI), the co-expression and KEGG pathway enrichment results are presented in S16 Fig. The above results and analysis verify the authenticity and interpretability of CancerSD in cancer subtype diagnosis, which also prove the potential of CancerSD in assisting clinical diagnosis. Outcomes of CancerSD are associated with stemness features of gastric cancer subtypes and patient prognosis In the previous analyses, CancerSD demonstrates the capability to accurately diagnose cancer subtypes using incomplete multi-omics data. Experiments conducted on the GC dataset also indicate its ability to identify key molecular signatures associated with GC. These results provide preliminary evidence of its reliability in assisting clinical diagnosis. To further investigate the role of CancerSD in diagnostic decision-making, we explored its relationship with gastric cancer subtypes and patient prognosis. Stem cells are characterized by their capacity for self-renewal, either infinitely or perpetually, alongside their ability for multi-lineage differentiation, while stemness is defined as the potential of stem cells in these two aspects [57]. Within tumor tissues, a small proportion of relatively stable cells possessing both proliferative and tumor-reconstructing abilities are identified as cancer stem cells or cancer stem-like cells [58]. These cells may cause various tumor malignancies, such as recurrence, metastasis, multidrug resistance, and radioresistance [59]. Thus, determining the stem-cell characteristic of each GC subtype is of significant importance for gaining deeper insights into mechanisms underlying tumor initiation and progression, as well as for the development of effective therapeutic strategies. To this end, we employed the stemness index model [57], known as mRNAsi, to score the stemness features of GC samples and then conduct further analysis. We first collected gene expression profiles of pluripotent stem cells from the Progenitor Cell Biology Consortium dataset [60, 61] (syn2701943). The data are preprocessed with mean-centering. Subsequently, the stemness signature is identified through the one-class logistic regression algorithm. Next, spearman correlation analysis is performed between the normalized expression matrix of GC samples and the stemness signature. The resulting correlation coefficients are scaled to the range [0, 1] to determine the stemness index. Finally, we assessed the relationship between mRNAsi scores and our CancerSD. From Fig 6a, we observed an association between the stemness index and clinical features in GC patients. In particular, there are significant differences in mRNAsi among patients of distinct GC subtypes (see Fig 6b). While there is typically a negative correlation between mRNAsi and the prognosis of cancer patients [62, 63], an opposite trend is noted in GC [64]. This is further highlighted in Fig 6c, where patients with higher mRNAsi tend to exhibit a favorable prognosis. Consistently, mRNAsi is the lowest in samples of the GS subtype, which corresponds to the poorest prognosis among the four subtypes [65]. Interestingly, a correlation analysis of the stemness index with the CancerSD score for each subtype shows that the GS subtype is significantly negatively correlated with the mRNAsi (r = −0.353, p = 4.94 × −103, Fig 6d), where CancerSD scores represent the probability of a patient being diagnosed with a certain cancer subtype. There is currently no consensus on why the GS subtype often corresponds to the lowest mRNAsi. Considering the high overlap between samples of GS subtype and of diffuse-type GC (see Fig 6e), we might gain insights into the mechanisms behind this phenomenon from diffuse-type GC [66], which similarly obtains the lowest mRNAsi scores within its corresponding Lauren [67] classification system (see Fig 6f). In addition, CancerSD scores of samples with other subtypes also show significant correlations with mRNAsi. Given the significant correlation between mRNAsi and patient prognosis, the aforementioned observations suggest that GC subtype scores may be associated with the prognosis of GC patients. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. The relationship among CancerSD outcomes, mRNAsi scores, and patient clinical characteristics. (a) An overview of the association between the mRNAsi and clinical features. The median of mRNAsi score is used to categorize mRNAsi levels.(b) mRNAsi scores across different molecular subtypes.(c) Kaplan-Meier survival curves of different mRNAsi levels. Among them, HR and 95CI are abbreviations of Hazard Ratio and 95% Confidence Interval, respectively.(d) The relationship between CancerSD scores (with normalization) and mRNAsi scores in GC patients. The former is derived from the output of CancerSD before the softmax layer, while the latter is obtained through the mRNAsi model.(e) The integrated sankey diagram portrays the underlying correlations across the mRNAsi, molecular subtypes and Lauren classification.(f) mRNAsi scores across different Lauren subtypes.(g) The relationship between Integrated CancerSD Score (ICS) and mRNAsi scores in GC patients.(h) Kaplan-Meier survival curves of different ICS levels.(i) Correlation of mRNAsi and expression levels of important genes identified by CancerSD. The regression lines in figures are fitted by the corresponding data. The significance in the figure is estimated by pearson correlation coefficient. https://doi.org/10.1371/journal.pcbi.1012710.g006 To delve deeper into this association, we aggregated CancerSD scores for different subtypes in a manner analogous to TCGA Risk Score [65], yielding the Integrated CancerSD Score (ICS) that estimates patient prognosis risk. To ensure a smoother ICS, we replaced the softmax function with the sigmoid function, constraining all CancerSD scores to the (0, 1) interval. Specifically, we formulated ICS as follow: ICS = eICSraw, where ICSraw = CIN score + (1—MSI score) + (2 × GS score) + (1—EBV score). Among that, the coefficients preceding the subtype prediction scores were determined based on their relationship with prognosis [65]. Since EBV and MSI are commonly associated with favorable prognosis, we used the inverse of their CancerSD scores. The weighting coefficient for the GS score was assigned to 2, reflecting its strong correlation with poor prognosis. The CIN score remained unchanged, as this subtype is only moderately associated with poor prognosis. Using 16.2 (the optimal cutoff value confirmed by maxstat R package [68]) as the cutoff point, we categorized ICS into low and high levels. As illustrated in Fig 6g, ICS exhibits a significant negative correlation with mRNAsi (r = −0.601, p = 6.5 × −1037), indicating that a high level of ICS may correspond to a poorer patient prognosis. Fig 6h validates this perspective, demonstrating a significant difference in overall survival among patients with different ICS levels (p = 0.04). While the stratification of patients based on ICS levels does not show a significant difference in disease-free survival (p = 0.16), noticeable distinctions can be observed in Fig 6h. This lack of significance may be attributed to the problem of insufficient data related to disease-free survival, with only 201 out of 415 GC patients possessing corresponding disease-free survival information. To further evaluate the prognostic value of the ICS, we conducted univariate and multivariate Cox proportional hazards regression analyses on ICS and other seven clinical variables. As listed in Table R in S1 Text, in addition to well-known prognostic factors such as T stage, N stage, M stage, and TNM stage, ICS emerged as a significant predictor for overall survival in univariate analysis. Even when incorporating all relevant clinical variables in a multivariate Cox regression analysis, ICS remains an important prognostic factor (HR, 1.7; 95% CI, 1.07–2.70, p = 0.03). Collectively, ICS may serve as a potential predictive factor for overall survival and even disease-free survival, highlighting the utility of CancerSD. We further extended our investigation to the relationship between mRNAsi and the top-ranked genes identified by CancerSD. Specifically, we analyzed the correlation between the expression levels of the top 25 genes (including those identified from DNA methylation and mRNA expression profiles) across all samples and mRNAsi scores. The results are presented in Fig 6i and S17 Fig. We observed that 13 genes exhibit a significant correlation with mRNAsi. Among them, five genes are directly or indirectly involved in various biological processes, exerting impacts on the stemness characteristics of cancer cells. For example, c-Met, the protein product of the MET proto-oncogene, has been demonstrated to promote tumor angiogenesis, growth, and metastasis [69]. Several studies [70, 71] reported that c-Met is implicated in the stemness of cancer stem cells in various cancers. In gastric cancer, Yashiro et al. [72] found that the combination of c-Met inhibitors with SN38 may effectively target cancer stem cells in diffuse-type GC. Bahrami et al. [73] also reported that c-Met/ALK inhibitors could reduce the expression of cancer stem cell markers in gastrointestinal cancers. These evidences suggest that elevated expression of MET can promote the characteristics of GC stem cells, thereby positively influencing mRNAsi scores. Moreover, ECRG4 serves as an inhibitory upstream regulator of the NF-κB pathway [74], while the latter is persistently activated in cancer stem cells across various malignancies, participating in several crucial biological processes of cancer stem cells [75]. The role of NF-κB in GC has been widely reported, where its activation can stimulate the proliferation and stemness of GC cells [76]. Ding et al. [77] found that the PEAK1-PPP1R12B axis can inhibit cell growth and metastasis in colorectal cancer by attenuating the Grb2/PI3K/Akt signaling pathway, and a similar mechanism might exist in GC. Considering the activation effect of PI3K/Akt pathway on the NF-κB system [78], the high expression of genePPP1R12B might indirectly play a role in suppressing the stemness of GC cells. Consequently, we observe a significant negative correlation between the expressions of ECRG4 and PPP1R12B and mRNAsi scores. Besides MET mentioned above, ECRG4 and PPP1R12B, another two genes (see Table S in S1 Text), whose expression levels are significantly correlated with mRNAsi scores, are also associated with the stemness features of cancer cells. Although not all of these genes have been confirmed to play a role in GC, there might be similar mechanisms promoting or inhibiting the stemness of GC cells. In summary, CancerSD scores for samples show a significant correlation with the stemness features of different GC subtypes and patient prognosis. Moreover, the majority of top-ranked important genes identified by CancerSD are closely associated with cancer cell stemness features. These findings once again validate the authenticity and reliability of CancerSD in GC subtype diagnosis, suggesting its potential to assist real-world clinical decision-making. CancerSD maintains good performance on multiple cancer datasets In the above analyses, we primarily discussed experimental results related to GC. To investigate the generalization capability of CancerSD, we also conducted a series of experiments on lung cancer and breast cancer datasets. The relevant results are presented in Tables B, L, T in S1 Text and S8–S9 Figs. Overall, we could draw similar observations from these results as those observed in GC datasets. First, CancerSD exhibits superior (or comparable) performance in subtype diagnosis for lung cancer and breast cancer than the competitive methods. Second, experiments involving multi-omics integration and missing omics imputation on these two cancer datasets further highlight the effectiveness of our method in these regards. Furthermore, we observed that the importance of different omics varies across different cancers during the diagnostic process. Lastly, experiments involving knowledge transfer between two lung cancer datasets once again demonstrate the capabilities of CancerSD in addressing sample scarcity and mitigating negative transfer. In addition to these observations, a more detailed exposition of the relevant experiments and corresponding results is provided in Section J in S2 Text. In summary, CancerSD emerges as an effective and authentic model for cancer subtype diagnosis, which can be readily deployed to different cancers. Methodology overview of CancerSD Our CancerSD has the advantage to accurately diagnose cancer subtypes by leveraging incomplete multi-omics data of patients and to reduce its dependence on the quantity of in-house cancer samples for training by absorbing knowledge from external datasets, as depicted in Fig 1. The CancerSD pipeline comprises four components dedicated to accomplishing reliable and flexible cancer subtype diagnosis in scenarios characterized by incomplete data and scarce samples. (i) It firstly establishes the patient feature encoder, a tensor-based fusion network, to efficiently integrate multi-omics data from samples. (ii) Then, it constructs the missing omics imputation network to reliably impute missing omics of samples, which consists of an encoder, a projector, and multiple omics-specific generators. After that, it defines Contrastive Learning tasks alongside Masking-And-Reconstruction (MAR) tasks to optimize this imputation network. The former explores the consistent patient representations across different augmented views, while the latter utilizes such representations to impute the missing omics data. (iii) Next, it introduces the cancer subtype diagnosis network that fuses available and imputed omics data to calculate the probability of each patient suffered from a particular subtype. (iv) To enable model optimization on the scarce in-house clinical samples, CancerSD further proposes a knowledge transfer network to extract meta-knowledge from external datasets. We wanted to remark that the first three networks are collectively referred as CancerSD backbone or the base learner (CancerSDb), while the last network is designated as the meta learner (CancerSDm). The detailed description of the CancerSD framework is presented in S1 Fig. For clarity, we illustrated the operational workflow of CancerSD using the GC dataset as a paradigmatic example. It is important to emphasize that our CancerSD framework can be readily extended to subtype diagnosis for various cancers. The framework takes incomplete multi-omics from GC patients as input and finally outputs the probability of them being diagnosed with a certain subtype. Here, incomplete multi-omics implies that certain omics data for some patients are absent due to loss or lack of measurement. M denotes the number of omics types, and is the data matrix for the m-th omics with N patients (or samples) and dm-dimensional features. Specifically, CancerSD starts by constructing a shared encoder, which encodes the multi-omics features of each patient into an integrated representation. The encoder captures distinctive features of different omics through the omics-specific encoding networks and explores inter-omics relationships using tensor outer products. Afterward, CancerSD extracts cross-omics underlying information from available omics data of GC patients to reliably impute their missing omics. To this end, CancerSD first generates diverse augmented views by masking certain omics in samples with completely-paired multi-omics data. Then, it employs contrastive learning to guide the patient feature encoder, thereby discerning consistency across different views of the same GC patient. This acquired information is subsequently fed into generators to reconstruct the masked omics and recover the missing omics. Finally, CancerSD inputs the imputed multi-omics data along with available ones into its diagnostic network to discriminate the GC subtypes of patients. To address the practical challenge of collecting a sufficient number of well-annotated GC samples, CancerSD extends the meta-learning framework to facilitate the circulation of knowledge from external public datasets to the in-house GC samples. It introduces a category-level contrastive loss to minimize differences between samples of the same subtype across different datasets at the distribution level, which aims to selectively learn knowledge from external datasets. Finally, CancerSD utilizes the assimilated knowledge to initialize the backbone, enabling rapid adaptation to the target dataset with limited samples and consequently improving the diagnosis performance. CancerSD outperforms existing comparison cancer subtype diagnosis methods in the standard supervised learning setting To assess the effectiveness of CancerSD, we first evaluated and compared it against representative cancer subtype diagnosis methods. Considering that the existing algorithms are based on conventional supervised learning models, we adjusted the training pipeline of CancerSD to adapt to this scenario for a fair comparison. Specifically, we designated the training set as an external dataset for training and fine-tuning, while the testing set serves as the target dataset for evaluating diagnostic performance. Since the experiments conducted here do not involve cross-dataset knowledge transfer, they validated the effectiveness of CancerSD backbone. Overall, the comparison methods can be broadly categorized into three groups: Traditional Machine Learning (TML), Multi-omics Integration (MI)-based methods, and incomplete Multi-omics Integration (iMI)-based methods. A detailed description of these methods is provided in Section A in S2 Text. The experimental results presented in Fig 2a and Tables B-C in S1 Text demonstrate that CancerSD consistently makes top performance in subtype diagnosis. Then, we utilized t-SNE [33] to reduce dimensionality and visualized the embedding spaces of better-performing methods (MOGONET, DCP, and CancerSD). As depicted in Fig 2b, the challenge posed by incomplete data manifests in the distortion of sample distributions, leading to a noticeable division into two distinct clusters. This distortion greatly heightens the difficulty of subtype diagnosis. Despite these challenges, CancerSD still exhibits the best clustering result, with samples of the same subtype more concentrated in the same cluster. These observations underscore the capability of CancerSD in representation learning and its superiority in cancer subtype diagnosis using incomplete multi-omics data. In Section B in S2 Text, we conducted more analyses of the above results, thus gaining a more comprehensive and detailed perspective on the performance differences among different methods. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Diagnostic performance on the GC dataset in the standard supervised learning setting. (a) The cancer subtype diagnosis performance of CancerSD vs. comparison methods, including kNN, RFC, AE-XGBoost, MOMA, MOGONET, DCP, and APADC on the STAD dataset.(b) Sample clustering using original data and embedded representations given by CancerSD and other methods.(c) The diagnosis performance of the tested methods under different degrees of omics missingness.(d) The diagnostic accuracy of CancerSD for different GC subtypes.(e) The diagnostic probability of CancerSD for different GC subtypes. https://doi.org/10.1371/journal.pcbi.1012710.g002 Next, we evaluated the diagnostic performance of these methods under diverse degrees of multi-omics missingness. We intentionally simulated omics missing scenarios by randomly masking specific omics in samples with completely-paired multi-omics data. Specifically, the simulated missing rates are set at 0%, 25%, 50%, and 75% in sequence, indicating the specified proportion of samples selected for missing certain omics. Fig 2c and Table D in S1 Text demonstrate that CancerSD consistently makes the top performance across all the missing rates. In line with the previous analyses, the performance of the tested methods at different rates manifests a step-like distribution. iMI-based methods generally outshine their MI-based counterparts, while traditional machine learning methods display relatively poor performance. As the simulated missing rate increases, there is a gradual performance decline in all methods, with the most significant decrease witnessed in methods that involve sample similarity calculations, such as kNN and MOGONET. Moreover, it is noteworthy that when multi-omics data for samples is either complete or involves minor incomplete issues, iMI-based methods also marginally outperform MI-based ones, primarily due to the auxiliary tasks constructed for recovering the missing omics. As the degree of omics missing intensifies, a substantial performance gap becomes evident between them. Besides the simulated scenarios of multi-omics data with random missing, we also recognize that the missingness of omics data may not be entirely random in the clinical practice. Therefore, we have conducted experiments focusing on the specific omics absence, displayed the results in Tables E-G in S1 Text, and provided further analysis in Section C in S2 Text. Finally, we investigated the identification preferences of CancerSD for different GC subtypes, including Epstein-Barr virus (EBV), microsatellite instability (MSI), genomically stable (GS), and chromosomal instability (CIN) categorized by the Cancer Genome Atlas (TCGA) Research Network [34]. As shown in Fig 2d, CancerSD can easily and accurately diagnose the EBV and CIN subtypes while maintaining high accuracy for the MSI subtype. However, CancerSD struggles with the identification of the GS subtype. In Fig 2e, it is evident that the accuracy of CancerSD in diagnosing the GS subtype is significantly lower than others. Based on Fig 2b, we noted that the clusters containing samples of GS and CIN subtypes are consistently close to each other, even overlapping. We further analyzed samples where diagnostic errors occurred in the experiments and find that compared to other subtype pairs, these two subtypes are more likely to be misdiagnosed as each other (see S2 Fig). In fact, Lee et al. [35] observed cases of subtype transition between these two subtypes after metastasis (transitions from CIN to GS and vice versa), while the subtype of metastatic tumors is generally the same as the primary tumor. These findings suggest a potential similarity between the GS and CIN subtypes, which gives rise to the confusion of CancerSD toward these two subtypes. As mentioned above, the experiments conducted under standard supervised learning settings actually highlight the capabilities of CancerSDb. Furthermore, we studied more details for CancerSDb, including evaluation its robustness (see S3 Fig and Section D in S2 Text), exploring its optimal architecture (see Table H in S1 Text and Section E in S2 Text), investigating the impact of different data augmentation operations (see S4 Fig and Section F in S2 Text), and examining its sensitivity to changes in hyper-parameters (see S5 Fig and Section G in S2 Text). These experiments and analyses provide a more in-depth and comprehensive perspective on why CancerSD can make superior subtype diagnosis performance. CancerSD demonstrates superior diagnostic performance in the few-sample scenario Cancer subtype diagnosis is a classical few-sample scenario, where in-house datasets often contain only a limited number of samples, posing challenges in optimizing an accurate diagnostic model. Moreover, variations in sample collection sources and biases in sample selection contribute to significant differences among different datasets (i.e., sample distribution disparities, as illustrated in S6 Fig). Disregarding this situation and directly transferring knowledge from external datasets to in-house ones can lead to negative transfer, which potentially undermines the performance of the diagnostic model. To assess the effectiveness of CancerSD in addressing these issues, we constructed knowledge transfer tasks across different datasets. The detailed experimental setups and description of comparison approaches are presented in Section Material and methods. First, we evaluated the performance of certain cancer subtype diagnosis methods on the GSE62254 dataset under the conventional supervised learning settings, which serves as the baseline performance. In this scenario, all methods only use the data from GSE62254, with no knowledge learned from external datasets. As shown in Fig 3a, it is observed that various methods exhibit similar diagnostic performance. Particularly, there is minimal difference among the performance of iMI-based methods, including CancerSD, DCP, and APADC. This phenomenon mainly arises from the fact that GSE62254 contains mRNA expression profiles as the sole omics data, leading all methods to degrade into simple classifiers. Nonetheless, owing to auxiliary tasks such as data reconstruction, iMI-based methods still perform slightly better than others. Additionally, we observed a significant performance decline when training CancerSD with only a small amount of samples (4-way 10-shot, ten samples for each subtype). In fact, by referring to Fig 3a and Table I in S1 Text, we noted that all methods perform poorly under this scenario, which could be attributed to two main reasons. On the one hand, since only mRNA data is accessible in GSE62254 dataset, MI-based and iMI-based methods essentially degrade into single-model classifier, losing their advantage in modeling multi-omics interactions. On the other hand, all compared methods can only use a limited amount of data from GSE62254, preventing sufficient optimization and causing underfitting. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Diagnostic performance on the GC datasets in the few-sample scenario. (a) The cancer subtype diagnosis performance of CancerSD vs. comparison methods on the GSE62254 dataset under the standard supervised learning setting.(b) The cancer subtype diagnosis performance (Accuracy and F1 Score) of different methods under the multi2mRNA (upper figure) and mRNA2mRNA (lower figure) settings. Here, the red dot dash line represents the performance obtained by optimizing CancerSD with the entire training set, while the gray dotted line indicates that obtained by optimizing CancerSD with a 4-way 10-shot set.(c) The similarity of representations for samples from different datasets. https://doi.org/10.1371/journal.pcbi.1012710.g003 Next, we attempted to transfer knowledge from the STAD dataset, supporting the optimization of diagnostic models on GSE62254. Specifically, we initialized and trained diagnostic models on STAD and subsequently select a small number of samples (4-way 10-shot) from the training set of GSE62254 to fine-tune these models. To include traditional machine learning methods into the experiment, we combined STAD and the data used for fine-tune from GSE62254 into a unified training set, which was then provided to these methods. Here, we explored knowledge transfer under two strategies: from multi-omics data of STAD to the mRNA expression profile of GSE62254 (multi2mRNA) and from the mRNA expression profile of STAD to that of GSE62254 (mRNA2mRNA). As illustrated in Fig 3b, and Tables J-K in S1 Text, We observed that even if TML approaches are granted access to external datasets, they struggle to make reasonable use of these data, due to the lack of effective knowledge extraction strategies. This demonstrates the difficulty of TML methods in adapting to the few-sample scenario, compared to few-shot learning-based approaches. We also found that meta-learning-based methods generally exhibit superior transfer performance compared to pretraining-based ones. This is likely due to the emphasis of meta-learning on the cross-dataset generalization ability of the models, while pretraining strategies focus on models’ performance on the current dataset. Notably, with only a 4-way 10-shot set sampled from the training set, our CancerSD achieves performance comparable to using the entire training set. This observation emphasizes the capability of CancerSDm in cross-dataset knowledge transfer. Moreover, the performance of tested methods under the mRNA2mRNA strategy is superior (or comparable) to that under the multi2mRNA. This could be attributed to the fact that embedding spaces generated from mRNA data in different datasets are more similar in distribution than those generated separately from multi-omics and mRNA data. Consequently, under the mRNA2mRNA strategy, the models can more easily absorb knowledge from external datasets. Taking results under mRNA2mRNA as example, we further analyzed the effectiveness of CancerSDm and the importance of category-level contrastive loss. In the knowledge transfer task, CancerSD achieves an Accuracy of 67.5%, AUROC of 83.7%, Precision of 73.3%, and F1 Score of 67.6%, outperforming its comparison methods and variations across almost all metrics. The superior performance can be attributed to its powerful backbone and domain-specific knowledge transfer capabilities. Specifically, CancerSD outperforms its variants CancerSD-MOMA and CancerSD-DCP, whose backbone networks are replaced with MOMA and DCP, respectively. This superiority underscores the capability of CancerSDb to effectively impute missing omics data and integrate multiple omics for accurate and reliable cancer subtype diagnosis. On the other hand, the significant performance decline of CancerSD-w/oCLC highlights the importance of the distribution-based category-level contrastive loss. By leveraging this loss, CancerSD can alleviate the sample distribution discrepancy across different datasets and focus on extracting knowledge relevant to the assigned downstream diagnosis tasks from external datasets. Fig 3c supports this perspective by showing that representations obtained by CancerSD for samples of the same subtype from different datasets exhibit higher similarity. This observation indicates that CancerSD effectively captures the consistency of cancer subtypes across different datasets and integrates the consistency into sample representations, thereby improving the similarity among samples of the same subtype. The comparison results also demonstrate that the similarity-based category-level contrastive loss fails to acquire knowledge from datasets with more samples effectively and may even hamper the generalization ability of CancerSDb. This is because CancerSD-SIM utilizes the instance-level similarity to cluster samples of the same subtype, which potentially leads to severe overfitting problems and is susceptible to noise and outliers. In contrast, CancerSD attempts to cluster samples of the same subtype at the distribution level, thereby alleviating such issues. The above analyses provide insights into why CancerSDm can effectively extract knowledge from other datasets. In summary, CancerSDm adopts a meta-learning strategy to mine and transfer meta-knowledge from external datasets and utilizes the category-level contrastive loss to maximize the agreement of distributions between samples with the same subtype across different datasets, thereby improving the diagnostic performance of the model on target dataset. Diagnostic performance of CancerSD under different omics data types Although CancerSD fuses three types of omics data (DNA methylation profiles, miRNA expression profiles, and mRNA expression profiles) for cancer subtype diagnosis in the above experiments, it can readily adapt to different numbers of omics data types. To verify the importance of multi-omics integration in improving the diagnosis performance and to assess the capability of CancerSD in multi-omics integration, we evaluate CancerSD using various combinations of omics data. Here, we only consider samples with completely-paired multi-omics data. Fig 4a shows that the diagnostic performance of CancerSD is improved continuously by integrating more omics data. In concrete terms, CancerSD trained with all three omics outperforms the model using a combination of two types of omics. The performance of CancerSD trained with two omics is also superior to that of only single omics. These results highlight the advantages of integrating multiple omics data for more accurate subtype diagnosis. Moreover, it is worth noting that CancerSD trained with mRNA expression data performs best when employing only single omics for training. This finding suggests that mRNA features contain information conducive to distinguishing GC subtypes, potentially harboring valuable biomarkers. In contrast, the performance of CancerSD trained using miRNA data is the poorest. This may be attributed to the lower dimensionality of its original data compared to the other two types of omics (702 vs. 3278 and 4089), which provide less discriminative information for subtyping. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Diagnostic performance under different omics data types. (a) Performance comparison for subtype diagnosis using different types of omics. Among them, methylation, miRNA, and mRNA refer to make diagnosis via CancerSD using DNA methylation data, miRNA expression data, and mRNA expression data, respectively; meth+miRNA, miRNA+mRNA, and meth+mRNA refer to diagnosis with two types of omics; meth+miRNA+mRNA refers to diagnosis with three types of omics.(b) Sample similarity heatmaps obtained from representation at different levels.(c) Diagnostic performance of CancerSD using only a single type of omics under different training strategies.(d) Sample clustering based on omics and fusion representations output by CancerSD under multi-omics joint training strategy.(e) Sample clustering based on omics and fusion representations output by CancerSD under single-omics independent training strategy. https://doi.org/10.1371/journal.pcbi.1012710.g004 Expanding upon the results mentioned above, we delve deeper into the effectiveness of CancerSD in integrating multi-omics data. First, we focus on the sample similarity pattern as an illustrative example to elucidate the capability of CancerSD in extracting cross-omics consistency information. As depicted in Fig 4b, after mining subtype-related discriminative information in the corresponding omics data using omics-specific feature extractors, samples of the same subtype exhibit relatively high similarity across different omics. CancerSD adeptly identifies and captures this cross-omics consistent pattern of sample similarity, incorporating these patterns into the multi-omics fusion representation. Meanwhile, CancerSD takes into account discrepant patterns across multiple omics to mitigate the impact of erroneously high similarity (i.e., the globally high similarity from miRNA embedding). Consequently, the fusion representation learned by CancerSD more accurately reflects the correct similarity relationships between samples. Next, we conduct extensive experiments to assess the capability of CancerSD in exploring cooperation between diverse omics data. To this end, we compare the subtype diagnosis performance of CancerSD in two distinct training strategies: (i) multi-omics joint learning, which uses multiple omics data simultaneously to optimize the model; (ii) single-omics independent learning, which uses only a single type of omics to optimize the model each time. As shown in Fig 4c and Table L in S1 Text, it is evident that under the identical condition of diagnosing, CancerSD optimized by multi-omics joint learning more accurately identifies patient subtypes compared to the model optimized by single-omics independent learning strategy. To gain further insights, we investigate the omics embeddings obtained under different training strategies and evaluate the diagnostic potential of multi-omics fusion representations derived from these embeddings. Specifically, we separately construct and optimize multi-omics fusion networks (see Experimental Section) to integrate multiple omics embeddings obtained under joint learning and independent learning strategies. Then, we utilize the resulting fusion representations to perform cancer subtype diagnosis tasks. The results presented in Table L in S1 Text suggest that the fusion representations integrated from omics embeddings learned by joint learning are more conducive to identifying subtypes. Furthermore, we visualize sample clustering for different training strategies. From Fig 4d and 4e, we can find that, in comparison with the output obtained by initially performing single-omics independent learning and then integrating the resulting omics embeddings, the fusion representations obtained through multi-omics joint learning can generate a more compact clustering structure, with more apparent margins between clusters. These observations prove that CancerSD has effectively learned cooperation between different omics during the multi-omics joint optimization process, thereby enhancing the performance of multi-omics fusion. In summary, multi-omics data can offer more prosperous and more comprehensive patient features. CancerSD effectively integrates these data by extracting cross-omics consistency and cooperation information, significantly improving the performance of cancer subtype diagnosis. To further highlight the superiority of our CancerSD, we also evaluated the diagnostic performance of several comparison methods across different omics types (see Table M in S1 Text), with a concise analysis provided in Section H in S2 Text. In addition to integrating multi-omics, CancerSD also possesses a notable capability in handling missing omics data. Even in cases of extensive omics data missingness (see Table N in S1 Text), CancerSD effectively recovers biologically meaningful expression values, affirming its authenticity and effectiveness in imputing missing omics. The details regarding this aspect are presented in S7–S13 Figs and are discussed more extensively in Section I in S2 Text. CancerSD identifies important molecules related to gastric cancer Identifying important biomarkers is crucial for understanding the underlying mechanisms of GC and interpreting the corresponding diagnostic decision made by CancerSD. To this end, we investigated the importance of each molecular characteristic on the diagnostic outcomes to find potential biomarkers. Specifically, we systematically shuffled the values of each molecular characteristic across all samples in the testing set and then evaluate the diagnostic performance of CancerSD using these modified features. After that, we compared the performance with results obtained when using all features, allowing us to discern the contribution of each molecule to diagnosis tasks, where the diagnosis loss (Eq 10) serves as a quantitative indicator for multi-classification tasks. The more loss increases, the more important the currently permutated molecule is. For a more robust result, we conduct ten random experiments and take the average performance degradation as the final result. Within each type of omics, we selected the top-ranked molecular characteristics for further analysis and validation. First, we presented importance scores of the top ten ranked molecules from each omics. As shown in Fig 5a and 5b, it is evident that there are significant importance differences of molecules across various omics. Among them, mRNA features obtain the highest importance scores, while miRNA features have the lowest, indicating that CancerSD relies more on mRNA expression profiles in the diagnostic decision-making process. This perspective is further highlighted in Fig 5c, where clusters from mRNA embeddings are closer to clusters from fusion representations than from other omics embeddings for patients. We speculated that the prominence of mRNA features may be mainly attributed to two reasons. On the one hand, owing to the higher dimensionality of the raw data, mRNA features can provide richer discriminative information for subtype diagnosis. On the other hand, mRNA expression data more directly reflects the activity of genes and cellular functions. Meanwhile, mRNA expression is influenced by multiple regulatory layers, including DNA methylation and miRNA regulation, among others, potentially more comprehensively reflecting the integrated effects of gene expression regulation. The division of cancer subtypes is often associated with changes in genes. Therefore, mRNA features play a more crucial role in the diagnostic process. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Important molecules identified by CancerSD. (a) Importance scores of the top-ranked molecules identified by CancerSD in various omics.(b) Differences in the importance scores of molecules across various omics.(c) Clustering for samples of different subtypes using omics embeddings and fusion representation output by CancerSD, respectively.(d) The methylation levels of the top 100 CpG sites ranked by importance, where the CpG sites are secondary sorting based on the average values across all samples.(e) The expression levels of mRNA characteristics across different GC subtypes.(f) The expression levels of mRNA and miRNA characteristics across different GC subtypes, where the expression values subjected to log2 transformation and normalization. Wilcoxon rank-sum test is employed to evaluate the differences in the expression levels of specific molecules among patients of distinct subtypes.(g) Gene co-expression analysis result for EBV subtype.(h) KEGG Pathway Enrichment results for module-2 (ME-2, top) and module-4 (ME-4, bottom), respectively. https://doi.org/10.1371/journal.pcbi.1012710.g005 Then, we visualized the expression of molecules identified by CancerSD in each omics type to preliminarily showcase the authenticity of CancerSD in making diagnostic decisions. The molecules selected in this step are detailed in Table 1 (genes and miRNAs are listed) and Table O in S1 Text (CpG sites are listed). For the DNA methylation profile, we explored differences in methylation patterns at CpG sites among patients of distinct subtypes, focusing on the top 100 ranked features. As depicted in Fig 5d, differences in methylation patterns emerge across GC subtypes. Specifically, EBV patients exhibit elevated methylation levels at most CpG sites in comparison to other subtypes, with MSI patients ranking second. Patients with the other two subtypes show a relatively similar DNA methylation pattern. These observations are consistent with the previous study [34]. For miRNA and mRNA expression profiles, we investigated the differences in the expression levels of the top 10 important molecules among various subtypes. Based on Fig 5e–5f and S14 Fig, we could observe that most top-ranked characteristics exhibit significantly different expression levels across various subtypes. These observations indicate that CancerSD primarily relies on molecules that exhibit distinction across different subtypes in the diagnostic process. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Important molecules identified by CancerSD. https://doi.org/10.1371/journal.pcbi.1012710.t001 Next, we explored the relationship between top-10 important molecules in each omics (see Table 1) and GC. Among them, characteristics derived from DNA methylation and mRNA expression profiles are inferred as their corresponding genes. Notably, nearly half of these molecules have been extensively studied. For instance, Yasumoto et al. [36] discovered that the CXCR4/CXCL12 axis plays a role in the development of peritoneal carcinomatosis, which is an incurable complication during the advanced stages of GC. Subsequent research by Hashimoto et al. [37] revealed that blocking the CXCR4/mTOR signaling pathway may contribute to the treatment of this complication. Moreover, Xiang et al. [38] demonstrated that CXCR4 can cross-activate with CXCR2, promoting the epithelial-mesenchymal transition, metastasis, and invasion in GC. Simultaneous inhibition of these two genes has been shown to reduce the metastasis of GC effectively. These studies highlight the close association between CXCR4 and GC. Besides important genes, the miRNAs identified by CancerSD have also been confirmed to have profound associations with GC. Taking hsa-mir-488 as an example, several studies [39, 40] demonstrated that its overexpression can delay the malignant progression of GC, suggesting that hsa-mir-488 holds promise as a valuable biomarker for the diagnosis and treatment for GC. Alongside the molecules mentioned above, existing studies (see Table P in S1 Text) have reported that CD9, KLK6, HLA-B, MUC1, MET, hsa-mir-7–2, hsa-mir-944, hsa-mir-942, hsa-mir-219a-2, and hsa-mir-1305 also play an important role in the occurrence, progression, or diagnosis and treatment of GC. While the relationships between certain molecules in Table 1 and GC remain unclear, some are closely associated with other digestive tract cancers. For example, Fujita et al. [41] found that the overexpression of ENC1 may suppress the differentiation of colon cells, potentially leading to the development of colorectal cancer. This process could be achieved through JAK2/STAT5/AKT axis-mediated epithelial-mesenchymal transition and stemness. [42] In addition, Than et al. [43] indicated that CFTR is a tumor suppressor gene in intestinal cancer. Similarity mechanisms might exist in GC, and further exploration of these genes could contribute to a more profound understanding of GC, as well as elucidating connections and distinctions among various digestive tract cancers. Finally, we conducted a detailed analysis of important genes identified by CancerSD for each GC subtype, including genes derived from DNA methylation and mRNA expression profiles. During the calculation of diagnosis loss, we individually assessed the impact of feature shuffling on the diagnosis for each subtype. These processes involve partitioning the testing set based on subtypes and separately calculating diagnosis loss for each subtype. By quantifying the increase of the loss, we assign importance to specific characteristics. For the top-10% ranked important genes (619 / 6186) identified for each subtype, we utilized the WGCNA R package [44] and OmicVerse [45] to conduct gene co-expression analysis and select certain co-expression modules for subsequent analysis. Taking the EBV subtype as an example, the selected genes are clustered into six co-expression modules (as illustrated in Fig 5g). Among these modules, genes in module-2, 3, 4, 5, and 6 exhibit significant co-expression relationships, with 75, 54, 50, 33, and 32 genes within each module. We further conducted KEGG enrichment analysis on these five modules, and the results are presented in Fig 5h and Table Q in S1 Text. In module-2 (ME-2), pathways such as ECM-receptor interaction (ko04512) and cell adhesion molecules (ko04514) are prevalent in cancer and play crucial roles in GC [46, 47]. Some pathways enriched in ME-2 are also highly associated with EBV. Liang et al. [48] reported that focal adhesion (ko04510) signal pathways are often dysregulated due to EBV-associated genomic and epigenomic alterations, which may play a crucial role in the development of EBV-associated GC. Other three myocarditis-related pathways (ko05412, ko05410, and ko05414) are associated with a rare but severe complication of EBV infection [49]. In module-4 (ME-4), numerous immune-related pathways are significantly enriched, such as PD-L1 expression and PD-1 checkpoint pathway in cancer (ko05235), TNF signaling pathway (ko04668), antigen processing and presentation (ko04612). Among them, NF-κB signaling pathway (ko04064) exhibits higher positivity in EBV-positive GC than EBV-negative one [50]. It promotes the proliferation of GC cells infected with EBV, which could be attributed to the regulation of the EBV-encoded BARF1 [51]. Similarly, the overexpression of PD-L1 has been reported as a typical characteristic of the EBV subtype [34], and PD-1 inhibition is an effective treatment for patients of this subtype [52]. These two mechanisms are closely related to ko05235 pathway. Cytokine-cytokine receptor interaction (ko04060) is also one of the core pathways dysregulated in EBV-associated GC [48]. Collectively, the aforementioned pathways may play important roles in the development of EBV-associated GC. Therefore, we hypothesized that key genes within these pathways could serve as potential biomarkers or therapeutic targets for the EBV subtype. For example, CXCL10 and CXCL11, both small-molecule cytokines in the CXC chemokine family, are significantly overexpressed in the EBV subtype compared to other subtypes (see S15 Fig). These two genes regulate the migration, differentiation, and activation of immune cells through the CXCL9/10/11/CXCR3 axis, which is also directly involved in the proliferation and metastasis of cancer cells. [53] Given their roles in guiding immune cells such as T cells and leukocytes to move towards inflammatory or infected sites, [54]CXCL10/11 may contribute to better immunotherapeutic effects in EBV-positive GC patients. In more detail, CXCL10/11 are regulated by EBV-related miRNAs, with the former being regulated by ebv-miR-BART1–3p [55] and the latter being regulated by ebv-mir-BHRF1–3. [56] It is possible that EBV promotes the occurrence and development of GC through these pathways, implying the potential of CXCL10/11 as diagnostic factors for the EBV subtype. Moreover, for other GC subtypes (CIN, GS, MSI), the co-expression and KEGG pathway enrichment results are presented in S16 Fig. The above results and analysis verify the authenticity and interpretability of CancerSD in cancer subtype diagnosis, which also prove the potential of CancerSD in assisting clinical diagnosis. Outcomes of CancerSD are associated with stemness features of gastric cancer subtypes and patient prognosis In the previous analyses, CancerSD demonstrates the capability to accurately diagnose cancer subtypes using incomplete multi-omics data. Experiments conducted on the GC dataset also indicate its ability to identify key molecular signatures associated with GC. These results provide preliminary evidence of its reliability in assisting clinical diagnosis. To further investigate the role of CancerSD in diagnostic decision-making, we explored its relationship with gastric cancer subtypes and patient prognosis. Stem cells are characterized by their capacity for self-renewal, either infinitely or perpetually, alongside their ability for multi-lineage differentiation, while stemness is defined as the potential of stem cells in these two aspects [57]. Within tumor tissues, a small proportion of relatively stable cells possessing both proliferative and tumor-reconstructing abilities are identified as cancer stem cells or cancer stem-like cells [58]. These cells may cause various tumor malignancies, such as recurrence, metastasis, multidrug resistance, and radioresistance [59]. Thus, determining the stem-cell characteristic of each GC subtype is of significant importance for gaining deeper insights into mechanisms underlying tumor initiation and progression, as well as for the development of effective therapeutic strategies. To this end, we employed the stemness index model [57], known as mRNAsi, to score the stemness features of GC samples and then conduct further analysis. We first collected gene expression profiles of pluripotent stem cells from the Progenitor Cell Biology Consortium dataset [60, 61] (syn2701943). The data are preprocessed with mean-centering. Subsequently, the stemness signature is identified through the one-class logistic regression algorithm. Next, spearman correlation analysis is performed between the normalized expression matrix of GC samples and the stemness signature. The resulting correlation coefficients are scaled to the range [0, 1] to determine the stemness index. Finally, we assessed the relationship between mRNAsi scores and our CancerSD. From Fig 6a, we observed an association between the stemness index and clinical features in GC patients. In particular, there are significant differences in mRNAsi among patients of distinct GC subtypes (see Fig 6b). While there is typically a negative correlation between mRNAsi and the prognosis of cancer patients [62, 63], an opposite trend is noted in GC [64]. This is further highlighted in Fig 6c, where patients with higher mRNAsi tend to exhibit a favorable prognosis. Consistently, mRNAsi is the lowest in samples of the GS subtype, which corresponds to the poorest prognosis among the four subtypes [65]. Interestingly, a correlation analysis of the stemness index with the CancerSD score for each subtype shows that the GS subtype is significantly negatively correlated with the mRNAsi (r = −0.353, p = 4.94 × −103, Fig 6d), where CancerSD scores represent the probability of a patient being diagnosed with a certain cancer subtype. There is currently no consensus on why the GS subtype often corresponds to the lowest mRNAsi. Considering the high overlap between samples of GS subtype and of diffuse-type GC (see Fig 6e), we might gain insights into the mechanisms behind this phenomenon from diffuse-type GC [66], which similarly obtains the lowest mRNAsi scores within its corresponding Lauren [67] classification system (see Fig 6f). In addition, CancerSD scores of samples with other subtypes also show significant correlations with mRNAsi. Given the significant correlation between mRNAsi and patient prognosis, the aforementioned observations suggest that GC subtype scores may be associated with the prognosis of GC patients. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. The relationship among CancerSD outcomes, mRNAsi scores, and patient clinical characteristics. (a) An overview of the association between the mRNAsi and clinical features. The median of mRNAsi score is used to categorize mRNAsi levels.(b) mRNAsi scores across different molecular subtypes.(c) Kaplan-Meier survival curves of different mRNAsi levels. Among them, HR and 95CI are abbreviations of Hazard Ratio and 95% Confidence Interval, respectively.(d) The relationship between CancerSD scores (with normalization) and mRNAsi scores in GC patients. The former is derived from the output of CancerSD before the softmax layer, while the latter is obtained through the mRNAsi model.(e) The integrated sankey diagram portrays the underlying correlations across the mRNAsi, molecular subtypes and Lauren classification.(f) mRNAsi scores across different Lauren subtypes.(g) The relationship between Integrated CancerSD Score (ICS) and mRNAsi scores in GC patients.(h) Kaplan-Meier survival curves of different ICS levels.(i) Correlation of mRNAsi and expression levels of important genes identified by CancerSD. The regression lines in figures are fitted by the corresponding data. The significance in the figure is estimated by pearson correlation coefficient. https://doi.org/10.1371/journal.pcbi.1012710.g006 To delve deeper into this association, we aggregated CancerSD scores for different subtypes in a manner analogous to TCGA Risk Score [65], yielding the Integrated CancerSD Score (ICS) that estimates patient prognosis risk. To ensure a smoother ICS, we replaced the softmax function with the sigmoid function, constraining all CancerSD scores to the (0, 1) interval. Specifically, we formulated ICS as follow: ICS = eICSraw, where ICSraw = CIN score + (1—MSI score) + (2 × GS score) + (1—EBV score). Among that, the coefficients preceding the subtype prediction scores were determined based on their relationship with prognosis [65]. Since EBV and MSI are commonly associated with favorable prognosis, we used the inverse of their CancerSD scores. The weighting coefficient for the GS score was assigned to 2, reflecting its strong correlation with poor prognosis. The CIN score remained unchanged, as this subtype is only moderately associated with poor prognosis. Using 16.2 (the optimal cutoff value confirmed by maxstat R package [68]) as the cutoff point, we categorized ICS into low and high levels. As illustrated in Fig 6g, ICS exhibits a significant negative correlation with mRNAsi (r = −0.601, p = 6.5 × −1037), indicating that a high level of ICS may correspond to a poorer patient prognosis. Fig 6h validates this perspective, demonstrating a significant difference in overall survival among patients with different ICS levels (p = 0.04). While the stratification of patients based on ICS levels does not show a significant difference in disease-free survival (p = 0.16), noticeable distinctions can be observed in Fig 6h. This lack of significance may be attributed to the problem of insufficient data related to disease-free survival, with only 201 out of 415 GC patients possessing corresponding disease-free survival information. To further evaluate the prognostic value of the ICS, we conducted univariate and multivariate Cox proportional hazards regression analyses on ICS and other seven clinical variables. As listed in Table R in S1 Text, in addition to well-known prognostic factors such as T stage, N stage, M stage, and TNM stage, ICS emerged as a significant predictor for overall survival in univariate analysis. Even when incorporating all relevant clinical variables in a multivariate Cox regression analysis, ICS remains an important prognostic factor (HR, 1.7; 95% CI, 1.07–2.70, p = 0.03). Collectively, ICS may serve as a potential predictive factor for overall survival and even disease-free survival, highlighting the utility of CancerSD. We further extended our investigation to the relationship between mRNAsi and the top-ranked genes identified by CancerSD. Specifically, we analyzed the correlation between the expression levels of the top 25 genes (including those identified from DNA methylation and mRNA expression profiles) across all samples and mRNAsi scores. The results are presented in Fig 6i and S17 Fig. We observed that 13 genes exhibit a significant correlation with mRNAsi. Among them, five genes are directly or indirectly involved in various biological processes, exerting impacts on the stemness characteristics of cancer cells. For example, c-Met, the protein product of the MET proto-oncogene, has been demonstrated to promote tumor angiogenesis, growth, and metastasis [69]. Several studies [70, 71] reported that c-Met is implicated in the stemness of cancer stem cells in various cancers. In gastric cancer, Yashiro et al. [72] found that the combination of c-Met inhibitors with SN38 may effectively target cancer stem cells in diffuse-type GC. Bahrami et al. [73] also reported that c-Met/ALK inhibitors could reduce the expression of cancer stem cell markers in gastrointestinal cancers. These evidences suggest that elevated expression of MET can promote the characteristics of GC stem cells, thereby positively influencing mRNAsi scores. Moreover, ECRG4 serves as an inhibitory upstream regulator of the NF-κB pathway [74], while the latter is persistently activated in cancer stem cells across various malignancies, participating in several crucial biological processes of cancer stem cells [75]. The role of NF-κB in GC has been widely reported, where its activation can stimulate the proliferation and stemness of GC cells [76]. Ding et al. [77] found that the PEAK1-PPP1R12B axis can inhibit cell growth and metastasis in colorectal cancer by attenuating the Grb2/PI3K/Akt signaling pathway, and a similar mechanism might exist in GC. Considering the activation effect of PI3K/Akt pathway on the NF-κB system [78], the high expression of genePPP1R12B might indirectly play a role in suppressing the stemness of GC cells. Consequently, we observe a significant negative correlation between the expressions of ECRG4 and PPP1R12B and mRNAsi scores. Besides MET mentioned above, ECRG4 and PPP1R12B, another two genes (see Table S in S1 Text), whose expression levels are significantly correlated with mRNAsi scores, are also associated with the stemness features of cancer cells. Although not all of these genes have been confirmed to play a role in GC, there might be similar mechanisms promoting or inhibiting the stemness of GC cells. In summary, CancerSD scores for samples show a significant correlation with the stemness features of different GC subtypes and patient prognosis. Moreover, the majority of top-ranked important genes identified by CancerSD are closely associated with cancer cell stemness features. These findings once again validate the authenticity and reliability of CancerSD in GC subtype diagnosis, suggesting its potential to assist real-world clinical decision-making. CancerSD maintains good performance on multiple cancer datasets In the above analyses, we primarily discussed experimental results related to GC. To investigate the generalization capability of CancerSD, we also conducted a series of experiments on lung cancer and breast cancer datasets. The relevant results are presented in Tables B, L, T in S1 Text and S8–S9 Figs. Overall, we could draw similar observations from these results as those observed in GC datasets. First, CancerSD exhibits superior (or comparable) performance in subtype diagnosis for lung cancer and breast cancer than the competitive methods. Second, experiments involving multi-omics integration and missing omics imputation on these two cancer datasets further highlight the effectiveness of our method in these regards. Furthermore, we observed that the importance of different omics varies across different cancers during the diagnostic process. Lastly, experiments involving knowledge transfer between two lung cancer datasets once again demonstrate the capabilities of CancerSD in addressing sample scarcity and mitigating negative transfer. In addition to these observations, a more detailed exposition of the relevant experiments and corresponding results is provided in Section J in S2 Text. In summary, CancerSD emerges as an effective and authentic model for cancer subtype diagnosis, which can be readily deployed to different cancers. Discussion In this study, we proposed CancerSD, an end-to-end model designed for cancer subtype diagnosis using limited incomplete multi-omics data. By leveraging the tensor fusion network and contrastive learning, CancerSD can extract more informative representations from available multi-omics data of patients. Then, it employs omics-specific generators with masking and reconstruction mechanisms to reliably recover missing omics. Finally, CancerSD integrates the available and imputed omics data to make accurate subtype diagnoses. To address the issue of limited cancer samples, CancerSD extends the meta-learning framework and introduces a distribution-based category-level contrastive loss, effectively mining relevant knowledge from external datasets. To demonstrate the effectiveness and versatility of CancerSD, we conducted a series of experiments on multiple challenging cancer datasets. The experimental results show that CancerSD significantly outperforms thirteen subtype diagnosis methods and four knowledge transfer methods in most cases. Besides its superior diagnostic performance, CancerSD exhibits good interpretability and maintains high authenticity. It can effectively integrate incomplete multi-omics data and recover biologically meaningful omics data, enabling accurate and reliable subtype diagnosis. More in-depth experiments conducted on the GC dataset further highlight the potential of CancerSD in clinical applications. For important molecular characteristics and related pathways identified by CancerSD, several studies have confirmed their close association with the occurrence and progression of GC, indicating their predictive and therapeutic value. Moreover, our defined Integrated CancerSD Score shows a close association with the prognosis of GC patients and holds the potential to serve as an independent predictive factor for patient prognosis. Despite the notable advancements of CancerSD in cancer subtype diagnosis, there remains room for further improvement. For instance, the usage of tensor fusion may overlook certain prior knowledge contained in multi-omics data, such as regulatory relationships among genes, miRNAs, and mRNAs. Considering these knowledge during modeling contributes to more effective integration of multi-omics data and obtaining better interpretability. Furthermore, CancerSD is currently confined to knowledge transfer across different datasets of the same cancer type. Recognizing potential differences and correlations among different types of cancers, we can further explore more extensive transfer, for example, transferring knowledge from other digestive tract cancers to improve gastric cancer subtype diagnosis. Addressing these aspects in future studies will contribute to the continued refinement and expansion of CancerSD. Materials and methods Datasets To study the effectiveness of CancerSD, we apply it to subtype three representative cancers using benchmark datasets: STAD and GSE62254 for gastric cancer (GC) TCGA subtype [34] diagnosis; LUAD, LUSC and CPTAC [79, 80] for lung cancer subtype diagnosis; and BRCA for breast invasive carcinoma PAM50 subtype [81] diagnosis. Three types of omics data are employed for experiments, including DNA methylation profile, miRNA expression profile, and mRNA (protein-coding gene) expression profile. Among the aforementioned datasets, only samples with molecular subtype diagnoses are selected. An overview of these datasets is provided in Table A in S1 Text. Data preprocessing In the data preprocessing stage, we first apply a log2 transformation to the miRNA and mRNA expression data. Then, we filter out features with low variance [14] (the threshold set as 0.2 for the DNA methylation profile, 0.1 for the miRNA expression profile, and 0.8 for the mRNA expression profile). These variance thresholds are consistently used across all experiments. Additionally, we select specific molecular characteristics following the analysis configuration proposed by Hoadley et al. [82] and incorporate them into the filtered features. Consequently, we retain 3287 DNA methylation characteristics, 702 miRNA expression characteristics, and 4089 mRNA expression characteristics. Finally, we individually normalize miRNA and mRNA data to a range of [0, 1] (the original range of DNA methylation data is already within the interval [0, 1], where 0 represents a lower level of methylation, and 1 represents a high level of methylation). The shared patient feature encoder To integrate multi-omics data of patients, CancerSD constructs a module-shared patient feature encoder, which is capable of providing high-quality patient representations for both missing omics imputation and cancer subtype diagnosis. The encoder comprises M omics-specific feature extractors and a multi-omics fusion network. The omics-specific feature extractors are feedforward networks designed to reduce dimensionality and capture discriminative characteristics and patterns of the corresponding omics data as: (1) where is the embedded representation for the m-th omics data of the i-th patient. fm(⋅) corresponds to the feature extractor of the m-th omics. To capitalize on the advantages of multi-omics data, we employ LMF [83] (Low-rank Multimodal Fusion) and formulate our multi-omics fusion network for data integration. In comparison to alternative methods [14, 84], LMF stands out by its ability to explore cross-omics cooperation while retaining omics-specific information, allowing more informative integration. We first concatenate an extra constant value of 1 after , namely , which can prevent the loss of the original features of each omics. Next, the fusion sub-network performs the outer product operation on and transforms the resulting fused tensor into a low-dimensional space as: (2) where is a (M + 1)-order transformation tensor, along with a bias term , and dh is the dimensionality of patient embedded representations. denotes the tensor outer product operation. is a form of decomposition of W, and R is the number of low-rank tensors obtained after decomposition. By decomposing W to make a separate linear transformation for , we can significantly reduce the computational complexity of the multi-omics integration. denotes the Hadamard product of M vectors: . In this way, the module-shared patient feature encoder can be formalized as: (3) where fenc represents the patient feature encoder, is the multi-omics data for the i-th patient, and is a concatenation operator. In detail, we first split x i into M omics features, such as DNA methylation, miRNA, and mRNA features used here. Then, each omics data is fed into the corresponding feature extractor fm(⋅). Finally, the resulting omics embeddings are integrated as a fusion representation z i through the fusion sub-network. Incomplete data imputation module In real clinical scenarios, it is common to have access to only several or even a single type of omics data for some patients. The multi-omics data for these patients are incomplete, potentially leading to information loss and data bias, resulting in misleading diagnostic outcomes. To avoid risks arising from such incomplete data, we utilize available patient omics to impute their missing ones. To this end, we partition X into two parts to perform different training tasks. Specifically, Ncp samples with completely-paired multi-omics data, denoted as , are utilized to execute contrastive learning and masking-and-reconstruction (MAR) tasks. Meanwhile, Ninc samples with incomplete data are exclusively employed for the reconstruction tasks. Firstly, we devise contrastive learning tasks to enhance the representational capacity of information learned by CancerSD for incomplete multi-omics data, providing generators with more informative representations. These processes begin by randomly generating two distinct masking strategies, which are then applied to . For each patient, we can garner two different but correlated augmented views, denoted as and , which together constitute the positive pair. Subsequently, and are fed into the patient feature encoder (see Eq 3), yielding and . These patient representations are further mapped into a projection space, where the contrastive loss is applied. In this way, we obtain higher-level representations and for the augmented patient views. When taking as the input, a representation set can be assembled, encompassing 2Ncp augmented representations. Finally, we formulate contrastive learning tasks within the projection space with the objective of maximizing the agreement among different augmented views of the same sample. Specifically, given a positive pair , the remaining 2(Ncp − 1) representations in Su are treated as negative samples [85]. Thus, the loss of the pair can be formulated as: (4) where τ denotes a temperature parameter. represents the set without from Su. Consequently, the instance-level contrastive loss can be further calculated as: (5) Next, we proceed to construct M omics-specific generators based on feedforward networks to purposefully impute the missing omics data using latent features extracted from available patient omics as: (6) where is the generated omics data and gm(⋅) denotes the generator designed for the m-th omics. The binary variable indicates whether the m-th omics of the i-th patient is missing or not. To acquire the imputation ability of the generators, we define MAR tasks on patient features. On the one hand, we utilize Ncp samples with completely-paired multi-omics data to perform the MAR tasks. These tasks involve randomly masking out certain omics of some samples (setting all data within the masked omics to zero) and subsequently reconstructing these masked values. The loss can be calculated as: (7) where MSE(⋅, ⋅) denotes the mean square error loss function, and indicates whether the m-th omics is masked out or not for the i-th patient. On the other hand, for Ninc samples with incomplete data, we only consider the reconstruction loss pertaining to the existing omics as: (8) Finally, the generation loss can be calculated as: (9) Cancer subtype diagnosis module To alleviate adverse efforts posed by the absence of certain omics, CancerSD fuses both available and imputed omics data to make cancer subtype diagnosis and calculates the diagnosis loss as: (10) (11) where the representation z i is obtained by feeding the multi-omics data into the patient feature encoder fenc. In cases where all omics for a patient are available, we input them directly into the encoder; otherwise, we first impute the missing omics and then encode them. fdiag denotes the subtype diagnosis network, yi is the subtype label of the i-th patient, and CE(⋅, ⋅) represents the cross entropy loss function. is the weight of subtype yi when calculating the diagnosis loss, which is used to alleviate the problem of sample imbalance. Ntr is the number of samples in the training set, Ns is the number of subtypes, and is the number of samples of subtype yi. In a word, CancerSDb is an end-to-end architecture involving two modules that are optimized simultaneously in one stage, and the total loss can be calculated based on the individual loss of each module as: (12) where λ1, and λ2 are trade-off parameters among three individual losses. In practice, we set both of them to 1 by default. It is worth noting that although we integrate the losses from each module into , each loss affects different sub-networks during the backpropagation process. For instance, updates fenc solely and does not impact the other modules. optimizes both fenc and the omics-specific generator gm. Meanwhile, updates the parameters of fenc as well as fdiag. By optimizing CancerSD, the missing omics data can be reliably imputed from available ones. Meanwhile, CancerSD extracts essential and meaningful information from incomplete multi-omics data of samples. As a result, it can make a more accurate and flexible cancer subtype diagnosis. Knowledge transfer module Cancer subtype diagnosis is a typical few-sample scenario where well-annotated cancer samples are challenging to collect. To cope with scarce training samples, meta-learning strategies emerge as promising solutions, which enable the backbone model to learn and adapt to new tasks with limited data rapidly. Among them, MAML [86] stands out as a renowned optimization-based [87] meta-learning algorithm. Its exceptional performance, flexibility, and model-agnostic nature make it widely applicable across various tasks and domains. However, the direct application of meta-learning strategies potentially leads to negative transfer issues when significant differences exist in sample distributions across different datasets. In such cases, knowledge learned from external datasets may fail to assist the model in adapting to the target dataset and even mislead its optimization. To address this issue, we aim to align the representations of samples of the same subtype and push away that of samples of different subtypes at the distribution level. Inspired by MAML and Eq 4, we construct a meta learner and formulate a distribution-based category-level contrastive loss to facilitate the desired distribution alignment during knowledge transfer. As illustrated in Fig 1c, given an external dataset and a target dataset (where and share the same label space ), the meta learner CancerSDm aims to learn a better initialization ψini for the backbone CancerSDb parameterized by ψ. In detail, CancerSDm first forms a set of sub-batches from the external dataset. Each sub-batch contains a support set and a query set , both of which are N-way K-shot (each subtype sample K patients). Then, CancerSDm undergoes a bi-level optimization procedure with two nested loops: an inner loop for learning sub-batch-specific knowledge and an outer loop for improving the model generalization capability based on multiple sub-batches. The two loops operate on a batch at each iteration, which is composed of Nsub related sub-batches [88]. Meanwhile, to facilitate the distribution alignment, CancerSDm uses a fine-tuning query set sampled from along with query sets from the current batch to calculate the category-level contrastive loss in the outer loop. In this step, is only involved in optimizing the encoder. Finally, we use to fine-tune the entire initialization ψini learned by CancerSDm and utilize the remaining data of the target dataset to evaluate CancerSDb characterized by these further refined parameters. Specifically, in the inner loop, CancerSDm changes ψini to sub-batch-specific for the t-th sub-batch by gradient descent on the support set as: (13) where α is the inner loop learning rate, represents the sub-batch-related training loss of the base learner (see Eq 12). In this process, CancerSDm separately acquires knowledge from each sub-batch. To further explore cross-sub-batch knowledge, CancerSDm comprehensively considers all sub-batches within and calculates the loss using query sets and fine-tuning query set to update ψini in the outer loop as: (14) where β is the outer loop learning rate. is a subset of , which represents the parameters in the patient feature encoder fenc. In addition, represents the category-level contrastive loss, which can be formulated as: (15) (16) where KDE(⋅) denotes Kernel Density Estimation [89], which is used to construct sample distribution. and are subsets of samples with subtype l. dist(⋅, ⋅) is used to measure the distance between two distributions, and we employ Jensen-Shannon divergence in this context, which offers advantages over the Kullback-Leible divergence here due to its symmetry and boundedness properties. When dealing with multiple external datasets, we abstain from considering relationships among them. Instead, we designate the target dataset as the anchor and calculate the contrastive loss individually between and each . In brief, the category-level contrastive loss allows CancerSDm to focus on extracting knowledge relevant to the target dataset from external datasets. In summary, CancerSDm inherits the merits of meta-learning. It leverages abundant samples available in external datasets to gain a better initialized CancerSDb, reduces the dependence on the quantity of training samples, and rapidly adapts to cancer subtype diagnosis tasks on the target dataset with limited samples. Experimental settings To evaluate the effectiveness of CancerSD, we evaluated and compared its diagnostic performance for cancer subtyping in both standard supervised learning and few-sample learning scenarios. On the one hand, we evaluated and compared various cancer subtype diagnostic methods under the standard supervised learning setting. Specifically, we first applied a random 80/20 split to each cancer dataset, where 80% of the samples were used for training and 20% for testing. To highlight the advantages of CancerSD, we then selected thirteen representative methods for comparison, covering wide-range popular and state-of-the-art approaches: (i) traditional machine learning, including k-Nearest Neighbor classifier (kNN) and Random Forest Classifier (RFC) [90]; (ii) Multi-omics Integration based methods, including AE-XGBoost [91], MOGONET [14], MOMA [15], MOFA+ [92], FactorCL [93], VICReg [94]; (iii) incomplete Multi-omics Integration based methods, including Subtype-GAN [95], scVAEIT [96], DCP [20], and APADC [21]. More detailed descriptions of these methods are provided in Section A in S2 Text. Among the comparison methods, kNN and RFC are trained with the direct concatenation of the preprocessed multi-omics data, while other methods explored effective integration of multi-omics data. It is worth noting that we included all comparative methods in the critical performance comparison experiments. However, considering the architecture or integration strategy similarities among these compared methods, we only deployed seven methods: KNN, RFC, AE-XGBoost, MOMA, MOGONET, DCP, and APADC in the subsequent experiments. On the other hand, we designed few-sample learning scenarios to investigate the cross-dataset knowledge transfer capability of CancerSD. Taking knowledge transfer tasks on GC datasets as examples, we conducted extensive experiments on the TCGA-STAD and GSE62254 datasets. In these experiments, STAD serves as the external dataset from which we sample N-way K-shot [97] sub-batches to optimize the models. Each sub-batch consists of a support set and a query set , both and include K samples for each of the N subtypes. Meanwhile, GSE62254 is treated as the target dataset, which is split into a training set containing one N-way K-shot set for fine-tuning and a testing set for evaluation. Since GSE62254 only contains mRNA data for samples, we conduct experiments to transfer knowledge from mRNA data and from multi-omics data of STAD to GSE62254. For a more comprehensive evaluation, we also conducted aforementioned experiments on lung cancer datasets (including TCGA-LUAD, TCGA-LUSC, and CPTAC [79, 80]). We selected four competitive approaches for comparison, including MOMA [15]-PT, DCP [20]-PT, QSFormer [98], and DeepBDC [99]. Among them, MOMA-PT and DCP-PT follow a pretraining strategy, where they first undergo pretraining on the external dataset and subsequently fine-tune themselves using a limited amount of samples from the target dataset. QSFormer and DeepBDC are few-shot classification methods based on the meta-learning framework. Moreover, five variants of CancerSD are developed for a more comprehensive evaluation, including (i) CancerSD-PT replaces the meta-learning framework with a pretraining strategy; (ii) CancerSD-MOMA replaces the CancerSD backbone with MOMA; (iii) CancerSD-DCP replaces the CancerSD backbone with DCP; (iv) CancerSD-SIM utilizes representation similarity in the category-level contrastive loss; (v) CancerSD-w/oCLC ignores sample distribution differences among different datasets. Each experiment randomly repeats ten times to take the average performance and standard deviations, where the diagnosis performance is measured in terms of Accuracy, AUROC, Precision, and average F1 Score weighted by the proportion of corresponding categories. Datasets To study the effectiveness of CancerSD, we apply it to subtype three representative cancers using benchmark datasets: STAD and GSE62254 for gastric cancer (GC) TCGA subtype [34] diagnosis; LUAD, LUSC and CPTAC [79, 80] for lung cancer subtype diagnosis; and BRCA for breast invasive carcinoma PAM50 subtype [81] diagnosis. Three types of omics data are employed for experiments, including DNA methylation profile, miRNA expression profile, and mRNA (protein-coding gene) expression profile. Among the aforementioned datasets, only samples with molecular subtype diagnoses are selected. An overview of these datasets is provided in Table A in S1 Text. Data preprocessing In the data preprocessing stage, we first apply a log2 transformation to the miRNA and mRNA expression data. Then, we filter out features with low variance [14] (the threshold set as 0.2 for the DNA methylation profile, 0.1 for the miRNA expression profile, and 0.8 for the mRNA expression profile). These variance thresholds are consistently used across all experiments. Additionally, we select specific molecular characteristics following the analysis configuration proposed by Hoadley et al. [82] and incorporate them into the filtered features. Consequently, we retain 3287 DNA methylation characteristics, 702 miRNA expression characteristics, and 4089 mRNA expression characteristics. Finally, we individually normalize miRNA and mRNA data to a range of [0, 1] (the original range of DNA methylation data is already within the interval [0, 1], where 0 represents a lower level of methylation, and 1 represents a high level of methylation). The shared patient feature encoder To integrate multi-omics data of patients, CancerSD constructs a module-shared patient feature encoder, which is capable of providing high-quality patient representations for both missing omics imputation and cancer subtype diagnosis. The encoder comprises M omics-specific feature extractors and a multi-omics fusion network. The omics-specific feature extractors are feedforward networks designed to reduce dimensionality and capture discriminative characteristics and patterns of the corresponding omics data as: (1) where is the embedded representation for the m-th omics data of the i-th patient. fm(⋅) corresponds to the feature extractor of the m-th omics. To capitalize on the advantages of multi-omics data, we employ LMF [83] (Low-rank Multimodal Fusion) and formulate our multi-omics fusion network for data integration. In comparison to alternative methods [14, 84], LMF stands out by its ability to explore cross-omics cooperation while retaining omics-specific information, allowing more informative integration. We first concatenate an extra constant value of 1 after , namely , which can prevent the loss of the original features of each omics. Next, the fusion sub-network performs the outer product operation on and transforms the resulting fused tensor into a low-dimensional space as: (2) where is a (M + 1)-order transformation tensor, along with a bias term , and dh is the dimensionality of patient embedded representations. denotes the tensor outer product operation. is a form of decomposition of W, and R is the number of low-rank tensors obtained after decomposition. By decomposing W to make a separate linear transformation for , we can significantly reduce the computational complexity of the multi-omics integration. denotes the Hadamard product of M vectors: . In this way, the module-shared patient feature encoder can be formalized as: (3) where fenc represents the patient feature encoder, is the multi-omics data for the i-th patient, and is a concatenation operator. In detail, we first split x i into M omics features, such as DNA methylation, miRNA, and mRNA features used here. Then, each omics data is fed into the corresponding feature extractor fm(⋅). Finally, the resulting omics embeddings are integrated as a fusion representation z i through the fusion sub-network. Incomplete data imputation module In real clinical scenarios, it is common to have access to only several or even a single type of omics data for some patients. The multi-omics data for these patients are incomplete, potentially leading to information loss and data bias, resulting in misleading diagnostic outcomes. To avoid risks arising from such incomplete data, we utilize available patient omics to impute their missing ones. To this end, we partition X into two parts to perform different training tasks. Specifically, Ncp samples with completely-paired multi-omics data, denoted as , are utilized to execute contrastive learning and masking-and-reconstruction (MAR) tasks. Meanwhile, Ninc samples with incomplete data are exclusively employed for the reconstruction tasks. Firstly, we devise contrastive learning tasks to enhance the representational capacity of information learned by CancerSD for incomplete multi-omics data, providing generators with more informative representations. These processes begin by randomly generating two distinct masking strategies, which are then applied to . For each patient, we can garner two different but correlated augmented views, denoted as and , which together constitute the positive pair. Subsequently, and are fed into the patient feature encoder (see Eq 3), yielding and . These patient representations are further mapped into a projection space, where the contrastive loss is applied. In this way, we obtain higher-level representations and for the augmented patient views. When taking as the input, a representation set can be assembled, encompassing 2Ncp augmented representations. Finally, we formulate contrastive learning tasks within the projection space with the objective of maximizing the agreement among different augmented views of the same sample. Specifically, given a positive pair , the remaining 2(Ncp − 1) representations in Su are treated as negative samples [85]. Thus, the loss of the pair can be formulated as: (4) where τ denotes a temperature parameter. represents the set without from Su. Consequently, the instance-level contrastive loss can be further calculated as: (5) Next, we proceed to construct M omics-specific generators based on feedforward networks to purposefully impute the missing omics data using latent features extracted from available patient omics as: (6) where is the generated omics data and gm(⋅) denotes the generator designed for the m-th omics. The binary variable indicates whether the m-th omics of the i-th patient is missing or not. To acquire the imputation ability of the generators, we define MAR tasks on patient features. On the one hand, we utilize Ncp samples with completely-paired multi-omics data to perform the MAR tasks. These tasks involve randomly masking out certain omics of some samples (setting all data within the masked omics to zero) and subsequently reconstructing these masked values. The loss can be calculated as: (7) where MSE(⋅, ⋅) denotes the mean square error loss function, and indicates whether the m-th omics is masked out or not for the i-th patient. On the other hand, for Ninc samples with incomplete data, we only consider the reconstruction loss pertaining to the existing omics as: (8) Finally, the generation loss can be calculated as: (9) Cancer subtype diagnosis module To alleviate adverse efforts posed by the absence of certain omics, CancerSD fuses both available and imputed omics data to make cancer subtype diagnosis and calculates the diagnosis loss as: (10) (11) where the representation z i is obtained by feeding the multi-omics data into the patient feature encoder fenc. In cases where all omics for a patient are available, we input them directly into the encoder; otherwise, we first impute the missing omics and then encode them. fdiag denotes the subtype diagnosis network, yi is the subtype label of the i-th patient, and CE(⋅, ⋅) represents the cross entropy loss function. is the weight of subtype yi when calculating the diagnosis loss, which is used to alleviate the problem of sample imbalance. Ntr is the number of samples in the training set, Ns is the number of subtypes, and is the number of samples of subtype yi. In a word, CancerSDb is an end-to-end architecture involving two modules that are optimized simultaneously in one stage, and the total loss can be calculated based on the individual loss of each module as: (12) where λ1, and λ2 are trade-off parameters among three individual losses. In practice, we set both of them to 1 by default. It is worth noting that although we integrate the losses from each module into , each loss affects different sub-networks during the backpropagation process. For instance, updates fenc solely and does not impact the other modules. optimizes both fenc and the omics-specific generator gm. Meanwhile, updates the parameters of fenc as well as fdiag. By optimizing CancerSD, the missing omics data can be reliably imputed from available ones. Meanwhile, CancerSD extracts essential and meaningful information from incomplete multi-omics data of samples. As a result, it can make a more accurate and flexible cancer subtype diagnosis. Knowledge transfer module Cancer subtype diagnosis is a typical few-sample scenario where well-annotated cancer samples are challenging to collect. To cope with scarce training samples, meta-learning strategies emerge as promising solutions, which enable the backbone model to learn and adapt to new tasks with limited data rapidly. Among them, MAML [86] stands out as a renowned optimization-based [87] meta-learning algorithm. Its exceptional performance, flexibility, and model-agnostic nature make it widely applicable across various tasks and domains. However, the direct application of meta-learning strategies potentially leads to negative transfer issues when significant differences exist in sample distributions across different datasets. In such cases, knowledge learned from external datasets may fail to assist the model in adapting to the target dataset and even mislead its optimization. To address this issue, we aim to align the representations of samples of the same subtype and push away that of samples of different subtypes at the distribution level. Inspired by MAML and Eq 4, we construct a meta learner and formulate a distribution-based category-level contrastive loss to facilitate the desired distribution alignment during knowledge transfer. As illustrated in Fig 1c, given an external dataset and a target dataset (where and share the same label space ), the meta learner CancerSDm aims to learn a better initialization ψini for the backbone CancerSDb parameterized by ψ. In detail, CancerSDm first forms a set of sub-batches from the external dataset. Each sub-batch contains a support set and a query set , both of which are N-way K-shot (each subtype sample K patients). Then, CancerSDm undergoes a bi-level optimization procedure with two nested loops: an inner loop for learning sub-batch-specific knowledge and an outer loop for improving the model generalization capability based on multiple sub-batches. The two loops operate on a batch at each iteration, which is composed of Nsub related sub-batches [88]. Meanwhile, to facilitate the distribution alignment, CancerSDm uses a fine-tuning query set sampled from along with query sets from the current batch to calculate the category-level contrastive loss in the outer loop. In this step, is only involved in optimizing the encoder. Finally, we use to fine-tune the entire initialization ψini learned by CancerSDm and utilize the remaining data of the target dataset to evaluate CancerSDb characterized by these further refined parameters. Specifically, in the inner loop, CancerSDm changes ψini to sub-batch-specific for the t-th sub-batch by gradient descent on the support set as: (13) where α is the inner loop learning rate, represents the sub-batch-related training loss of the base learner (see Eq 12). In this process, CancerSDm separately acquires knowledge from each sub-batch. To further explore cross-sub-batch knowledge, CancerSDm comprehensively considers all sub-batches within and calculates the loss using query sets and fine-tuning query set to update ψini in the outer loop as: (14) where β is the outer loop learning rate. is a subset of , which represents the parameters in the patient feature encoder fenc. In addition, represents the category-level contrastive loss, which can be formulated as: (15) (16) where KDE(⋅) denotes Kernel Density Estimation [89], which is used to construct sample distribution. and are subsets of samples with subtype l. dist(⋅, ⋅) is used to measure the distance between two distributions, and we employ Jensen-Shannon divergence in this context, which offers advantages over the Kullback-Leible divergence here due to its symmetry and boundedness properties. When dealing with multiple external datasets, we abstain from considering relationships among them. Instead, we designate the target dataset as the anchor and calculate the contrastive loss individually between and each . In brief, the category-level contrastive loss allows CancerSDm to focus on extracting knowledge relevant to the target dataset from external datasets. In summary, CancerSDm inherits the merits of meta-learning. It leverages abundant samples available in external datasets to gain a better initialized CancerSDb, reduces the dependence on the quantity of training samples, and rapidly adapts to cancer subtype diagnosis tasks on the target dataset with limited samples. Experimental settings To evaluate the effectiveness of CancerSD, we evaluated and compared its diagnostic performance for cancer subtyping in both standard supervised learning and few-sample learning scenarios. On the one hand, we evaluated and compared various cancer subtype diagnostic methods under the standard supervised learning setting. Specifically, we first applied a random 80/20 split to each cancer dataset, where 80% of the samples were used for training and 20% for testing. To highlight the advantages of CancerSD, we then selected thirteen representative methods for comparison, covering wide-range popular and state-of-the-art approaches: (i) traditional machine learning, including k-Nearest Neighbor classifier (kNN) and Random Forest Classifier (RFC) [90]; (ii) Multi-omics Integration based methods, including AE-XGBoost [91], MOGONET [14], MOMA [15], MOFA+ [92], FactorCL [93], VICReg [94]; (iii) incomplete Multi-omics Integration based methods, including Subtype-GAN [95], scVAEIT [96], DCP [20], and APADC [21]. More detailed descriptions of these methods are provided in Section A in S2 Text. Among the comparison methods, kNN and RFC are trained with the direct concatenation of the preprocessed multi-omics data, while other methods explored effective integration of multi-omics data. It is worth noting that we included all comparative methods in the critical performance comparison experiments. However, considering the architecture or integration strategy similarities among these compared methods, we only deployed seven methods: KNN, RFC, AE-XGBoost, MOMA, MOGONET, DCP, and APADC in the subsequent experiments. On the other hand, we designed few-sample learning scenarios to investigate the cross-dataset knowledge transfer capability of CancerSD. Taking knowledge transfer tasks on GC datasets as examples, we conducted extensive experiments on the TCGA-STAD and GSE62254 datasets. In these experiments, STAD serves as the external dataset from which we sample N-way K-shot [97] sub-batches to optimize the models. Each sub-batch consists of a support set and a query set , both and include K samples for each of the N subtypes. Meanwhile, GSE62254 is treated as the target dataset, which is split into a training set containing one N-way K-shot set for fine-tuning and a testing set for evaluation. Since GSE62254 only contains mRNA data for samples, we conduct experiments to transfer knowledge from mRNA data and from multi-omics data of STAD to GSE62254. For a more comprehensive evaluation, we also conducted aforementioned experiments on lung cancer datasets (including TCGA-LUAD, TCGA-LUSC, and CPTAC [79, 80]). We selected four competitive approaches for comparison, including MOMA [15]-PT, DCP [20]-PT, QSFormer [98], and DeepBDC [99]. Among them, MOMA-PT and DCP-PT follow a pretraining strategy, where they first undergo pretraining on the external dataset and subsequently fine-tune themselves using a limited amount of samples from the target dataset. QSFormer and DeepBDC are few-shot classification methods based on the meta-learning framework. Moreover, five variants of CancerSD are developed for a more comprehensive evaluation, including (i) CancerSD-PT replaces the meta-learning framework with a pretraining strategy; (ii) CancerSD-MOMA replaces the CancerSD backbone with MOMA; (iii) CancerSD-DCP replaces the CancerSD backbone with DCP; (iv) CancerSD-SIM utilizes representation similarity in the category-level contrastive loss; (v) CancerSD-w/oCLC ignores sample distribution differences among different datasets. Each experiment randomly repeats ten times to take the average performance and standard deviations, where the diagnosis performance is measured in terms of Accuracy, AUROC, Precision, and average F1 Score weighted by the proportion of corresponding categories. Supporting information S1 Fig. Detailed framework of CancerSD. (a) CancerSD is an end-to-end deep learning model for cancer subtype diagnosis using limited data with missingness. The initial phase introduces a multi-module shared patient feature encoder to integrate diverse omics data from samples. Then it constructs the imputation and diagnosis modules upon this encoder to perform cancer subtype diagnosis tasks. In addition, it designs a plug-and-play knowledge transfer module to acquire additional knowledge for these two modules in scenarios of scarce samples. Finally, a series of downstream analyses can be conducted based on the outcomes of CancerSD.(b) Incomplete data imputation module uses contrastive learning to extract cross-omics consistency features from available patient data and then feeds these features into the generator, facilitating the imputation of missing omics in samples.(c) Cancer subtype diagnosis module leverages available and imputed omics of samples to diagnose cancer subtypes.(d) Knowledge transfer module follows the meta-learning paradigm, it develops a meta learner and a category-level contrastive loss to mine domain-specific knowledge from external datasets and to initialize backbone network composed with the representation and diagnosis modules. https://doi.org/10.1371/journal.pcbi.1012710.s001 (TIF) S2 Fig. Actual subtype of patients and the corresponding misdiagnosed subtype. We collect samples that were misdiagnosed in ten repeated experiments and visualize both their true afflictions and the subtypes diagnosed by CancerSD. https://doi.org/10.1371/journal.pcbi.1012710.s002 (TIF) S3 Fig. Diagnostic performance of CancerSD with random initialization. We fix the dataset (STAD) split and randomly initialize the parameters in CancerSD, thereby evaluating the robustness of CancerSD. In the figure, the red line represents the mean of all experimental results (with ten random initializations for each of the ten random dataset splits, totaling 100 experiments), and the colored shaded area represents the mean±std. https://doi.org/10.1371/journal.pcbi.1012710.s003 (TIF) S4 Fig. Analysis of the use of different data augmentation. (a) F1 Score of CancerSD in gastric cancer subtype diagnosis task under combination of different data augmentation operations.(b) The similarity between features resulting from different data augmentation operations and the original features. The p-value indicates the significance of the difference (evaluated by Mann-Whitney U test) between similarities obtained from various operations and those from omics-level masking. https://doi.org/10.1371/journal.pcbi.1012710.s004 (TIF) S5 Fig. Hyper-parameters analysis. (a) Performance of CancerSD in gastric cancer subtype diagnosis tasks under different values of temperature factor τ.(b) Performance of CancerSD in gastric cancer subtype diagnosis tasks under different values of rank R.(c) Performance of CancerSD in gastric cancer subtype diagnosis task under different values of λ1 (weight for instance-level contrastive loss).(d) Performance of CancerSD in gastric cancer subtype diagnosis task under different values of λ2 (weight for the missing omics generation loss).(e) The impact of combining different values for τ and R (left), and for λ1 and λ2 (right) on CancerSD (F1 Score).(f) Sample clustering under different values of λ1. https://doi.org/10.1371/journal.pcbi.1012710.s005 (TIF) S6 Fig. Sample clustering on different datasets. The STAD and GSE62254 dataset are for gastric cancer molecular subtype classification with EBV, MSI, GS, and CIN subtypes. The ADSC and CPTAC datasets are for lung cancer classification with lung adenocarcinoma (LUAD) and lung squamous cell carcinoma (LUSC), where the CPTAC luad and CPTAC lusc represent samples of these two subtypes in the CPTAC dataset. The BRCA dataset is for breast invasive carcinoma PAM50 subtype classification with Luminal A, Liminal B, Basal-like, HER2-enriched, and Normal-like subtypes. https://doi.org/10.1371/journal.pcbi.1012710.s006 (TIF) S7 Fig. The imputation performance of CancerSD on the gastric cancer dataset (STAD). (a) Sample clustering under different scenarios.(b) Similarity between the original samples and the samples after simulating missingness and imputation.(c) Differentially expressed genes are obtained separately from the original mRNA data, following by Gene Ontology functional enrichment analysis.(d) Differentially expressed genes are obtained separately from the imputed mRNA data, following by Gene Ontology functional enrichment analysis. https://doi.org/10.1371/journal.pcbi.1012710.s007 (TIF) S8 Fig. The imputation performance of CancerSD on the lung cancer dataset (ADSC). (a) Sample clustering under different scenarios.(b) Similarity between the original samples and the samples after simulating missingness and imputation.(c) Differentially expressed genes are obtained separately from the original mRNA data, following by Gene Ontology functional enrichment analysis.(d) Differentially expressed genes are obtained separately from the imputed mRNA data, following by Gene Ontology functional enrichment analysis. https://doi.org/10.1371/journal.pcbi.1012710.s008 (TIF) S9 Fig. The imputation performance of CancerSD on the breast cancer dataset (BRCA). (a) Sample clustering under different scenarios.(b) Similarity between the original samples and the samples after simulating missingness and imputation.(c) Differentially expressed genes are obtained separately from the original mRNA data, following by Gene Ontology functional enrichment analysis.(d) Differentially expressed genes are obtained separately from the imputed mRNA data, following by Gene Ontology functional enrichment analysis. https://doi.org/10.1371/journal.pcbi.1012710.s009 (TIF) S10 Fig. The imputation performance of CancerSD on STAD dataset with missingness occuring in methylation data. (a) Mean Absolute Error (MAE) between original and imputed methylation data in the testing set at different missing rates, and corresponding Root Mean Square Error (RMSE) at each rate.(b) Similarity between original and imputed methylation data under various missing rates across different subtypes.(c) Sample clustering using the original methylation data.(d) Sample clustering using the imputed methylation data under different missing rates. https://doi.org/10.1371/journal.pcbi.1012710.s010 (TIF) S11 Fig. The imputation performance of CancerSD on STAD dataset with missingness occuring in miRNA data. (a) Mean Absolute Error (MAE) between original and imputed miRNA data in the testing set at different missing rates, and corresponding Root Mean Square Error (RMSE) at each rate.(b) Similarity between original and imputed miRNA data under various missing rates across different subtypes.(c) Sample clustering using the original miRNA data.(d) Sample clustering using the imputed miRNA data under different missing rates. https://doi.org/10.1371/journal.pcbi.1012710.s011 (TIF) S12 Fig. The imputation performance of CancerSD on STAD dataset with missingness occuring in mRNA data. (a) Mean Absolute Error (MAE) between original and imputed mRNA data in the testing set at different missing rates, and corresponding Root Mean Square Error (RMSE) at each rate.(b) Similarity between original and imputed mRNA data under various missing rates across different subtypes.(c) Sample clustering using the original mRNA data.(d) Sample clustering using the imputed mRNA data under different missing rates. https://doi.org/10.1371/journal.pcbi.1012710.s012 (TIF) S13 Fig. Efficiency of imputation algorithms. https://doi.org/10.1371/journal.pcbi.1012710.s013 (TIF) S14 Fig. The expression levels of mRNA and miRNA characteristics across different gastric cancer subtypes. The expression values subjected to log2 transformation and normalization. Wilcoxon rank-sum test is employed to evaluate the differences in the expression levels of specific molecules among patients of distinct subtypes. https://doi.org/10.1371/journal.pcbi.1012710.s014 (TIF) S15 Fig. The gene expression levels of CXCL10 and CXCL11 across different gastric cancer subtypes. The expression values subjected to log2 transformation and normalization. Wilcoxon rank-sum test is employed to evaluate the differences in the expression levels of specific molecules among patients of distinct subtypes. https://doi.org/10.1371/journal.pcbi.1012710.s015 (TIF) S16 Fig. Analysis of gene co-expression and KEGG pathway enrichment results in gastric cancer subtype of CIN, GS, and MSI. https://doi.org/10.1371/journal.pcbi.1012710.s016 (TIF) S17 Fig. Correlation of mRNAsi and expression levels of top-25 important genes identified by CancerSD. The regression lines in figures are fitted by the corresponding data. The significance in the figure is estimated by pearson correlation coefficient. https://doi.org/10.1371/journal.pcbi.1012710.s017 (TIF) S1 Text. Supplementary Tables. Tables A-V. https://doi.org/10.1371/journal.pcbi.1012710.s018 (PDF) S2 Text. Supplementary Discussions and Analyses. Sections A-J. https://doi.org/10.1371/journal.pcbi.1012710.s019 (PDF) Acknowledgments The work described in this paper was substantially supported by Shandong Provincial Key Research and Development Program to YG (NO. 2021CXGC010506) and National Natural Science Foundation of China (62072380 to JW, and 62272276 to JW). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Graham, Andrea L.;Regoes, Roland R.

doi: 10.1371/journal.pcbi.1012652pmid: 39642189

Introduction Following exposure to parasites (here defined broadly to include viruses, bacteria, fungi, protozoa, helminths, or ectoparasites), the dynamics of parasite growth and immune response induction are important determinants of infection success (i.e., the probability of establishing infection) as well as severity and chronicity of infection. For example, hosts that exhibit extremely rapid induction and decay of immune responses are expected to clear infection quickly while minimizing costs of defense [1]. However, the dose of parasites to which a host is exposed is likely to alter these within-host dynamics, both qualitatively and quantitatively: inoculating dose may determine whether infection is established at all [2–6] as well as the duration or severity of infections that do establish [7–10]. Quantitative analysis of dose-dependence in the establishment or mortality risk posed by a given infectious agent is therefore considered a crucial component of public health risk analysis [11]. General principles governing dose-dependence of infection and immunity merit formal investigation. A prevalent and useful conceptualisation of the quantitative process of infection is the hypothesis of independent action. According to this hypothesis, each individual parasite has the same small chance to initiate an infection, and this chance is independent of how many other parasites are present in the inoculum [12, 13]. Because each parasite can initiate an infection independently if the hypothesis of independent action holds, the cumulative infection probability is predicted to increase monotonically with the inoculum dose [13]. Many systems have been found to conform to the hypothesis of independent action, including classic work on Haemophilus influenzae in rats [14] (but see [15]), as well as diverse virus infections in plants [4], insects [3], and fish [5]). Deviations from the independent action hypothesis have also been conceived and documented. Halvorson [12], who first formulated the hypothesis of independent action, contrasted it with a hypothesis that a critical number of parasites is required for infection. Requiring a critical, or threshold, number is not consistent with parasite individuals acting independently. Rather it constitutes a form of synergy–e.g., when a quorum of bacteria in insects [16] or a quorum of bacteria [17] or protozoa [18] in mammals must cooperatively signal or differentiate to achieve or sustain infection. Mechanistically, such cooperativity could arise directly by signaling among individual parasites, or indirectly [15], for example as the result of the interaction between the parasite population and the immune system of the host, which is focal to this study. Another apparent deviation from independent action arises when hosts differ in susceptibility (e.g., due to variation among individuals in rates of immune response induction). In this case, the increase in infection success with inoculum dose is flatter than predicted by the hypothesis of independent action, and this slope can be used to quantify the variation in susceptibility among hosts in the population [19–24]. This apparent deviation does not require a lack of independence of parasites during infection, but arises from the population-level effects of susceptibility differences. Such explanations for the shape of the relationship between inoculum dose and infection success, though informative, do not fully account for the essence of the host-parasite interaction: e.g., the reciprocal quantitative dependence of immune response induction on parasite abundance and of parasite abundance on immune responses. Indeed, even though non-independent action has been invoked as a contrast to independent action, the interactive, mechanistic basis of dose-dependence has rarely been clarified [25]. A notable exception was provided by Pujol et al. [2], in which cooperativity arose as a consequence of the interaction between the parasite population and a single-tiered immune response: in their model, the parasite elicited an immune effector that in turn curbed its growth. Allowing such feedbacks generated complex relationships between inoculum dose and infection success that were consistent with data on poliovirus, among other infections [2]. The interval of time over which parasites were inoculated was crucial to their results, which is perhaps unsurprising, given the time-dependence of immune processes as well as parasite replication. We were therefore motivated to further explore within-host feedbacks and differential time signatures of different tiers of immune defense. More broadly, we were intrigued by the idea that, due to these feedbacks, infection outcomes could vary, and in the most extreme cases, be bimodal, leading to either immediate parasite clearance or persistent infection. Such bimodal infection outcomes could arise as a consequence of the interplay between stochasticity and within-host feedbacks. Alternatively, they could arise even without stochasticity in a manner similar to Allee effects, which are commonly observed in population ecology and conservation biology [26]. Allee effects arise when positive feedback loops generate ever-higher per-capita growth rates as population density escalates, leading to persistence thresholds for the population (e.g., [27]): below the density threshold, the population will go extinct; above it, the population will persist. Such rules of population ecology could apply to parasite populations growing (or going extinct) within hosts, just as for frog or orchid populations growing (or going extinct) within a forest. Here, we formulate mathematical models to investigate how the inoculating dose of parasites interacts with multi-tiered immune defenses to determine duration of infection, including whether or not infection establishes at all (i.e., if infection fails to establish, the duration is arguably zero). Like Pujol et al. [2] and a few others (e.g., [28–30]), we thus go beyond the simplification of the classic independent action hypothesis, and mathematically capture more realistic interplay between the parasites and the immune system within the host. The range of inoculating doses that we consider spans many orders of magnitude, because the range of infective doses empirically observed–for example, across various bacterial infections of human hosts–is vast (>6 orders of magnitude) [31], and because we aim to uncover general rules of within-host engagement between any type of parasite and multiple immune components, rather than using a model to explain dose dependence in any one particular infectious disease across a narrower range. We consider that not only do parasites induce immune responses (roughly according to “mass action” of the rates of encounter between parasites and immune cells [32]), but also that different immune system components differ in how and when they are triggered, when they act, and how effective they are (following, e.g., [33, 34]). Specifically, we model host defenses that are three-tiered, with barrier, innate, and adaptive defenses, as described in mammals (e.g., [35, 36]), but with likely analogues, if not homologues, among other animals as well as plants. These three tiers have distinct temporal properties. First, barrier defenses are constitutive (like skin) or continuously maintained at steady-state levels (like mucosal IgA antibodies exhibiting cross-(microbial)-species reactivity [37]). Such defenses are immediately ready upon exposure but can be overcome or eroded at sufficiently high inoculating doses [38]. Next, non-specific innate defenses such as macrophages consuming microbes are rapidly induced (within hours) and subject to handling time saturation (such that large numbers of parasites can overwhelm them [39, 40]). Finally, specific adaptive defenses like killer T cell or antibody responses are induced more slowly (days to weeks [36]) but are capable of achieving extremely high concentrations that might conquer large numbers of parasites. While these defenses can also have distinct spatial aspects–e.g., local expression of barriers and innate defenses versus more systemic distribution of adaptive defenses–we, for the most part, ignore the spatial aspects in this initial model (aside from in our robustness analysis reported in S1 Code). Our model of how inoculating dose interacts with these three tiers of defense predicts relationships for the establishment and chronicity of infection with inoculum dose that are inconsistent with the hypothesis of independent action. The inconsistencies we report are more dramatic than in previous studies: not only do we predict that infection success and duration do not increase linearly with dose, we also find profiles with multiple peaks at intermediate parasite doses. The prediction of multiple peaks in the dose-dependence of infection success and duration, if empirically confirmed, is likely to be exacerbated by spatial compartmentalization of induced immune responses, especially for localized infections. If infection duration doesn’t generally increase monotonically with inoculum dose, this would have important implications for the estimation of host susceptibility, for epidemiological dynamics, and for the evolution of strategies for both attack and defense, as we outline in the Discussion. Methods Model definition We developed a simple model that describes the growth of the parasite population within the host, and its inhibition by the immune system. Here, we assume no anatomical structure within the host, so, after inoculation, the population dynamics unfold in a single, well-mixed compartment. Because induced innate and adaptive immune effectors migrate to the site of infection, we need not invoke anatomical complexity to study how parasites interact with these three tiers of defense. Let, P denote the number of parasites in this compartment. Parasite population dynamics are described by the following differential equation: (1) Hereby η(t) is a time-dependent function that captures the exposure of the host. The integral over this function, , is the total number of parasites that enter the host, either through natural exposure or experimental inoculation. Because, in this study, we are focusing on the implications of dosing on the within host dynamics, this function is central. The form of the function η(t) allows very flexible dosing schedules. Inoculations with a single dose (“bolus”) can be described by function with a high short peak for a short duration, where the height multiplied by the duration gives the overall dose. Alternatively, continuous (“trickle”) or repeated inoculations can also be described. Once the host has been seeded with parasites, they start to replicate exponentially at a rate rP and are inhibited by three immune effectors, the number of which we denote by EA, EB, and EC and the efficacy of which we denote by γA, γB, and γC. These three immune effector tiers, rather than describing specific actors that can differ among infections or host types, cover the relevant dynamical range of immune responses. In particular, we consider a constitutively expressed barrier effector EA, and two inducible effectors, EB and EC, that differ in the speed at which they are induced. The slower one, EC, is ultimately more effective (i.e., gamma C is larger than gamma B). The dynamics of these immune effectors is governed by the following equations: (2) (3) (4) EA is constitutively expressed at the level EA(0). This effector is depleted when it kills the parasite at a rate γAPEA. It is replenished at a maximum rate σA. The other two effectors. EB and EC, are induced at rates that depend on the parasite load P. At low parasite loads, P and low levels of EB and EC, they are induced at exponential rates σBP and σCP, respectively. The parameters hB or hC denote the parasite loads at which these two rates are half the maximum. The induction rate goes to zero if the levels of the immune effectors reach their respective carrying capacities and . Unlike EA, EB and EC are not assumed to be decimated by killing parasites. The dynamics of EB and EC are structurally identical, but they differ in terms of the speed of induction (σB and σC) and maximum efficacy ( and ) Model parameterization and scenarios The model is parameterized generically. This means that, rather than simulating any specific infection, we choose our parameters such that they recapitulate the typical time scales of the generation of the three tiers of immune responses upon exposure. In the first instance, we assumed that, following arrival at an array of inoculating doses, the parasite replicates at a per capita rate of 1 per day, which corresponds to a doubling time of approximately 0.7 days, or a 10-fold increase in approximately 2.3 days. Parasite growth is not assumed to be limited by a carrying capacity. When we instead simulate macroparasitic infection, we assume that parasites enter the hosts at various numbers, but do not replicate. We also ran simulations for singular ("bolus") or continuous ("trickle") exposure to both micro- and macroparasites. The first, barrier tier of the immune system is constitutively expressed at the arbitrarily chosen level of 100 effectors, EA. As for mucosal IgA antibodies, by exerting their effect, they are depleted, and at a rate that depends on the inoculum size, leading to either an immediate depletion of the effectors when facing a large inoculum dose, or to a fast clearance of a small inoculum. Once the parasites are cleared, these first-tier effectors grow back to the initial level of 100 at an initial rate of σA = 30 per day. The rate is reduced linearly until the homeostatic level is reached. The second tier of the immune system, EB, is conceived as innate immune components that are initially set to a single effector. They are stimulated by parasites that overcome the first-tier response to proliferate/recruit at a logistic rate that depends on the parasite load. A single second-tier effector is assumed to be slightly less effective than a single first-tier effector. These second tier effectors can reach a carrying capacity of 10’000. At this level they are still assumed to be 20% less potent than the barrier response at its constitutively expressed level, to capture handling time constraints for effectors such as macrophages [39, 40] compared to mucosal antibodies at the barrier [41, 42]. Unlike the first-tier effectors, however, second-tier effectors are not depleted by exerting their effect. Thus, the second tier is effective across a larger range of parasite doses. The third tier of the immune system, EC, is conceived as adaptive immune effectors that are assumed to differ from the second tier only quantitatively, in terms of both a slower induction rate and higher effector efficacy. They are elicited to proliferate by the parasites that overcome the first-tier responses at a rate that is two orders of magnitude lower than the induction rate of the second tier. Their per-effector potency to clear parasites is assumed to be the same as that of the first-tier effectors. Their carrying capacity is set to 106 but in practice they stop proliferating before that cap because they typically lead to fast parasite clearance. In these model formulations for the innate and adaptive tiers of defense, we assume a dependence of the induced immune responses on the dose of antigen. For both innate and adaptive responses, there is clear evidence for such dose-dependence. For example, the magnitude of CD8+ T cell responses in mice is affected by the dose of the antigen (e.g., see Fig 1 in [43]). The magnitude of antibody responses are also clearly affected by dose. A prominent case in point comes from the phase I/II trials for the mRNA vaccines against COVID that showed a clear dose-dependence of the antibody response (e.g., see [44] and [45]; in this context, we refer to the dose of the vaccine antigen rather than of the whole parasite, but those are immunologically similar in terms of quantitative effects of increasing antigen exposure on the induced immune response). The magnitude of some innate responses also depends on the antigen dose [46], as reflected in the implementations of innate immunity in previous studies [30, 47] and in our model. Such evidence is the basis for our assumption that parasite load affects the quantity of induced immune effectors. Interestingly, it has been established that the dynamics of the induced response for some adaptive immune components can unfold according to an intrinsic, parasite-load-independent program after initial activation [48]. Our implementation of the adaptive response does not take this program into account, but dose-dependence of the different stages of induced lymphocyte responses (from activation to proliferation) would be an interesting avenue to explore in future work. We deviate from our default parameterization to investigate the infection dynamics in immune-tier “knockout" hosts. These tier "knockouts" are implemented by setting the initial concentration of the respective immune effectors, EA, EB, or EC, to zero. This procedure mirrors knocking out genes responsible for various immune components in mice in experimental immunology [49], but is, due to our conceptual approach, more generic and cleaner because it is without pleiotropic effects on other traits. Model implementation We implemented the deterministic population dynamical model in Eqs 1–4 in the R language for statistical computing [50] using the function lsoda in the package deSolve [51]. We also implemented a stochastic version of the model using the implementation of the Gillespie algorithm in the R-package adaptivetau [52] that implements the adaptive tau-leaping approximation for simulating the trajectory of a continuous-time Markov [53]. For the stochastic implementation, the logistic growth terms in Eqs 3 and 4 have been partitioned into two terms, on describing the population expansion ( and ), the other describing death due to crowding ( and ). (This partitioning of the growth terms is generally required for the stochastic implementation of logistic growth to prevent the population turnover to be zero at the carrying capacity.) For all details on our simulation models, the parameterization, and the alternative, more complex model we used to assess the robustness of our inferences, please refer to code and results that are provided in S1 Code. Model definition We developed a simple model that describes the growth of the parasite population within the host, and its inhibition by the immune system. Here, we assume no anatomical structure within the host, so, after inoculation, the population dynamics unfold in a single, well-mixed compartment. Because induced innate and adaptive immune effectors migrate to the site of infection, we need not invoke anatomical complexity to study how parasites interact with these three tiers of defense. Let, P denote the number of parasites in this compartment. Parasite population dynamics are described by the following differential equation: (1) Hereby η(t) is a time-dependent function that captures the exposure of the host. The integral over this function, , is the total number of parasites that enter the host, either through natural exposure or experimental inoculation. Because, in this study, we are focusing on the implications of dosing on the within host dynamics, this function is central. The form of the function η(t) allows very flexible dosing schedules. Inoculations with a single dose (“bolus”) can be described by function with a high short peak for a short duration, where the height multiplied by the duration gives the overall dose. Alternatively, continuous (“trickle”) or repeated inoculations can also be described. Once the host has been seeded with parasites, they start to replicate exponentially at a rate rP and are inhibited by three immune effectors, the number of which we denote by EA, EB, and EC and the efficacy of which we denote by γA, γB, and γC. These three immune effector tiers, rather than describing specific actors that can differ among infections or host types, cover the relevant dynamical range of immune responses. In particular, we consider a constitutively expressed barrier effector EA, and two inducible effectors, EB and EC, that differ in the speed at which they are induced. The slower one, EC, is ultimately more effective (i.e., gamma C is larger than gamma B). The dynamics of these immune effectors is governed by the following equations: (2) (3) (4) EA is constitutively expressed at the level EA(0). This effector is depleted when it kills the parasite at a rate γAPEA. It is replenished at a maximum rate σA. The other two effectors. EB and EC, are induced at rates that depend on the parasite load P. At low parasite loads, P and low levels of EB and EC, they are induced at exponential rates σBP and σCP, respectively. The parameters hB or hC denote the parasite loads at which these two rates are half the maximum. The induction rate goes to zero if the levels of the immune effectors reach their respective carrying capacities and . Unlike EA, EB and EC are not assumed to be decimated by killing parasites. The dynamics of EB and EC are structurally identical, but they differ in terms of the speed of induction (σB and σC) and maximum efficacy ( and ) Model parameterization and scenarios The model is parameterized generically. This means that, rather than simulating any specific infection, we choose our parameters such that they recapitulate the typical time scales of the generation of the three tiers of immune responses upon exposure. In the first instance, we assumed that, following arrival at an array of inoculating doses, the parasite replicates at a per capita rate of 1 per day, which corresponds to a doubling time of approximately 0.7 days, or a 10-fold increase in approximately 2.3 days. Parasite growth is not assumed to be limited by a carrying capacity. When we instead simulate macroparasitic infection, we assume that parasites enter the hosts at various numbers, but do not replicate. We also ran simulations for singular ("bolus") or continuous ("trickle") exposure to both micro- and macroparasites. The first, barrier tier of the immune system is constitutively expressed at the arbitrarily chosen level of 100 effectors, EA. As for mucosal IgA antibodies, by exerting their effect, they are depleted, and at a rate that depends on the inoculum size, leading to either an immediate depletion of the effectors when facing a large inoculum dose, or to a fast clearance of a small inoculum. Once the parasites are cleared, these first-tier effectors grow back to the initial level of 100 at an initial rate of σA = 30 per day. The rate is reduced linearly until the homeostatic level is reached. The second tier of the immune system, EB, is conceived as innate immune components that are initially set to a single effector. They are stimulated by parasites that overcome the first-tier response to proliferate/recruit at a logistic rate that depends on the parasite load. A single second-tier effector is assumed to be slightly less effective than a single first-tier effector. These second tier effectors can reach a carrying capacity of 10’000. At this level they are still assumed to be 20% less potent than the barrier response at its constitutively expressed level, to capture handling time constraints for effectors such as macrophages [39, 40] compared to mucosal antibodies at the barrier [41, 42]. Unlike the first-tier effectors, however, second-tier effectors are not depleted by exerting their effect. Thus, the second tier is effective across a larger range of parasite doses. The third tier of the immune system, EC, is conceived as adaptive immune effectors that are assumed to differ from the second tier only quantitatively, in terms of both a slower induction rate and higher effector efficacy. They are elicited to proliferate by the parasites that overcome the first-tier responses at a rate that is two orders of magnitude lower than the induction rate of the second tier. Their per-effector potency to clear parasites is assumed to be the same as that of the first-tier effectors. Their carrying capacity is set to 106 but in practice they stop proliferating before that cap because they typically lead to fast parasite clearance. In these model formulations for the innate and adaptive tiers of defense, we assume a dependence of the induced immune responses on the dose of antigen. For both innate and adaptive responses, there is clear evidence for such dose-dependence. For example, the magnitude of CD8+ T cell responses in mice is affected by the dose of the antigen (e.g., see Fig 1 in [43]). The magnitude of antibody responses are also clearly affected by dose. A prominent case in point comes from the phase I/II trials for the mRNA vaccines against COVID that showed a clear dose-dependence of the antibody response (e.g., see [44] and [45]; in this context, we refer to the dose of the vaccine antigen rather than of the whole parasite, but those are immunologically similar in terms of quantitative effects of increasing antigen exposure on the induced immune response). The magnitude of some innate responses also depends on the antigen dose [46], as reflected in the implementations of innate immunity in previous studies [30, 47] and in our model. Such evidence is the basis for our assumption that parasite load affects the quantity of induced immune effectors. Interestingly, it has been established that the dynamics of the induced response for some adaptive immune components can unfold according to an intrinsic, parasite-load-independent program after initial activation [48]. Our implementation of the adaptive response does not take this program into account, but dose-dependence of the different stages of induced lymphocyte responses (from activation to proliferation) would be an interesting avenue to explore in future work. We deviate from our default parameterization to investigate the infection dynamics in immune-tier “knockout" hosts. These tier "knockouts" are implemented by setting the initial concentration of the respective immune effectors, EA, EB, or EC, to zero. This procedure mirrors knocking out genes responsible for various immune components in mice in experimental immunology [49], but is, due to our conceptual approach, more generic and cleaner because it is without pleiotropic effects on other traits. Model implementation We implemented the deterministic population dynamical model in Eqs 1–4 in the R language for statistical computing [50] using the function lsoda in the package deSolve [51]. We also implemented a stochastic version of the model using the implementation of the Gillespie algorithm in the R-package adaptivetau [52] that implements the adaptive tau-leaping approximation for simulating the trajectory of a continuous-time Markov [53]. For the stochastic implementation, the logistic growth terms in Eqs 3 and 4 have been partitioned into two terms, on describing the population expansion ( and ), the other describing death due to crowding ( and ). (This partitioning of the growth terms is generally required for the stochastic implementation of logistic growth to prevent the population turnover to be zero at the carrying capacity.) For all details on our simulation models, the parameterization, and the alternative, more complex model we used to assess the robustness of our inferences, please refer to code and results that are provided in S1 Code. Results We studied the dose-dependence of immune response induction and within-host dynamics of infection using a mathematical model. The model describes how the parasite invades the host, replicates in it, and is confronted with three tiers of defense–roughly corresponding to the constitutively maintained barrier, the induced innate, and the induced adaptive tiers. Initially, we ran deterministic model simulations varying the inoculum size (dose) across four orders of magnitude. Parasites and the three immune components exhibited diverse dose-dependent dynamics (Fig 1), as follows. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. (A) Time courses of parasite load (top row) and immune effectors of all three tiers (second, third, and fourth rows, corresponding to barrier, innate, and adaptive tiers, respectively) for varying inoculum sizes (doses). Intermediate inoculum sizes cause the longest infections due to how they interact with the series of defense tiers. The parasite loads in the top row start out at 0 and ramp up quickly to the inoculum dose within one hour, consistent with our implementation of the bolus inoculation (see S1 Code). (B) Time-course of parasite load for the varying doses displayed as a contour plot. (These are the same simulation data as shown in (A)). In this contour plot, it is more clearly visible that infection duration (in days since inoculation, on the y-axis) is longest at doses 20 and 2000, even if peak parasite load (colored according to heatmap P, at right) is highest in hosts receiving the largest dose. These simulations are based on the default parameterization of our model (see Model Parameterization and Scenarios as well as S1 Code.) (C) Infection duration versus inoculating dose for varying strengths of the first- and second-tier immune responses. The efficacy parameters of the first and second tier defenses, γA/γB, have been scaled to keep their relative efficacies constant. The two peaks of infection duration are at lower doses for weaker immune responses, and for very weak responses they disappear completely, showing that the peaks in the relationship between duration and inoculum size are determined by the strength of first- and second-tier defenses. https://doi.org/10.1371/journal.pcbi.1012652.g001 The dynamics of parasites (Fig 1A) were non-monotonic: very low doses generated low peak parasite densities and short-lived infections; at higher doses, parasites persisted longer, with a peak in duration at the inoculum dose 23; at yet higher doses, infection duration decreased approximately two-fold, only to increase and peak again at an inoculum dose of 2000; beyond that, infection duration decreased again (Fig 1B). We also found that the multi-peaked relationship between inoculating dose and duration was robust to variation in the efficacy of barrier and induced defenses, with more potent defenses simply reducing the height of the peaks (Fig 1C). We also tracked immune effector dynamics separately for the three tiers of defense. The effectors of the barrier defense (EA in Fig 1A) exhibited a depletion that depended on the size of the inoculum. For inoculum sizes of 2000 and larger, this depletion was complete. By contrast, dynamics of the rapidly inducible (innate-like) and slowly-inducible (adaptive-like) defenses varied in a more straight-forward fashion with dose. Innate immune effectors (EB in Fig 1A) were induced increasingly rapidly with increasing dose, up to a maximum level of defense. Intermediate doses thereby induced middling innate defenses. Adaptive immune effectors (EC in Fig 1A) likewise exhibited straightforward dose dependence: both rates of induction and peaks of adaptive immune effectors increased monotonically with increasing parasite dose. Together, these results suggest that intermediate doses achieved long duration of infection by escaping the first or second tier defenses and then inducing little further defense. In this way, such doses found a “wormhole” through 3-tiered host defense systems. (Hereby, we are referring to a as described in the movie “Stargate,”as a hypothesized portal through the space/time continuum, rather than a hole in old cupboards.) We next systematically studied the peak parasite load and infection duration within 4 host types: “wild type” hosts with all 3 lines of defense intact; “barrier knockout” hosts lacking the first tier defense; “innate knockout” hosts lacking the rapidly inducible tier of defense; and “adaptive knockout” hosts lacking the slowly inducible tier of defense (Fig 2). To obtain sufficient resolution, we studied dose variation across six orders of magnitudes of inoculum size in each of these hosts. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Peak parasite load (top row), infection duration (middle row), and cumulative parasite load (AUC, or area under the curve; bottom row) statistics for deterministic simulations across a range of inoculum sizes in 4 host types: (A) wildtype (“default”), (B) barrier knockouts (“EAko”), (C) second-tier (innate) knockouts (“EBko”), and (D) third-tier (adaptive) knockouts (“ECko”). Note the non-monotonic profiles of infection duration, especially in wildtype or innate-knockout hosts ("EBko”). https://doi.org/10.1371/journal.pcbi.1012652.g002 In wild-type “default” hosts (Fig 2A), we found that peak parasite load (top row) exhibited straightforward dose-dependence, increasing gradually with increasing dose. The duration of infection (second row), in contrast, exhibited non-monotonic dose dependence. Most interestingly, infection duration peaked twice across dose, as described above. The cumulative parasite load (third row) shows the same pattern as the peak parasite load. Thus, the non-monotonic pattern in duration is over-ridden by increases in peak parasite load to generate increases in cumulative parasite load with increasing dose. The three knockout host types provide insight into the causes of these patterns. Generally, for all types of knockouts (Fig 2B–2D), the average duration of infections becomes longer across the range of doses compared to wild-type (see y-axis ranges). Barrier knockout hosts exhibited simpler dose dependence than the wild-types, with peak parasite load (top row) increasing with increasing dose (Fig 2B). Most notably, however, infection duration (second row) did not display two peaks anymore but continuously decreased with dose. This decrease became more pronounced beyond a inoculation dose of approximately 500, echoing the second peak in the duration profile in the wild-type hosts (Fig 2A), while the first peak had disappeared. This change of pattern can be explained by the fact that knocking out barrier defenses deletes the transition between first and second tier that is responsible for the first peak of the duration at low inoculum doses. Knocking out innate responses (second tier), in contrast, makes the second peak in infection duration (second row) at high inoculum doses disappear (Fig 2C). The profiles of peak parasite load (top row) did not increase gradually anymore, but displayed jumps at an inoculum dose of 20, at which barrier responses are overcome and adaptive immune responses are triggered. The patterns in adaptive immune knockouts are similar to those in innate immune knockouts: the second peak in the profile of infection duration versus dose (second row) disappears (Fig 2D). Unlike in innate immune knockouts, however, the infection duration does not continuously decrease with further dose escalation, but instead turns around and increases again. This difference in the patterns between innate versus adaptive immune knockouts may be due to the lower potency of innate compared to adaptive immune effectors in our model. These results show the link between the peaks in the profile of infection duration and tier transitions, and support a central role for tier transitions in explaining why intermediate doses of parasites lead to infections that persist longest. In all cases, cumulative parasite load exhibited the same monotonic dose-dependence as peak parasite load. Our final set of analyses of the deterministic system of equations aimed to assess the generality of the dose-dependent dynamics (Fig 1) and summary statistics (Fig 2) described thus far. We compared microparasites versus macroparasites (which, by definition, are non-replicating within a given host), and different patterns of exposure (a bit each day in a “trickle” versus all at once in a “bolus”). For the macroparasite bolus exposures, the qualitative conclusion that duration depends non-monotonically on dose was the same as for microparasites, but details differed: e.g., intermediate doses generated a local, if not global, maximum for duration of infection even in barrier and innate response knockouts (Fig 3); the macroparasite dose-dependence is remarkably parallel to the microparasite dose-dependence in this system. Trickle exposures induced complexities, especially when parasites continued to arrive after considerable immune response induction and in hosts with a tier of defense knocked out (S1 and S2 Figs). In no case was duration of infection monotonically associated with dose. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Peak parasite load (top row), infection duration (middle row), and cumulative parasite load (AUC, or area under the curve; bottom row) statistics for deterministic simulations across a range of inoculum sizes in 4 host types: (A) wildtype (“default”), (B) barrier knockouts (“EAko”), (C) second-tier knockouts (“EBko”), and (D) third-tier knockouts (“ECko”). Here, all parasites are non-replicating macroparasites, shorthanded here as worms, that arrive all at once in a bolus inoculation. https://doi.org/10.1371/journal.pcbi.1012652.g003 We next undertook stochastic simulations, to investigate how random variation in the processes hypothesized in our model would impact outcomes. We particularly focused on the response variable of infection duration, and we used finer-grain variation in inoculum size than before, but again across four orders of magnitude of dose. Consistent with the deterministic results, the stochastic results show non-monotonic dose dependence of infection duration with two peaks and tremendous variance associated with overcoming the first two tiers of defense. Again, doses intermediate between the extremes often generated the most persistent infections (Fig 4). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Infection duration versus inoculum size in stochastic simulations of the model, again showing non-monotonic changes in infection duration with increasing dose. Note that inoculum sizes of ~50 that overcome the first-tier responses lead to less variable durations than inoculum sizes of 20. https://doi.org/10.1371/journal.pcbi.1012652.g004 Finally, we drew upon our stochastic formulation to investigate how infection success depends upon inoculum size. We yet again observed non-monotonic, two-peaked dose-dependence, but time of observation emerged as crucial to the inference, as follows. Observed early after exposure, infection success appears to have a sigmoidal relationship to dose, with increasing success with dose up until a first peak. Observed later after exposure, hosts receiving high doses and thus more stimulation of inducible defense began to clear their infections, which generated a double-peaked relationship between dose and infection success (Fig 5). In other words, even infection success has two peaks at different intermediate doses, but detecting that relationship depends upon the timepoint at which hosts are observed and thus declared infected or not. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Infection success against inoculum size and time of observation. As expected, infection success increases overall with the inoculum size, and is higher when observed sooner. Note, however, the non-monotonic, double-peaked profile of infection success versus inoculum for observation times of 10–15 days. https://doi.org/10.1371/journal.pcbi.1012652.g005 Further simulations revealed that the stochastic version of our mathematical model can display bimodal variation in infection dynamics, such as those observed previously [6, 33]. S3 Fig shows simulations of the time course of the parasite population in hosts, in which the second-tier innate responses are assumed to be knocked out. At the inoculum size that breaches the barrier response, the time courses are extremely variable, leading to bi-modal infection duration distributions. Finally, stochastic simulations of the model also revealed that, in wild-type (i.e., immunologically intact) hosts, both peak and cumulative parasite load escalate monotonically with increasing inoculum size (S4 and S5 Figs). In a final suite of both deterministic and stochastic models, we investigated the robustness of our findings to the particular implementation of the three immune response tiers in our model. To this end, we developed an alternative model (see S1 Code). In the alternative model, we added an exterior compartment to allow the barrier defense to act in its own spatial compartment, we assumed alternative stimulation terms of the innate defense (drawing upon the approach of [47] and [30]), and we linked the stimulation of the adaptive defense to the innate defense, leading to a clearer separation in the timing of these two immune response tiers. In each case, the qualitative patterns we observed using the original, simpler model above held: dose-dependence in duration and infection success were in all cases non-monotonic (e.g., see S6 and S7 Figs). Discussion Immunoepidemiology links the dynamics of parasites within the host to their epidemiological spread. The two most essential parameters at this interface between immunology and epidemiology are exposure and susceptibility of hosts. Complicating immunoepidemiological analysis, however, is the fact that exposure and susceptibility are not independent of each other. Indeed, the number of parasites (or parasites per unit time [2]) to which hosts are exposed (here, inoculating dose) may in fact determine how susceptible the host is to infection per se [19–24], as well as susceptibility to chronic infection or severe symptoms [7, 8, 54–56]. Studies of dose-dependence are therefore of applied relevance to risk analysis [11]. In our view, dose-dependence studies are also at the very heart of immunoepidemiology, because they enable testing of basic science hypotheses such as the independent action of microbes [12, 13] or the optimal blend of constitutive and inducible defenses [57]. The value of studies of dose is greatly enhanced when physiological mechanisms are incorporated [25], as we have attempted here, with coarse but realistic incorporation of three tiers of host defense meeting varied inoculating doses. Using a deterministic model that considers immune responses dynamically, we found that both the establishment and duration of infection exhibited complex dose-dependence. Two intermediate inoculating doses generated infections of longest duration, while very low and very high doses led to infections of shorter duration. We found that this is explained by certain doses being sufficient to clear the first and even the second tier defenses but then only slightly/slowly inducing subsequent tiers of effectors. This pattern is supported in at least some empirical systems (e.g., dose-dependent induction and efficacy of immune defenses by malaria parasites [40, 58]). Similarly, vaccine-induced immune responses are maximal at intermediate doses of the attenuated parasite strains included in vaccines [30]. Our findings thus suggest that parasites arriving at intermediate doses may be more likely to establish and persist in a host. Simulated knockout of barrier or innate defenses turned the complex dose dependence with two peaks into simpler form, usually with a single peak of chronic infection associated with intermediate doses. Our stochastic simulations revealed threshold doses of approximately 23 and 2000 (for the parameters used) around which duration peaked in many cases. This corresponds to the dose necessary to clear either barrier or innate defenses, and the variation suggests within-host feedbacks and stochasticity leading to variation in trajectories of infection. We note that the hypothesis of independent action [12] generates a very particular, monotonically-increasing profile of infection success versus inoculum dose. Such monotonic escalation of infection success or mortality rate with inoculating dose is empirically supported in various host-parasite systems (e.g., [3–5]) and provides important public health risk assessment tools [11], as cataloged on the Quantitative Microbial Risk Assessment Wiki (QMRI). Yet there are exceptions: for example, mortality risks associated with Naegleria fowleri and nontyphoid Salmonella typhimurium infections escalate more rapidly with dose than the considered monotonic models of QMRI can capture. In the case of S. typhimurium, the dose-response data appear downright non-monotonic. Such exceptions could be due to experimental uncertainty, but may also be taken as motivation to use mathematical tools to explore non-monotonic relationships more systematically, as we have done here. In any case, the results of our analysis are inconsistent with the generality of the hypothesis of independent action. We go beyond other studies that conceptualize deviations from the independent action hypothesis [2, 15, 29] by revealing rules of within-host ecology that generate non-monotonic profiles of infection success and duration with escalating dose. Furthermore, our model suggests that the “one is enough” corollary of the independent action hypothesis, in which a single parasite has a considerable chance of establishing infection [59], is not general and may depend upon the absence of key defenses. Importantly, our model assumed no cooperativity to the parasites themselves. Instead, an “apparent cooperativity" arose due to dynamical interactions of the parasite population with each of the tiers of host defense. Indeed, our central finding is that the first and second tiers of immune defenses are crucial to both the non-independent action of parasites and chronicity-permitting “wormholes” in host defense. It is not new to suggest that constitutive defenses are as important as inducible in explaining biotic interactions [60] including when defenses against infection incur costs (e.g., [57, 61]). What is new here is the inference that this suite of defenses may be what confers complex dose-dependence. Others have emphasized the importance of immune response growth terms that are independent of parasite load [28, 33, 34], and we extend this to barrier defense biology which may, in general, be central to predicting the probability and course of infection when host exposure varies. We find that these conclusions are robust to various alternative model formulations that make more complex and realistic assumptions about the anatomical compartment in which a barrier might act, the functional form of innate response induction, and the dependence of adaptive response induction upon the innate tier (see S1 Code). Relevant barrier defenses are diverse, ranging from constitutive physical and chemical barriers (such as skin or stomach acid) through to continuously produced mucins, antimicrobial peptides (AMPs), or IgA on mucosal surfaces [36]. Mucus, for instance, provides vital multi-pronged barrier defense across the enormous surface areas of the gut, lung, and reproductive tract [62]. Although it is produced by B cells, IgA is continuously secreted in mass quantities, is often antigen-independent (and thus doesn’t require induction of an adaptive response), cross-reactive across many microbial antigens, and also functions as a barrier defense [37]; such IgA can, for example, alter establishment of gut commensals [41] or prevent infection by pathogenic pneumococci [42]. Dose-response studies of barrier defenses are rare but have revealed, for example, that even barriers of intact skin and conjunctival mucosa are overcome when a host is exposed to sufficiently high doses of Leptospira bacteria [38]. There are, of course, important caveats to acknowledge, but some of these also lead to insights and predictions for the evolution of immune systems. For example, we modeled just three coarse tiers, but in reality, within each coarse tier, there are finer grained tiers. Such granularity is evident in differential dependence of different induced defenses upon parasite load (e.g., [33, 34]) or immunisation rate [63] as well as varied innate defenses of, for example, the lung [64, 65] or the diversity of T cell types that do not neatly fall in the classic innate vs adaptive categories [66]. This array of finer tiers, including tiers of extremely rapidly induced defenses, imply that “constitutive defenses” actually are part of a continuum [36] of rates of defensive readiness. Our model revealed that three coarse tiers leave vulnerabilities around the barrier and innate defenses; perhaps more realistic redundancies and finer-scaled tiers would mitigate such vulnerabilities. It is tempting to speculate that evolution has led to such redundancies and fine-scale tiers for the very reason that tier transitions open vulnerabilities. Nonetheless, animals such as cnidarians [67] are thought to have just two coarse tiers of defense (barrier & innate). It would be fascinating to investigate whether there are systematic differences in the relationship between inoculum dose and infection success or chronicity across hosts with two vs. three coarse tiers of defenses. Our study suggests that in organisms with only two tiers of defense, there should be only one “wormhole” through which parasites might travel. It would also be useful to test whether/how multiple anatomical compartments might interact with tiers of defense to affect dose-dependence of infection duration, perhaps by comparing localized versus systemic infections. The fact that plants have barrier defenses as well as synergistic recognition and effector defenses [68] plus adaptive RNAi capability (e.g., [69]) suggests that our three-tier predictions might be interesting to test in plants as well as animals. Furthermore, dose-dependent probabilities and durations of infection are likely to have implications for the evolution of virulence, so we second Schmid-Hempel and Frank’s suggestion [31] that this is a rich area for further research. Virulence is generally defined as fitness effects of infection upon the host, and is hypothesized to trade off against transmission potential–i.e., parasites that kill hosts too quickly might lose out on transmission opportunities, while parasites that cause little damage to the host might transmit a lot, as long as they avoid rapid immune clearance [70]. It would therefore be interesting to investigate how non-monotonic dose-dependence of the probability and duration of infection (as reported here) might affect host fitness, parasite transmission, and thereby the evolution of virulence. For example, would the highest doses accelerate host death, enhancing the benefit to parasites arriving at an intermediate dose? Such investigation is beyond the scope of this study, however, in part because the effects of parasite load and infection duration on host fitness are complex and system-dependent. For instance, virulence might be maximized by maximal cumulative parasite load in some systems, but other relationships are conceivable: e.g., if a threshold number of parasites triggers death [71] or if virulence is actually maximized at low parasite burden due to severe immunopathology [72]. The implications of dose-dependence for transmission are likewise complex but worthy of future investigation. For instance, we found that duration, but NOT cumulative parasite load, is maximized at intermediate doses. In empirical systems where cumulative parasite load is linearly related to cumulative transmission probability, transmission would therefore not be predicted to be maximized at intermediate doses. We would nonetheless still advocate for the importance of dose-response studies in such systems, to better predict how exposure maps onto onward transmission even in simpler systems. However, a linear relationship between cumulative parasite load and cumulative transmission probability does not always hold. Indeed, there are examples of parasites for which duration per se rather than cumulative parasite burden maximizes transmission. For instance, when the infectious dose is small, as for norovirus, the sheer number of days of shedding exceeds the importance of load in predicting the risk that a host will seed an outbreak [73]. Human Immunodeficiency Virus (HIV) provides another example, because of the nonlinear relationship between set-point viral load and the probability of onward transmission [74] as well as the disconnect between transmission probability and the HIV load in the host at the time of transmission [75]. Furthermore, because we found that infection success is also maximized at intermediate doses, this arguably represents a strong effect of dose upon onward transmission because an effective duration of zero could preclude the development of any differences in cumulative parasite load. Such non-monotonic dose-dependence in infection success, if borne out experimentally, would therefore be likely to affect transmission and virulence of all parasite species. We look forward to further work on the evolutionary ecology of dose-dependence, both theoretical (to further explore the rules governing the evolution of defenses and virulence) and empirical (to test the predictions). One testable prediction is the two-peaked dose-dependence in infection duration that should hold if there are three tiers of immune responses with two clearcut transitions among them. Empirical dose-response studies with a wide range of doses could reveal such patterns. Empiricists designing dose-response studies are often logistically constrained to select just a few doses, however, and it may be challenging to identify a suitable range that will reveal any hidden nonlinearities, or even non-monotonicities. Indeed, the dose that is intermediate will vary hugely across parasite taxa, in ways that may map on to their life histories, virulence, and/or extent of spatial compartmentalization [31]. The predictions generated here might thus best be tested via experimental approaches that track bottlenecks of founder bacteria in hosts deficient in immune defenses of various speeds of activation and efficacy–e.g., barcoded isogenic Listeria monocytogenes combined with host immune manipulation as in the elegant work of Zhang et al. [76]. One could also investigate how dose-dependencies change in hosts with altered immune systems. Actual in vivo barrier knockouts analogous to our in silico barrier knockouts may be rarely practical, but knockdowns may be: as in the work on leptospirosis of Gostic and colleagues [38], barrier defenses like skin or mucosa could perhaps be experimentally eroded for dose-response studies across a broader array of infections. We conclude by noting that empirically-grounded theory has made a great deal of recent progress in revealing rules of host-parasite interaction. These include stochasticity around inoculating dose thresholds or other early dynamics that lead to life or death for the host [6, 77]. Another insight that has emerged from recent theory is that acute and chronic infections can emerge from the same set of equations so long as suitable feedback mechanisms (e.g., between parasite growth and induced immune defenses, or among different induced immune responses) are built in [78–80]. Here, we add that complex dose-dependence of the probability and duration of infection also arises from such feedbacks when combined with the stochasticity inherent in the within-host parasite population dynamics. We note that, in the present work, variable or bimodal infection outcomes are not technically related to the Allee effect because our model does not feature multiple equilibria that characterize this effect. Nonetheless, Allee effects and other dynamical patterns arising from within-host feedbacks may provide a general framework for understanding rules governing patterns of dose-dependence across a wide array of infectious diseases (such as those represented in Quantitative Microbial Risk Assessment [10, 11]), including to help explain observed deviations from monotonic dose-dependence. These tractable insights from within-host dynamics are likely to have major implications for the immunoepidemiology of diverse infections as well as the evolution of defense systems. These are exciting times indeed for quantitative studies of host-parasite interactions. Supporting information S1 Fig. Peak parasite load (top row), infection duration (middle row), and cumulative parasite load (AUC, or area under the curve; bottom row) statistics for deterministic simulations across a range of inoculum sizes in 4 host types: (A) wildtype (“default”), (B) barrier knockouts (“EAko”), (C) second-tier knockouts (“EBko”), and (D) third-tier knockouts (“ECko”). Here, all doses arrive slowly over time, in a trickle rather than bolus inoculation. https://doi.org/10.1371/journal.pcbi.1012652.s001 (TIF) S2 Fig. Peak parasite load (top row), infection duration (middle row), and cumulative parasite load (AUC, or area under the curve; bottom row) statistics for deterministic simulations across a range of inoculum sizes in 4 host types: (A) wildtype (“default”), (B) barrier knockouts (“EAko”), (C) second-tier knockouts (“EBko”), and (D) third-tier knockouts (“ECko”). Here, all parasites are non-replicating macroparasites/worms, that arrive slowly in a trickle inoculation. https://doi.org/10.1371/journal.pcbi.1012652.s002 (TIF) S3 Fig. Stochastic simulations of the time course of the parasite population in hosts in which the second-tier innate responses are assumed to be knocked out. https://doi.org/10.1371/journal.pcbi.1012652.s003 (TIF) S4 Fig. Peak parasite load from stochastic simulations of wild type hosts, in which peak parasite load escalates monotonically with increasing inoculating dose. https://doi.org/10.1371/journal.pcbi.1012652.s004 (TIF) S5 Fig. Cumulative parasite load (AUC, or area under the curve) from stochastic simulations of wild type hosts of wild type hosts, in which cumulative parasite load escalates monotonically with increasing inoculating dose. https://doi.org/10.1371/journal.pcbi.1012652.s005 (TIF) S6 Fig. Peak parasite load (top row), infection duration (middle row), and cumulative parasite load (AUC, or area under the curve; bottom row) statistics for deterministic simulations in our alternative model, across a range of inoculum sizes in 4 host types: (A) wildtype (“default”), (B) barrier knockouts (“EAko”), (C) second-tier knockouts (“EBko”), and (D) third-tier knockouts (“ECko”). Here, all parasites are microparasites that arrive all at once in a bolus. https://doi.org/10.1371/journal.pcbi.1012652.s006 (TIF) S7 Fig. Infection success against inoculum size and time of observation, following simulations of our alternative model. Infection success again increases overall with the inoculum size, and, in general, is higher when observed sooner. Note, however, the non-monotonic, double-peaked profile of infection success versus inoculum for observation times of approximately 10–16 days. https://doi.org/10.1371/journal.pcbi.1012652.s007 (TIF) S1 Code. Supplementary Online Material which includes all code necessary for the reported analyses and figures. https://doi.org/10.1371/journal.pcbi.1012652.s008 (PDF) Acknowledgments ALG thanks ETH for sabbatical fellowship support. Both authors thank Aaron King for valuable discussion of these ideas as well as code that aided plotting of our results.

Iwamoto, Masami;Hirabayashi, Satoko;Atsumi, Noritoshi

doi: 10.1371/journal.pcbi.1012645pmid: 39689064

Introduction While humans can voluntarily regulate their respiratory state, heart rate (HR) and blood pressure (BP) remain beyond their conscious control. Voluntary breathing (VB), short-term exercise (STE), and mental stress (MS) have been shown to influence breathing rate (BR), HR, and BP dynamics [1–6]. Understanding the mechanism underlying these changes is crucial for promoting overall physical and mental well–being [7–10]. Particularly, emotions interact with autonomic nervous control [11] and induce cardiovascular changes [12]. Therefore, a better understanding of the mechanisms by which VB, STE, or MS alter the HR and BP is critical for enhancing physical and mental health. VB impacts HR, BP, and heart rate variability (HRV) [2, 3, 13, 14], while MS also influences the BR, HR, and BP [5, 13, 15, 16], and vice versa [9, 10]. STE induces hyperventilation and affects HR/BP [1, 4]. Especially, the relationship between minute ventilation (MV) and carbon dioxide partial pressures (PaCO2) in the blood is critical to understand the respiratory and cardiovascular responses during STE [17] and increase in MV with dead space, that is, with larger airway resistance, generates short–term and long–term potentiation of exercise ventilatory response [18]. However, due to inter–individual variability and challenges in simultaneous measurements (particularly during physical activity), these studies offer only partial insight. Despite the ability to measure the physiological parameters, such as BR, HR, BP, and HRV, in the respiratory–circulatory system, establishing a precise relationship among these variables remains challenging due to their strong correlations. Considering physiological values, a computational approach can offer a comprehensive understanding. Previous studies have developed computational models of the respiratory control system or cardiovascular model to simulate the effect of dynamic exercise [19, 20]. The model reported by Waldrop et al. (2011) [19] is a computer-based mathematical model of the respiratory control system including rapidly active neural mechanisms, short–term potentiation, and serum [K+] in addition to the feedback mechanisms of PaCO2 and oxygen partial pressures in the blood (PaO2), and generates ventilatory and PaCO2 responses to dynamic exercise for 6 min. However, it does not include the cardiovascular system and thereby cannot simulate the responses of HR and BP during exercise. The model reported by Roy et al. (2023) [20] is also a computer simulation-based mathematical model of the cardiovascular hemodynamic system including its parameters such as HR, cardiac output, and mean arterial pressure, and simulates the exercise effect on the cardiac parameters relevant to cardiac rehabilitation. However, it does not include the respiratory control system and thereby cannot simulate respiratory responses, such as the BR and MV, during exercise. Given the established connection between emotions or affective states and parameters such as HR, BP, and HRV [11], recent studies have applied deep learning techniques to associate these parameters [21–23]. Deep learning methods leverage substantial experimental data from physiological measurements, including HR and facial expression, often captured in static or passive states, and supplemented by subjective ratings. However, the accuracy of emotion or affective state estimation remains at approximately 70%, which is insufficient [21–23]. This limitation arises due to inter–individual variability and complex interactions among physiological values. Moreover, deep learning algorithm cannot explain the influence of external stimuli on physiological values and emotions. Recent advancements propose alternative computational approaches to explore the interplay between external stimulation, physiological parameters, and brain activity [24–26]. For example, a dynamic system model utilizing coupled autonomous oscillators that emulate brain and cardiorespiratory dynamics has been employed for emotion recognition. This model demonstrated high recognition accuracy using the circumplex model of affect for arousal and valence [24], suggesting the strong correlations between the cardiorespiratory system and emotion. Similarly, investigations into attention mechanisms revealed a positive relationship between attention and tonic locus coeruleus (LC) frequency entrained by respiration using a similar model [25]. Spiking neural network models have explored the LC–amygdala interaction and cardiorespiratory centers, elucidating the effects of external body vibrations on BR, HR, and arousal levels [26]. However, we lack models capable of predicting BR, HR, BP, and HRV simultaneously during VB, STE, or MS, while also examining the impact of these stimulations on physiological responses. This study presents a computational approach for in–silico simultaneous measurements of physiological values, such as the BR, tidal volume (TV), MV, PaO2, HR, BP, and HRV during VB, STE, or MS by predicting the responses of the respiratory–circulatory system to the external stimulations, aiming to elucidate the mechanisms altering the physiological values. The validity of this approach is assessed by comparing predictions with experimental data from the literature. Additionally, we explore the influence of VB, STE, or MS on physiological values impacting human physical or mental state. The proposed method reveals the interactions among these physiological values during respiratory–circulatory responses, which are not fully captured by in–vivo physiological measurements or artificial intelligence (AI)–based data analysis requiring extensive datasets. This approach also enhances understanding of the ways in which human activities such as, slow, deep breathing, improve HRV and performance. Results Physiological changes during active or passive short-term exercises The first condition models the respiratory and circulatory changes during active and passive exercises, based on a prior study by Ishida et al. (2000) [1]. This study involved 13 healthy young adult males (mean age: 22.9 years) and 13 elderly males (mean age: 66.8 years), who either voluntarily flexed their knees or had their knees passively flexed at a constant velocity by an experimenter. Measurements included the BR, TV, MV, HR, systolic blood pressure (SBP), and diastolic blood pressure (DBP). We adjusted the PONS, RAMPI, UMAXE, TML, EXGAIN, and PCO2 parameters iteratively to replicate the respiratory and circulatory states observed in the young adult subjects [1] (Materials and methods and Table 1). Fig 1 compares the simulation results (red solid lines) with experimental data (blue solid lines) [1] for physiological values over time during active STE involving knee flexions. Yellow rectangles indicate the responses during the exercise. The physiological parameters include the BR, TV, MV, HR, and BP. Additionally, Fig 1 also presents the simulation results (red solid line) of the relationship between the heartbeat frequency and power spectral density (PSD) of HRV, noting that experimental HRV data are not reported in the referenced study [1]. The exercise commenced 20 s after the simulation began. From 20 to 40 s, four parameters—UMAXE = 0.07, TML = 2.0, EXGAIN = 1.0, and PCO2 = 0.02—were applied to simulate active STE. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Comparisons between simulation results (red solid lines) and experimental data of young and elderly adult males (blue solid and dotted lines, respectively) [1] with mean values (solid lines) and standard error (error bars) exclusively for young adult males of physiological values versus time during active short–term exercise involving knee flexions. The yellow rectangles correspond the responses observed during the exercise. (a) Breathing rate (BR), (b) Tidal volume (TV), (c) Minute ventilation (MV), (d) Heart rate (HR), (e) Blood pressure (BP) vs. time, and (f) Power spectral density (PSD) of heart rate variability (HRV) vs. heart beat frequency. The light green, blue, and red rectangles in (f) represent the very–low–frequency (VLF: ∼0.05 Hz), low frequency (LF: 0.05–0.15 Hz), and high-frequency (HF: 0.15–0.4 Hz) components, respectively. Experimental data on HRV are not available in the referenced study [1]. The black dotted lines show the results of Model A simulating elderly males whose systemic arterial resistance was set to 0.082 mmHg⋅s/ml whereas the green dotted lines denote the results of model B simulating adult females whose vital capacity was set to 3.1 L. https://doi.org/10.1371/journal.pcbi.1012645.g001 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Parameters to control respiratory states. A parameter of APSR is set to 1.0 in all conditions. https://doi.org/10.1371/journal.pcbi.1012645.t001 The predicted results for the BR, TV, MV, HR, and SBP generally aligned with the experimental data for young adult males [1]. However, the predicted DBP was 10–20 mmHg higher than the experimental data. In Fig 1f, the light green, blue, and red rectangles represent the very–low–frequency (VLF: ∼0.05 Hz), low–frequency (LF: 0.05–0.15 Hz), and high–frequency (HF: 0.15–0.4 Hz) components, respectively, with the VLF component being predominant during active STE. Fig 1 also presents the outcome of two parametric simulations involving Models A and B. In Model A, we simulated elderly males, adjusting their systemic arterial resistance (indicated by Ri, a parameter of the circulatory system described in Materials and methods, in Table 2) to 0.082 mmHg⋅s/ml, while young adult males had a resistance of 0.06 mmHg⋅s/ml. This choice was informed by the observation that resistance in elderly individuals increases by 37% across the age range from 20 to 80 years old [27]. In Model B, we focused on adult females setting their vital capacity at 3.1 L, while adult males had a vital capacity of 4.8 L, as referenced from Tortora and Derrickson [28]. These simulation results revealed that elderly males (indicated by black dotted lines) exhibited similar changes in HR, SBP, and DBP to the experimental data of elderly males (blue dotted lines). However, they also experienced increased BR, TV, and MV. In contrast, adult female (green dotted lines) demonstrated reduced MV due to decreased TV and lower HR, SBP, and DBP. Notably, the VLF component of HRV in elderly males (black dotted line) and adult females (green dotted line) was lower than that observed in young adult males (red dotted line). Download: PPT PowerPoint slide PNG larger image TIFF original image Table 2. Parameters characterizing the circulatory system [66]. https://doi.org/10.1371/journal.pcbi.1012645.t002 Fig 2 presents a comparative analysis between the simulation results (indicated by red solid lines) and experimental data (represented by blue solid lines) [1]. These data pertain to physiological values versus time during passive STE involving knee flexions. The yellow rectangles highlight the responses observed during the exercise. The physiological parameters considered included the BR, TV, MV, HR, and BP. Fig 2 also presents simulation results (indicated by a red solid line) illustrating the relationship between the heartbeat frequency and the PSD of HRV. Notably, the referenced study [1] did not report experimental data on the HRV. The simulation commenced 20 s after the start of the exercise. During the time interval from 20 to 40 s interval, four parameters—UMAXE = 0.01, TML = 2.0, EXGAIN = 0.001, and PCO2 = 0.02—were utilized to replicate passive STE. The predicted outcome for BR, TV, MV, and SBP closely aligned with the experimental data [1]. While the predicted HR exhibited peaky waveforms slightly exceeding those in the experimental data, the peak of the predicted DBP was 15 mmHg higher. A comparison between Figs 1f and 2f reveals that the VLF component of HRV during passive STE was one–third lower than that during active STE, while the LF and HF components during passive STE were higher. Simulation results for the first condition indicate that adjusting the parameters of RAMPI, UMAXE, and EXGAIN (see Table 1) can replicate physiological values during active and passive STE. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Comparisons of the simulation results (red solid lines) and experimental data (blue solid lines) [1] with mean values (solid lines) and standard error (error bars) of physiological values vs. time in passive short–term exercise involving knee flexions. The yellow rectangles represent the responses during the exercise. (a) Breathing rate (BR), (b) Tidal volume (TV), (c) Minute ventilation (MV), (d) Heart rate (HR), (e) Blood pressure (BP) vs. time, and (f) Power spectral density (PSD) of heart rate variability (HRV) vs. heart beat frequency. Experimental data on HRV are not available in the referenced study [1]. https://doi.org/10.1371/journal.pcbi.1012645.g002 Physiological changes during voluntary adjustment of breathing rate The second condition emulates voluntary BR adjustments, drawing from a prior experimental study [2, 3]. Nuckowska et al. (2019) conducted experiments with eight male and 12 female subjects (mean age: 25.3 years), allowing them to voluntarily control their BR at 6 or 12 beats per minute (bpm) for 5 minutes. Similarly, Song et al. (2003) explored the BR control in five female subjects (mean age: 29 years) across a range of rates (3, 4, 6, 8, 10, 12, and 14 bpm) for 5 minutes. While their study measured HR and BP, they did not report TV data. To maintain a constant MV of 8 bpm⋅L (as the typical MV for healthy individuals at rest is approximately 8 bpm⋅L [29]), we adjusted the parameters of PONS, RAMPI, APSR, and UMAX using trial and error to replicate BRs of 6, 8, 10, 12, and 14 bpm and an MV of 8 bpm⋅L. Fig 3 presents the simulation results for VB controls across five respiratory states A (BR = 14 bpm, TV = 0.57 L), B (BR = 12 bpm, TV = 0.67 L), C (BR = 10 bpm, TV = 0.8 L), D (BR = 8 bpm, TV = 1.0 L), and E (BR = 6 bpm, TV = 1.33 L) under the second condition. The figure encompasses the time histories of BR, TV, MV, HR, and BP, along with the relationship between the PSD of HRV and heartbeat frequency. Fig 3a, 3b and 3c show that the targeted values of BR, TV, and MV were nearly achieved by adjusting the parameters of PONS and RAMPI. Despite efforts to adjust the parameters to achieve an MV of 8 bpm⋅L in the respiratory state E, TV increased to over 1.4 L, resulting in an MV increase to 10 bpm⋅L. The variation in HR increased as BR decreased, aligning with the experimental test data from Song et al. [2] (Fig 3d). However, BP remained relatively unchanged (Fig 3e), contrary to Nuckowska et al. [3], who reported a 5 mmHg increase in BP with a decrease in the BR. In the PSD of HRV, the HF component was predominant at BRs of 12 or 14 bpm, while the LF component was predominant at BRs of 6 or 8 bpm. The LF component increased with decreasing BR, consistent with Song et al. [2], who found that LF component is higher at 4 to 8 bpm than at 10 to 14 bpm whereas HF component is higher at 10 to 14 bpm than at 4 to 8 bpm. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Simulation results of voluntary breathing controls in five respiratory states: Case A (BR = 14 bpm, TV = 0.5 L), B (BR = 12 bpm, TV = 0.67 L), C (BR = 10 bpm, TV = 0.8 L), D (BR = 8 bpm, TV = 1.0 L), E (BR = 6 bpm, TV = 1.33 L). (a) Breathing rate (BR), (b) Tidal volume (TV), (c) Minute ventilation (MV), (d) Heart rate (HR) with experimental test data from Song et al. [2], including mean values (solid circles) and peak and trough values (dotted lines), (e) Blood pressure (BP) vs. time, and (f) Power spectral density (PSD) of heart rate variability (HRV) vs. heart beat frequency. https://doi.org/10.1371/journal.pcbi.1012645.g003 Physiological changes during mental stress loads The third condition simulates the changes in the HR and BP induced by the MS loads to evaluate the outcomes of an experimental study [15]. Carroll et al. (2000) [15] measured the HR and BP at rest and in response to a 3–minute mental arithmetic stress in 1900 participants using a semi–automatic sphygmomanometer. In our simulations, we were unable to replicate the mental arithmetic stress. Instead, we modeled MS as an electrical input to the amygdala based on prior studies indicating that stress leads to amygdala hyperactivity [30]. We conducted parametric simulations by inputting constant values of 0.1, 0.2, 0.3, 0.4, and 0.5 to the central nucleus of the amygdala (CeA) within the respiratory–circulatory system model, which reproduced automatic breathing at 16 bpm. Fig 4 presents the simulation outcomes of physiological changes under five MS loads, ranging from 0.1 to 0.5 in increments of 0.1, within an automatic breathing condition (BR = 16 bpm, TV = 0.5 L). The figure shows the time histories of BR, TV, MV, HR, and BP, alongside the relationship between the PSD of HRV and heartbeat frequency. Yellow rectangles denote responses to mental stress loads. The predicted respiratory parameters (BR, TV, and MV) remained constant with increasing mental stress load (Fig 4a, 4b and 4c). HR peaked 5 s post–MS load (Fig 4d), while SBP and DBP gradually increased peaking at 40 s, coinciding with the end of the MS load (Fig 4e), This trend aligns with Carroll et al’s [15] experimental data, indicating higher HR, SBP, and DBP under the mental arithmetic stress compared to resting state. HRV increased with MS load, with the VLF component being predominant (Fig 4f). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Simulation results for mental stress loads: (a) Breathing rate (BR), (b) Tidal volume (TV), (c) Minute ventilation (MV), (d) Heart rate (HR), (e) Blood pressure (BP) vs. time, (f) Power spectral density (PSD) of heart rate variability (HRV) vs. heart beat frequency. The yellow rectangles represent the responses during the mental stress loads. Five mental stress loads varying from 0.1 to 0.5 in steps of 0.1 were applied from 20 to 40 s in the automatic breathing condition (BR = 16 bpm, TV = 0.5 L). https://doi.org/10.1371/journal.pcbi.1012645.g004 Fig 5a compares the PaO2 levels during active or passive STE, showing higher PaO2 during active STE. Fig 5b compares PaO2 at breathing rates of 6 and 14 bpm, indicating higher PaO2 during slow breathing (6 bpm) compared to rapid breathing (14 bpm). Fig 5c compares PaO2 under MS loads of 0.1 and 0.5, revealing similar PaO2 levels for both stress loads. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Comparative analysis of blood oxygen partial pressures during short–term exercise, voluntary controlled breathing, and mental stress loading: (a) active vs passive exercise, (b) controlled breathing at 6 or 14 bpm, and (c) mental stress loads of 0.1 and 0.5. The yellow rectangles indicate data during short–term exercise and mental stress. https://doi.org/10.1371/journal.pcbi.1012645.g005 Fig 6 presents the HRV metrics: standard deviation of the NN intervals (SDNN), Sample Entropy (SamEn), total HRV power, and LF/HF power ratio across all the simulation conditions. These metrics were derived using the Open–Source Python Toolbox for Heart Rate Variability, pyHRV [31] from RRI time histories. SDNN reflects variability, while SamEn indicates regularity in the RRI time history curve [32]. SDNN was higher in VB controls, in particular, control to slower BR, and lower in passive STE and MS loading (Fig 6a). Conversely, SamEn was higher in passive STE and MS loading and lower in active STE and VB controls (Fig 6b). Total HRV power correlates with cognitive performance [33] and the LF/HF ratio represents autonomic nervous system balance [32]. The total power of HRV was greater during slower BR and active STE, whereas it was reduced during faster BR and MS loading (Fig 6c). The LF/HF power ratio was higher in active STE and slower BR (Fig 6d). Fig 7 presents Poincaré plots for STE, VB control, and MS loading. The ellipse area of Poincaré plot (EAPP) indicates stress or relaxation states, while the ellipse shape denotes the predominance of sympathetic or parasympathetic tone [32, 34]. Increased MS loading enhanced sympathetic tone and shortened RR intervals. The EAPP was largest during controlled breathing at 6 bpm, followed by controlled breathing at 14 bpm, passive STE, active STE, MS load of 0.5, and MS load of 0.1. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. HRV metrics of SDNN, Sample Entropy, and total power of HRV and LF/HF power ratio for all simulation conditions. (a) SDNN, (b) Sample Entropy, (c) Total power of HRV, and (d) LF/HF power ratio. https://doi.org/10.1371/journal.pcbi.1012645.g006 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 7. Comparisons of the Poincaré plots during short–term exercise, voluntary breathing control, and mental stress loading. (a) active short–term exercise, (b) passive short–term exercise, (c) controlled breathing of 6 bpm, (d) controlled breathing of 14 bpm, (e) mental stress load of 0.1, and (f) mental stress load of 0.5. NNI indicates normal–to–normal interval. https://doi.org/10.1371/journal.pcbi.1012645.g007 Physiological changes during active or passive short-term exercises The first condition models the respiratory and circulatory changes during active and passive exercises, based on a prior study by Ishida et al. (2000) [1]. This study involved 13 healthy young adult males (mean age: 22.9 years) and 13 elderly males (mean age: 66.8 years), who either voluntarily flexed their knees or had their knees passively flexed at a constant velocity by an experimenter. Measurements included the BR, TV, MV, HR, systolic blood pressure (SBP), and diastolic blood pressure (DBP). We adjusted the PONS, RAMPI, UMAXE, TML, EXGAIN, and PCO2 parameters iteratively to replicate the respiratory and circulatory states observed in the young adult subjects [1] (Materials and methods and Table 1). Fig 1 compares the simulation results (red solid lines) with experimental data (blue solid lines) [1] for physiological values over time during active STE involving knee flexions. Yellow rectangles indicate the responses during the exercise. The physiological parameters include the BR, TV, MV, HR, and BP. Additionally, Fig 1 also presents the simulation results (red solid line) of the relationship between the heartbeat frequency and power spectral density (PSD) of HRV, noting that experimental HRV data are not reported in the referenced study [1]. The exercise commenced 20 s after the simulation began. From 20 to 40 s, four parameters—UMAXE = 0.07, TML = 2.0, EXGAIN = 1.0, and PCO2 = 0.02—were applied to simulate active STE. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Comparisons between simulation results (red solid lines) and experimental data of young and elderly adult males (blue solid and dotted lines, respectively) [1] with mean values (solid lines) and standard error (error bars) exclusively for young adult males of physiological values versus time during active short–term exercise involving knee flexions. The yellow rectangles correspond the responses observed during the exercise. (a) Breathing rate (BR), (b) Tidal volume (TV), (c) Minute ventilation (MV), (d) Heart rate (HR), (e) Blood pressure (BP) vs. time, and (f) Power spectral density (PSD) of heart rate variability (HRV) vs. heart beat frequency. The light green, blue, and red rectangles in (f) represent the very–low–frequency (VLF: ∼0.05 Hz), low frequency (LF: 0.05–0.15 Hz), and high-frequency (HF: 0.15–0.4 Hz) components, respectively. Experimental data on HRV are not available in the referenced study [1]. The black dotted lines show the results of Model A simulating elderly males whose systemic arterial resistance was set to 0.082 mmHg⋅s/ml whereas the green dotted lines denote the results of model B simulating adult females whose vital capacity was set to 3.1 L. https://doi.org/10.1371/journal.pcbi.1012645.g001 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Parameters to control respiratory states. A parameter of APSR is set to 1.0 in all conditions. https://doi.org/10.1371/journal.pcbi.1012645.t001 The predicted results for the BR, TV, MV, HR, and SBP generally aligned with the experimental data for young adult males [1]. However, the predicted DBP was 10–20 mmHg higher than the experimental data. In Fig 1f, the light green, blue, and red rectangles represent the very–low–frequency (VLF: ∼0.05 Hz), low–frequency (LF: 0.05–0.15 Hz), and high–frequency (HF: 0.15–0.4 Hz) components, respectively, with the VLF component being predominant during active STE. Fig 1 also presents the outcome of two parametric simulations involving Models A and B. In Model A, we simulated elderly males, adjusting their systemic arterial resistance (indicated by Ri, a parameter of the circulatory system described in Materials and methods, in Table 2) to 0.082 mmHg⋅s/ml, while young adult males had a resistance of 0.06 mmHg⋅s/ml. This choice was informed by the observation that resistance in elderly individuals increases by 37% across the age range from 20 to 80 years old [27]. In Model B, we focused on adult females setting their vital capacity at 3.1 L, while adult males had a vital capacity of 4.8 L, as referenced from Tortora and Derrickson [28]. These simulation results revealed that elderly males (indicated by black dotted lines) exhibited similar changes in HR, SBP, and DBP to the experimental data of elderly males (blue dotted lines). However, they also experienced increased BR, TV, and MV. In contrast, adult female (green dotted lines) demonstrated reduced MV due to decreased TV and lower HR, SBP, and DBP. Notably, the VLF component of HRV in elderly males (black dotted line) and adult females (green dotted line) was lower than that observed in young adult males (red dotted line). Download: PPT PowerPoint slide PNG larger image TIFF original image Table 2. Parameters characterizing the circulatory system [66]. https://doi.org/10.1371/journal.pcbi.1012645.t002 Fig 2 presents a comparative analysis between the simulation results (indicated by red solid lines) and experimental data (represented by blue solid lines) [1]. These data pertain to physiological values versus time during passive STE involving knee flexions. The yellow rectangles highlight the responses observed during the exercise. The physiological parameters considered included the BR, TV, MV, HR, and BP. Fig 2 also presents simulation results (indicated by a red solid line) illustrating the relationship between the heartbeat frequency and the PSD of HRV. Notably, the referenced study [1] did not report experimental data on the HRV. The simulation commenced 20 s after the start of the exercise. During the time interval from 20 to 40 s interval, four parameters—UMAXE = 0.01, TML = 2.0, EXGAIN = 0.001, and PCO2 = 0.02—were utilized to replicate passive STE. The predicted outcome for BR, TV, MV, and SBP closely aligned with the experimental data [1]. While the predicted HR exhibited peaky waveforms slightly exceeding those in the experimental data, the peak of the predicted DBP was 15 mmHg higher. A comparison between Figs 1f and 2f reveals that the VLF component of HRV during passive STE was one–third lower than that during active STE, while the LF and HF components during passive STE were higher. Simulation results for the first condition indicate that adjusting the parameters of RAMPI, UMAXE, and EXGAIN (see Table 1) can replicate physiological values during active and passive STE. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Comparisons of the simulation results (red solid lines) and experimental data (blue solid lines) [1] with mean values (solid lines) and standard error (error bars) of physiological values vs. time in passive short–term exercise involving knee flexions. The yellow rectangles represent the responses during the exercise. (a) Breathing rate (BR), (b) Tidal volume (TV), (c) Minute ventilation (MV), (d) Heart rate (HR), (e) Blood pressure (BP) vs. time, and (f) Power spectral density (PSD) of heart rate variability (HRV) vs. heart beat frequency. Experimental data on HRV are not available in the referenced study [1]. https://doi.org/10.1371/journal.pcbi.1012645.g002 Physiological changes during voluntary adjustment of breathing rate The second condition emulates voluntary BR adjustments, drawing from a prior experimental study [2, 3]. Nuckowska et al. (2019) conducted experiments with eight male and 12 female subjects (mean age: 25.3 years), allowing them to voluntarily control their BR at 6 or 12 beats per minute (bpm) for 5 minutes. Similarly, Song et al. (2003) explored the BR control in five female subjects (mean age: 29 years) across a range of rates (3, 4, 6, 8, 10, 12, and 14 bpm) for 5 minutes. While their study measured HR and BP, they did not report TV data. To maintain a constant MV of 8 bpm⋅L (as the typical MV for healthy individuals at rest is approximately 8 bpm⋅L [29]), we adjusted the parameters of PONS, RAMPI, APSR, and UMAX using trial and error to replicate BRs of 6, 8, 10, 12, and 14 bpm and an MV of 8 bpm⋅L. Fig 3 presents the simulation results for VB controls across five respiratory states A (BR = 14 bpm, TV = 0.57 L), B (BR = 12 bpm, TV = 0.67 L), C (BR = 10 bpm, TV = 0.8 L), D (BR = 8 bpm, TV = 1.0 L), and E (BR = 6 bpm, TV = 1.33 L) under the second condition. The figure encompasses the time histories of BR, TV, MV, HR, and BP, along with the relationship between the PSD of HRV and heartbeat frequency. Fig 3a, 3b and 3c show that the targeted values of BR, TV, and MV were nearly achieved by adjusting the parameters of PONS and RAMPI. Despite efforts to adjust the parameters to achieve an MV of 8 bpm⋅L in the respiratory state E, TV increased to over 1.4 L, resulting in an MV increase to 10 bpm⋅L. The variation in HR increased as BR decreased, aligning with the experimental test data from Song et al. [2] (Fig 3d). However, BP remained relatively unchanged (Fig 3e), contrary to Nuckowska et al. [3], who reported a 5 mmHg increase in BP with a decrease in the BR. In the PSD of HRV, the HF component was predominant at BRs of 12 or 14 bpm, while the LF component was predominant at BRs of 6 or 8 bpm. The LF component increased with decreasing BR, consistent with Song et al. [2], who found that LF component is higher at 4 to 8 bpm than at 10 to 14 bpm whereas HF component is higher at 10 to 14 bpm than at 4 to 8 bpm. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Simulation results of voluntary breathing controls in five respiratory states: Case A (BR = 14 bpm, TV = 0.5 L), B (BR = 12 bpm, TV = 0.67 L), C (BR = 10 bpm, TV = 0.8 L), D (BR = 8 bpm, TV = 1.0 L), E (BR = 6 bpm, TV = 1.33 L). (a) Breathing rate (BR), (b) Tidal volume (TV), (c) Minute ventilation (MV), (d) Heart rate (HR) with experimental test data from Song et al. [2], including mean values (solid circles) and peak and trough values (dotted lines), (e) Blood pressure (BP) vs. time, and (f) Power spectral density (PSD) of heart rate variability (HRV) vs. heart beat frequency. https://doi.org/10.1371/journal.pcbi.1012645.g003 Physiological changes during mental stress loads The third condition simulates the changes in the HR and BP induced by the MS loads to evaluate the outcomes of an experimental study [15]. Carroll et al. (2000) [15] measured the HR and BP at rest and in response to a 3–minute mental arithmetic stress in 1900 participants using a semi–automatic sphygmomanometer. In our simulations, we were unable to replicate the mental arithmetic stress. Instead, we modeled MS as an electrical input to the amygdala based on prior studies indicating that stress leads to amygdala hyperactivity [30]. We conducted parametric simulations by inputting constant values of 0.1, 0.2, 0.3, 0.4, and 0.5 to the central nucleus of the amygdala (CeA) within the respiratory–circulatory system model, which reproduced automatic breathing at 16 bpm. Fig 4 presents the simulation outcomes of physiological changes under five MS loads, ranging from 0.1 to 0.5 in increments of 0.1, within an automatic breathing condition (BR = 16 bpm, TV = 0.5 L). The figure shows the time histories of BR, TV, MV, HR, and BP, alongside the relationship between the PSD of HRV and heartbeat frequency. Yellow rectangles denote responses to mental stress loads. The predicted respiratory parameters (BR, TV, and MV) remained constant with increasing mental stress load (Fig 4a, 4b and 4c). HR peaked 5 s post–MS load (Fig 4d), while SBP and DBP gradually increased peaking at 40 s, coinciding with the end of the MS load (Fig 4e), This trend aligns with Carroll et al’s [15] experimental data, indicating higher HR, SBP, and DBP under the mental arithmetic stress compared to resting state. HRV increased with MS load, with the VLF component being predominant (Fig 4f). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Simulation results for mental stress loads: (a) Breathing rate (BR), (b) Tidal volume (TV), (c) Minute ventilation (MV), (d) Heart rate (HR), (e) Blood pressure (BP) vs. time, (f) Power spectral density (PSD) of heart rate variability (HRV) vs. heart beat frequency. The yellow rectangles represent the responses during the mental stress loads. Five mental stress loads varying from 0.1 to 0.5 in steps of 0.1 were applied from 20 to 40 s in the automatic breathing condition (BR = 16 bpm, TV = 0.5 L). https://doi.org/10.1371/journal.pcbi.1012645.g004 Fig 5a compares the PaO2 levels during active or passive STE, showing higher PaO2 during active STE. Fig 5b compares PaO2 at breathing rates of 6 and 14 bpm, indicating higher PaO2 during slow breathing (6 bpm) compared to rapid breathing (14 bpm). Fig 5c compares PaO2 under MS loads of 0.1 and 0.5, revealing similar PaO2 levels for both stress loads. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Comparative analysis of blood oxygen partial pressures during short–term exercise, voluntary controlled breathing, and mental stress loading: (a) active vs passive exercise, (b) controlled breathing at 6 or 14 bpm, and (c) mental stress loads of 0.1 and 0.5. The yellow rectangles indicate data during short–term exercise and mental stress. https://doi.org/10.1371/journal.pcbi.1012645.g005 Fig 6 presents the HRV metrics: standard deviation of the NN intervals (SDNN), Sample Entropy (SamEn), total HRV power, and LF/HF power ratio across all the simulation conditions. These metrics were derived using the Open–Source Python Toolbox for Heart Rate Variability, pyHRV [31] from RRI time histories. SDNN reflects variability, while SamEn indicates regularity in the RRI time history curve [32]. SDNN was higher in VB controls, in particular, control to slower BR, and lower in passive STE and MS loading (Fig 6a). Conversely, SamEn was higher in passive STE and MS loading and lower in active STE and VB controls (Fig 6b). Total HRV power correlates with cognitive performance [33] and the LF/HF ratio represents autonomic nervous system balance [32]. The total power of HRV was greater during slower BR and active STE, whereas it was reduced during faster BR and MS loading (Fig 6c). The LF/HF power ratio was higher in active STE and slower BR (Fig 6d). Fig 7 presents Poincaré plots for STE, VB control, and MS loading. The ellipse area of Poincaré plot (EAPP) indicates stress or relaxation states, while the ellipse shape denotes the predominance of sympathetic or parasympathetic tone [32, 34]. Increased MS loading enhanced sympathetic tone and shortened RR intervals. The EAPP was largest during controlled breathing at 6 bpm, followed by controlled breathing at 14 bpm, passive STE, active STE, MS load of 0.5, and MS load of 0.1. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. HRV metrics of SDNN, Sample Entropy, and total power of HRV and LF/HF power ratio for all simulation conditions. (a) SDNN, (b) Sample Entropy, (c) Total power of HRV, and (d) LF/HF power ratio. https://doi.org/10.1371/journal.pcbi.1012645.g006 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 7. Comparisons of the Poincaré plots during short–term exercise, voluntary breathing control, and mental stress loading. (a) active short–term exercise, (b) passive short–term exercise, (c) controlled breathing of 6 bpm, (d) controlled breathing of 14 bpm, (e) mental stress load of 0.1, and (f) mental stress load of 0.5. NNI indicates normal–to–normal interval. https://doi.org/10.1371/journal.pcbi.1012645.g007 Discussion The computational model developed for predicting the responses of the respiratory and circulatory systems, along with the VB control method using actor–critic reinforcement learning (ACRL) successfully reproduced hyperventilation at the onset of exercise. Figs 1c and 2c show that the model accurately replicated the changes in MV during the STEs exercises, with MV rapidly increasing after 20 s and remaining constant until 40 s in both active and passive STEs. Except for DBP, the predicted BR, TV, MV, HR, and SBP closely matched the experimental data. DBP is primarily influenced by cardiac output and peripheral vascular resistance, which increases and decreases, respectively, during exercises due to the vasodilation of resistance vessels within the exercising skeletal muscles [35]. Therefore, the peripheral systemic resistance Rep in Table 3, one of the regulation effectors for circulatory system described in Materials and methods, must be adjusted to reproduce the DBP. Adjusting G4 of Rep to 0.106 achieved the DBP observed in the experimental data. However, the HR decreased by approximately 10 bpm compared to the experimental data. Further research is required to adjust other parameters to reproduce both HR and DBP simultaneously. The VLF component of HRV was predominant during active STE. However, direct comparison of the predicted HRV with experimental data was impossible due to the absence of HRV data in the referenced study [1]. Several studies have indicated that LF components are elevated during active STE compared to static STE [36], and they increase with higher exercise load from the resting state [37]. Conversely, VLF and LF components increase while HF components decrease in sitting or standing posture with greater muscle activity compared to the supine posture [38, 39]. Additionally, VLF components during rhythmic or random activity are three– to five–fold higher than during rest [40]. PaO2 levels during active STE were higher than during passive STE (Fig 5a), and arterial oxygen partial pressure during STE was higher than at rest [41]. This suggests that our model can reliably predict respiratory and circulatory responses, including HRV and PaO2 during STE. Among HRV metrics, SamEn was higher during passive STE, whereas the LF/HF power ratio was higher during active STE (Fig 6b and 6d). These results indicate that active STE enhances sympathetic nerve activity and reduces regularity in compared to passive STE. Additionally, elderly males with higher systemic arterial resistance than young males replicated HR, SBP, and DBP in experimental data but not BR, TV, and MV. In adult females with lower vital capacity than young males, MV and TV were lower than in young males, reflecting gender differences in ventilatory responses to progressive exercise [42]. The discrepancy in respiratory states for elderly males likely arises from difference in ribcage geometry and stiffness between young and elderly adults. Further research is required to examine the impact of age–related changes in ribcage geometry and stiffness on respiratory function. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 3. Parameters describing the regulation effectors [66]. https://doi.org/10.1371/journal.pcbi.1012645.t003 The model successfully replicated various respiratory states, from rapid to slow breathing. The maximum heart rate was higher during slow breathing compared to rapid breathing (Fig 3d). Song et al. [2] examined the peak, mean, and trough HR values for BR ranging from 3 to 14 bpm. Comparing the results for 14 bpm with those for 6 bpm as reported by Song et al. [2], the mean and trough values for 6 bpm were lower by 5 and 6 bpm, respectively. The predictions of the model demonstrated that the trough values at 6 bpm were 6 bpm lower than those at 14 bpm, consistent with the experimental data [2], although the mean values at 6 bpm were 5 bpm higher than those at 14 bpm, differing from the experimental data [2]. In the simulation results, the peak and mean values were higher during slow breathing compared to rapid breathing. However, the HR controlled at 6 bpm varied significantly from 100 bpm to 70 bpm within less than 5 s, replicating the cardiac oscillations during deep slow breathing at 6 bpm as reported by Sevoz–Couche and Laborde [43]. No predicted changes in SBP and DBP were observed in any respiratory state. Contradictory results exist regarding changes in SBP and DBP with increasing BR. SBP and DBP decreased by approximately 3 mmHg with an increase in BR from 6 bpm to 12 bpm [3], whereas they increased by 2 to 4 mmHg with an increase in BR from 6 bpm to 15 bpm [44]. The model predicted an increase in LF components with a decrease in breathing speed (Fig 3f). Experimental data corroborated this, showing increased LF components during slow breathing [2, 45]. PaO2 levels were higher during slow breathing (6 bpm) compared to rapid breathing (14 bpm) (Fig 5b). Slow breathing enhances HRV [2] and PaO2 [46]. Among HRV metrics, SDNN, total HRV power, and LF/HF power ratio were higher during slow breathing (6 and 8 bpm) than during rapid breathing (10–14 bpm) (Fig 6a, 6c and 6d). EAPP was highest during slow breathing at 6 bpm, followed by rapid breathing at 14 bpm across all simulation conditions (Fig 7). These findings indicate that our model reliably predicts respiratory and circulatory responses, including HRV and PaO2, during VB. Slow breathing enhances variability, cognitive performance, and relaxation. The model predicted increases in HR and SBP for MS loads (Fig 4d and 4e), consistent with previous experimental studies [8, 15, 47, 48]. The predicted respiratory states–BR, TV, and MV–remained stable across all stress conditions, despite reports of hyperventilation in response to stress [49] and increased BR due to anxiety anticipation [16]. The predicted HRV, particularly the VLF components, increased with an increase in the MS load. Experimental studies showed that HRV variables changed in response to stress due to low parasympathetic activity, characterized by a decrease in the HF band and an increase in the LF band [50], aligning with our predictions. However, differences were observed in the VLF components, which decreased with increasing depression [51] and contributed to MS recovery [52]. The peak VLF component predicted by the model was approximately 0.15 s2/Hz, less than half of those during active STE and voluntary slow, deep breathing. The PaO2 under an MS load of 0.5 was almost similar to that under an MS load of 0.1 (Fig 5c). All HRV metrics and the EAPP were higher under an MS load of 0.5 than under an MS load of 0.1 (Figs 6 and 7). SamEn was highest under an MS load of 0.5 among all simulation conditions, and the total power of HRV was lower under MS loading than during active STE and slow breathing. The EAPP under MS loads of 0.1 and 0.5 was significantly lower than during slow breathing at 6 bpm. These findings indicate that our model can reliably predict HR and BP under MS loading and that MS loading enhances regularity while reducing relaxation levels. However, the current model does not account for the amygdala–LC interaction, where fear– and stress– induced activity in the amygdala affects sensory brain regions via LC connections [53] and the entrained LC neurons via respiration [25, 54], nor the amygdala–prefrontal cortex (PFC)–hippocampus (HPC) interaction, which may relate to anxiety anticipation [55]. Further research is required to explore the amygdala–LC and amygdala–PFC–HPC interactions, examine the relationship between respiration and MS, and investigate the VLF components’ relation to MS loads. These simulation results demonstrated that voluntary slow deep breathing at 6 bpm increased the PaO2, total HRV power, SDNN, and EAPP, thereby enhancing the cognitive performance, variability, and relaxation. Conversely, MS loading elevated HR, SBP, and SamEn while reducing total HRV power and EAPP, leading to increased physical risk and decreased relaxation. Individuals with high resting HRV outperformed those with low HRV, exhibiting faster reaction times in stimulus identification [33]. Exercise and meditation with controlled VB can elevate resting HRV level. Lower HRV is associated with increases SBP, hypertension and cardiovascular disease, as shown in a longitudinal study on rheumatoid arthritis patients [56]. Both active STE [41] and slow breathing [46] elevate PaO2. Thus, slow breathing promotes mental health whereas MS loading increases the physical risk and reduces relaxation. While our proposed model does not predict emotional arousal or valence levels differently from existing literature [21–26], it uniquely predicts temporal changes in physiological values such as BR, TV, MV, HR, SBP, DBP, HRV, and blood oxygen and carbon dioxide partial pressures. It elucidates the mutual interactions among these physiological values during resting and active states, including active STE and VB control. This capability may enhance the prediction accuracy of emotional arousal or valence in AI–based data analysis using large datasets of physiological or psychological measurement. Additionally, it provides insight into the relationship between (1) the activity of emotion–related brain regions (e.g., amygdala, anterior cingulate cortex, and PFC) modeled by spiking neuron models, and (2) physiological values such as HR and HRV, which have been experimentally verified [57–61]. Furthermore, the model parameters for systemic arterial resistance and vital capacity reproduce HR and BP in age–related differences and MV and TV in gender–related difference. Thus, our approach has the potential to elucidate the mutual interactions among physiological values and investigate the effect of individual differences during VB, STE, or MS. This study has several limitations. First, the model only includes the diaphragm and abdominal muscles, focusing solely on abdominal breathing. Normal breathing involves both abdominal and chest breathing, necessitating the inclusion of other respiratory muscles such as the external intercostal muscles, inspiratory accessory muscles (scalene and sternocleidomastoid), and expiratory muscles (internal intercostal muscles) to simulate chest breathing accurately. Second, the current model cannot adjust the inspiratory–to–expiratory ratio and only simulates mouth breathing, thus precluding the investigation of the effects of the inspiratory–to–expiratory ratio on nasal breathing during slow deep breathing. Nasal breathing reduces mean blood pressure and DBP but does not affect SBP or HR while increasing parasympathetic contributions to HRV [62]. Further research is needed to model nasal breathing, adjust the inspiratory–to–expiratory ratio during slow deep breathing, and investigate the effects of slow, deep breathing on the autonomic nervous system and MS reduction. Third, the current model can predict respiratory and circulatory responses during long–term exercise exceeding four minutes; however, it predicts MV fluctuations with an amplitude of 1 bpm, which is not observed in experimental data. Additional studies are required to minimize these fluctuations for accurate long–term exercise simulation. Lastly, only four parameters were used as inputs to the respiratory–circulatory system model to represent active and passive STE. Future research should integrate this model with a musculoskeletal model, including muscle controllers [63], to simulate physiological values such as BR, HR, BP, and HRV during various motions and posture stabilizations. Materials and methods Integrated respiratory and circulatory systems Fig 8 shows a computational model for predicting the responses of the respiratory and circulatory systems. The model comprises the nucleus of the solitary tract (NTS), respiratory center and system as rhythm–generation and pattern formation regions, and circulatory center and system. The respiratory center and system were developed based on a mathematical model of the closed–loop control of breathing proposed by Molkov et al. [64]. Lung ventilation, or breathing, involves the exchange of air between the lungs and the environment, driven by the rhythmic contraction of the diaphragm and abdominal muscles. The firing activities of the phrenic and abdominal motor neurons, which control the diaphragm and abdominal muscles, respectively, are the primary outputs of the brainstem respiratory central pattern generator (CPG) that generates respiratory oscillations. The closed–loop respiratory system comprises two primary feedback pathways from the lungs to the respiratory CPG: mechanical and chemical feedback. Mechanical feedback is mediated by pulmonary stretch receptors (PSR) in the lungs, which convey lung volume information to the brainstem via the vagus nerve. Chemical feedback is provided by central chemoreceptors in the retrotrapezoid nucleus (RTN) neurons, which are highly sensitive to oxygen (O2) and carbon dioxide (CO2) levels in brain tissues. Additionally, we incorporated chemical feedback from second–order peripheral chemoreceptors, which are sensitive to O2 and CO2 levels in the blood and project directly the RTN and pre–I/I (pre–BötC) via chemoreceptors in the NTS (NTS Chemo) as described by Barnett et al. [65], thereby modulating the respiratory CPG in a CO2–dependent manner. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 8. Respiratory–circulatory system model. The model integrates the respiratory CPG system with spiking neurons, as proposed by Molkov et al. (2014) [64], and the circulatory system utilizing a rate–coding approach, as described by Ursino (1998) [66]. The red, blue, light blue, and purple lines denote excitatory and inhibitory signals, CO2 signals as chemical feedback, and breathing control signals, respectively. The light green line indicates pulmonary stretch receptors as mechanical feedback. Thick black and orange lines represent blood vessels and the interactions between the respiratory and circulatory systems, respectively. https://doi.org/10.1371/journal.pcbi.1012645.g008 In the respiratory center and system (Fig 8), each neuron in the respiratory CPG can be fundamentally modeled using Hodgkin–Huxley (HH) neurons. The model is represented by the following equations: (1) (2) where C is the membrane capacitance (C = 20 pF), and gK and gL are the peak conductance of potassium and leak conductance (leak), respectively. gKdr = 5 nS and gL = 2.8 nS, where EK and EL represent the reversal potential of potassium and leakage reverse potential, respectively. Specifically, EK = –85 mV and EL = –75 mV. The currents Ii represents the Pre–I/I (i = 1), Early–I (i = 2), Post–I (i = 3), Aug–E (i = 4), Late–E (i = 5), and Ramp–I (i = 6), and are defined as follows: (3) where gNaP and gAD are the maximum conductances, and ENa and EK are the reversal potentials for sodium and potassium, respectively. The values are gNaP = 5 nS, gAD = 10 nS, and ENa = 50 mV. The gating parameters mNa, mi(i = 2, 3, 4), and hi(i = 1, 5) are functions of the membrane variable Vi and are defined by the following equations: (4) (5) (6) (7) where τh = 4.0/cosh((Vi + 55.0)/10.0) s and (i = 2,3,4) are time constants. The variable represents the neuronal activation level and is defined as follows: (8) The neuronal activation levels of late–E (i = 5) and ramp–I (i = 6) were utilized to simulate the abdominal muscles and diaphragm, respectively. This study introduced the power exponent k to regulate the TV, corresponding to the breathing control parameter RAMPI described in the Results section; k = 1 for i = 1 ∼ 5 and k = RAMPI for i = 6. The gating variables si and qi for synaptic conductances were derived from the activity of the presynaptic neurons and other input sources using the following equations: (9) (10) (11) (12) (13) where is defined in Eq (8). The constant drive D1 originates from the pons and corresponds to the breathing control parameter PONS described in the Results section. Drive D2 is related to the partial pressure of CO2 in the blood that projects to each neuron in the respiratory CPG from the RTN (see [64] for more detailed information). The synaptic weights (a, b, c, e, f) were modified only for c1i from the original model [64] and are listed in Table 4. Eqs (1), (6), (7) and (12) were updated using the Euler method. In this study, the time step was set to 0.0001s. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 4. Parameters of respiratory CPG model. https://doi.org/10.1371/journal.pcbi.1012645.t004 For VB control, we adapted the diaphragm and abdominal muscles models using a Hill–type muscle model as described by O’Connor et al. [67]. We applied Eqs (14) and (15) to calculate the recoil pressures of the diaphragm and abdominal wall. (14) where is the static diaphragm recoil pressure at optimal length and maximum activation; udi is the phrenic activation of the diaphragm; represents the pressure due to the passive resistance of the diaphragm; is the static pressure–volume relationship of the diaphragm; and is the pressure–flow relationship of the diaphragm with velocity replaced by flow. is the passive transdiaphragmatic pressure as a function of the diaphragm volume. Detailed descriptions of , , and are provided by O’Connor et al. [67]. (15) where uab represents the activation of the abdominal muscle, and FCEmax is the maximal force capacity of the contractile element for a 1.5 cm2 cross–section of the canine external oblique muscle. is the static force–length relationship of the abdominal wall, and is the force–velocity relationship of the abdominal wall muscle. accounts for the pressure due to the passive resistance of the abdominal wall muscles. The constant kab converts force to surface tension, while (1/rs + 1/rt) translates the surface tension to pressure via Laplace’s law. The term (Vab − Vab0)/Cab represents the passive recoil pressure of the abdominal wall, where Vab0 is the volume at which the recoil pressure is zero, and Cab denotes the compliance of the abdominal wall. The respiratory CPG model determines the lung volume VL by solving Eq (16) for the pressure equilibrium involving abdominal pressure σab, pleural pressure Ppl, and diaphragm recoil pressure σdi. (16) where, Ppl is related to lung volume VL through Eq (17). (17) where Rrs is the airway resistance, VL0 is the lung volume at zero recoil pressure, and CL is the lung compliance. In addition, we modified several equations from the original model by O’Connor et al. [67] as follows: (18) (19) (20) where the vital capacity (VC), functional residual capacity (FRC) , and residual volume (RV) are set to 4.8, 2.2, and 1.2 L, respectively. To adapt the original adult rat respiratory system model [64] to the adult human respiratory system, we adjusted the parameters listed in Table 5 and revised the equation for the activity of the PSR (ref. Eq. (25) in [64]) as follows: (21) where is the residual volume of the lung. cPSR is a coefficient that adjusts the effect of the PSR on the lung volume change, corresponding to the breathing control parameter APSR described in the Results section. We set . Download: PPT PowerPoint slide PNG larger image TIFF original image Table 5. Parameters adjusted for adult human respiratory system. https://doi.org/10.1371/journal.pcbi.1012645.t005 The circulatory center was developed based on the pathways by which the CeA influences blood pressure during mental stress or anxiety, as proposed by [69], and the autonomic chronotropic control of the heart, as described by [70]. During stress, the CeA may inhibit the baroreceptive neurons in the NTS, potentially deactivating inhibitory inputs from the caudal ventrolateral medulla (CVLM) to the rostral ventrolateral medulla (RVLM). This could activate RVLM neurons, increasing sympathetic outflow, BP, and HR. The nucleus ambiguus (AMB) also receives excitatory signals from the NTS, enhancing parasympathetic outflow and reducing HR. Baroreceptors in the carotid arteries and aortic arch detect increased in the arterial blood pressure, activating afferent nerves (sinus nerves) and NTS neurons. The circulatory center and system were modeled using a rate–coding approach based on Ursino (1998) [66] with parameters for each segment in the circulatory system (Fig 8), listed in Table 2. In the circulatory center and system (Fig 8), the afferent baroreflex pathway is modeled as a first–order linear partial differential equation (Eq 22) using the carotid sinus pressure Pcs. The frequency of spikes in the afferent fibers, fcs, is represented as a sigmoidal function (Eq 23) with an intermediate variable . (22) (23) where τp and τz are time constants τp = 2.076 s and τz = 6.37 s. fmax and fmin are the upper and lower saturations limits of the frequency discharge, respectively; fmin = 2.52 spikes/s, fmax = 47.78 spikes/s, Pn is the intrasinus pressure at the central point of the sigmoidal function, and Ka is a parameter with pressure dimensions Pn = 92.0 mmHg and Ka = 11.758 mmHg. The frequencies of spikes in the efferent sympathetic nerves, fes, and the efferent vagal fibers, fev, are described by the frequency of spikes in the afferent fibers, fcs, as follows: (24) (25) where , , and kes are constants with values spikes/s, spikes/s, and kes = 0.0675 s. Similarly, , , , kev, and kresp are constants with values spikes/s, spikes/s, spikes/s, kev = 7.06 spikes/s, and kresp = 0.08 × 10–3 m3. As the state variables of the circulatory system are changed by efferent sympathetic nerve stimulation, the controlled variables θj, are defined as follows: and , represent the end–systolic elastances of the left and right ventricles (j = 1, 2); Rsp and Rep denote the hydraulic resistances of the splanchnic and extrasplanchnic peripheral circulations (j = 3, 4), respectively; and indicate the unstressed volumes of the splanchnic and extrasplanchnic venous circulation (j = 5, 6), respectively; and T signifies the heart period (j = 7). The parameters of θj are detailed in Table 3 [66]. The responses of resistances, unstressed volumes, and cardiac elastances to sympathetic drive encompass pure latency, a monotonic logarithmic static function, and low–pass first–order dynamics. Consequently, the following equations apply: (26) (27) Here, is an intermediate variable with static characteristics, and Gj represents the constant gain factor of each component. The parameters τj and Dj denote the time constant and pure latency, respectively. fesmin is the minimum sympathetic stimulation and fesmin = 2.66 Hz according to [66]. The change in heart period due to efferent vagus nerve stimulation is given by: (28) (29) is an intermediate variable that exhibits static characteristics. Gv = 0.09 s/Hz, Dv = 0.2 s, τv = 1.5 s. (30) In Eq (30), ΔTv and Δθ7 denote the variations in the heart period regulated by the sympathetic and parasympathetic nerves, respectively. The parameter represents the baseline heart period without efferent nerves stimulation, set to 0.58 s. Although the partial pressures of CO2 and O2 in the blood are reinitialized with each heartbeat in the model by Molkov et al. [64], the heartbeat initiation time was derived from changes in heart period due to autonomic nerve activity described in Eq (30) for j = 7. The lung volume VL from the respiratory system influenced parasympathetic nerve activity, as detailed in Eq (26). The total pressure and concentration of O2 and CO2 in the mouth were set to 760 [mmHg], 21%, and 0.03%, respectively. A comprehensive description of the circulatory center and system is available in our previous study [71]. Voluntary breathing control method using reinforcement learning Fig 9 shows the VB control model. Anatomical connections between the substantia nigra in the basal ganglia and the respiratory control centers, including a direct pathway to the pre–Bötzinger complex, are illustrated. The output of the substantia nigra indirectly informs the respiratory control centers about other ongoing movements [72, 73]. Consequently, we determined the voluntary activation levels of the diaphragm and abdominal muscles using ACRL, a mathematical model of the basal ganglia, based on [63]. In contrast, the involuntary activation levels of these muscles were derived from previously described respiratory and circulatory system models. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 9. Voluntary breathing control model. The model incorporates a muscle controller utilizing actor–critic reinforcement learning to simulate the activity of respiratory muscles (diaphragm and abdominal muscles) during voluntary breathing. Additionally, it includes a breathing rhythm generation model analogous to the respiratory CPG system. The red lines denote muscle activation signals for the respiratory muscles. https://doi.org/10.1371/journal.pcbi.1012645.g009 The VB control model (Fig 9) was developed based on a previous study by the authors [63]. Critic and actor networks were implemented using a normalized Gaussian network (NGnet) and a continuous time–space formulation for reinforcement learning [74]. NGnet models continuous state space using a Gaussian softmax network, which can generalize the state space by extrapolation, even outside the range, as a base function of the radial basis function network. Two state spaces were set: the first, s1, is the difference between the current partial pressure of CO2 in the blood (pc in Eq (9)) and the mean of the maximum value (45 mmHg) and minimum value (35 mmHg) of pc. The second, s2, is the difference between the percentage of CO2 in the air inside the mouth and the reference value, set to 0.03% in this study. The range of the first state space was set from –5 to 5 mmHg and the second from –2 to 2%. Using NGnet, the state value function V(s(t)) in the critic and actor value function am(s(t)) for the mth muscle in the actor are represented as follows: (31) (32) Here, bk(s(t)) denotes the base function represented by the following equation: (33) ci denotes the coordinates (s1, s2) of the center of the activation function, The parameter , and n represent a constant, the number of base functions, and the number of states s(t), respectively. The number of base functions K was set to 144. In an environment where the CO2 percentage in the mouth ranges from 0 to 2% and the partial pressure of CO2 in the blood ranges from 35 to 45 mmHg, the agent observes the current state s(t). This state includes the partial pressure of CO2 in the blood pc obtained from the respiratory and circulatory systems, and the CO2 percentage in the mouth (ranging randomly from 0 to 2% with 0.1 intervals) at the start of the simulation. The agent then determines the activation level input um for the diaphragm and abdominal muscles to control lung volume using Eq (38) voluntarily. The agent receives the reward r(t) as described by Eq (34). (34) where errorpco2max is the difference between the maximum calculated pc and 45 mmHg, and errorpco2min is the difference between the minimum calculated pc and 35 mmHg. Here, σr = 6, and c = 0.1. This reward function is used because the partial pressure of CO2 in the blood exhibits minimal variation compared to changes in BR and HR, and blood oxygen partial pressure in normal human subjects [75]. The critic network computes the value function V(s(t)) from the current state s(t) using Eq (31) and aims to minimize the prediction error, specifically, the TD error δ(t) as defined by Eq (35). (35) γ denotes the discount factor (0 ≤ γ ≤ 1) and τ denotes the time constant of the evaluation. In the calculation of the TD error δ(t) using Eq (35) with online learning, which updates at each time step, the backward Euler approximation of the time derivative is often employed. This involves the eligibility trace ek(t) updated using Eq (36) [74]. (36) where κ denotes the eligibility trace time constant. The value function V(s(t)) is updated using Eq (37), incorporating the eligibility trace ek(t). (37) where αV denotes the learning rate of the critic. The TD error δ(t) is then computed using Eq (35). The actor–network computes the action value function am(s(t)) for the mth muscle from the current state s(t) using Eq (32). It learns to enhance the value function V(s(t)), and maximizes the expected cumulative reward. In calculating am(s(t)) via Eq (32), the weight is updated using Eq (39), incorporating the TD error δ(t). The activation level um(t), for the mth muscle, is derived from Eq (38), which includes the weight of the action value function am(s(t)), specifically, . (38) (39) (40) where represents the maximum activation level of the mth muscle; m = 1 or 2. m1 and m2 correspond to the diaphragm and abdominal muscles, respectively; is obtained using Eq (40), and ranges from 0 to 1. The sigmoid function is denoted by sig(), with constants A and B. The actor’s learning rate is αa. The white noise function nm(t) is randomly determined for each muscle m from zero to one at each time step to explore the control output. Parameters A, B, τ, κ, αV, and αa are set to 1.0, 0.0, 0.053, 0.053, 0.3, and 0.1, respectively. UMAX determines the upper limit of voluntary activation, as described in the Results section. Four control parameters—UMAXE, TML, EXGAIN, and PCO2—were incorporated to represent the BR, TV, and MV derived from experimental exercise data. UMAXE simulates hyperventilation at exercise onset, aligning with the central command hypothesis [76]. TML, used in Eq (24), muscle length change afferent feedback based on the peripheral neural reflex hypothesis [76], directly inputting to the RVLM to increase BR and HR via RVLM presympathetic neuron activation [77]. EXGAIN and PCO2, as defined in Eq (9), correspond to excitatory signals from NTS Chemo to RTN and pre–I/I, introduced as humoral inputs essential for initiating hyperventilation at exercise onset. Experimental design Simulations were conducted for 50 s to predict the responses of respiratory and circulatory system using the specified model. Before these simulations, the four breathing control parameters—PONS, RAMPI, APSR, and UMAX—were determined. Involuntary breathing simulations were then performed for 100 s to stabilize the breathing conditions. Subsequently, simulations utilizing the online ACRL were executed with a time step of 0.0001 s under three conditions: Active or passive STEs with both knees flexed as per Ishida et al. experimental setup [1]; VB control to adjust the breathing rate from 6 to 14 bpm; and Physiological changes under MS loads. The simulation conditions commenced 20 s after the start due to the initial respiratory state instability observed until approximately 15 s. The model, incorporating the closed–loop system by Molkov et al. [64], autonomously regulates respiration akin to living organisms. Consequently, after inputting the initial four parameters, approximately three respiratory cycles are required to achieve stable respiration at rest. The parameters for controlling respiratory states under the three conditions are detailed in Table 1. In Condition 1, the BR was 14.5 bpm, which represents the BR before STEs, almost consistent with experimental data [1]. PONS, RAMPI, and APSR adjusting automatic/involuntary breathing; UMAX adjusts VB; UMAXE, TML, EXGAIN, and PCO2 adjust respiratory states during STE. The computational model of the respiratory–circulatory system as shown in Figs 8 and 9 was implemented using Python 3.8. All simulations were executed on a Dell OptiPlex 5050 computer equipped with an Intel Core i7–6700 and 16 GB Memory (DDR4). Integrated respiratory and circulatory systems Fig 8 shows a computational model for predicting the responses of the respiratory and circulatory systems. The model comprises the nucleus of the solitary tract (NTS), respiratory center and system as rhythm–generation and pattern formation regions, and circulatory center and system. The respiratory center and system were developed based on a mathematical model of the closed–loop control of breathing proposed by Molkov et al. [64]. Lung ventilation, or breathing, involves the exchange of air between the lungs and the environment, driven by the rhythmic contraction of the diaphragm and abdominal muscles. The firing activities of the phrenic and abdominal motor neurons, which control the diaphragm and abdominal muscles, respectively, are the primary outputs of the brainstem respiratory central pattern generator (CPG) that generates respiratory oscillations. The closed–loop respiratory system comprises two primary feedback pathways from the lungs to the respiratory CPG: mechanical and chemical feedback. Mechanical feedback is mediated by pulmonary stretch receptors (PSR) in the lungs, which convey lung volume information to the brainstem via the vagus nerve. Chemical feedback is provided by central chemoreceptors in the retrotrapezoid nucleus (RTN) neurons, which are highly sensitive to oxygen (O2) and carbon dioxide (CO2) levels in brain tissues. Additionally, we incorporated chemical feedback from second–order peripheral chemoreceptors, which are sensitive to O2 and CO2 levels in the blood and project directly the RTN and pre–I/I (pre–BötC) via chemoreceptors in the NTS (NTS Chemo) as described by Barnett et al. [65], thereby modulating the respiratory CPG in a CO2–dependent manner. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 8. Respiratory–circulatory system model. The model integrates the respiratory CPG system with spiking neurons, as proposed by Molkov et al. (2014) [64], and the circulatory system utilizing a rate–coding approach, as described by Ursino (1998) [66]. The red, blue, light blue, and purple lines denote excitatory and inhibitory signals, CO2 signals as chemical feedback, and breathing control signals, respectively. The light green line indicates pulmonary stretch receptors as mechanical feedback. Thick black and orange lines represent blood vessels and the interactions between the respiratory and circulatory systems, respectively. https://doi.org/10.1371/journal.pcbi.1012645.g008 In the respiratory center and system (Fig 8), each neuron in the respiratory CPG can be fundamentally modeled using Hodgkin–Huxley (HH) neurons. The model is represented by the following equations: (1) (2) where C is the membrane capacitance (C = 20 pF), and gK and gL are the peak conductance of potassium and leak conductance (leak), respectively. gKdr = 5 nS and gL = 2.8 nS, where EK and EL represent the reversal potential of potassium and leakage reverse potential, respectively. Specifically, EK = –85 mV and EL = –75 mV. The currents Ii represents the Pre–I/I (i = 1), Early–I (i = 2), Post–I (i = 3), Aug–E (i = 4), Late–E (i = 5), and Ramp–I (i = 6), and are defined as follows: (3) where gNaP and gAD are the maximum conductances, and ENa and EK are the reversal potentials for sodium and potassium, respectively. The values are gNaP = 5 nS, gAD = 10 nS, and ENa = 50 mV. The gating parameters mNa, mi(i = 2, 3, 4), and hi(i = 1, 5) are functions of the membrane variable Vi and are defined by the following equations: (4) (5) (6) (7) where τh = 4.0/cosh((Vi + 55.0)/10.0) s and (i = 2,3,4) are time constants. The variable represents the neuronal activation level and is defined as follows: (8) The neuronal activation levels of late–E (i = 5) and ramp–I (i = 6) were utilized to simulate the abdominal muscles and diaphragm, respectively. This study introduced the power exponent k to regulate the TV, corresponding to the breathing control parameter RAMPI described in the Results section; k = 1 for i = 1 ∼ 5 and k = RAMPI for i = 6. The gating variables si and qi for synaptic conductances were derived from the activity of the presynaptic neurons and other input sources using the following equations: (9) (10) (11) (12) (13) where is defined in Eq (8). The constant drive D1 originates from the pons and corresponds to the breathing control parameter PONS described in the Results section. Drive D2 is related to the partial pressure of CO2 in the blood that projects to each neuron in the respiratory CPG from the RTN (see [64] for more detailed information). The synaptic weights (a, b, c, e, f) were modified only for c1i from the original model [64] and are listed in Table 4. Eqs (1), (6), (7) and (12) were updated using the Euler method. In this study, the time step was set to 0.0001s. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 4. Parameters of respiratory CPG model. https://doi.org/10.1371/journal.pcbi.1012645.t004 For VB control, we adapted the diaphragm and abdominal muscles models using a Hill–type muscle model as described by O’Connor et al. [67]. We applied Eqs (14) and (15) to calculate the recoil pressures of the diaphragm and abdominal wall. (14) where is the static diaphragm recoil pressure at optimal length and maximum activation; udi is the phrenic activation of the diaphragm; represents the pressure due to the passive resistance of the diaphragm; is the static pressure–volume relationship of the diaphragm; and is the pressure–flow relationship of the diaphragm with velocity replaced by flow. is the passive transdiaphragmatic pressure as a function of the diaphragm volume. Detailed descriptions of , , and are provided by O’Connor et al. [67]. (15) where uab represents the activation of the abdominal muscle, and FCEmax is the maximal force capacity of the contractile element for a 1.5 cm2 cross–section of the canine external oblique muscle. is the static force–length relationship of the abdominal wall, and is the force–velocity relationship of the abdominal wall muscle. accounts for the pressure due to the passive resistance of the abdominal wall muscles. The constant kab converts force to surface tension, while (1/rs + 1/rt) translates the surface tension to pressure via Laplace’s law. The term (Vab − Vab0)/Cab represents the passive recoil pressure of the abdominal wall, where Vab0 is the volume at which the recoil pressure is zero, and Cab denotes the compliance of the abdominal wall. The respiratory CPG model determines the lung volume VL by solving Eq (16) for the pressure equilibrium involving abdominal pressure σab, pleural pressure Ppl, and diaphragm recoil pressure σdi. (16) where, Ppl is related to lung volume VL through Eq (17). (17) where Rrs is the airway resistance, VL0 is the lung volume at zero recoil pressure, and CL is the lung compliance. In addition, we modified several equations from the original model by O’Connor et al. [67] as follows: (18) (19) (20) where the vital capacity (VC), functional residual capacity (FRC) , and residual volume (RV) are set to 4.8, 2.2, and 1.2 L, respectively. To adapt the original adult rat respiratory system model [64] to the adult human respiratory system, we adjusted the parameters listed in Table 5 and revised the equation for the activity of the PSR (ref. Eq. (25) in [64]) as follows: (21) where is the residual volume of the lung. cPSR is a coefficient that adjusts the effect of the PSR on the lung volume change, corresponding to the breathing control parameter APSR described in the Results section. We set . Download: PPT PowerPoint slide PNG larger image TIFF original image Table 5. Parameters adjusted for adult human respiratory system. https://doi.org/10.1371/journal.pcbi.1012645.t005 The circulatory center was developed based on the pathways by which the CeA influences blood pressure during mental stress or anxiety, as proposed by [69], and the autonomic chronotropic control of the heart, as described by [70]. During stress, the CeA may inhibit the baroreceptive neurons in the NTS, potentially deactivating inhibitory inputs from the caudal ventrolateral medulla (CVLM) to the rostral ventrolateral medulla (RVLM). This could activate RVLM neurons, increasing sympathetic outflow, BP, and HR. The nucleus ambiguus (AMB) also receives excitatory signals from the NTS, enhancing parasympathetic outflow and reducing HR. Baroreceptors in the carotid arteries and aortic arch detect increased in the arterial blood pressure, activating afferent nerves (sinus nerves) and NTS neurons. The circulatory center and system were modeled using a rate–coding approach based on Ursino (1998) [66] with parameters for each segment in the circulatory system (Fig 8), listed in Table 2. In the circulatory center and system (Fig 8), the afferent baroreflex pathway is modeled as a first–order linear partial differential equation (Eq 22) using the carotid sinus pressure Pcs. The frequency of spikes in the afferent fibers, fcs, is represented as a sigmoidal function (Eq 23) with an intermediate variable . (22) (23) where τp and τz are time constants τp = 2.076 s and τz = 6.37 s. fmax and fmin are the upper and lower saturations limits of the frequency discharge, respectively; fmin = 2.52 spikes/s, fmax = 47.78 spikes/s, Pn is the intrasinus pressure at the central point of the sigmoidal function, and Ka is a parameter with pressure dimensions Pn = 92.0 mmHg and Ka = 11.758 mmHg. The frequencies of spikes in the efferent sympathetic nerves, fes, and the efferent vagal fibers, fev, are described by the frequency of spikes in the afferent fibers, fcs, as follows: (24) (25) where , , and kes are constants with values spikes/s, spikes/s, and kes = 0.0675 s. Similarly, , , , kev, and kresp are constants with values spikes/s, spikes/s, spikes/s, kev = 7.06 spikes/s, and kresp = 0.08 × 10–3 m3. As the state variables of the circulatory system are changed by efferent sympathetic nerve stimulation, the controlled variables θj, are defined as follows: and , represent the end–systolic elastances of the left and right ventricles (j = 1, 2); Rsp and Rep denote the hydraulic resistances of the splanchnic and extrasplanchnic peripheral circulations (j = 3, 4), respectively; and indicate the unstressed volumes of the splanchnic and extrasplanchnic venous circulation (j = 5, 6), respectively; and T signifies the heart period (j = 7). The parameters of θj are detailed in Table 3 [66]. The responses of resistances, unstressed volumes, and cardiac elastances to sympathetic drive encompass pure latency, a monotonic logarithmic static function, and low–pass first–order dynamics. Consequently, the following equations apply: (26) (27) Here, is an intermediate variable with static characteristics, and Gj represents the constant gain factor of each component. The parameters τj and Dj denote the time constant and pure latency, respectively. fesmin is the minimum sympathetic stimulation and fesmin = 2.66 Hz according to [66]. The change in heart period due to efferent vagus nerve stimulation is given by: (28) (29) is an intermediate variable that exhibits static characteristics. Gv = 0.09 s/Hz, Dv = 0.2 s, τv = 1.5 s. (30) In Eq (30), ΔTv and Δθ7 denote the variations in the heart period regulated by the sympathetic and parasympathetic nerves, respectively. The parameter represents the baseline heart period without efferent nerves stimulation, set to 0.58 s. Although the partial pressures of CO2 and O2 in the blood are reinitialized with each heartbeat in the model by Molkov et al. [64], the heartbeat initiation time was derived from changes in heart period due to autonomic nerve activity described in Eq (30) for j = 7. The lung volume VL from the respiratory system influenced parasympathetic nerve activity, as detailed in Eq (26). The total pressure and concentration of O2 and CO2 in the mouth were set to 760 [mmHg], 21%, and 0.03%, respectively. A comprehensive description of the circulatory center and system is available in our previous study [71]. Voluntary breathing control method using reinforcement learning Fig 9 shows the VB control model. Anatomical connections between the substantia nigra in the basal ganglia and the respiratory control centers, including a direct pathway to the pre–Bötzinger complex, are illustrated. The output of the substantia nigra indirectly informs the respiratory control centers about other ongoing movements [72, 73]. Consequently, we determined the voluntary activation levels of the diaphragm and abdominal muscles using ACRL, a mathematical model of the basal ganglia, based on [63]. In contrast, the involuntary activation levels of these muscles were derived from previously described respiratory and circulatory system models. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 9. Voluntary breathing control model. The model incorporates a muscle controller utilizing actor–critic reinforcement learning to simulate the activity of respiratory muscles (diaphragm and abdominal muscles) during voluntary breathing. Additionally, it includes a breathing rhythm generation model analogous to the respiratory CPG system. The red lines denote muscle activation signals for the respiratory muscles. https://doi.org/10.1371/journal.pcbi.1012645.g009 The VB control model (Fig 9) was developed based on a previous study by the authors [63]. Critic and actor networks were implemented using a normalized Gaussian network (NGnet) and a continuous time–space formulation for reinforcement learning [74]. NGnet models continuous state space using a Gaussian softmax network, which can generalize the state space by extrapolation, even outside the range, as a base function of the radial basis function network. Two state spaces were set: the first, s1, is the difference between the current partial pressure of CO2 in the blood (pc in Eq (9)) and the mean of the maximum value (45 mmHg) and minimum value (35 mmHg) of pc. The second, s2, is the difference between the percentage of CO2 in the air inside the mouth and the reference value, set to 0.03% in this study. The range of the first state space was set from –5 to 5 mmHg and the second from –2 to 2%. Using NGnet, the state value function V(s(t)) in the critic and actor value function am(s(t)) for the mth muscle in the actor are represented as follows: (31) (32) Here, bk(s(t)) denotes the base function represented by the following equation: (33) ci denotes the coordinates (s1, s2) of the center of the activation function, The parameter , and n represent a constant, the number of base functions, and the number of states s(t), respectively. The number of base functions K was set to 144. In an environment where the CO2 percentage in the mouth ranges from 0 to 2% and the partial pressure of CO2 in the blood ranges from 35 to 45 mmHg, the agent observes the current state s(t). This state includes the partial pressure of CO2 in the blood pc obtained from the respiratory and circulatory systems, and the CO2 percentage in the mouth (ranging randomly from 0 to 2% with 0.1 intervals) at the start of the simulation. The agent then determines the activation level input um for the diaphragm and abdominal muscles to control lung volume using Eq (38) voluntarily. The agent receives the reward r(t) as described by Eq (34). (34) where errorpco2max is the difference between the maximum calculated pc and 45 mmHg, and errorpco2min is the difference between the minimum calculated pc and 35 mmHg. Here, σr = 6, and c = 0.1. This reward function is used because the partial pressure of CO2 in the blood exhibits minimal variation compared to changes in BR and HR, and blood oxygen partial pressure in normal human subjects [75]. The critic network computes the value function V(s(t)) from the current state s(t) using Eq (31) and aims to minimize the prediction error, specifically, the TD error δ(t) as defined by Eq (35). (35) γ denotes the discount factor (0 ≤ γ ≤ 1) and τ denotes the time constant of the evaluation. In the calculation of the TD error δ(t) using Eq (35) with online learning, which updates at each time step, the backward Euler approximation of the time derivative is often employed. This involves the eligibility trace ek(t) updated using Eq (36) [74]. (36) where κ denotes the eligibility trace time constant. The value function V(s(t)) is updated using Eq (37), incorporating the eligibility trace ek(t). (37) where αV denotes the learning rate of the critic. The TD error δ(t) is then computed using Eq (35). The actor–network computes the action value function am(s(t)) for the mth muscle from the current state s(t) using Eq (32). It learns to enhance the value function V(s(t)), and maximizes the expected cumulative reward. In calculating am(s(t)) via Eq (32), the weight is updated using Eq (39), incorporating the TD error δ(t). The activation level um(t), for the mth muscle, is derived from Eq (38), which includes the weight of the action value function am(s(t)), specifically, . (38) (39) (40) where represents the maximum activation level of the mth muscle; m = 1 or 2. m1 and m2 correspond to the diaphragm and abdominal muscles, respectively; is obtained using Eq (40), and ranges from 0 to 1. The sigmoid function is denoted by sig(), with constants A and B. The actor’s learning rate is αa. The white noise function nm(t) is randomly determined for each muscle m from zero to one at each time step to explore the control output. Parameters A, B, τ, κ, αV, and αa are set to 1.0, 0.0, 0.053, 0.053, 0.3, and 0.1, respectively. UMAX determines the upper limit of voluntary activation, as described in the Results section. Four control parameters—UMAXE, TML, EXGAIN, and PCO2—were incorporated to represent the BR, TV, and MV derived from experimental exercise data. UMAXE simulates hyperventilation at exercise onset, aligning with the central command hypothesis [76]. TML, used in Eq (24), muscle length change afferent feedback based on the peripheral neural reflex hypothesis [76], directly inputting to the RVLM to increase BR and HR via RVLM presympathetic neuron activation [77]. EXGAIN and PCO2, as defined in Eq (9), correspond to excitatory signals from NTS Chemo to RTN and pre–I/I, introduced as humoral inputs essential for initiating hyperventilation at exercise onset. Experimental design Simulations were conducted for 50 s to predict the responses of respiratory and circulatory system using the specified model. Before these simulations, the four breathing control parameters—PONS, RAMPI, APSR, and UMAX—were determined. Involuntary breathing simulations were then performed for 100 s to stabilize the breathing conditions. Subsequently, simulations utilizing the online ACRL were executed with a time step of 0.0001 s under three conditions: Active or passive STEs with both knees flexed as per Ishida et al. experimental setup [1]; VB control to adjust the breathing rate from 6 to 14 bpm; and Physiological changes under MS loads. The simulation conditions commenced 20 s after the start due to the initial respiratory state instability observed until approximately 15 s. The model, incorporating the closed–loop system by Molkov et al. [64], autonomously regulates respiration akin to living organisms. Consequently, after inputting the initial four parameters, approximately three respiratory cycles are required to achieve stable respiration at rest. The parameters for controlling respiratory states under the three conditions are detailed in Table 1. In Condition 1, the BR was 14.5 bpm, which represents the BR before STEs, almost consistent with experimental data [1]. PONS, RAMPI, and APSR adjusting automatic/involuntary breathing; UMAX adjusts VB; UMAXE, TML, EXGAIN, and PCO2 adjust respiratory states during STE. The computational model of the respiratory–circulatory system as shown in Figs 8 and 9 was implemented using Python 3.8. All simulations were executed on a Dell OptiPlex 5050 computer equipped with an Intel Core i7–6700 and 16 GB Memory (DDR4). Acknowledgments This research was conducted as part of a collaborative project between the Intelligent Mobility Society Design/Social Cooperation Program and the Next Generation Artificial Intelligence Research Center at the University of Tokyo, and Toyota Central R&D Labs Inc. We acknowledge Editage (www.editage.com) for their assistance with English language editing.

Glynatsi, Nikoleta E.;Knight, Vincent;Harper, Marc

doi: 10.1371/journal.pcbi.1012644pmid: 39724033

Introduction The Iterated Prisoner’s Dilemma (IPD) is a repeated two-player game that models behavioral interactions, specifically interactions where self-interest clashes with collective interest. It encompasses a wide range of social and biological phenomena. In each turn of the game, both players simultaneously and independently decide between cooperation (C) and defection (D). This decision is made with the memory of all prior interactions. The payoffs for each player at each turn are influenced by their own choice and the choice of the other player. To this end, the payoffs of the game are defined by (1) where typically T > R > P > S and 2R > T + S. The most common values used in the literature [1] are R = 3, P = 1, T = 5, S = 0, and these are the values also used in this work. Conceptualizing strategies and understanding the best way to play the game have been of interest to the scientific community since the formulation of the game [2–13]. This extends to both tournament settings and population dynamics. Computer tournaments became a common evaluation technique for newly designed strategies following Axelrod’s computer tournaments in the 1980s [2, 14]. The winner of both of Axelrod’s tournaments [2, 14] was the simple strategy Tit For Tat (TFT). TFT cooperates on the first turn and thereafter copies the previous action of its opponent, retaliating against defections with a defection and forgiving a defection if followed by cooperation. Axelrod concluded that the strategy’s robustness was due to four properties, which he adapted into four suggestions for success in an IPD tournament: Do not be envious by striving for a payoff larger than the opponent’s payoff. Be “nice” by not being the first to defect. Reciprocate both cooperation and defection; Be provocable to retaliation and forgiveness. Do not be too clever by scheming to exploit the opponent. Forgiveness, in this context, is a strategy’s ability to cooperate after a DC outcome to achieve mutual cooperation again. In environments without noise, TFT would end up in DC only if it had received a defection and then retaliated. Subsequently, TFT would forgive an opponent that apologizes (in a DC round) by returning to cooperation, as mutual cooperation is deemed better than mutual defection. Due to the strategy’s strong performance in both tournaments and a series of evolutionary experiments [1], TFT was often claimed to be a highly robust (and sometimes the most robust) strategy for the IPD. There are strategies that have built upon TFT and the reciprocity-based approach. In [5], a strategy called Gradual was introduced, constructed to have the same qualities as those of TFT with one addition. Gradual has a memory of the previous rounds of play in the game, recording the number of defections by the opponent and punishing them with a growing number of defections. It then enters a calming state in which it cooperates for two rounds. A strategy with the same intuition as Gradual is Adaptive Tit for Tat [15]. Adaptive Tit for Tat maintains a continually updated estimate of the opponent’s behavior and uses this estimate to condition its future actions. Other research has built upon the limitations of TFT. For example, in [16–19], it was shown that TFT suffered in environments with noise. This was mainly due to the strategy being too provocable and its lack of generosity and contrition. Since TFT immediately punishes a defection, in a noisy environment, it can get stuck in a repeated cycle of defections and cooperations. Some new strategies, more robust in tournaments with noise, were soon introduced, including Nice and Forgiving [16], Generous Tit For Tat [3], and Pavlov (aka Win Stay Lose Shift) [4], as well as later variants such as OmegaTFT [20]. Finally, others introduced strategies deviating completely from the originally suggested properties of success. For example, a set of “envious” Iterated Prisoner’s Dilemma (IPD) strategies were introduced, called zero-determinant strategies (ZDs), in [6]. These strategies attempt to force a linear relationship between stationary payoffs against their opponents, potentially ensuring that they receive a higher average payout. While ZDs were introduced with a small tournament in which some were reportedly successful [21], this result has not generally held in future work [22]. Furthermore, in [23], a series of “clever” strategies trained using reinforcement learning were introduced. These strategies were trained using lookup tables [24], hidden Markov models [23], and finite-state automata [25], on a set of 170 strategies. One thing that has remained the same is that the introduction of a new strategy is often accompanied by a claim that the new strategy is the best performing strategy for the IPD, often without extensive testing against a broad spectrum of opponents or representative classes of opponents. The lack of testing against formally defined strategies and tournament winners is understandable given the effort required to implement the hundreds of published IPD strategies. Implementing prior strategies faithfully is often extremely difficult or impossible due to insufficient descriptions and lack of published implementations or code. Despite these challenges, the absence of thorough testing raises concerns about claims regarding the superiority or robustness of newly introduced strategies. Beyond these difficulties, we believe that limited comprehensive analyses are rooted in field conventions. Tournaments or evolutionary dynamics often rely on a select list of hand-picked strategies chosen by modelers, typically based on specific properties they wish to examine. This practice may stem from misconceptions, such as the assumption that because TFT performs relatively well, it is sufficient to test only against TFT variants. Another misconception may stem from the Press & Dyson result [6], which implies that lower memory strategies will always dominate pairwise interactions, leading some to consider only memory-one strategies. However, this result holds strictly only in pairwise interactions. It is not only the set of strategies or the tournament parameters such as noise that may impact results but also the design of the round-robin tournament itself. To address this [26] separately examined the effects of changes in format, objective criteria, and payoff values on tournament outcomes. They demonstrated that TFT’s performance declined under certain conditions. To our knowledge, this is the only study that has reanalyzed the tournament structure and critically evaluated it. However, in this work, we employ an extensive list of strategies made possible by the Axelrod-Python package, an approach that would have been difficult to achieve previously. Unlike the authors of that study, we do not consider a new tournament design. In this paper, we evaluate the performance of a significant number of IPD strategies across a diverse array of tournaments. Many of the strategies used in our analysis are drawn from well-known and named strategies in IPD literature, including previous tournament winners. This contrasts with other work that is often constrained to specific classes such as as memory-one strategies or those of a certain structural form like finite state machines or deterministic memory-two strategies. Furthermore, our tournaments encompass variations, including standard tournaments resembling Axelrod’s original ones, tournaments with noise, probabilistic match length, and both noise and probabilistic match length. This diversity in strategies and tournament types provides new insights and tests earlier claims in alternative settings against known powerful strategies. More specifically, we show that the previous tournament winners are lacking against large enough opponent pools; they do not appear among the top-performing strategies anymore. This could be due to likely suffering from a lack of diversity in the strategies they were trained/tested against, finding it hard to adapt to the new strategies. It is important to note that we do not assert the existence of a single best-performing strategy across all tournaments or tournament types. On the contrary, our work demonstrates that such a strategy does not exist (notwithstanding a few strategies with broadly high performance). The primary objective of this paper, presented in the latter parts of the paper, is to continue the discussion on the properties of successful strategies, a conversation started by Axelrod. The results of our analysis conclude that the properties of a successful strategy in the Iterated Prisoner’s Dilemma (IPD) are: Be a little bit envious Be “nice” in non-noisy environments or when game lengths are longer Reciprocate both cooperation and defection appropriately; Be provocable in tournaments with short matches, and generous in tournaments with noise It’s ok to be clever Adapt to the environment; Adjust to the mean population cooperation We believe that the discussion on the properties of winning strategies holds significant importance. It aims to provide guidance to researchers designing new strategies and those training strategies. Specifically, much like the recognized value of diversity in training datasets [27], such as variations in image perspective, skin color, etc., are critical in training accurate and generalizable machine learning models, we show that diversity in the population of opponent strategies is of paramount importance in the construction and evaluation of game theory strategies. Moreover, conducting a similar analysis can shed light on already trained strategies, aiding in understanding the key features they have autonomously developed during their training processes. Model The data collection of various types of tournaments and the use of different strategies are made possible due to an open-source library called Axelrod-Python [28] (version 3.0.0). Axelrod-Python enables the simulation of IPD tournaments and contains an extensive list of strategies. Most of these strategies are described in the literature, with a few exceptions contributed specifically to the package. In this paper, we use a total of 195 strategies, which can be found in the Supplementary Material (S1 Text). The package supports several tournament types, and this work considers standard, noisy, probabilistic ending, and noisy probabilistic ending tournaments. Standard tournaments are similar to Axelrod’s well-known tournaments [2]. In these tournaments, there are N strategies, and each strategy plays an iterated game with n turns against all other strategies, not including self-interactions. Noisy tournaments also involve N strategies and n turns, but in each turn, there is a probability pn that a player’s action is flipped. Compared to these two tournaments, in probabilistic ending tournaments the number of turns is not fixed. Instead, a match between strategies ends with a given probability pe. Finally, noisy probabilistic ending tournaments incorporate both a noise probability pn and an ending probability pe. For smoother results, each tournament is repeated k times, and this repetition factor was allowed to vary to assess the impact of smoothing. The winner of each tournament is determined based on the average score achieved by a strategy from the entire set of repetitions, not by the number of wins. To run a tournament, only a few lines of code are required (Fig 1). Specifically, one needs to define the list of strategies that the players use when participating in the tournament, the number of repetitions, the number of turns or the probability of each match ending, and the probability of noise. We demonstrate two examples in Fig 1: one with a fixed number of turns and one with a probabilistic ending. A tournament is an instance of the Tournament class. To execute the tournament, users need to run the play() method, which returns an instance of the ResultSet class. This instance includes many details of the tournament, such as the winner and the average score of the participants. Additionally, the instance contains a more detailed summary of the tournament, which we use in our analysis. We describe the results summary in detail below. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Example usage of the Axelrod-Python package for running tournaments. The strategies that players use are saved in a list, which is then passed to the Tournament class. In this example, strategies we previously discussed, such as TFT, Generous Tit-for-Tat, Gradual, and a stochastic strategy evolved through reinforcement learning, are included. We create a standard tournament where noise is set to 0, as well as a noisy tournament with probabilistic ending. Once an instance of the tournament class is defined, executing the tournament is straightforward. The play() method generates all possible pairs from the list of strategies, including pairs where each strategy plays against itself, and then iterates through each match to play it. Each match is repeated according to the specified number of repetitions, with results aggregated to a tournament summary. https://doi.org/10.1371/journal.pcbi.1012644.g001 The process of collecting tournament results is outlined in Algorithm 1. For each trial, a random size N is selected, and a random list of N strategies from the 195 available. Subsequently, one standard, one noisy, one probabilistic ending, and one noisy probabilistic ending tournament are conducted for the selected list of strategies. The parameters for the tournaments, as well as the number of repetitions, are chosen once for each trial. We have run a total of 11400 trials of Algorithm 1. For each trial, we collect the results for four different tournaments, resulting in a total of 45600 (11400 × 4) tournament results. Each tournament outputs a result summary in the form of Table 1. Algorithm 1: Tournament Data Collection Algorithm for seed ∈ [0, 11420] do  N ← randomly select integer ∈ [3, 195];  players ← randomly select N players;  k ← randomly select integer ∈ [10, 100];  n ← randomly select integer ∈ [1, 200];  pn ← randomly select float ∈ [0, 1];  pe ← randomly select float ∈ [0, 1];  result standard ← Axelrod.tournament(players, n, k);  result noisy ← Axelrod.tournament(players, n, pn, k);  result probabilistic ending ← Axelrod.tournament(players, pe, k);  result noisy probabilistic ending ← Axelrod.tournament(players, pn, pe, k); return result standard, result noisy, result probabilistic ending, result noisy probabilistic ending; Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Result summary example of a tournament. A result summary consists of N rows, with each row containing information for each strategy that participated in the tournament. This information includes the strategy’s rank (R), median score, the cooperation rate (Cr), the number of match wins, and the probability that the strategy cooperated in the opening move. Additionally, it provides the probabilities of a strategy being in any of the four states (CC, CD, DC, DD) and the cooperation rate after each state. https://doi.org/10.1371/journal.pcbi.1012644.t001 The summary contains statistics regarding each strategy that participated in the tournament, such as its rank, cooperation rate, time spent in each state when only a single past round is considered, and the probability of cooperating after each of the four possible outcomes of the previous round. In our analysis, we will use the measures for each strategy provided in the summary, as well as additional measures we calculated. Namely, these include the SSE error, the average, median, maximum, and minimum cooperation rates in each tournament. The SSE (introduced in [29]) shows how closely a strategy behaves as a zero-determinant strategy and subsequently in an extortionate way. We also consider how each strategy’s cooperation rate Cr compares to those of the tournament as a whole, for example, by comparing Cr to Cmax. During the data collection process, the probabilities of noise (pn) and tournament ending (pe) were allowed to take values between 0 and 1. However, commonly used values for these probabilities are pn ≤ 0.1 and pe ≤ 0.1. This is to make the results more interpretable. For example, consider a strategy competing in an environment with pn > 0.1. In cases with a high value of noise, most of the actions the strategy takes are the complete opposite of what the strategy is designed to do. Therefore, we will focus on the tournaments for which pn ≤ 0.1 and pe ≤ 0.1. Thus, the results presented here pertain to subsets of the noisy and probabilistic ending tournaments. Specifically, the results rely on 1150 tournaments with noise, 1134 tournaments with a probabilistic ending, and 117 tournaments with both noise and a probabilistic ending. We also provide an analysis of the paper considering the entire datasets, and these results are presented in the Supplementary Material (S1 Text). The general results of the analysis are not affected by the restriction of the noise and probabilistic ending probabilities. Results Top ranked strategies across tournaments A strategy has participated in multiple tournaments of each type, and to evaluate its overall performance, we introduce a measure called the normalized rank. In each tournament, the strategies receive a rank (R), where 0 denotes that the strategy was the winner, and N − 1 indicates that the strategy came last in the tournament. The normalized rank, denoted as r, is calculated as . Thus, the rank a strategy achieved over the number of players in the tournament. The performance of the strategies is assessed based on the median of the normalized rank, denoted as . For example, let’s consider the well-known strategies TFT and Gradual. Each strategy participated in several tournaments of each type. In Fig 2 we show the distribution of the normalised ranks of these strategies in each of the four tournaments. We can observe that TFT looks to be normally distributed normalized rank. In comparison, Gradual’s performance has longer tails, indicating that there were tournaments where the strategy performed very well or very poorly. Overall, Gradual achieves a lower median rank, signifying that it performs better than TFT except in the case of noisy and probabilistic ending tournaments (lower rank is better). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Examples of normalized rank distributions for two strategies, TFT and Gradual. We plot the distributions of r for the two strategies in the four tournament types. As a reminder, lower values of r correspond to better performances. The top left quadrant of each plot shows the distribution for standard tournaments (fixed number of turns and no noise). The top right quadrant shows the distribution for noisy tournaments (fixed number of turns and noise). The bottom left quadrant shows the distribution for probabilistic ending tournaments (no noise and probabilistic ending). Finally, the bottom right quadrant shows the distribution for noisy probabilistic ending tournaments (noise and probabilistic ending). In each quadrant, we also show the number of data points. Both strategies participated in a similar number of tournaments. Based on the median rank, which we use in this work to define overall performance, TFT performs best in probabilistic ending tournaments, whereas Gradual was in standard tournaments. https://doi.org/10.1371/journal.pcbi.1012644.g002 The top 15 strategies for each tournament type, based on , are presented in Table 2, while the r distributions for the top-ranked strategies can be found in Fig 3. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. r distributions of the top 15 strategies in different environments. A lower value of corresponds to a more successful performance. A strategy’s r distribution skewed towards zero indicates that the strategy ranked highly in most tournaments it participated in. Most distributions are skewed towards zero. https://doi.org/10.1371/journal.pcbi.1012644.g003 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 2. Top performances for each tournament type based on . The results of each type are based on 11420 unique tournaments. The results for noisy tournaments with pn < 0.1 are based on 1151 tournaments, and for probabilistic ending tournaments with pe < 0.1 on 1139. The top ranks indicate that trained strategies perform well in a variety of environments, but so do simple deterministic strategies. For noisy tournaments DBS is the top ranked strategy with , thus DBS won every tournament it participated in. The same for Evolved FSM 16 Noise 05 in probabilistic ending. https://doi.org/10.1371/journal.pcbi.1012644.t002 In standard tournaments dominating strategies were those trained using reinforcement learning techniques. 10 out of the 15 top strategies were introduced in [23]. These strategies are based on finite state automata (FSM), hidden Markov models (HMM), artificial neural networks (ANN), lookup tables (LookerUp), and stochastic lookup tables (Gambler). They have been trained using reinforcement learning algorithms (evolutionary and particle swarm algorithms) to perform well against a subset of the strategies in Axelrod-Python in a standard tournament. Thus, their performance in the specific setting was anticipated, although still noteworthy given the random sampling of tournament participants. DoubleCrosser and BackStabber, both from the Axelrod-Python, use the number of turns and are set to defect in the last two rounds. These strategies can be characterized as “cheaters” because their source code allows them to know the number of turns (unless the match has a probabilistic ending). These strategies were expected to not perform as well in tournaments where the number of turns is not specified. Finally, Winner 12 [22] and DBS [30] are both from the literature. DBS is a strategy specifically designed for noisy environments; however, it ranks highly in standard tournaments as well. Similarly, the fourth-ranked player, Evolved FSM 16 Noise 05, was trained for noisy tournaments yet performs well in standard tournaments. In the case of noisy tournaments, the top-performing strategies include strategies specifically designed for noisy tournaments. These are DBS, Evolved FSM 16 Noise 05, Evolved ANN 5 Noise 05, PSO Gambler 2 2 2 Noise 05, and Omega Tit For Tat [20]. Omega TFT, a strategy designed to break the deadlocking cycles of CD and DC that TFT can fall into in noisy environments, places 10th. The rest of the top ranks are occupied by strategies that performed well in standard tournaments and deterministic strategies such as Spiteful Tit For Tat [31], Level Punisher [32], Eugine Nier [33]. Furthermore, in tournaments with probabilistic endings, the highly ranked strategies leaned towards defecting strategies and trained finite state automata, as demonstrated by the works of Ashlock et al. [34, 35]. The most effective strategies in probabilistic ending tournaments are also a series of ensemble Meta strategies, trained strategies that performed well in standard tournaments, and Grudger [28] and Spiteful Tit for Tat [31]. The Meta strategies [28] utilize a team of strategies and aggregate the potential actions of the team members into a single action in various ways. While no single strategy consistently outperforms all others in any of the distinct tournament types or across various tournament types, certain types of strategies consistently achieve top rankings. These include strategies that have undergone training, those that retaliate, and those that adapt their behavior based on preassigned rules to optimize outcomes. These findings challenge some of Axelrod’s suggestions, particularly the advice to “not be clever” and “not be envious”. The effect of strategy features on performance For each strategy, we have a variety of features as described in Table 3. These features capture measures related to a strategy’s behavior in the tournaments it competed in, as well as intrinsic properties, such as whether a strategy is deterministic or stochastic. The correlation coefficients between the features for performance evaluation, the median score and the median normalised rank are given by Table 4. The correlation coefficients between all features have also been calculated and a graphical representation can be found in the Supplementary Material (S1 Text). Download: PPT PowerPoint slide PNG larger image TIFF original image Table 3. Included features for performance evaluation analysis. Stochastic, makes use of length and makes use of game are APL classifiers that determine whether a strategy is stochastic or deterministic, whether it makes use of the number of turns or the game’s payoffs. The memory usage is calculated as the number of turns the strategy considers to make an action (which is specified in the APL) divided by the number of turns. The SSE (introduced in [29]) shows how close a strategy is to behaving as a ZDs, and subsequently, in an extortionate way. The method identifies the ZDs closest to a given strategy and calculates the algebraic distance between them as the sum of squared error (SSE). A SSE value of 1 indicates no extortionate behaviour at all whereas a value of 0 indicates that a strategy is behaving as a ZDs. The memory usage of strategies is the number of rounds of play used by the strategy when deciding on an action, divided by the number of turns in each match. For example, Winner12 uses the previous two rounds of play, and if participating in a match with 100 turns its memory usage would be 2/100. For strategies with an infinite memory size, for example Evolved FSM 16 Noise 05, memory usage is equal to 1. Note that for tournaments with a probabilistic ending the number of turns was not collected, so the memory usage feature is not used for probabilistic ending tournaments. The rest of the features considered are the CC to C, CD to C, DC to C, and DD to C rates as well as cooperating ratio of a strategy, the minimum (Cmin), maximum (Cmax), mean (Cmean) and median (Cmedian) cooperating ratios of each tournament. https://doi.org/10.1371/journal.pcbi.1012644.t003 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 4. Correlations between the features of Table 3 and the normalised rank and the median score. For each type of tournament, standard, noisy, probabilistic ending, and noisy probabilistic ending, we conduct a correlation analysis. For each tournament, we check the correlation between each feature used in our analysis and the normalized random and median scores. Note that the correlation coefficients are calculated using Spearman’s rank correlation coefficient. A negative value indicates a negative correlation, and in the case of the normalized rank, a smaller rank translates to a better position in the tournament. https://doi.org/10.1371/journal.pcbi.1012644.t004 In standard tournaments, the features CC to C, Cr, Cr/Cmax, and the cooperating ratio compared to Cmedian and Cmean have a moderately negative effect on the normalized rank (a smaller rank is better) and a moderate positive effect on the median score. The SSE error and the DD to C rate have the opposite effects. Thus, in standard tournaments, behaving cooperatively corresponds to a more successful performance. Even though being nice generally pays off, that does not hold against defective strategies. Being more cooperative after a mutual defection, that is not retaliating, is associated with lesser overall success in terms of normalized rank. Compared to standard tournaments, in both noisy and noisy probabilistic ending tournaments, the higher the rates of cooperation, the lower a strategy’s success and median score. A strategy would not want to cooperate more than both the mean and median cooperator in such settings. In probabilistic ending tournaments, the cooperation rate of the winners and its relative comparison to the cooperation rates of the tournament have no effect. The only features that have an effect are the CD to C rate, which is the tendency of a strategy to forgive, and the SSE rate, which has a positive effect on the normalized rank. A multivariate linear regression has been fitted to model the relationship between the features and the normalized rank. Based on the graphical representation of the correlation matrices given in the Supplementary Material (S1 Text), several features are highly correlated and have been removed before fitting the linear regression model. The features included are given in Table 5 alongside their corresponding p values in distinct tournaments and their regression coefficients. The CD to C rate has a positively statistically significant effect on the normalized rank across all tournament types. This suggests that being generous tends to lower one’s performance. In the case of probabilistic ending tournaments, the coefficient of the CD to C rate is the highest, indicating that one should be more provocative in this setting. Similarly, the SEE error rate has a positive effect on the normalized rank, suggesting that being extortionate pays off, especially in noisy tournaments. The measures of cooperation, Cr and Cr/Cmax, also exhibit a significant effect. In noisy probabilistic ending tournaments, this effect is positive; however, the coefficient is very close to zero. In other tournament types, the effect is negative, indicating that one should aim to be less cooperative than the mean cooperator of the tournament. However, we cannot interpret the result as suggesting that a strategy should be as uncooperative as possible. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 5. Results of multivariate linear regressions with r as the dependent variable. R squared is reported for each model. The R scores of the fitted models indicate their capability to explain some of the variation in the median rank. Most of the features have a statistically significant effect on the normalized rank. A multivariate linear regression has also be fitted on the median score. The coefficients and p values of the features can be found in Supplementary Material (S1 Text). Both approaches lead to similar conclusions. https://doi.org/10.1371/journal.pcbi.1012644.t005 The results presented here suggest that generosity/provocation and a strategy’s cooperation rate, particularly in comparison to the tournament averages, are significant features. The analysis suggests that strategies should be more generous in noisy tournaments and less generous in probabilistic ending tournaments. Moreover, strategies should aim to not cooperate more than the mean cooperator in their tournaments. We note the analysis is limited as we only consider a linear relationship between these parameters and the rank. To further investigate the effects of the parameters discussed in this section, we have conducted a more detailed analysis in the next section, focusing on the performances of the winners of the tournaments. Features of top performing strategies In Fig 4, we present the distributions of the cooperation ratio and Cr/Cmean for the winners of tournaments. A value of Cr/Cmean = 1 implies that the cooperation ratio of the winner was the same as the mean cooperating ratio of the tournament, and we observe that this occurs for most tournament types, apart from the case of noisy and probabilistically ending tournaments. In the case of probabilistic ending tournaments, there are several winners that cooperated much less than that, confirming the results of the previous section that defecting strategies can be winners in probabilistic ending tournaments. The distribution of the cooperation rates showcases a high cooperation rate in standard tournaments and probabilistic ending tournaments. In tournaments with noise, we observe a much less cooperative behavior, which could result from strategies being cautious of potential flip actions by the co-player or strategies not suited for noise holding grudges against defections. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Distributions of Cr and Cr/Cmean for the winners of tournaments. In this distribution, we consider the winners of the tournaments, specifically the strategies that ranked first in each tournament. For each type of tournament, we plot the cooperation rate of the winner in the tournament they won, as well as the ratio of the winner’s cooperation rate to that of the entire tournament. A value of Cr/Cmean = 1 implies that the cooperation ratio of the winner was the same as the mean cooperation ratio of the tournament. https://doi.org/10.1371/journal.pcbi.1012644.g004 Analyzing the SSE distributions across different tournament types (Fig 5) suggests that successful strategies exhibit some extortionate behavior, though not consistently. ZDs are a set of strategies that are often envious, as they attempt to exploit their opponents. The winners of the tournaments considered in this work demonstrate envious behavior, but not to the extent observed in many ZDs. While the exact interactions between matches are not recorded here, the work of [23], which introduced the trained strategies appearing in the top-ranked strategies of Section, did record such interactions. In [23], it was shown that clever strategies managed to achieve mutual cooperation with stronger strategies while exploiting weaker ones. This could explain the clever winners in our analysis and the observed SSE distributions. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Distributions of SSE error for the winners of tournaments. Here, we again consider the winners of the tournaments, separated by type of tournament, and plot their SSE error. As a reminder, the SSE error indicates how closely a strategy behaves like a Zero-Determinant (ZD) strategy, and subsequently, in an extortionate way. An SSE value of 1 indicates no extortionate behavior at all, whereas a value of 0 indicates that a strategy is behaving as a ZD. https://doi.org/10.1371/journal.pcbi.1012644.g005 This might also be the reason why ZDs fail to appear in the top ranks—they attempt to exploit all opponents and cannot actively adapt back to mutual cooperation against stronger strategies, which requires a deeper memory. It’s worth noting that ZDs tend to perform poorly in population games for a similar reason: they aim to exploit other players using ZDs, failing to form a cooperative subpopulation [36]. This makes them effective invaders but poor at resisting invasion. Finally, we examine the distributions of the cooperation rates after the outcomes CC, CD, DC, and DD, as shown in Fig 6. In the case of cooperating after mutual cooperation, the results align with expectations; the distributions skew towards 1, indicating that the winners of the tournaments are more likely to cooperate after mutual cooperation. Regarding the CD outcome and the likelihood to cooperate after such a result, capturing generosity, the distributions skew towards 1/2, not 1, suggesting that strategies need to reduce their readiness to forgive. This aligns with the known result that Generous Tit For Tat generally outperforms TFT in most settings. In probabilistic ending tournaments, there is a peak at 0, suggesting that strategies should not be too generous in tournaments with short matches. Such a peak also appears in standard tournaments; however, not in tournaments with noise, where a strategy should be more generous. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Distributions of rates CC to C, CD to C, DC to C, and DD to C for the winners of tournaments. The result summary from the tournaments records how often each strategy cooperated after each possible outcome of the previous round. Specifically, we analyze the probability with which a strategy chose C following the outcomes. Here, we plot the distributions of these probabilities for the winners of the tournaments. We separate them by type of tournament, from top to bottom: standard, noisy, probabilistic ending, and noisy probabilistic ending. From left to right, we plot the distributions of cooperation after the outcomes CC, CD, DC, and DD. https://doi.org/10.1371/journal.pcbi.1012644.g006 Part of a strategy’s envious behavior can be captured by the rate of DC to C. In noisy tournaments, winners are not too envious, but in tournaments without noise, we can see that winners behave in two ways. Some are a bit envious, whereas others are very envious. In the DD to D, we can observe that, expectedly, the results are skewed towards 0. However, there are winners that attempt to recover from a DD outcome. The remaining results are as expected, skewed towards 0. Top ranked strategies across tournaments A strategy has participated in multiple tournaments of each type, and to evaluate its overall performance, we introduce a measure called the normalized rank. In each tournament, the strategies receive a rank (R), where 0 denotes that the strategy was the winner, and N − 1 indicates that the strategy came last in the tournament. The normalized rank, denoted as r, is calculated as . Thus, the rank a strategy achieved over the number of players in the tournament. The performance of the strategies is assessed based on the median of the normalized rank, denoted as . For example, let’s consider the well-known strategies TFT and Gradual. Each strategy participated in several tournaments of each type. In Fig 2 we show the distribution of the normalised ranks of these strategies in each of the four tournaments. We can observe that TFT looks to be normally distributed normalized rank. In comparison, Gradual’s performance has longer tails, indicating that there were tournaments where the strategy performed very well or very poorly. Overall, Gradual achieves a lower median rank, signifying that it performs better than TFT except in the case of noisy and probabilistic ending tournaments (lower rank is better). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Examples of normalized rank distributions for two strategies, TFT and Gradual. We plot the distributions of r for the two strategies in the four tournament types. As a reminder, lower values of r correspond to better performances. The top left quadrant of each plot shows the distribution for standard tournaments (fixed number of turns and no noise). The top right quadrant shows the distribution for noisy tournaments (fixed number of turns and noise). The bottom left quadrant shows the distribution for probabilistic ending tournaments (no noise and probabilistic ending). Finally, the bottom right quadrant shows the distribution for noisy probabilistic ending tournaments (noise and probabilistic ending). In each quadrant, we also show the number of data points. Both strategies participated in a similar number of tournaments. Based on the median rank, which we use in this work to define overall performance, TFT performs best in probabilistic ending tournaments, whereas Gradual was in standard tournaments. https://doi.org/10.1371/journal.pcbi.1012644.g002 The top 15 strategies for each tournament type, based on , are presented in Table 2, while the r distributions for the top-ranked strategies can be found in Fig 3. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. r distributions of the top 15 strategies in different environments. A lower value of corresponds to a more successful performance. A strategy’s r distribution skewed towards zero indicates that the strategy ranked highly in most tournaments it participated in. Most distributions are skewed towards zero. https://doi.org/10.1371/journal.pcbi.1012644.g003 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 2. Top performances for each tournament type based on . The results of each type are based on 11420 unique tournaments. The results for noisy tournaments with pn < 0.1 are based on 1151 tournaments, and for probabilistic ending tournaments with pe < 0.1 on 1139. The top ranks indicate that trained strategies perform well in a variety of environments, but so do simple deterministic strategies. For noisy tournaments DBS is the top ranked strategy with , thus DBS won every tournament it participated in. The same for Evolved FSM 16 Noise 05 in probabilistic ending. https://doi.org/10.1371/journal.pcbi.1012644.t002 In standard tournaments dominating strategies were those trained using reinforcement learning techniques. 10 out of the 15 top strategies were introduced in [23]. These strategies are based on finite state automata (FSM), hidden Markov models (HMM), artificial neural networks (ANN), lookup tables (LookerUp), and stochastic lookup tables (Gambler). They have been trained using reinforcement learning algorithms (evolutionary and particle swarm algorithms) to perform well against a subset of the strategies in Axelrod-Python in a standard tournament. Thus, their performance in the specific setting was anticipated, although still noteworthy given the random sampling of tournament participants. DoubleCrosser and BackStabber, both from the Axelrod-Python, use the number of turns and are set to defect in the last two rounds. These strategies can be characterized as “cheaters” because their source code allows them to know the number of turns (unless the match has a probabilistic ending). These strategies were expected to not perform as well in tournaments where the number of turns is not specified. Finally, Winner 12 [22] and DBS [30] are both from the literature. DBS is a strategy specifically designed for noisy environments; however, it ranks highly in standard tournaments as well. Similarly, the fourth-ranked player, Evolved FSM 16 Noise 05, was trained for noisy tournaments yet performs well in standard tournaments. In the case of noisy tournaments, the top-performing strategies include strategies specifically designed for noisy tournaments. These are DBS, Evolved FSM 16 Noise 05, Evolved ANN 5 Noise 05, PSO Gambler 2 2 2 Noise 05, and Omega Tit For Tat [20]. Omega TFT, a strategy designed to break the deadlocking cycles of CD and DC that TFT can fall into in noisy environments, places 10th. The rest of the top ranks are occupied by strategies that performed well in standard tournaments and deterministic strategies such as Spiteful Tit For Tat [31], Level Punisher [32], Eugine Nier [33]. Furthermore, in tournaments with probabilistic endings, the highly ranked strategies leaned towards defecting strategies and trained finite state automata, as demonstrated by the works of Ashlock et al. [34, 35]. The most effective strategies in probabilistic ending tournaments are also a series of ensemble Meta strategies, trained strategies that performed well in standard tournaments, and Grudger [28] and Spiteful Tit for Tat [31]. The Meta strategies [28] utilize a team of strategies and aggregate the potential actions of the team members into a single action in various ways. While no single strategy consistently outperforms all others in any of the distinct tournament types or across various tournament types, certain types of strategies consistently achieve top rankings. These include strategies that have undergone training, those that retaliate, and those that adapt their behavior based on preassigned rules to optimize outcomes. These findings challenge some of Axelrod’s suggestions, particularly the advice to “not be clever” and “not be envious”. The effect of strategy features on performance For each strategy, we have a variety of features as described in Table 3. These features capture measures related to a strategy’s behavior in the tournaments it competed in, as well as intrinsic properties, such as whether a strategy is deterministic or stochastic. The correlation coefficients between the features for performance evaluation, the median score and the median normalised rank are given by Table 4. The correlation coefficients between all features have also been calculated and a graphical representation can be found in the Supplementary Material (S1 Text). Download: PPT PowerPoint slide PNG larger image TIFF original image Table 3. Included features for performance evaluation analysis. Stochastic, makes use of length and makes use of game are APL classifiers that determine whether a strategy is stochastic or deterministic, whether it makes use of the number of turns or the game’s payoffs. The memory usage is calculated as the number of turns the strategy considers to make an action (which is specified in the APL) divided by the number of turns. The SSE (introduced in [29]) shows how close a strategy is to behaving as a ZDs, and subsequently, in an extortionate way. The method identifies the ZDs closest to a given strategy and calculates the algebraic distance between them as the sum of squared error (SSE). A SSE value of 1 indicates no extortionate behaviour at all whereas a value of 0 indicates that a strategy is behaving as a ZDs. The memory usage of strategies is the number of rounds of play used by the strategy when deciding on an action, divided by the number of turns in each match. For example, Winner12 uses the previous two rounds of play, and if participating in a match with 100 turns its memory usage would be 2/100. For strategies with an infinite memory size, for example Evolved FSM 16 Noise 05, memory usage is equal to 1. Note that for tournaments with a probabilistic ending the number of turns was not collected, so the memory usage feature is not used for probabilistic ending tournaments. The rest of the features considered are the CC to C, CD to C, DC to C, and DD to C rates as well as cooperating ratio of a strategy, the minimum (Cmin), maximum (Cmax), mean (Cmean) and median (Cmedian) cooperating ratios of each tournament. https://doi.org/10.1371/journal.pcbi.1012644.t003 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 4. Correlations between the features of Table 3 and the normalised rank and the median score. For each type of tournament, standard, noisy, probabilistic ending, and noisy probabilistic ending, we conduct a correlation analysis. For each tournament, we check the correlation between each feature used in our analysis and the normalized random and median scores. Note that the correlation coefficients are calculated using Spearman’s rank correlation coefficient. A negative value indicates a negative correlation, and in the case of the normalized rank, a smaller rank translates to a better position in the tournament. https://doi.org/10.1371/journal.pcbi.1012644.t004 In standard tournaments, the features CC to C, Cr, Cr/Cmax, and the cooperating ratio compared to Cmedian and Cmean have a moderately negative effect on the normalized rank (a smaller rank is better) and a moderate positive effect on the median score. The SSE error and the DD to C rate have the opposite effects. Thus, in standard tournaments, behaving cooperatively corresponds to a more successful performance. Even though being nice generally pays off, that does not hold against defective strategies. Being more cooperative after a mutual defection, that is not retaliating, is associated with lesser overall success in terms of normalized rank. Compared to standard tournaments, in both noisy and noisy probabilistic ending tournaments, the higher the rates of cooperation, the lower a strategy’s success and median score. A strategy would not want to cooperate more than both the mean and median cooperator in such settings. In probabilistic ending tournaments, the cooperation rate of the winners and its relative comparison to the cooperation rates of the tournament have no effect. The only features that have an effect are the CD to C rate, which is the tendency of a strategy to forgive, and the SSE rate, which has a positive effect on the normalized rank. A multivariate linear regression has been fitted to model the relationship between the features and the normalized rank. Based on the graphical representation of the correlation matrices given in the Supplementary Material (S1 Text), several features are highly correlated and have been removed before fitting the linear regression model. The features included are given in Table 5 alongside their corresponding p values in distinct tournaments and their regression coefficients. The CD to C rate has a positively statistically significant effect on the normalized rank across all tournament types. This suggests that being generous tends to lower one’s performance. In the case of probabilistic ending tournaments, the coefficient of the CD to C rate is the highest, indicating that one should be more provocative in this setting. Similarly, the SEE error rate has a positive effect on the normalized rank, suggesting that being extortionate pays off, especially in noisy tournaments. The measures of cooperation, Cr and Cr/Cmax, also exhibit a significant effect. In noisy probabilistic ending tournaments, this effect is positive; however, the coefficient is very close to zero. In other tournament types, the effect is negative, indicating that one should aim to be less cooperative than the mean cooperator of the tournament. However, we cannot interpret the result as suggesting that a strategy should be as uncooperative as possible. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 5. Results of multivariate linear regressions with r as the dependent variable. R squared is reported for each model. The R scores of the fitted models indicate their capability to explain some of the variation in the median rank. Most of the features have a statistically significant effect on the normalized rank. A multivariate linear regression has also be fitted on the median score. The coefficients and p values of the features can be found in Supplementary Material (S1 Text). Both approaches lead to similar conclusions. https://doi.org/10.1371/journal.pcbi.1012644.t005 The results presented here suggest that generosity/provocation and a strategy’s cooperation rate, particularly in comparison to the tournament averages, are significant features. The analysis suggests that strategies should be more generous in noisy tournaments and less generous in probabilistic ending tournaments. Moreover, strategies should aim to not cooperate more than the mean cooperator in their tournaments. We note the analysis is limited as we only consider a linear relationship between these parameters and the rank. To further investigate the effects of the parameters discussed in this section, we have conducted a more detailed analysis in the next section, focusing on the performances of the winners of the tournaments. Features of top performing strategies In Fig 4, we present the distributions of the cooperation ratio and Cr/Cmean for the winners of tournaments. A value of Cr/Cmean = 1 implies that the cooperation ratio of the winner was the same as the mean cooperating ratio of the tournament, and we observe that this occurs for most tournament types, apart from the case of noisy and probabilistically ending tournaments. In the case of probabilistic ending tournaments, there are several winners that cooperated much less than that, confirming the results of the previous section that defecting strategies can be winners in probabilistic ending tournaments. The distribution of the cooperation rates showcases a high cooperation rate in standard tournaments and probabilistic ending tournaments. In tournaments with noise, we observe a much less cooperative behavior, which could result from strategies being cautious of potential flip actions by the co-player or strategies not suited for noise holding grudges against defections. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Distributions of Cr and Cr/Cmean for the winners of tournaments. In this distribution, we consider the winners of the tournaments, specifically the strategies that ranked first in each tournament. For each type of tournament, we plot the cooperation rate of the winner in the tournament they won, as well as the ratio of the winner’s cooperation rate to that of the entire tournament. A value of Cr/Cmean = 1 implies that the cooperation ratio of the winner was the same as the mean cooperation ratio of the tournament. https://doi.org/10.1371/journal.pcbi.1012644.g004 Analyzing the SSE distributions across different tournament types (Fig 5) suggests that successful strategies exhibit some extortionate behavior, though not consistently. ZDs are a set of strategies that are often envious, as they attempt to exploit their opponents. The winners of the tournaments considered in this work demonstrate envious behavior, but not to the extent observed in many ZDs. While the exact interactions between matches are not recorded here, the work of [23], which introduced the trained strategies appearing in the top-ranked strategies of Section, did record such interactions. In [23], it was shown that clever strategies managed to achieve mutual cooperation with stronger strategies while exploiting weaker ones. This could explain the clever winners in our analysis and the observed SSE distributions. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Distributions of SSE error for the winners of tournaments. Here, we again consider the winners of the tournaments, separated by type of tournament, and plot their SSE error. As a reminder, the SSE error indicates how closely a strategy behaves like a Zero-Determinant (ZD) strategy, and subsequently, in an extortionate way. An SSE value of 1 indicates no extortionate behavior at all, whereas a value of 0 indicates that a strategy is behaving as a ZD. https://doi.org/10.1371/journal.pcbi.1012644.g005 This might also be the reason why ZDs fail to appear in the top ranks—they attempt to exploit all opponents and cannot actively adapt back to mutual cooperation against stronger strategies, which requires a deeper memory. It’s worth noting that ZDs tend to perform poorly in population games for a similar reason: they aim to exploit other players using ZDs, failing to form a cooperative subpopulation [36]. This makes them effective invaders but poor at resisting invasion. Finally, we examine the distributions of the cooperation rates after the outcomes CC, CD, DC, and DD, as shown in Fig 6. In the case of cooperating after mutual cooperation, the results align with expectations; the distributions skew towards 1, indicating that the winners of the tournaments are more likely to cooperate after mutual cooperation. Regarding the CD outcome and the likelihood to cooperate after such a result, capturing generosity, the distributions skew towards 1/2, not 1, suggesting that strategies need to reduce their readiness to forgive. This aligns with the known result that Generous Tit For Tat generally outperforms TFT in most settings. In probabilistic ending tournaments, there is a peak at 0, suggesting that strategies should not be too generous in tournaments with short matches. Such a peak also appears in standard tournaments; however, not in tournaments with noise, where a strategy should be more generous. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Distributions of rates CC to C, CD to C, DC to C, and DD to C for the winners of tournaments. The result summary from the tournaments records how often each strategy cooperated after each possible outcome of the previous round. Specifically, we analyze the probability with which a strategy chose C following the outcomes. Here, we plot the distributions of these probabilities for the winners of the tournaments. We separate them by type of tournament, from top to bottom: standard, noisy, probabilistic ending, and noisy probabilistic ending. From left to right, we plot the distributions of cooperation after the outcomes CC, CD, DC, and DD. https://doi.org/10.1371/journal.pcbi.1012644.g006 Part of a strategy’s envious behavior can be captured by the rate of DC to C. In noisy tournaments, winners are not too envious, but in tournaments without noise, we can see that winners behave in two ways. Some are a bit envious, whereas others are very envious. In the DD to D, we can observe that, expectedly, the results are skewed towards 0. However, there are winners that attempt to recover from a DD outcome. The remaining results are as expected, skewed towards 0. Discussion This manuscript explores the performance of 195 strategies in the IPD in thousands of computer tournaments. The collection of computer tournaments presented here is the largest and most diverse in the literature. The 195 strategies are drawn from Axelrod-Python library and include strategies from the IPD literature. The computer tournaments encompass four different types. So, what is the best way to play the IPD? And is there a single dominant strategy for the IPD? There was not a single strategy within the collection of 195 strategies that managed to perform well in all the tournament variations it competed in. A strategy ranking highly in a specific environment did not guarantee its success over different tournament types, with a few exceptions—strategies that generalize better. Already well-known in the AI/ML literature, adding noise to training data leads to more robust models [37]. We see that clearly here, where the strategies trained for noise (or designed for noise) tend to be better generalists. There were instances where a few strategies trained in narrow conditions outperformed more generalist strategies, as they tend to overfit. However, the strategies trained with noise perform well in general, whilst the strategies trained specifically on no noise or small subpopulations do not. We also examined the best-performing strategies across various tournament types and analyzed their salient features. This demonstrated that there are properties associated with the success of strategies that contradict the originally suggested properties of Axelrod [1]. We showed that complex or clever strategies can be effective, whether trained against a corpus of possible opponents or purposely designed to mitigate the impact of noise such as the DBS strategy. Moreover, we found some strategies designed or trained for noisy environments were also highly ranked in noise-free tournaments which reinforces the idea that strategies’ complexity/cleverness is not necessarily a liability, rather it can confer adaptability to a more diverse set of environments. We also showed that while the type of exploitation attempted by ZDs is not typically effective in standard tournaments, envious strategies capable of both exploiting and not their opponents can be highly successful. Based on the results of [23] this could be because they are selectively exploiting weaker opponents while mutually cooperating with stronger opponents. Highly noisy or tournaments with short matches also favoured envious strategies. These environments mitigated the value of being nice. Uncertainty enables exploitation, reducing the ability of maintaining or enforcing mutual cooperation, while triggering grudging strategies to switch from typically cooperating to typically defecting. The features analysis of the best performing strategies demonstrated that a strategy should reciprocate, as suggested by Axelrod, but it should relax its readiness to do so and be more generous. For noisy environments this is inline with the results of [16–19], however, we also showed that generosity pays off even in standard settings, and that in fact the only setting a strategy would want to be too provocable is when the matches are not long. Forgiveness as defined by Axerlod was not explored here. This was mainly because the two round states were not recorded during the data collection. This could be a topic of future work that examines the impact of considering more rounds of history. The features analysis also concluded that there is a significant importance in adapting to the environment, and more specifically, to the mean cooperator. In most tournament types, the winner of the tournament was also the average cooperator. Even in tournaments with short matches where defecting behavior could secure a win, a large number of winners were average cooperators. This could potentially explain the early success of TFT. TFT naturally achieves a cooperation rate near Cmean by virtue of copying its opponent’s last move while also minimizing instances where it is exploited by an opponent (cooperating while the opponent defects), at least in non-noisy tournaments. It could also explain why Tit For N Tats does not fare well for N > 1—it fails to achieve the proper cooperation ratio by tolerating too many defections. Our results may also help explain the historically unexpected effectiveness of memory-one strategies [38]. The success of these strategies contradicts the intuitive assumption that a longer memory and therefore more information would yield better strategic performance [39]. Given that among the important features associated with success are the relative cooperation rate to the population average and the four memory-one probabilities of cooperating conditional on the previous round of play, these features can be optimized by a memory-one strategy such as TFT. Usage of more history becomes valuable when there are exploitable opponent patterns. This is indicated by the importance of SSE as a feature, showing that the first-approximation provided by a memory-one strategy is no longer sufficient. These results highlight a central idea in evolutionary game theory in this context: the fitness landscape is a function of the population (where fitness in this case is tournament performance) [40]. While that may seem obvious now, it shows why historical tournament results on small or arbitrary populations of strategies have so often failed to produce generalizable results. To this end, many strategies, such as Win-Stay-Lose-Shift and Generous Tit For Tat, emerged due to their strong performance in evolutionary dynamics. Axelrod’s original work relied on computer tournaments, so we chose to remain consistent with this approach, as a comprehensive study like ours had not yet been undertaken. However, evolutionary settings would be an exciting direction for future study. Overall, the five properties successful strategies need to have in a IPD competition based on the analysis that has been presented in this manuscript are: Be “nice” in non-noisy environments or when game lengths are longer Be provocable in tournaments with short matches, and generous in tournaments with noise Be a little bit envious Be clever Adapt to the environment (including the population of strategies). The results presented here were based only on a subset of the whole data we have collected. The analysis of the full dataset is discussed in the Supplementary Material (S1 Text). However, we can see that the general results of our work remain the same. In the Supplementary Material (S1 Text), we also evaluate the importance of features using a random forest classifier and a clustering approach. The results of these analyses are also in line with the results presented here. The data set described in this work contains the largest number of IPD tournaments, to the authors knowledge. The raw data set is available at [41] and the processed data at [42]. Further data mining could be applied and provide new insights in the field. Supporting information S1 Text. Supplementary material. This document provides details of further analysis, a summary of all parameters and notations used in the manuscript, and a comprehensive list of all strategies considered in this work. https://doi.org/10.1371/journal.pcbi.1012644.s001 (PDF) Acknowledgments A variety of open-source software have been used in this work. The authors would like to express their gratitude to the open-source software community, whose invaluable contributions significantly enhanced the development and execution of this research. Namely, the authors would like to thank the developers of the following software packages: Axelrod-Python library for IPD simulations, the Matplotlib library for visualisation, The Numpy library for data manipulation, and finally the scikit-learn library for data analysis.

Sun, Yutong;Qiu, Peng

doi: 10.1371/journal.pcbi.1012643pmid: 39666603

Introduction As scRNA-seq technologies continue to advance, analyzing gene expression patterns at the single-cell level has become an increasingly popular approach to understand cellular heterogeneity and differentiation [1, 2]. Identification of marker genes is a crucial component of the analysis, as the marker genes allow us to distinguish various cell clusters and identify their cell types and states based on their unique gene expression signatures [3]. Selection of marker genes is a non-trivial task. In the literature, existing marker gene selection approaches can be organized into two categories. One category adopts a one-vs-all strategy [4–6], while the other uses a hierarchical strategy [7]. The one-vs-all strategy is more commonly used to identify marker genes in scRNA-seq data. Methods adopting this strategy aim to identify marker genes that exhibit differential expression between one cell cluster and the combination of other cell clusters. For example, Seurat [4] identifies differentially expressed genes in each cell cluster compared to all other cells in the dataset using a Wilcoxon rank sum test. Genes with the highest differential expression are selected as marker genes for each cell cluster. Monocle [5] uses logistic regression to identify genes that are differentially expressed between each cell cluster and all other cells in the dataset. Genes with the highest probability of being expressed in a specific cell cluster are selected as marker genes. There are several ranking-based one-vs-all marker gene selection methods. SingleR [6] compares gene expression profiles in scRNA-seq datasets to reference bulk transcriptomic data of sorted cell clusters to identify marker genes that are differentially expressed between the target cell cluster and all other cell clusters. SingleR employs a ranking-based method that takes into account the degree of differential expression and the prevalence of the marker gene in the target cell cluster. The top-ranked genes for each cell cluster are selected as the marker genes for that cell cluster. COMET [3] is a ranking-based brute force approach that selects sets of up to four markers with the best predictive power to separate one cell cluster. RankCorr [8] applies a rank sum transformation that provides a non-parametric way of considering the counts eliminates the need to normalize the data, and selects an informative number of markers for each cluster in a one-vs-all fashion. Several existing methods apply consensus optimization to select marker genes. SC3 [9] proposes an unsupervised consensus clustering approach by combine binary classification based on mean cluster-expression values and Wilcoxon signed-rank test to compute p-values. Genes with high areas under ROC curve and low p-value are selected as marker genes. Both SCMarker [10] and scTIM [11] use consensus optimization strategy to identify marker genes. SCMarker uses a mixture distribution model to select genes that are individually discriminative across underlying cell clusters and are either co-expressed or mutually exclusively expressed with other genes. scTIM integrates ‘gene specificity’, ‘cell relation network’, and ‘gene redundancy’ into a multi-objective optimization problem. SMaSH filters and ranks genes according to an ensemble learning model or a deep neural network. These one-vs-all marker gene selection approaches do not take advantage of the hierarchical relationships that exist among cell clusters and the correlations in expression patterns among genes, which are crucial aspects in obtaining a comprehensive understanding of cell cluster identities and biological processes [7]. A relatively less popular strategy for marker gene selection is to incorporate the hierarchical structure of cell clusters, which not only identifies marker genes for individual cell clusters (leaves of a hierarchical tree), but also provides marker genes for subsets of closely related cell clusters (intermediate nodes of the hierarchy). This strategy has the potential to provide more interpretable markers, as genes that are differentially expressed across multiple subsets of cell clusters are more likely to be involved in key biological processes. One such algorithm is scGeneFit [7], which combined multi-variate projection and hierarchical ideas. Given a cell cluster hierarchy provided by expert users or hierarchical clustering algorithms, scGeneFit solves one projection problem for each split in the hierarchy, where each projection problem aims to find to the lowest-dimensional subspace to separate cells in different classes defined by the corresponding split in the hierarchy. In scGeneFit, defining the cell cluster hierarchy and finding marker genes are considered separately and consecutively, and hence, the definition of cell cluster hierarchy is not affected by the subsequent analysis of finding marker genes. In this paper, we explore a hierarchical approach for finding marker genes, where the definition of cell cluster hierarchy and the identification of marker genes are jointly considered. Our approach is motivated by one drawback of the one-vs-all strategy, which often generates overlapping marker genes for closely related cell clusters, capturing their common signature but providing limited information to distinguish them. One example is shown in Fig 1a. When one-vs-all cell cluster marker genes are visualized using heatmaps, diagonal blocks of high expression confirm that the identified cell cluster marker genes are indeed highly expressed in the corresponding cell clusters, while off-diagonal blocks of high expression indicate that marker genes for one cell cluster may also be highly expressed in other cell clusters which is undesirable. Therefore, we propose to identify an optimal grouping of cell clusters that minimizes off-diagonal expression signal, so that the one-vs-all strategy identifies marker genes specific to each cell cluster group. Within each cell cluster group, the same analysis can be applied to identify optimal subgroups, so that the one-vs-all strategy produces marker genes highly specific to each subgroup. Iterating such off-diagonal minimization analysis induces a cell cluster hierarchy, as well as marker genes for the cell clusters or cell cluster groups at each split of the hierarchy. Using real scRNA-seq datasets, we compared our hierarchical approach with the one-vs-all marker selection approach in Seurat and the hierarchical approach in scGeneFit, and demonstrated the advantage of our approach in terms of its interpretability and its performance in automated cell type mapping. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Overview of hierarchical marker gene selection in PBMC3k data. (a) Marker gene heatmap generated by the one-vs-all FindMarker approach in Seurat. (b) Our constructed hierarchical structure of cell clusters in the PBMC3k dataset. (c) Assembled heatmap that concatenates marker gene heatmaps for individual splits in the constructed cell cluster hierarchy. https://doi.org/10.1371/journal.pcbi.1012643.g001 Results Hierarchical marker genes selection framework To motivate the proposed hierarchical marker gene selection framework, we used an example scRNA-seq dataset of PBMC, which contained data for 2638 cells grouped into 9 cell clusters that correspond to 9 cell types. The cell types included Naive CD4 T cells, CD14+ Monocytes, Memory CD4 T cells, B cells, CD8 T cells, FCGR3A+ Monocytes, NK cells, Dendritic cells and Platelets. Applying the one-vs-all find marker approach implemented in Seurat, the expression patterns of the identified markers were shown as the heatmap in Fig 1a, where each vertical section corresponds to one cell cluster labeled by its cell type name. In addition to the diagonal blocks of high expression that confirmed the expression of identified marker genes in their corresponding cell clusters, multiple off-diagonal blocks of high expression were observed. For example, the marker genes for the first cell cluster (Naive CD4 T cells) were also highly expressed in the third cell cluster (Memory CD4 T cells). This is reasonable because the two CD4 T cell subtypes are closely related. However, these marker genes may not provide sufficient information to interpret and separate these two clusters corresponding to the two closely related CD4 T cell subtypes. We propose a scoring function defined as the average of diagonal expression minus the average of off-diagonal expression (details in Methods). This scoring function quantifies how much undesirable off-diagonal expression exists in the marker gene heatmap. We then combine two of the cell clusters, re-do the one-vs-all marker gene identification to re-generate the marker gene heatmap, and use the scoring functions to quantify how much off-diagonal expression exists after the two cell clusters are combined. We examine all possible pairs of cell clusters to find the best pair whose combination leads to the least off-diagonal expression in the marker gene heatmap. If combining this best cell cluster pair is able reduce off-diagonal expression compared to not combining them, we merge this pair of cell clusters, so that the number of cell clusters reduces by one. After that, we perform the same analysis to the resulting cell clusters, identify the best pair of cell clusters whose combination leads to least off-diagonal expression, and merge this pair if the off-diagonal expression is further reduced after merging. This process iterates until no merge is able to further reduce off-diagonal expression in the marker gene heatmap. This is essentially an agglomerative clustering process of the cell clusters, using the proposed scoring function as both distance metric and stopping criterion. In this example dataset, the agglomerative process stopped when the 9 original cell clusters were merged into two: one was the cell cluster corresponding to Platelets and the other was the remaining 8 cell clusters combined. As shown in Fig 1b, the first split of our cell clusters hierarchy had two branches, separating the Platelets and all other cell types. Performing one-vs-all marker finding for these two branches produced marker genes for Platelets and marker genes for other cell types combined, and the resulting heatmap is shown next to the first split in Fig 1b. To further construct the cell cluster hierarchy, we focused on the 8 cell clusters belonging to the left branch of first split, as if we were analyzing a new dataset composed of these 8 cell clusters. We agglomeratively merged these 8 cell clusters, and used the scoring function to stop the agglomeration when the off-diagonal expression was minimized. In this example, the 8 cell clusters were agglomeratively merged into two groups, so that the second split of the cell cluster hierarchy was also a two-way split: one branch was the combination of the 3 myeloid cell clusters (Monocytes and Dendritic cells), and the other branch was the combination of the remaining 5 lymphoid cell clusters (B cells, T cells and NK cells). Similarly, the branch containing the 3 myeloid cell clusters and the branch containing the 5 lymphoid cell clusters were examined separately to construct additional splits in the cell cluster hierarchy. This construction process iterated until all of the 9 original cell clusters were separated as leaf nodes in the cell cluster hierarchy. In summary, our cell cluster hierarchy is essentially a divisive hierarchical clustering process, where each split is determined by an agglormerative process to minimize undesirable off-diagonal expression and hence maximize specificity of the identified marker genes. The marker gene heatmaps for individual splits in the cell cluster hierarchy can be concatenated and assembled into Fig 1c, where each horizontal section corresponds to an individual split named by the corresponding cell types within the split. Since majority of the splits only considered a subset of the cell clusters, for a particular horizontal section corresponding to one split, expression data for cell clusters not considered in the split were zeroed out and shown as white areas in the assembled heatmap. This visualization provides a compact view of marker genes defined by our hierarchical approach. Data collections for evaluation To evaluate the proposed hierarchical marker gene selection approach, we applied it to three peripheral blood mononuclear cells (PBMC) datasets, namely PBMC3k [12], PBMC control, and PBMC stimuated [13]. PBMC3k is the dataset used as the illustrative example in Fig 1. The PBMC control dataset contained 6573 cells, and the PBMC stimulated dataset contained 7263 cells. Cells in both of these two datasets were grouped into 13 cell clusters. In addition to the PBMC datasets, we also included analysis based on a human pancreas dataset published by Xin [14, 15], which contained 1492 cells grouped into 4 cell clusters. The purpose is to provide an example dataset where the proposed hierarchical marker selection approach degenerated to the flat one-vs-all approach, because the cell clusters were sufficiently distinct and the flat one-vs-all approach did not produce much undesirable off-diagonal expression signal in the marker expression heatmap. Hierarchical marker genes capture more cell type differences We applied the proposed hierarchical marker gene selection to the three PBMC datasets (PBMC3k, PBMC control, PBMC stim). The hierarchical marker genes for PBMC3k data are visualized in Fig 1b. The hierarchical marker genes for the other two PBMC datasets are shown in Figs 2b and 3b. In addition, we applied the one-vs-all marker finding approach in Seurat [16], and both the flat and hierarchical versions of scGeneFit [7] to the three datasets. We compared the selected genes, as well as directly using all genes or the highly variable genes, in terms of their ability to separate the cell types annotated in these datasets. More specifically, each dataset was split into a training set and a testing set with a 7:3 ratio. K-Nearest Neighbor classifiers were trained based on genes selected by various approaches, and the classification accuracies were evaluated using the testing set. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Hierarchical marker gene selection in PBMC control dataset. (a) Marker gene heatmap generated by the one-vs-all FindMarker approach in Seurat. (b) Constructed hierarchy of cell clusters in PBMC control dataset. (c) Assembled heatmap that summarizes all marker genes for various splits in the cell cluster hierarchy. https://doi.org/10.1371/journal.pcbi.1012643.g002 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Hierarchical marker gene selection in PBMC stim dataset. (a) Marker gene heatmap generated by the one-vs-all FindMarker approach in Seurat. (b) Constructed hierarchy of cell clusters in PBMC stim dataset.(c) Assembled heatmap that summarizes all marker genes for various splits in the cell cluster hierarchy. https://doi.org/10.1371/journal.pcbi.1012643.g003 For the PBMC3k dataset, the classification accuracies are shown in Fig 4a. The first 9 sets of color bars show cell-type-wise classification accuracies. “Average” denotes the average value of cell-type-wise classification accuracies across all cell types. We observed that the cell-type-wise classification performance varied across cell types. As baseline references, classification based on all genes or the top 1406 highly variable genes (the number of highly variable genes were determined automatically by Scanpy with its default parameters) achieved accuracy between 39% and 98% for various cell types, with an average accuracy of around 80%. The one-vs-all marker genes from Seurat improved classification accuracy by around 4%, even though the number of the one-vs-all marker genes was 118, far fewer than the highly variable genes. The marker genes generated by the flat and hierarchical versions of scGeneFit did not outperform the baselines. Finally, with the same number of marker genes compared to the one-vs-all approach in Seurat, the proposed hierarchical marker genes achieved the highest classification performance, which is 10.5% improvement over marker genes found by one-vs-all approach in Seurat. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Comparison of hierarchical marker genes with two baselines and three existing marker genes selection methods. Baselines are either all genes or highly variable genes. The three existing approaches are the flat one-vs-all FindMarker in Seurat, the flat version of scGeneFit, and the hierarchical version of scGeneFit. For each evaluation datasets, we trained a K-Nearest Neighbor classifier on 70% of the cells, and tested classification accuracy on the remaining 30% cells. (a) Classification accuracies for the PBMC3k dataset; (b) Classification accuracies for the PBMC control dataset; (c) Classification accuracies for the PBMC stim dataset. https://doi.org/10.1371/journal.pcbi.1012643.g004 The marker genes from the PBMC3k dataset were also evaluated based on UMAP visualizations in Fig 5a colored by cell types. UMAP visualization based on all genes is shown in the first column of Fig 5a, where major lineages were well separated but closely related cell types were co-located. More specifically, T cell subtypes formed one island, Monocytes formed one island, and B cells formed its own island. In the second and third columns of Fig 5a, UMAP based on high variable genes and one-vs-all marker genes found by Seurat produced tighter clusters and better separation among the major lineages. Although closely related cell types were still co-located, CD8 T cells were better separated from Naive CD4 T and Memory CD4 T. In the fourth and fifth columns, UMAP based on flat and hierarchical versions of scGeneFit showed poor cell type separation, consistent with the evaluation based on classification performance in Fig 4a. The last column of Fig 5a showed UMAP based on the proposed hierarchical marker genes. This UMAP was drawn based on the assembled data behind the heatmap in Fig 1c. In the last column of Fig 5a, we observed that many closely related cell types formed their own clusters. For examples, CD8 T cells, CD4 T cells and NK cells formed three isolated clusters, Monocytes and Dendritic cells formed two isolated clusters, while these cell types were co-located in UMAPs based on other marker gene selection methods. Although we still observed co-localization of Memory CD4 T and Naive CD4 T and co-localization of the two Monocyte subtypes, UMAP based on our hierarchical marker genes showed significantly better cell type separation compared to UMAPs based on other gene selection algorithms, which was consistent to the comparison based on classification accuracies. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. UMAP visualization of hierarchical marker genes, two baselines and three existing marker genes selection methods, applied to three datasets. (a) UMAP visualizations of PBMC3k dataset colored by cell types; (b) UMAP visualizations of PBMC control dataset; (c) UMAP visualizations of PBMC stim dataset. https://doi.org/10.1371/journal.pcbi.1012643.g005 For the PBMC control dataset, evaluation based on classification accuracy is shown in Fig 4b. We noticed that classification based on the proposed hierarchical marker genes achieved the highest average accuracy of 88%, while classification based on the one-vs-all marker genes from Seurat achieved an average accuracy of 74%. This difference is mainly contributed by the fact that the proposed hierarchical marker genes achieved much higher classification accuracies than the one-vs-all marker genes for several cell types, including T activated, CD16 Mono, Eryth and Mk. The marker genes in the PBMC control dataset were also visualized using UMAP shown in Fig 5b. Once again, based on the one-vs-all marker genes from Seurat (third column), UMAP showed islands that separated major lineages, while closely related cell types were co-located. In contrast, in the last column of Fig 5b, UMAP based on the hierarchical marker genes showed more islands that separated more cell types. Similar result was also observed in the PBMC stim dataset, as shown in Figs 4c and 5c. Hierarchical marker genes improve automated cell type mapping We further compared various marker gene selection approaches in the context of cell type annotation, using the PBMC control dataset and the PBMC stim dataset. We first considered the PBMC control dataset as reference, applied various marker gene selection approaches to the reference dataset, and then used the reference dataset to train K-Nearest-Neighbors classifiers based on marker genes selected by those approaches. After that, we applied the classifiers to predict the cell type labels of cells in the PBMC stim dataset as query data. The resulting prediction accuracies are shown in Fig 6a. We noticed that the prediction accuracies varied across cell types. In average, the one-vs-all marker genes from Seurat achieved an average accuracy of 72.3%, while the hierarchical marker genes achieved an average prediction accuracy of 88.1%. In multiple cell types, the hierarchical marker genes achieved much higher prediction accuracy compared to one-vs-all marker genes. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Compare hierarchical marker genes with two baselines and three existing marker genes selection methods, in the context of cell type mapping. Given two scRNA-seq datasets with significant batch effect between them, we trained a K-Nearest Neighbor classifier on one dataset (reference), and tested classification accuracy on the other dataset (query). (a) Classification accuracies with PBMC control as reference and PBMC stim as query; (b) Classification accuracies with PBMC stim as reference and PBMC control as query. https://doi.org/10.1371/journal.pcbi.1012643.g006 We also reversed the cell type mapping prediction, treating the PBMC stim dataset as reference and the PBMC control dataset as query. From the cell type prediction accuracies shown in Fig 6b, we again observed that hierarchical marker genes achieved higher prediction accuracy compared to one-vs-all marker genes, in majority of cell types and in average. The average prediction accuracy for one-vs-all marker genes from Seurat and the proposed hierarchical marker genes were 75% and 91%, respectively. This comparison showed that the hierarchical marker genes led to improved cell type mapping accuracy across datasets. Hierarchical marker genes selection framework To motivate the proposed hierarchical marker gene selection framework, we used an example scRNA-seq dataset of PBMC, which contained data for 2638 cells grouped into 9 cell clusters that correspond to 9 cell types. The cell types included Naive CD4 T cells, CD14+ Monocytes, Memory CD4 T cells, B cells, CD8 T cells, FCGR3A+ Monocytes, NK cells, Dendritic cells and Platelets. Applying the one-vs-all find marker approach implemented in Seurat, the expression patterns of the identified markers were shown as the heatmap in Fig 1a, where each vertical section corresponds to one cell cluster labeled by its cell type name. In addition to the diagonal blocks of high expression that confirmed the expression of identified marker genes in their corresponding cell clusters, multiple off-diagonal blocks of high expression were observed. For example, the marker genes for the first cell cluster (Naive CD4 T cells) were also highly expressed in the third cell cluster (Memory CD4 T cells). This is reasonable because the two CD4 T cell subtypes are closely related. However, these marker genes may not provide sufficient information to interpret and separate these two clusters corresponding to the two closely related CD4 T cell subtypes. We propose a scoring function defined as the average of diagonal expression minus the average of off-diagonal expression (details in Methods). This scoring function quantifies how much undesirable off-diagonal expression exists in the marker gene heatmap. We then combine two of the cell clusters, re-do the one-vs-all marker gene identification to re-generate the marker gene heatmap, and use the scoring functions to quantify how much off-diagonal expression exists after the two cell clusters are combined. We examine all possible pairs of cell clusters to find the best pair whose combination leads to the least off-diagonal expression in the marker gene heatmap. If combining this best cell cluster pair is able reduce off-diagonal expression compared to not combining them, we merge this pair of cell clusters, so that the number of cell clusters reduces by one. After that, we perform the same analysis to the resulting cell clusters, identify the best pair of cell clusters whose combination leads to least off-diagonal expression, and merge this pair if the off-diagonal expression is further reduced after merging. This process iterates until no merge is able to further reduce off-diagonal expression in the marker gene heatmap. This is essentially an agglomerative clustering process of the cell clusters, using the proposed scoring function as both distance metric and stopping criterion. In this example dataset, the agglomerative process stopped when the 9 original cell clusters were merged into two: one was the cell cluster corresponding to Platelets and the other was the remaining 8 cell clusters combined. As shown in Fig 1b, the first split of our cell clusters hierarchy had two branches, separating the Platelets and all other cell types. Performing one-vs-all marker finding for these two branches produced marker genes for Platelets and marker genes for other cell types combined, and the resulting heatmap is shown next to the first split in Fig 1b. To further construct the cell cluster hierarchy, we focused on the 8 cell clusters belonging to the left branch of first split, as if we were analyzing a new dataset composed of these 8 cell clusters. We agglomeratively merged these 8 cell clusters, and used the scoring function to stop the agglomeration when the off-diagonal expression was minimized. In this example, the 8 cell clusters were agglomeratively merged into two groups, so that the second split of the cell cluster hierarchy was also a two-way split: one branch was the combination of the 3 myeloid cell clusters (Monocytes and Dendritic cells), and the other branch was the combination of the remaining 5 lymphoid cell clusters (B cells, T cells and NK cells). Similarly, the branch containing the 3 myeloid cell clusters and the branch containing the 5 lymphoid cell clusters were examined separately to construct additional splits in the cell cluster hierarchy. This construction process iterated until all of the 9 original cell clusters were separated as leaf nodes in the cell cluster hierarchy. In summary, our cell cluster hierarchy is essentially a divisive hierarchical clustering process, where each split is determined by an agglormerative process to minimize undesirable off-diagonal expression and hence maximize specificity of the identified marker genes. The marker gene heatmaps for individual splits in the cell cluster hierarchy can be concatenated and assembled into Fig 1c, where each horizontal section corresponds to an individual split named by the corresponding cell types within the split. Since majority of the splits only considered a subset of the cell clusters, for a particular horizontal section corresponding to one split, expression data for cell clusters not considered in the split were zeroed out and shown as white areas in the assembled heatmap. This visualization provides a compact view of marker genes defined by our hierarchical approach. Data collections for evaluation To evaluate the proposed hierarchical marker gene selection approach, we applied it to three peripheral blood mononuclear cells (PBMC) datasets, namely PBMC3k [12], PBMC control, and PBMC stimuated [13]. PBMC3k is the dataset used as the illustrative example in Fig 1. The PBMC control dataset contained 6573 cells, and the PBMC stimulated dataset contained 7263 cells. Cells in both of these two datasets were grouped into 13 cell clusters. In addition to the PBMC datasets, we also included analysis based on a human pancreas dataset published by Xin [14, 15], which contained 1492 cells grouped into 4 cell clusters. The purpose is to provide an example dataset where the proposed hierarchical marker selection approach degenerated to the flat one-vs-all approach, because the cell clusters were sufficiently distinct and the flat one-vs-all approach did not produce much undesirable off-diagonal expression signal in the marker expression heatmap. Hierarchical marker genes capture more cell type differences We applied the proposed hierarchical marker gene selection to the three PBMC datasets (PBMC3k, PBMC control, PBMC stim). The hierarchical marker genes for PBMC3k data are visualized in Fig 1b. The hierarchical marker genes for the other two PBMC datasets are shown in Figs 2b and 3b. In addition, we applied the one-vs-all marker finding approach in Seurat [16], and both the flat and hierarchical versions of scGeneFit [7] to the three datasets. We compared the selected genes, as well as directly using all genes or the highly variable genes, in terms of their ability to separate the cell types annotated in these datasets. More specifically, each dataset was split into a training set and a testing set with a 7:3 ratio. K-Nearest Neighbor classifiers were trained based on genes selected by various approaches, and the classification accuracies were evaluated using the testing set. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Hierarchical marker gene selection in PBMC control dataset. (a) Marker gene heatmap generated by the one-vs-all FindMarker approach in Seurat. (b) Constructed hierarchy of cell clusters in PBMC control dataset. (c) Assembled heatmap that summarizes all marker genes for various splits in the cell cluster hierarchy. https://doi.org/10.1371/journal.pcbi.1012643.g002 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Hierarchical marker gene selection in PBMC stim dataset. (a) Marker gene heatmap generated by the one-vs-all FindMarker approach in Seurat. (b) Constructed hierarchy of cell clusters in PBMC stim dataset.(c) Assembled heatmap that summarizes all marker genes for various splits in the cell cluster hierarchy. https://doi.org/10.1371/journal.pcbi.1012643.g003 For the PBMC3k dataset, the classification accuracies are shown in Fig 4a. The first 9 sets of color bars show cell-type-wise classification accuracies. “Average” denotes the average value of cell-type-wise classification accuracies across all cell types. We observed that the cell-type-wise classification performance varied across cell types. As baseline references, classification based on all genes or the top 1406 highly variable genes (the number of highly variable genes were determined automatically by Scanpy with its default parameters) achieved accuracy between 39% and 98% for various cell types, with an average accuracy of around 80%. The one-vs-all marker genes from Seurat improved classification accuracy by around 4%, even though the number of the one-vs-all marker genes was 118, far fewer than the highly variable genes. The marker genes generated by the flat and hierarchical versions of scGeneFit did not outperform the baselines. Finally, with the same number of marker genes compared to the one-vs-all approach in Seurat, the proposed hierarchical marker genes achieved the highest classification performance, which is 10.5% improvement over marker genes found by one-vs-all approach in Seurat. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Comparison of hierarchical marker genes with two baselines and three existing marker genes selection methods. Baselines are either all genes or highly variable genes. The three existing approaches are the flat one-vs-all FindMarker in Seurat, the flat version of scGeneFit, and the hierarchical version of scGeneFit. For each evaluation datasets, we trained a K-Nearest Neighbor classifier on 70% of the cells, and tested classification accuracy on the remaining 30% cells. (a) Classification accuracies for the PBMC3k dataset; (b) Classification accuracies for the PBMC control dataset; (c) Classification accuracies for the PBMC stim dataset. https://doi.org/10.1371/journal.pcbi.1012643.g004 The marker genes from the PBMC3k dataset were also evaluated based on UMAP visualizations in Fig 5a colored by cell types. UMAP visualization based on all genes is shown in the first column of Fig 5a, where major lineages were well separated but closely related cell types were co-located. More specifically, T cell subtypes formed one island, Monocytes formed one island, and B cells formed its own island. In the second and third columns of Fig 5a, UMAP based on high variable genes and one-vs-all marker genes found by Seurat produced tighter clusters and better separation among the major lineages. Although closely related cell types were still co-located, CD8 T cells were better separated from Naive CD4 T and Memory CD4 T. In the fourth and fifth columns, UMAP based on flat and hierarchical versions of scGeneFit showed poor cell type separation, consistent with the evaluation based on classification performance in Fig 4a. The last column of Fig 5a showed UMAP based on the proposed hierarchical marker genes. This UMAP was drawn based on the assembled data behind the heatmap in Fig 1c. In the last column of Fig 5a, we observed that many closely related cell types formed their own clusters. For examples, CD8 T cells, CD4 T cells and NK cells formed three isolated clusters, Monocytes and Dendritic cells formed two isolated clusters, while these cell types were co-located in UMAPs based on other marker gene selection methods. Although we still observed co-localization of Memory CD4 T and Naive CD4 T and co-localization of the two Monocyte subtypes, UMAP based on our hierarchical marker genes showed significantly better cell type separation compared to UMAPs based on other gene selection algorithms, which was consistent to the comparison based on classification accuracies. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. UMAP visualization of hierarchical marker genes, two baselines and three existing marker genes selection methods, applied to three datasets. (a) UMAP visualizations of PBMC3k dataset colored by cell types; (b) UMAP visualizations of PBMC control dataset; (c) UMAP visualizations of PBMC stim dataset. https://doi.org/10.1371/journal.pcbi.1012643.g005 For the PBMC control dataset, evaluation based on classification accuracy is shown in Fig 4b. We noticed that classification based on the proposed hierarchical marker genes achieved the highest average accuracy of 88%, while classification based on the one-vs-all marker genes from Seurat achieved an average accuracy of 74%. This difference is mainly contributed by the fact that the proposed hierarchical marker genes achieved much higher classification accuracies than the one-vs-all marker genes for several cell types, including T activated, CD16 Mono, Eryth and Mk. The marker genes in the PBMC control dataset were also visualized using UMAP shown in Fig 5b. Once again, based on the one-vs-all marker genes from Seurat (third column), UMAP showed islands that separated major lineages, while closely related cell types were co-located. In contrast, in the last column of Fig 5b, UMAP based on the hierarchical marker genes showed more islands that separated more cell types. Similar result was also observed in the PBMC stim dataset, as shown in Figs 4c and 5c. Hierarchical marker genes improve automated cell type mapping We further compared various marker gene selection approaches in the context of cell type annotation, using the PBMC control dataset and the PBMC stim dataset. We first considered the PBMC control dataset as reference, applied various marker gene selection approaches to the reference dataset, and then used the reference dataset to train K-Nearest-Neighbors classifiers based on marker genes selected by those approaches. After that, we applied the classifiers to predict the cell type labels of cells in the PBMC stim dataset as query data. The resulting prediction accuracies are shown in Fig 6a. We noticed that the prediction accuracies varied across cell types. In average, the one-vs-all marker genes from Seurat achieved an average accuracy of 72.3%, while the hierarchical marker genes achieved an average prediction accuracy of 88.1%. In multiple cell types, the hierarchical marker genes achieved much higher prediction accuracy compared to one-vs-all marker genes. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Compare hierarchical marker genes with two baselines and three existing marker genes selection methods, in the context of cell type mapping. Given two scRNA-seq datasets with significant batch effect between them, we trained a K-Nearest Neighbor classifier on one dataset (reference), and tested classification accuracy on the other dataset (query). (a) Classification accuracies with PBMC control as reference and PBMC stim as query; (b) Classification accuracies with PBMC stim as reference and PBMC control as query. https://doi.org/10.1371/journal.pcbi.1012643.g006 We also reversed the cell type mapping prediction, treating the PBMC stim dataset as reference and the PBMC control dataset as query. From the cell type prediction accuracies shown in Fig 6b, we again observed that hierarchical marker genes achieved higher prediction accuracy compared to one-vs-all marker genes, in majority of cell types and in average. The average prediction accuracy for one-vs-all marker genes from Seurat and the proposed hierarchical marker genes were 75% and 91%, respectively. This comparison showed that the hierarchical marker genes led to improved cell type mapping accuracy across datasets. Discussion In this study, we proposed a hierarchical marker gene selection approach for interpreting cell clusters derived from scRNA-seq data. Given a clustered scRNA-seq dataset, our approach aimed to construct of a hierarchy of the cell clusters, so that marker genes could be defined for each split in the hierarchy. With a well-constructed hierarchy, our approach was able to identify marker genes for not only individual cell clusters, but also intermediate nodes that represented lineages consisted of closely related cell clusters. In addition, the identified marker genes tended to be more specific to the corresponding cluster or lineage, which was manifested by reduced off-diagonal signals in heatmap visualizations of the identified marker genes. To evaluate the effectiveness of our proposed hierarchical marker gene selection approach, we compared it with several other gene selection approaches, such as directly using all genes, the highly variable genes, the marker genes selected by one-vs-all approach implemented in Seurat, as well as flat and hierarchical versions of scGeneFit. We applied two strategies for the comparison. The first strategy is to evaluate the selected marker genes within one scRNA-seq dataset. By splitting one scRNA-seq dataset into training and testing sets, a K-Nearest Neighbors classifier was able to quantify how well the selected marker genes could separate cell clusters. We also generated UMAP visualizations that provided a qualitative comparison of separations among cell clusters. The second strategy is to evaluate the selected marker genes across two scRNA-seq datasets that harbored batch effect. Training and testing across different scRNA-seq datasets allowed quantitative evaluation of the robustness of the identify marker genes with respect to batch effects. In both strategies, we demonstrated the benefit of hierarchical marker genes over the one-vs-all flat approach for marker gene selection. To further evaluate our proposed hierarchical marker gene selection approach, we compared the marker genes selected by our approach against those selected by the one-vs-all method. We observed that our method identified a substantial number of genes not selected by the one-vs-all method, particularly within the lineages closer to the bottom of the hierarchy. This indicates that our algorithm effectively identifies new marker genes that can help distinguish closely related cell subtypes. Two specific examples illustrate this capability. In the PBMC3k dataset, our hierarchical method selected RPL21 as a marker gene for Naïve CD4T. RPL21 was not selected as a marker gene in the one-vs-all method, and also not selected as a marker gene in higher levels of the hierarchy, which means RPL21 is a marker gene specifically helpful for separating Naïve CD4 T and Memory CD4 T. Data from the Human Protein Atlas corroborates this, showing higher RPL21 expression in Naïve CD4 T cells compared to other T cells across three datasets [17]. Similarly, our hierarchical method selected S100A11 as a marker gene for Memory CD4T. S100A11 was not selected as a marker gene in the one-vs-all method, not selected as a marker gene in higher levels of the hierarchy. According to CD4+ T lymphocyte reclustering and subcluster analysis in [18], S100A11 is exclusively expressed by for memory CD4+ T cells, highlighting its role as a specific marker for this subtype. The advantage of the hierarchical marker genes over the one-vs-all marker genes is dependent on the heterogeneity of the data. When analyzing datasets containing distinct cell clusters without any hierarchical structure, the proposed hierarchical marker gene approach would degenerate to the one-vs-all approach. To provide a concrete example of this scenario, we applied the proposed hierarchical method on pancreas dataset (Xin et al [15]), which included 4 distinct cell types (i.e., alpha cells, beta cells, gamma cells and delta cells). Fig 7 shows the heatmap visualization of marker genes identified by the one-vs-all find marker approach in Seurat, where each vertical section corresponds to one cluster labeled by its corresponding annotated cell type name. Although some off-diagonal expression can be observed in the heatmap, merging the cell clusters did not lead to reduced off-diagonal signal. The proposed hierarchical marker gene algorithm stopped at its first iteration before merging any cell clusters, and the resulting hierarchy was identical to the flat structure of the one-vs-all marker genes approach. Therefore, when there is no hierarchical structure among cell clusters, the proposed hierarchical marker gene approach is the same as the one-vs-all find marker approach. Although the Xin dataset shows one instance where hierarchical and one-vs-all marker genes are equivalent, we believe the proposed hierarchical marker gene approach is advantageous in many biological applications and contexts, because cellular heterogeneity in most biological contexts is hierarchically organized into lineages, cell types and subtypes. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 7. Marker gene heatmap generated by the one-vs-all FindMarker approach in Seurat. https://doi.org/10.1371/journal.pcbi.1012643.g007 Methods Data preprocessing We performed data preprocessing using the standard Seurat preprocessing pipeline. We first conducted quality control and removed cells with low gene detection or high mitochondrial gene expression. We then applied library size normalization and log transformation to the data. Following this, we utilized data scaling to eliminate the expression level differences specific to each cell. Scoring function In this study, we introduce a hierarchical marker gene selection method to identify better marker genes for distinguishing cell clusters compared to the popular one-vs-all methods. We proposed a scoring function to determine the hierarchical structure among cell clusters, by evaluating the heatmap visualization of their marker genes. The scoring function aims to encourage new definition of cell cluster groups, such that the one-vs-all selected marker genes based on these cell cluster groups are highly specific to the corresponding cell cluster groups and lowly expressed in other cell cluster groups. Specifically, given a collection of cells and definition of cell clusters for these cells, we first apply the one-vs-all FindMarkers approach in Seurat, and visualize the selected marker genes as a heatmap. We then compute the average the values of diagonal blocks of the heatmap, and sum up all those averages to obtain Vdiagonal. For the off-diagonal blocks, we average the expression values of the marker genes corresponding to a specific cell cluster (or group), and then choose the maximum average value across all cell clusters (or groups). We sum up all the maximum off-diagonal average values across all cell clusters (or groups) to obtain Voff−diagonal. Finally, we define our scoring function as s = (Vdiagonal − k * Voff−diagonal)/celltype_number, where k is a pre-defined parameter that determines the weights of the two terms, in our application, we set k to be 5. Overall, this scoring function measures the expression signals in the diagonal blocks minus large expression signals in the off-diagonal blocks. This is essentially a quantification of the amount of undesirable off-diagonal expression signals. Each split in our cell cluster hierarchy is constructed by iteratively grouping cell clusters to minimize this scoring function, and hence minimizing the undesirable off-diagonal expression signals and making the gene markers highly specific to each cell group. Highly variable genes selection For the purpose of quantitative evaluation of marker gene selection approaches, we decided to examine the highly variable genes, which serves as a baseline gene selection approach for the comparisons. We applied the scanpy.pp.highly_variable_genes() function in Scanpy on the pre-porcessed data to select highly variable genes (HVGs). All the parameters were kept as default settings within the function. In detail, this function calculated the mean and a dispersion measure (variance/mean) for each gene across all single cells, placed genes into 20 bins based on their average expression, performed z-normalization for the dispersions within each bin, and applied a threshold to the z-scores to identify highly variable genes. We found 1406 HVG in PBMC3k dataset, 1398 HVGs in PBMC control dataset, and 1459 HVGs in PBMC stim dataset. For the cell mapping experiment, we kept the 824 overlapping HVGs between PBMC control and PBMC stim. Assembled heatmap generation For each split in our constructed cell cluster hierarchy, marker genes for the branches can be identified and visualized using a heatmap. We concatenated and assembled the marker gene heatmaps for all splits in the cell cluster hierarchy into one heatmap matrix, where each horizontal section corresponds to an individual split. Other than the first split at the root of the hierarchy, each split only includes a subset of the cell clusters. So, for a particular horizontal section corresponding to one split, expression data for cell clusters not considered in the split are set to 0. The assembled heatmap summarizes all splits of the cell cluster hierarchy, and the data matrix behind this assembled heatmap is what we used to generate the UMAP visualization and classification accuracy in our comparison and evaluation analysis. Data preprocessing We performed data preprocessing using the standard Seurat preprocessing pipeline. We first conducted quality control and removed cells with low gene detection or high mitochondrial gene expression. We then applied library size normalization and log transformation to the data. Following this, we utilized data scaling to eliminate the expression level differences specific to each cell. Scoring function In this study, we introduce a hierarchical marker gene selection method to identify better marker genes for distinguishing cell clusters compared to the popular one-vs-all methods. We proposed a scoring function to determine the hierarchical structure among cell clusters, by evaluating the heatmap visualization of their marker genes. The scoring function aims to encourage new definition of cell cluster groups, such that the one-vs-all selected marker genes based on these cell cluster groups are highly specific to the corresponding cell cluster groups and lowly expressed in other cell cluster groups. Specifically, given a collection of cells and definition of cell clusters for these cells, we first apply the one-vs-all FindMarkers approach in Seurat, and visualize the selected marker genes as a heatmap. We then compute the average the values of diagonal blocks of the heatmap, and sum up all those averages to obtain Vdiagonal. For the off-diagonal blocks, we average the expression values of the marker genes corresponding to a specific cell cluster (or group), and then choose the maximum average value across all cell clusters (or groups). We sum up all the maximum off-diagonal average values across all cell clusters (or groups) to obtain Voff−diagonal. Finally, we define our scoring function as s = (Vdiagonal − k * Voff−diagonal)/celltype_number, where k is a pre-defined parameter that determines the weights of the two terms, in our application, we set k to be 5. Overall, this scoring function measures the expression signals in the diagonal blocks minus large expression signals in the off-diagonal blocks. This is essentially a quantification of the amount of undesirable off-diagonal expression signals. Each split in our cell cluster hierarchy is constructed by iteratively grouping cell clusters to minimize this scoring function, and hence minimizing the undesirable off-diagonal expression signals and making the gene markers highly specific to each cell group. Highly variable genes selection For the purpose of quantitative evaluation of marker gene selection approaches, we decided to examine the highly variable genes, which serves as a baseline gene selection approach for the comparisons. We applied the scanpy.pp.highly_variable_genes() function in Scanpy on the pre-porcessed data to select highly variable genes (HVGs). All the parameters were kept as default settings within the function. In detail, this function calculated the mean and a dispersion measure (variance/mean) for each gene across all single cells, placed genes into 20 bins based on their average expression, performed z-normalization for the dispersions within each bin, and applied a threshold to the z-scores to identify highly variable genes. We found 1406 HVG in PBMC3k dataset, 1398 HVGs in PBMC control dataset, and 1459 HVGs in PBMC stim dataset. For the cell mapping experiment, we kept the 824 overlapping HVGs between PBMC control and PBMC stim. Assembled heatmap generation For each split in our constructed cell cluster hierarchy, marker genes for the branches can be identified and visualized using a heatmap. We concatenated and assembled the marker gene heatmaps for all splits in the cell cluster hierarchy into one heatmap matrix, where each horizontal section corresponds to an individual split. Other than the first split at the root of the hierarchy, each split only includes a subset of the cell clusters. So, for a particular horizontal section corresponding to one split, expression data for cell clusters not considered in the split are set to 0. The assembled heatmap summarizes all splits of the cell cluster hierarchy, and the data matrix behind this assembled heatmap is what we used to generate the UMAP visualization and classification accuracy in our comparison and evaluation analysis.

Grosse-Wentrup, Moritz;Kumar, Akshey;Meunier, Anja;Zimmer, Manuel

doi: 10.1371/journal.pcbi.1012674pmid: 39680605

Introduction At least since the work of David Marr [1], it is widely acknowledged that complex systems can be described at different levels, e.g., at the implementational, the algorithmic, and the computational level. Understanding the neuronal basis of cognition amounts to bridging the explanatory gap between the level of neuronal activity patterns and the level of cognitive states [2]. However, despite the long history of research into this problem, a mathematically rigorous framework for bridging this gap is still lacking. Traditionally, observations of single-unit neuronal activities and behaviors, and statistical relations between the two, have formed the basis for identifying neuronal circuits and hand-crafting mechanistic models of their computations, resulting in models that range from the level of individual neurons [3] over models that explain the neuronal implementation of simple mathematical operations [4] to models that describe how neuronal circuits control behavior [5]. Due to increasing awareness of the importance of (potentially widely distributed) neuronal activity patterns for behavior, and the difficulty in scaling up hand-crafted models to large-scale neuronal recordings, machine learning methods (also referred to as decoding- or multivariate pattern analysis (MVPA) models) have been developed to uncover relations between complex neuronal activity patterns, cognitive states, and behaviors [6–8]. More recently, these models have been complemented by algorithms for learning neuronal manifolds that enable the visualization of high-dimensional neuronal dynamics [9] and their relation to behavior [12, 13]. However, the ability to visualize and decode behavior from complex neuronal activity patterns does not imply that these activity patterns represent the behaviorally relevant neuronal computations [14], e.g., because of a common but latent neuronal cause of the behavior and the observed activation patterns [15]. Under certain conditions, decoding models can be endowed with a causal interpretation, enabling experimentally testable predictions on causal relations between neuronal activity and behavior [15]. However, we argue that such causal models are not suitable to study relationships between neuronal activity patterns and cognition because causal relationships between these two levels would be irreconcilable with physicalism: A causal relation between two variables X → Y implies that manipulations of X can alter the probability of observing specific values of Y [18]. This relation is only possible if X and Y are two separate processes that are linked by a mechanism (an in-depth discussion of the nature of causal variables is given in [19]). A causal relation between X and Y thus implies that it must be conceptually possible to intervene on Y without intervening on X, thereby breaking the mechanism that links the two variables. For instance, if X and Y represent two different neurons, one could construct an experimental setup that controls the membrane potential of Y, thereby eliminating any influence action potentials of X can have on the membrane potential of Y. If X and Y were to represent a neuronal- and a cognitive state, however, a causal relation of the form X → Y would imply that we can (at least conceptually) manipulate the cognitive state without changing the neuronal state that causes it. As such, causal relations between neuronal- and cognitive states would only be meaningful if we adopted a strong dualistic viewpoint in which neuronal- and cognitive states co-existed as independent physical processes. In the present work, we adhere to physicalism (even though our framework may be compatible with a weaker form of predicate dualism [20]), i.e., we adopt the view that all mental phenomena are ultimately physical phenomena and hence reject the notion that neuronal activity causes cognition. Instead, we argue in the following that it is reasonable to consider relationships between neuronal activity and cognitive states as constitutive or, in more philosophical terms, that cognitive states supervene on neuronal states. While we consider causal models unsuitable for representing relations between neuronal- and cognitive states, we consider them well-suited to represent relations within each level. We first consider the neuronal level. Building on the example in the previous paragraph, it is perfectly reasonable to consider two neurons as separate entities that can be manipulated individually and that are linked by a physical mechanism. For instance, the sentence neuron X’s action potential is a cause of neuron Y’s membrane potential is a meaningful causal statement in the sense that it implies an empirically testable prediction that is consistent with our physical understanding of neuronal processes: Manipulating the spiking probability of neuron X, e.g., by injecting a current into neuron X, will alter the membrane potential of neuron Y. We argue that the same holds true for relations between cognitive states. We employ cognitive states to causally reason about our own and other people’s mental processes. For instance, when we say I am unhappy because I am bored, we express the causal relation boredom → happiness. This causal statement implies the empirically testable prediction that if we reduce boredom in a person, we increase their probability of being happy. It is further consistent with the view that boredom and happiness are two separate processes that can be manipulated independently. If causal models are well-suited to explain relations within the neuronal and cognitive levels but unsuitable for explaining relations between these two levels, which type of relations should hold between causal models on the neuronal- and causal models on the cognitive level? The solution we advocate here is causally consistent transformations between causal models on each level [21]. Causally consistent transformations are functional mappings between states and interventions of two causal models chosen so that we can reason about the observational effects of causal interventions consistently across both models. For instance, assume the cognitive concepts boredom and happiness to have the two neuronal realizations x and y (with the bold notation indicating that x and y may represent elements of high-dimensional sets of neuronal activity patterns), with the causal relations boredom → happiness and x → y. Intervening on x, e.g., by electrical stimulation, would then be equivalent to modulating boredom and, via the causal effect x exerts on y, affect happiness. In this conceptual framework, which we term neuro-cognitive multilevel causal modeling (NC-MCM), neuronal cause-effect relations play out in parallel to cognitive cause-effect relations in a consistent manner that enables us to causally reason and explain observations interchangeably on each level. In the NC-MCM framework, neuronal- and cognitive states thus do not co-exist as separate processes but are linked via functional mappings. In other words, neuronal states do not cause but are constitutive of cognitive states. The NC-MCM framework thereby provides a theoretical framework to bridge the explanatory gap between neuronal activity and cognition while maintaining a physicalist position. When attempting to map neuronal- to cognitive states, we must consider that there is no universally agreed-upon set of cognitive states [22]. One question thus seems of particular importance: What is the set of possible cognitive states? Is there, for example, a single cognitive state of being in pain, or should having a headache and feeling back pain be considered two different states? Do all individuals share the same sets of cognitive states? And if so, are the mappings from neuronal to cognitive levels equivalent across subjects? The NC-MCM framework does not assume any fixed set of cognitive states, and we do not argue for the existence of such a universal set. Instead, and in agreement with a recent argument in [23] that neuroscience needs behavior, we let the behavioral context determine the relevant set of cognitive states in a purely data-driven approach. We illustrate the relationship between cognition, neuronal activity, and behavior in the MCM framework with a (slightly simplified) analogy from physics. The temperature of a gas is proportional to the average kinetic energy of its molecules. To reason about the conditions under which the gas ignites, both levels of description are equivalent, e.g., the two sentences the gas will ignite if its temperature is raised by T degrees Celsius and the gas will ignite if the average kinetic energy of its molecules is increased by K Joules are causally consistent statements. Importantly, many kinetic energy configurations give rise to the same temperature. The gas temperature can not be altered, however, without also changing the kinetic energies. As such, the macroscopic concept of temperature is a causally consistent abstraction of the microscopic kinetic energy configuration with respect to the behavior of the gas. In the MCM framework, the analogies of temperature, kinetic energies, and ignition are cognitive states, neuronal states, and behaviors in the sense that the macroscopic cognitive states are causally consistent abstractions of the microscopic neuronal states that cause behavior. In the following, we introduce the MCM framework in a mathematically rigorous fashion and show how causally consistent transformations between the neuronal- and the cognitive level can be learned from empirical data. We then illustrate the application of the MCM framework on calcium imaging data from the nematode C. elegans. We conclude our work by discussing future extensions and the philosophical implications of the MCM framework. Multilevel causal modeling in cognitive neuroscience We begin this section by discussing the construction of causal models on the neuronal level and elucidating how they are linked to behavior. We then consider the nature of cognitive states and causal models thereof before explaining how causal models on the neuronal- and on the cognitive level can be linked via causally consistent transformations. We conclude the section by discussing how to learn MCMs from empirical data. Causal models on the neuronal level To construct causal models on the neuronal level, we first need to decide on a framework for causal modeling. A variety of causal modeling frameworks have been developed for and evaluated on neuronal data [24]. To be applicable in the MCM framework, we require causal models that can make empirically testable predictions on the effects of experimental interventions. The framework of Causal Bayesian Networks (CBNs) [18, 25] fulfills this requirement and hence serves as the causal modeling framework in the present setting. Causal relations in CBNs are modeled by structural causal models (SCMs): Definition 1 (Structural Causal Model—SCM). A SCM is a triple with X a set of N random variables endogenous to the model, E a set of N exogenous noise variables, and F a set of N functions defining each endogenous variable as a function of its direct causes (i.e., parents—pa()) and its corresponding exogenous noise variable, so that for each i ∈ {1, …, N} we have Xi ≔ Fi(pa(Xi), Ei) where the Fi are chosen such that no variable is a (direct or indirect) cause of itself. The joint probability distribution P(E) over the exogenous noise variables induces a joint probability distribution P(X) over the endogenous variables (via the pushforward measure). If the noise variables are mutually independent, the SCM is causally sufficient, i.e., no unobserved confounders that influence multiple endogenous variables exist. In the present setting, each endogenous variable Xi represents the activity of one neuron. We note that in general, X and E may represent continuous- as well as discrete variables, implying that P(X) and P(E) denote probability densities or distributions, respectively. In the following, we assume all variables to be discrete. We consider an extension of the MCM framework to continuous-valued random variables feasible but beyond the scope of the present work. We denote random variables by upper- and their realizations by lower-case letters, e.g., we write P(Xi = x) to indicate the probability that the neuron represented by the random variable Xi takes on the value . Causal relations in SCMs are commonly depicted by directed acyclic graphs (DAGs), with an arrow drawn from node A into node B if the endogenous variable represented by node A is a parent of the endogenous variable represented by node B. An arrow from A into B indicates a direct causal influence of A on B, and a directed path from node A to node B indicates an indirect causal influence of A and B. We note that direct- and indirect causal relations are relative to the set of variables endogenous to the model and may change when dropping or adding nodes. Knowledge of the DAG, in combination with the joint probability distribution over all endogenous variables (represented by the nodes of the DAG), enables us to reason about the probabilistic effects of experimental interventions on variable subsets [18]. Experimental interventions are represented mathematically by the do()-operator, e.g., do(Xi = x) represents the experimental intervention of setting variable Xi to the value x. In empirical settings, the DAG and the joint probability distribution are usually not known and have to be inferred from a combination of experimental and observational data. We show in the section on learning MCMs that knowledge of the causal model on the neuronal level is not a prerequisite for constructing a causally consistent mapping between the neuronal- and the cognitive level. In their original form, SCMs model independent and identically distributed (i.i.d.) data, i.e., data without temporal structure. It is straightforward, however, to extend SCMs to model dynamical neuronal systems by unrolling the endogenous variables across time, as exemplified in the middle row of Fig 1. Here, the set of random variables X[t0] represents the global state of all N neurons at time t0 and the arrow from X[t0] into X[t1] indicates that the global neuronal state at time t0 is a cause of the global neuronal state at time t1, in the sense that the state of each neuron at time t1 is a function of a subset of the neuronal states at time t0 and their respective exogenous noise term. We remark that this representation also allows feedback loops, e.g., as in Xi[t] → Xj[t + 1] → Xi[t + 2]. More formally, we define the extension of SCMs to dynamic settings as follows: Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Relations between cognitive-, neuronal-, and behavioral states in MCM. Solid and dotted arrows denote causal and constitutive relations, respectively. https://doi.org/10.1371/journal.pcbi.1012674.g001 Definition 2 (Dynamic Structural Causal Model—dSCM) A dynamic SCM (dSCM) is family of T triples , indexed by t ∈ {1, …, T}, with X[t] a set of N random variables endogenous to the model, E[t] a set of N exogenous noise variables, and F[t] a set of N functions defining each endogenous variable as a function of its direct causes (i.e., parents—pa()) and its corresponding exogenous noise variable, so that for each i ∈ {1, …, N} and t ∈ {1, …, T} we have Xi[t] ≔ Fi[t](pa(Xi[t]), Ei[t]), where pa(Xi[t]) may contain any endogenous variables prior to time t. We note that we do not allow instantaneous causal relations in Definition 2, i.e., situations where pa(Xi[t1]) may include other neurons at time t1, because our current understanding of physical processes posits that causes must precede effects. This constraint may have to be reconsidered when modeling neuronal dynamics that are observed at sampling rates slower than the system’s dynamics. If the exogenous noise terms are mutually independent and pa(X[t]) only includes variables at the previous time step for all t, as in the example in Fig 1, the dSCM represents a first-order Markov process with a transition probability distribution (1) Markov processes are of particular relevance in the MCM framework, because they define causally sufficient state spaces, in the sense that the global state at any given point in time is causally sufficient for the probability distribution over the global state at the next time point. More formally, Markovianity implies (under the faithfulness assumption) that the exogenous noise terms at different time points are mutually statistically independent and hence (via the backdoor criterion, see [18]) P(X[t]|X[t − 1]) = P(X[t]|do(X[t − 1])) for all t, i.e., the observational- and interventional conditional distributions are identical. A counter-example would be the case of an exogenous variable, e.g., a stimulus, affecting both X[t − 2] and X[t]. In this case, the Markov property would be violated (X[t ∓ 2] ⫫̸ X[t]|X[t − 1]) and hence P(X[t]|X[t − 1]) ≠ P(X[t]|do(X[t − 1])) (we use the symbols ⫫̸ and ⫫ to denote statistical dependence and independence, respectively). As we discuss in the section on how to learn an MCM, empirical tests for Markovianity are essential to ensure that cognitive variables are causally consistent abstractions. In the following, we assume that the functional mappings between states, as well as the probability distribution over their exogenous variables, do not change over time, i.e., we assume the process to be time-homogeneous. We further assume that any process we consider has converged to a stationary probability distribution. These assumptions are not required from a theoretical perspective but greatly simplify empirical inference. To model the causal effect of neuronal activity on behavior, we extend the DAG in Fig 1 by a behavioral state vector B[t] and let X[t] → B[t], i.e., we model the behavioral states at time t to be caused by (a subset of) the neuronal states at time t. We note that it is reasonable to consider relations between neuronal activity and behavior as causal because we can, in principle, manipulate behavior independently of neuronal activity, e.g., we can fixate an animal and thereby prevent neuronal activity from causing an actual movement. We further note that we represent the behavioral states across time by measurement nodes, i.e., by nodes that have no causal effect on any other variables [26]. As such, the behavioral states do not form a Markov process, and B[t] ⫫̸ B[t − 2]|B[t − 1] due to the common effect of X[t − 2] on both B[t − 2] and B[t]. We denote the extension of the dSCM by the behavioral state vector as . To model feedback loops of the neuronal system with its environment, the DAG could be further extended by a state vector S[t] that represents stimuli, which could be influenced by the system’s past behavior, e.g., S[t0] → X[t0] → B[t0] → S[t1]. We leave such an extension for future work. Bridging the neuronal- and the cognitive level Before discussing how to bridge the neuronal- and the cognitive level, we must first consider the nature of cognitive states. Somewhat surprisingly, there currently exists no generally agreed-upon definition of what a cognitive state is (cf. [27], pp. 41 ff.). The broader term cognition is commonly used to denote any kind of mental operation or structure that can be studied in precise terms [28]. To arrive at a working definition that allows us to operationalize cognitive states for the MCM framework, we consider three refinements of this concept of cognition. First, we propose that a cognitive state must be mathematically quantified to be studied in precise terms. Second, we consider an operation to represent a mechanism that translates a system from one state to another. And third, we consider the addendum or structure to indicate that a physical substrate supports any mental operation. These considerations lead us to the following working definition, which we will render mathematically precise towards the end of this section: Definition 3 (Cognitive state) A cognitive state is a causally meaningful abstraction of a neuronal state. To elucidate this definition, we need to explain what the terms causally meaningful and abstraction mean in this context. We consider a concept as causally meaningful if it is conceptually (but not necessarily technically) feasible to fix its state to a specific value by an external intervention. For instance, the concept of age, defined as the time since birth, is not causally meaningful because there is no intervention conceivable that would allow an experimenter to alter the age of a subject. In contrast, the concept of membrane potential is causally meaningful because it is conceivable (and in this case also technically feasible) to set the membrane potential of a neuron to a desired voltage. We consider a state to be an abstraction if it is a macroscopic description of the state of a system that is consistent with (potentially infinitely) many microscopic configurations of the system. Revisiting the example from the introduction, the temperature of a gas is a macroscopic abstraction of the microscopic states of the gas molecules because there are infinitely many configurations of kinetic energies of the molecules in the gas that give rise to the same temperature. This relation, however, is asymmetric. It is impossible to change the temperature of a gas without altering the kinetic energies of its molecules. Accordingly, the temperature of a gas and its molecular configuration do not stand in a causal but in a constitutive relationship. Combining these two concepts, we call an abstraction causally meaningful if the microscopic configurations that determine the macroscopic state are causally meaningful. For instance, consider the statement the gas will ignite if we raise its temperature above a certain threshold. Even though there exists no direct intervention on the macroscopic concept of temperature, the experimental intervention of raising the temperature is meaningful because there exist microscopic interventions (increasing kinetic energies) that are identical to raising the temperature. These causally meaningful microscopic interventions endow the macroscopic concept of temperature with causal meaning. Definition 3 expresses the notion that cognitive- and neuronal states stand in a similar constitutive relationship. While cognitive states are not intrinsically causally meaningful, e.g., it is unclear how one could set the cognitive state of anxiety to a desired level by an experimental intervention that directly acts on anxiety, the neuronal states that are consistent with a certain level of anxiety are intrinsically causally meaningful, in the sense that experimental interventions are conceivable (though not necessarily technically feasible) that realize one out of the consistent neuronal configurations. Furthermore, if intervening on neuronal states has a causal influence on future neuronal dynamics and on behavior, which we consider uncontroversial, this causal efficacy is inherited by cognitive states. To summarize, Definition 3 expresses our understanding that we use cognitive concepts to reason about mental processes because we consider them to be causally effective in terms of the future evolution and behavior of the system under study due to their realization through intrinsically causally meaningful neuronal dynamics. We now formalize the relations discussed above. In analogy to causal models on the neuronal level, we represent the cognitive state of a model system or organism by a cognitive dSCM with C[t] the cognitive state vector (note that E[t] and F[t] are specific to each model and not shared across neuronal- and cognitive dSCMs). To link the the neuronal- and the cognitive dSCM in a causally consistent manner, we require the two models to be behaviorally and dynamically causally consistent: Definition 4 (Behavioral Causal Consistency—BCC) Let a neuronal dSCM in a behavioral context and a cognitive dSCM. Denote the state space of X[t] and C[t] by and , respectively. We call the triple behaviorally causally consistent if is a surjective mapping such that for every pair x1, x2 ∈ τ−1(c) and for every we have (2) where τ−1(c) is the pre-image of c. In this case, we define do(C[t] = c)≔ do(X[t] = x|x ∈ τ−1(c)) and have (3) for all . If a neuronal- and a cognitive SCM are behaviorally causally consistent, every experimental intervention on the neuronal level has a matching intervention on the cognitive level and every cognitive-level intervention has at least one neuronal implementation, in the sense that both interventions lead to the same probability distribution over the behaviors. The surjectivity of τ ensures, first, that the cognitive level does not include any states that do not represent at least one neuronal state, and, second, that the cognitive level can form an abstraction of the neuronal level, i.e., that many neuronal states may map to the same cognitive state yet distinct cognitive states can only be linked to distinct neuronal states. Intuitively, the cognitive SCM constitutes a lossless compression of all information in the neuronal SCM that is causally relevant for a given set of behaviors. As such, behavioral causal consistency allows us to reason interchangeably about the causes of behaviors on the neuronal- and on the cognitive level. Importantly, behavioral causal consistency endows cognitive states with a causal meaning, in the sense that cognitive interventions, for which it may be unclear how the intervention can be experimentally implemented, e.g., increase happiness, can be translated into equivalent interventions on the neuronal level, for which a well-defined experimental procedure is available, e.g., stimulate a set of neurons. The definition of dynamic causal consistency is analogous to behavioral causal consistency except that it considers the probability distribution over the next cognitive state. Definition 5 (Dynamic Causal Consistency—DCC) We call a triple dynamically causally consistent if for all pairs of cognitive states (4) for all x1, x2 ∈ τ−1(c) where τ−1(c) is the pre-image of c. In this case, we define do(C[t] = c)≔ do(X[t] = x|x ∈ τ−1(c)) and have (5) for all and x ∈ τ−1(c). While behavioral causal consistency defines the consistency of neuronal- and cognitive states with respect to behavior, dynamic causal consistency defines consistency with respect to the dynamics of the neuronal system. Intuitively, a cognitive SCM that is DCC constitutes a lossless compression of all information in the neuronal SCM that is causally relevant to the dynamics of the neuronal system. In analogy to behavioral causal consistency, DCC grounds causal relations between cognitive states in the neuronal level, e.g., statements such as reducing boredom is likely to lead to increased happiness can be translated into the neuronal-level statement inducing a neuronal activity pattern x′ that represents low boredom, e.g., by electrical stimulation, is likely to lead to a neuronal state that corresponds to increased happiness. We are now in a position to formally introduce the primary contribution of this work: Definition 6 (Neuro-Cognitive Multilevel Causal Model—NC-MCM) We call a triple a neuro-cognitive multilevel causal model (NC-MCM) if it is behaviorally and dynamically causally consistent. We remark that we have chosen τ not to depend on time because we consider a constant mapping between levels more desirable. It would be straightforward to extend the definition to time-varying mappings, but this generalization would substantially complicate learning the mapping from empirical data. We further note that in contrast to Rubenstein et al. [21] we only consider experimental interventions on the full neuronal- and cognitive state vectors at each time step and leave the extension of NC-MCMs to interventions on subsets of state variables for future work. We illustrate the concept of a NC-MCM in the following toy example. Example 1 Consider a neuronal dSCM with one endogenous variable (N = |X| = 1) and discrete state space , where each state represents one out of a total of four neuronal activity patterns. Further, assume there are three distinct behaviors with conditional probabilities given the neuronal state Note that the conditional probability distribution over the three behaviors is identical for {x1, x2} and for {x3, x4}. As such, one may interpret the system as having two distinct macro-states, {x1, x2} and {x3, x4}, with each macro-state subserving a particular behavioral pattern that is expressed by a constant probability distribution over the three behaviors. We may thus define a mapping (6) These two macro-states form the cognitive state space , where we may choose to name the two cognitive states foraging and recuperating, respectively. Next, assume the neuronal dSCM has the steady-state transition probability distribution TX shown in the left-hand side of Fig 2. The transition matrix TX and the mapping τ induce a cognitive dSCM with the transition probabilities shown in the right-hand side of Fig 2. The triple is behaviorally causally consistent, because the cognitive macro-states condense all information in the neuronal micro-states that are causally relevant for the set of behaviors, i.e., for every intervention on the neuronal micro-level there is an equivalent intervention on the cognitive macro-level that entails the same probabilities over the behaviors (cf. Definition 4). The triple is further dynamically causally consistent because the probabilities of remaining in the current or progressing to another cognitive state are identical for each neuronal state that maps to the same cognitive state. Because the triple is behaviorally and dynamically causally consistent, it fulfills the criteria for a NC-MCM. As such, the cognitive dSCM is a causally meaningful abstraction of the underlying neuronal dSCM because causal statements on the effects of current cognitive states on behavior and on future cognitive states are grounded in the neuronal level, e.g., the cognitive-level statement ‘the animal is moving around and feeding because it is foraging’ can be translated into the neuronal-level statement ‘the animal is moving around and feeding because it is in neuronal state x1 or x2’. We note that behavioral- does not imply dynamic causal consistency, e.g., introducing an asymmetry in the neuronal-level transition probabilities in Fig 2 would break dynamic- but not behavioral causal consistency. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. State-space diagram with transition probabilities for Example 1. https://doi.org/10.1371/journal.pcbi.1012674.g002 We remark that the toy example above is constructed to illustrate the idea that in the NC-MCM framework redundancies in the neuronal dynamics and their relation to behaviors are exploited to build a cognitive-level model that preserves all causally relevant information. In practice, the challenge is to learn which (potentially widely-distributed and complex) neuronal activity patterns are redundant with respect to a behavioral context and with respect to the neuronal dynamics, a problem that we turn to in the next section. Learning neuro-cognitive multilevel causal models A NC-MCM model is specified by the triple , a neuronal-level dSCM in a behavioral context, a cognitive-level dSCM, and a causally consistent mapping between the two. A learning problem arises when any subset of these three components is not fully specified and needs to be inferred from data. In this section, we consider the case where we have access to a set of samples , generated by some unknown neuronal dSCM in a behavioral context, and our goal is to learn a mapping τ that induces a (behaviorally and dynamically) causally consistent cognitive dSCM. We consider this setting of particular interest because it amounts to learning the cognitive states that a neuronal system has developed in a specific behavioral context from empirical data. Other learning problems are briefly discussed in the final discussion section. Learning a mapping τ that gives rise to a causally consistent cognitive dSCM proceeds in two steps. The first step is to learn a causal model between neuronal states and behaviors. This model then forms the basis for learning the mapping τ that induces a behaviorally and dynamically consistent cognitive dSCM. In this section, we discuss these steps from a conceptual perspective. A particular instantiation of a pipeline that learns a causally consistent cognitive dSCM from empirical data is presented in a later section. Learning a causal model that relates neuronal activity to behavior amounts to learning the interventional distribution P(B[t]|do(X[t])). The gold standard to identify interventional distributions is by experimentation, i.e., by repeatedly setting X[t] to random values via an intervention and observing the induced behaviors. However, large-scale neuronal stimulation with concurrent recordings of neuronal activity and behavior remains a challenge [29]. Alternatively, we may attempt to learn the interventional distribution from observational data only. A variety of causal inference algorithms have been developed for this purpose [18, 25, 30] and applied to neuronal data [15, 17, 24]. However, causal structure learning from observational data is also a hard problem, the computational complexity of which typically grows exponentially in the number of variables. We thus follow a third approach in which we merely learn an observational prediction model P(B[t]|X[t]) from the set of samples . We discuss the implications of substituting an interventional model for an observational one at the end of this section. After learning the interventional or observational distribution between neuronal activity and behavior, the second step is to learn the actual mapping τ. Learning this mapping can be broken down into two further steps. The first step is to construct a behaviorally consistent mapping between neuronal activity and cognitive states. The second step is to empirically test whether the induced cognitive dSCM is also dynamically causally consistent. We note that designing a one-step algorithm that directly constructs a mapping that is guaranteed to be behaviorally and dynamically consistent would be preferable but is beyond the scope of the present work. To construct a behaviorally consistent mapping, we need to find subsets of the neuronal feature space, i.e., partitions, for which P(B[t]|X[t]) is constant. One way to solve this problem is to first learn a model that predicts the probability of each behavior as a function of the neuronal state, e.g., using a logistic regression model, a neural network model, or any other suitable modeling approach. We remark that the sampling rate of the neuronal dynamics must be faster than the transitions of the behavioral labels to allow for good decoding performance. In the second step, the predicted probabilities can then be clustered, e.g., using k-means or any other preferred clustering algorithm. The inferred clusters define a partition of the neuronal feature space with (approximately) constant conditional probabilities over all behaviors, i.e., the mapping τ. This mapping then induces a cognitive dSCM that is by construction behaviorally consistent, with the number of cognitive states being equal to the number of clusters. Because behavioral does not imply dynamic causal consistency, we additionally need to test whether the learned mapping τ also induces a dynamically causally consistent cognitive dSCM. To do so, we first note that our definition of dynamic causal consistency (Def. 5) is identical to condition (3) in Theorem 1 of [31], with the exception that the former and the latter are based on interventional- and observational transition probabilities, respectively. Accordingly, we term condition (3) in [31] dynamic observational consistency. We can then invoke Theorem 4 of [31] to conclude that Markovianity of the cognitive dSCM is sufficient for dynamic observational consistency. Finally, we recall that Markovianity of a dSCM implies that the observational- and interventional transition probabilities coincide. As such, Markovianity of the cognitive dSCM is also sufficient for dynamic causal consistency. To empirically test a cognitive dSCM for Markovianity, we test the null-hypothesis H0: C[t − 1] ⫫ C[t + 1]|C[t]. If we find sufficient evidence against Markovianity, we reject the null hypothesis and conclude that the cognitive dSCM is not dynamically causally consistent. Otherwise, we accept the null hypothesis and conclude that the cognitive dSCM induced by τ is behaviorally and dynamically causally consistent with the data-generating neuronal dSCM. We note that if we find a cognitive dSCM not to be dynamically causally consistent, we can vary the number of clusters to tune the granularity of the cognitive dSCM and repeat the test for Markovianity. If we base the procedure described above on the observational distribution P(B[t]|X[t]) (instead of on the interventional distribution P(B[t]|do{X[t]})), the cognitive dSCM is behaviorally observationally consistent (BOC) but not (in general) behaviorally causally consistent (BCC). However, the causal coarsening theorem states that a causal partitioning is almost always a coarsening of an observational partitioning [32, 33]. As such, a cognitive dSCM that is BCC can be obtained from a cognitive dSCM that is BOC by fusing subsets of cognitive states. While experimental interventions may be required to identify which cognitive states cause behaviors with identical probabilities and thus should be fused, the number of required interventions is on the order of the number of cognitive states and not on the order of the (potentially orders of magnitude larger) number of neuronal states. As such, learning a cognitive dSCM on observational data first and reducing the number of cognitive states by experimental interventions afterward is experimentally more tractable than directly constructing the interventional distribution P(B[t]|do(X[t])) on the neuronal level. If even a reduced number of experimental interventions is not feasible, we may have to accept that a cognitive dSCM is merely BOC. Causal models on the neuronal level To construct causal models on the neuronal level, we first need to decide on a framework for causal modeling. A variety of causal modeling frameworks have been developed for and evaluated on neuronal data [24]. To be applicable in the MCM framework, we require causal models that can make empirically testable predictions on the effects of experimental interventions. The framework of Causal Bayesian Networks (CBNs) [18, 25] fulfills this requirement and hence serves as the causal modeling framework in the present setting. Causal relations in CBNs are modeled by structural causal models (SCMs): Definition 1 (Structural Causal Model—SCM). A SCM is a triple with X a set of N random variables endogenous to the model, E a set of N exogenous noise variables, and F a set of N functions defining each endogenous variable as a function of its direct causes (i.e., parents—pa()) and its corresponding exogenous noise variable, so that for each i ∈ {1, …, N} we have Xi ≔ Fi(pa(Xi), Ei) where the Fi are chosen such that no variable is a (direct or indirect) cause of itself. The joint probability distribution P(E) over the exogenous noise variables induces a joint probability distribution P(X) over the endogenous variables (via the pushforward measure). If the noise variables are mutually independent, the SCM is causally sufficient, i.e., no unobserved confounders that influence multiple endogenous variables exist. In the present setting, each endogenous variable Xi represents the activity of one neuron. We note that in general, X and E may represent continuous- as well as discrete variables, implying that P(X) and P(E) denote probability densities or distributions, respectively. In the following, we assume all variables to be discrete. We consider an extension of the MCM framework to continuous-valued random variables feasible but beyond the scope of the present work. We denote random variables by upper- and their realizations by lower-case letters, e.g., we write P(Xi = x) to indicate the probability that the neuron represented by the random variable Xi takes on the value . Causal relations in SCMs are commonly depicted by directed acyclic graphs (DAGs), with an arrow drawn from node A into node B if the endogenous variable represented by node A is a parent of the endogenous variable represented by node B. An arrow from A into B indicates a direct causal influence of A on B, and a directed path from node A to node B indicates an indirect causal influence of A and B. We note that direct- and indirect causal relations are relative to the set of variables endogenous to the model and may change when dropping or adding nodes. Knowledge of the DAG, in combination with the joint probability distribution over all endogenous variables (represented by the nodes of the DAG), enables us to reason about the probabilistic effects of experimental interventions on variable subsets [18]. Experimental interventions are represented mathematically by the do()-operator, e.g., do(Xi = x) represents the experimental intervention of setting variable Xi to the value x. In empirical settings, the DAG and the joint probability distribution are usually not known and have to be inferred from a combination of experimental and observational data. We show in the section on learning MCMs that knowledge of the causal model on the neuronal level is not a prerequisite for constructing a causally consistent mapping between the neuronal- and the cognitive level. In their original form, SCMs model independent and identically distributed (i.i.d.) data, i.e., data without temporal structure. It is straightforward, however, to extend SCMs to model dynamical neuronal systems by unrolling the endogenous variables across time, as exemplified in the middle row of Fig 1. Here, the set of random variables X[t0] represents the global state of all N neurons at time t0 and the arrow from X[t0] into X[t1] indicates that the global neuronal state at time t0 is a cause of the global neuronal state at time t1, in the sense that the state of each neuron at time t1 is a function of a subset of the neuronal states at time t0 and their respective exogenous noise term. We remark that this representation also allows feedback loops, e.g., as in Xi[t] → Xj[t + 1] → Xi[t + 2]. More formally, we define the extension of SCMs to dynamic settings as follows: Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Relations between cognitive-, neuronal-, and behavioral states in MCM. Solid and dotted arrows denote causal and constitutive relations, respectively. https://doi.org/10.1371/journal.pcbi.1012674.g001 Definition 2 (Dynamic Structural Causal Model—dSCM) A dynamic SCM (dSCM) is family of T triples , indexed by t ∈ {1, …, T}, with X[t] a set of N random variables endogenous to the model, E[t] a set of N exogenous noise variables, and F[t] a set of N functions defining each endogenous variable as a function of its direct causes (i.e., parents—pa()) and its corresponding exogenous noise variable, so that for each i ∈ {1, …, N} and t ∈ {1, …, T} we have Xi[t] ≔ Fi[t](pa(Xi[t]), Ei[t]), where pa(Xi[t]) may contain any endogenous variables prior to time t. We note that we do not allow instantaneous causal relations in Definition 2, i.e., situations where pa(Xi[t1]) may include other neurons at time t1, because our current understanding of physical processes posits that causes must precede effects. This constraint may have to be reconsidered when modeling neuronal dynamics that are observed at sampling rates slower than the system’s dynamics. If the exogenous noise terms are mutually independent and pa(X[t]) only includes variables at the previous time step for all t, as in the example in Fig 1, the dSCM represents a first-order Markov process with a transition probability distribution (1) Markov processes are of particular relevance in the MCM framework, because they define causally sufficient state spaces, in the sense that the global state at any given point in time is causally sufficient for the probability distribution over the global state at the next time point. More formally, Markovianity implies (under the faithfulness assumption) that the exogenous noise terms at different time points are mutually statistically independent and hence (via the backdoor criterion, see [18]) P(X[t]|X[t − 1]) = P(X[t]|do(X[t − 1])) for all t, i.e., the observational- and interventional conditional distributions are identical. A counter-example would be the case of an exogenous variable, e.g., a stimulus, affecting both X[t − 2] and X[t]. In this case, the Markov property would be violated (X[t ∓ 2] ⫫̸ X[t]|X[t − 1]) and hence P(X[t]|X[t − 1]) ≠ P(X[t]|do(X[t − 1])) (we use the symbols ⫫̸ and ⫫ to denote statistical dependence and independence, respectively). As we discuss in the section on how to learn an MCM, empirical tests for Markovianity are essential to ensure that cognitive variables are causally consistent abstractions. In the following, we assume that the functional mappings between states, as well as the probability distribution over their exogenous variables, do not change over time, i.e., we assume the process to be time-homogeneous. We further assume that any process we consider has converged to a stationary probability distribution. These assumptions are not required from a theoretical perspective but greatly simplify empirical inference. To model the causal effect of neuronal activity on behavior, we extend the DAG in Fig 1 by a behavioral state vector B[t] and let X[t] → B[t], i.e., we model the behavioral states at time t to be caused by (a subset of) the neuronal states at time t. We note that it is reasonable to consider relations between neuronal activity and behavior as causal because we can, in principle, manipulate behavior independently of neuronal activity, e.g., we can fixate an animal and thereby prevent neuronal activity from causing an actual movement. We further note that we represent the behavioral states across time by measurement nodes, i.e., by nodes that have no causal effect on any other variables [26]. As such, the behavioral states do not form a Markov process, and B[t] ⫫̸ B[t − 2]|B[t − 1] due to the common effect of X[t − 2] on both B[t − 2] and B[t]. We denote the extension of the dSCM by the behavioral state vector as . To model feedback loops of the neuronal system with its environment, the DAG could be further extended by a state vector S[t] that represents stimuli, which could be influenced by the system’s past behavior, e.g., S[t0] → X[t0] → B[t0] → S[t1]. We leave such an extension for future work. Bridging the neuronal- and the cognitive level Before discussing how to bridge the neuronal- and the cognitive level, we must first consider the nature of cognitive states. Somewhat surprisingly, there currently exists no generally agreed-upon definition of what a cognitive state is (cf. [27], pp. 41 ff.). The broader term cognition is commonly used to denote any kind of mental operation or structure that can be studied in precise terms [28]. To arrive at a working definition that allows us to operationalize cognitive states for the MCM framework, we consider three refinements of this concept of cognition. First, we propose that a cognitive state must be mathematically quantified to be studied in precise terms. Second, we consider an operation to represent a mechanism that translates a system from one state to another. And third, we consider the addendum or structure to indicate that a physical substrate supports any mental operation. These considerations lead us to the following working definition, which we will render mathematically precise towards the end of this section: Definition 3 (Cognitive state) A cognitive state is a causally meaningful abstraction of a neuronal state. To elucidate this definition, we need to explain what the terms causally meaningful and abstraction mean in this context. We consider a concept as causally meaningful if it is conceptually (but not necessarily technically) feasible to fix its state to a specific value by an external intervention. For instance, the concept of age, defined as the time since birth, is not causally meaningful because there is no intervention conceivable that would allow an experimenter to alter the age of a subject. In contrast, the concept of membrane potential is causally meaningful because it is conceivable (and in this case also technically feasible) to set the membrane potential of a neuron to a desired voltage. We consider a state to be an abstraction if it is a macroscopic description of the state of a system that is consistent with (potentially infinitely) many microscopic configurations of the system. Revisiting the example from the introduction, the temperature of a gas is a macroscopic abstraction of the microscopic states of the gas molecules because there are infinitely many configurations of kinetic energies of the molecules in the gas that give rise to the same temperature. This relation, however, is asymmetric. It is impossible to change the temperature of a gas without altering the kinetic energies of its molecules. Accordingly, the temperature of a gas and its molecular configuration do not stand in a causal but in a constitutive relationship. Combining these two concepts, we call an abstraction causally meaningful if the microscopic configurations that determine the macroscopic state are causally meaningful. For instance, consider the statement the gas will ignite if we raise its temperature above a certain threshold. Even though there exists no direct intervention on the macroscopic concept of temperature, the experimental intervention of raising the temperature is meaningful because there exist microscopic interventions (increasing kinetic energies) that are identical to raising the temperature. These causally meaningful microscopic interventions endow the macroscopic concept of temperature with causal meaning. Definition 3 expresses the notion that cognitive- and neuronal states stand in a similar constitutive relationship. While cognitive states are not intrinsically causally meaningful, e.g., it is unclear how one could set the cognitive state of anxiety to a desired level by an experimental intervention that directly acts on anxiety, the neuronal states that are consistent with a certain level of anxiety are intrinsically causally meaningful, in the sense that experimental interventions are conceivable (though not necessarily technically feasible) that realize one out of the consistent neuronal configurations. Furthermore, if intervening on neuronal states has a causal influence on future neuronal dynamics and on behavior, which we consider uncontroversial, this causal efficacy is inherited by cognitive states. To summarize, Definition 3 expresses our understanding that we use cognitive concepts to reason about mental processes because we consider them to be causally effective in terms of the future evolution and behavior of the system under study due to their realization through intrinsically causally meaningful neuronal dynamics. We now formalize the relations discussed above. In analogy to causal models on the neuronal level, we represent the cognitive state of a model system or organism by a cognitive dSCM with C[t] the cognitive state vector (note that E[t] and F[t] are specific to each model and not shared across neuronal- and cognitive dSCMs). To link the the neuronal- and the cognitive dSCM in a causally consistent manner, we require the two models to be behaviorally and dynamically causally consistent: Definition 4 (Behavioral Causal Consistency—BCC) Let a neuronal dSCM in a behavioral context and a cognitive dSCM. Denote the state space of X[t] and C[t] by and , respectively. We call the triple behaviorally causally consistent if is a surjective mapping such that for every pair x1, x2 ∈ τ−1(c) and for every we have (2) where τ−1(c) is the pre-image of c. In this case, we define do(C[t] = c)≔ do(X[t] = x|x ∈ τ−1(c)) and have (3) for all . If a neuronal- and a cognitive SCM are behaviorally causally consistent, every experimental intervention on the neuronal level has a matching intervention on the cognitive level and every cognitive-level intervention has at least one neuronal implementation, in the sense that both interventions lead to the same probability distribution over the behaviors. The surjectivity of τ ensures, first, that the cognitive level does not include any states that do not represent at least one neuronal state, and, second, that the cognitive level can form an abstraction of the neuronal level, i.e., that many neuronal states may map to the same cognitive state yet distinct cognitive states can only be linked to distinct neuronal states. Intuitively, the cognitive SCM constitutes a lossless compression of all information in the neuronal SCM that is causally relevant for a given set of behaviors. As such, behavioral causal consistency allows us to reason interchangeably about the causes of behaviors on the neuronal- and on the cognitive level. Importantly, behavioral causal consistency endows cognitive states with a causal meaning, in the sense that cognitive interventions, for which it may be unclear how the intervention can be experimentally implemented, e.g., increase happiness, can be translated into equivalent interventions on the neuronal level, for which a well-defined experimental procedure is available, e.g., stimulate a set of neurons. The definition of dynamic causal consistency is analogous to behavioral causal consistency except that it considers the probability distribution over the next cognitive state. Definition 5 (Dynamic Causal Consistency—DCC) We call a triple dynamically causally consistent if for all pairs of cognitive states (4) for all x1, x2 ∈ τ−1(c) where τ−1(c) is the pre-image of c. In this case, we define do(C[t] = c)≔ do(X[t] = x|x ∈ τ−1(c)) and have (5) for all and x ∈ τ−1(c). While behavioral causal consistency defines the consistency of neuronal- and cognitive states with respect to behavior, dynamic causal consistency defines consistency with respect to the dynamics of the neuronal system. Intuitively, a cognitive SCM that is DCC constitutes a lossless compression of all information in the neuronal SCM that is causally relevant to the dynamics of the neuronal system. In analogy to behavioral causal consistency, DCC grounds causal relations between cognitive states in the neuronal level, e.g., statements such as reducing boredom is likely to lead to increased happiness can be translated into the neuronal-level statement inducing a neuronal activity pattern x′ that represents low boredom, e.g., by electrical stimulation, is likely to lead to a neuronal state that corresponds to increased happiness. We are now in a position to formally introduce the primary contribution of this work: Definition 6 (Neuro-Cognitive Multilevel Causal Model—NC-MCM) We call a triple a neuro-cognitive multilevel causal model (NC-MCM) if it is behaviorally and dynamically causally consistent. We remark that we have chosen τ not to depend on time because we consider a constant mapping between levels more desirable. It would be straightforward to extend the definition to time-varying mappings, but this generalization would substantially complicate learning the mapping from empirical data. We further note that in contrast to Rubenstein et al. [21] we only consider experimental interventions on the full neuronal- and cognitive state vectors at each time step and leave the extension of NC-MCMs to interventions on subsets of state variables for future work. We illustrate the concept of a NC-MCM in the following toy example. Example 1 Consider a neuronal dSCM with one endogenous variable (N = |X| = 1) and discrete state space , where each state represents one out of a total of four neuronal activity patterns. Further, assume there are three distinct behaviors with conditional probabilities given the neuronal state Note that the conditional probability distribution over the three behaviors is identical for {x1, x2} and for {x3, x4}. As such, one may interpret the system as having two distinct macro-states, {x1, x2} and {x3, x4}, with each macro-state subserving a particular behavioral pattern that is expressed by a constant probability distribution over the three behaviors. We may thus define a mapping (6) These two macro-states form the cognitive state space , where we may choose to name the two cognitive states foraging and recuperating, respectively. Next, assume the neuronal dSCM has the steady-state transition probability distribution TX shown in the left-hand side of Fig 2. The transition matrix TX and the mapping τ induce a cognitive dSCM with the transition probabilities shown in the right-hand side of Fig 2. The triple is behaviorally causally consistent, because the cognitive macro-states condense all information in the neuronal micro-states that are causally relevant for the set of behaviors, i.e., for every intervention on the neuronal micro-level there is an equivalent intervention on the cognitive macro-level that entails the same probabilities over the behaviors (cf. Definition 4). The triple is further dynamically causally consistent because the probabilities of remaining in the current or progressing to another cognitive state are identical for each neuronal state that maps to the same cognitive state. Because the triple is behaviorally and dynamically causally consistent, it fulfills the criteria for a NC-MCM. As such, the cognitive dSCM is a causally meaningful abstraction of the underlying neuronal dSCM because causal statements on the effects of current cognitive states on behavior and on future cognitive states are grounded in the neuronal level, e.g., the cognitive-level statement ‘the animal is moving around and feeding because it is foraging’ can be translated into the neuronal-level statement ‘the animal is moving around and feeding because it is in neuronal state x1 or x2’. We note that behavioral- does not imply dynamic causal consistency, e.g., introducing an asymmetry in the neuronal-level transition probabilities in Fig 2 would break dynamic- but not behavioral causal consistency. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. State-space diagram with transition probabilities for Example 1. https://doi.org/10.1371/journal.pcbi.1012674.g002 We remark that the toy example above is constructed to illustrate the idea that in the NC-MCM framework redundancies in the neuronal dynamics and their relation to behaviors are exploited to build a cognitive-level model that preserves all causally relevant information. In practice, the challenge is to learn which (potentially widely-distributed and complex) neuronal activity patterns are redundant with respect to a behavioral context and with respect to the neuronal dynamics, a problem that we turn to in the next section. Learning neuro-cognitive multilevel causal models A NC-MCM model is specified by the triple , a neuronal-level dSCM in a behavioral context, a cognitive-level dSCM, and a causally consistent mapping between the two. A learning problem arises when any subset of these three components is not fully specified and needs to be inferred from data. In this section, we consider the case where we have access to a set of samples , generated by some unknown neuronal dSCM in a behavioral context, and our goal is to learn a mapping τ that induces a (behaviorally and dynamically) causally consistent cognitive dSCM. We consider this setting of particular interest because it amounts to learning the cognitive states that a neuronal system has developed in a specific behavioral context from empirical data. Other learning problems are briefly discussed in the final discussion section. Learning a mapping τ that gives rise to a causally consistent cognitive dSCM proceeds in two steps. The first step is to learn a causal model between neuronal states and behaviors. This model then forms the basis for learning the mapping τ that induces a behaviorally and dynamically consistent cognitive dSCM. In this section, we discuss these steps from a conceptual perspective. A particular instantiation of a pipeline that learns a causally consistent cognitive dSCM from empirical data is presented in a later section. Learning a causal model that relates neuronal activity to behavior amounts to learning the interventional distribution P(B[t]|do(X[t])). The gold standard to identify interventional distributions is by experimentation, i.e., by repeatedly setting X[t] to random values via an intervention and observing the induced behaviors. However, large-scale neuronal stimulation with concurrent recordings of neuronal activity and behavior remains a challenge [29]. Alternatively, we may attempt to learn the interventional distribution from observational data only. A variety of causal inference algorithms have been developed for this purpose [18, 25, 30] and applied to neuronal data [15, 17, 24]. However, causal structure learning from observational data is also a hard problem, the computational complexity of which typically grows exponentially in the number of variables. We thus follow a third approach in which we merely learn an observational prediction model P(B[t]|X[t]) from the set of samples . We discuss the implications of substituting an interventional model for an observational one at the end of this section. After learning the interventional or observational distribution between neuronal activity and behavior, the second step is to learn the actual mapping τ. Learning this mapping can be broken down into two further steps. The first step is to construct a behaviorally consistent mapping between neuronal activity and cognitive states. The second step is to empirically test whether the induced cognitive dSCM is also dynamically causally consistent. We note that designing a one-step algorithm that directly constructs a mapping that is guaranteed to be behaviorally and dynamically consistent would be preferable but is beyond the scope of the present work. To construct a behaviorally consistent mapping, we need to find subsets of the neuronal feature space, i.e., partitions, for which P(B[t]|X[t]) is constant. One way to solve this problem is to first learn a model that predicts the probability of each behavior as a function of the neuronal state, e.g., using a logistic regression model, a neural network model, or any other suitable modeling approach. We remark that the sampling rate of the neuronal dynamics must be faster than the transitions of the behavioral labels to allow for good decoding performance. In the second step, the predicted probabilities can then be clustered, e.g., using k-means or any other preferred clustering algorithm. The inferred clusters define a partition of the neuronal feature space with (approximately) constant conditional probabilities over all behaviors, i.e., the mapping τ. This mapping then induces a cognitive dSCM that is by construction behaviorally consistent, with the number of cognitive states being equal to the number of clusters. Because behavioral does not imply dynamic causal consistency, we additionally need to test whether the learned mapping τ also induces a dynamically causally consistent cognitive dSCM. To do so, we first note that our definition of dynamic causal consistency (Def. 5) is identical to condition (3) in Theorem 1 of [31], with the exception that the former and the latter are based on interventional- and observational transition probabilities, respectively. Accordingly, we term condition (3) in [31] dynamic observational consistency. We can then invoke Theorem 4 of [31] to conclude that Markovianity of the cognitive dSCM is sufficient for dynamic observational consistency. Finally, we recall that Markovianity of a dSCM implies that the observational- and interventional transition probabilities coincide. As such, Markovianity of the cognitive dSCM is also sufficient for dynamic causal consistency. To empirically test a cognitive dSCM for Markovianity, we test the null-hypothesis H0: C[t − 1] ⫫ C[t + 1]|C[t]. If we find sufficient evidence against Markovianity, we reject the null hypothesis and conclude that the cognitive dSCM is not dynamically causally consistent. Otherwise, we accept the null hypothesis and conclude that the cognitive dSCM induced by τ is behaviorally and dynamically causally consistent with the data-generating neuronal dSCM. We note that if we find a cognitive dSCM not to be dynamically causally consistent, we can vary the number of clusters to tune the granularity of the cognitive dSCM and repeat the test for Markovianity. If we base the procedure described above on the observational distribution P(B[t]|X[t]) (instead of on the interventional distribution P(B[t]|do{X[t]})), the cognitive dSCM is behaviorally observationally consistent (BOC) but not (in general) behaviorally causally consistent (BCC). However, the causal coarsening theorem states that a causal partitioning is almost always a coarsening of an observational partitioning [32, 33]. As such, a cognitive dSCM that is BCC can be obtained from a cognitive dSCM that is BOC by fusing subsets of cognitive states. While experimental interventions may be required to identify which cognitive states cause behaviors with identical probabilities and thus should be fused, the number of required interventions is on the order of the number of cognitive states and not on the order of the (potentially orders of magnitude larger) number of neuronal states. As such, learning a cognitive dSCM on observational data first and reducing the number of cognitive states by experimental interventions afterward is experimentally more tractable than directly constructing the interventional distribution P(B[t]|do(X[t])) on the neuronal level. If even a reduced number of experimental interventions is not feasible, we may have to accept that a cognitive dSCM is merely BOC. Multilevel causal modeling in C. elegans In this section, we demonstrate the application and utility of the NC-MCM framework on calcium imaging data recorded in the nematode C. elegans. After introducing the data, we show how to learn cognitive dSCMs that are behaviorally observationally consistent and test them for dynamic causal consistency. We then study the NC-MCMs in terms of the behavioral motifs they uncover, the representations of C. elegans’ neuronal manifold as a graph, and the insights these models enable into decision making in C. elegans. C. elegans is an ideal model system to demonstrate the utility of the NC-MCM framework for multiple reasons. It has only 302 neurons, about a third of which can be simultaneously recorded by Ca2+ imaging in an individual animal at single-cell resolution [34–36]. Due to the stereotypical nature of the nematode’s neuronal system, many of these neurons can be identified and thus compared across animals. C. elegans further has a small behavioral repertoire of motor programs like, e.g., forward crawling, backward crawling and turning. This moderate complexity, in combination with extensive prior knowledge of relations between neuronal activity and behavior, allows us to demonstrate the neuropysiological plausibility of the insights derived from the NC-MCM framework, paving the way for its application to more complex model systems. Despite these many advantages of C. elegans as a model system, there are a few drawbacks regarding available data sets. To demonstrate the full potential of the NC-MCM framework, i.e., to learn a dynamically and behaviorally causally consistent model, we require data that combine external interventions on diverse sets of individual neurons with concurrent recordings of high-dimensional neuronal dynamics at single-cell resolution while tracking behavior in unconstrained animals. At the time of writing, such data are not publicly available. As described in detail below, the data available to us does not incorporate external neuronal stimulation and has been recorded in immobilized animals in which the behavioral labels are deduced through established equivalences between the activity of individual neurons and the worms’ behavior. As a result, we can present a model that fulfills dynamic causal consistency but not behavioral causal consistency; we are limited to behavioral observational consistency on reconstructed behaviors. However, this model can be used to make empirically testable predictions on the causal effects of neuronal dynamics on behavior, as we discuss in the section Towards understanding decision making in C. elegans. We remark that the framework for learning a causally consistent cognitive dSCM is model agnostic in the sense that each of the necessary computational steps can be accomplished by different algorithms that solve the same computational task, e.g., the behavioral probabilities can be estimated from the neuronal states via logistic regression, (deep) neuronal networks, or any other suitable decoding method. We follow the general guideline that simple models should be preferred over complex ones if they provide sufficiently accurate results. The pipeline described below is implemented as the function learn_mcm(neuronal_data, behavioral_labels) in the nilab toolbox available for the Julia programming language at https://github.com/moritzgw/nc-mcm. The code to reproduce the empirical results presented here, which is based on the nilab toolbox and customized Python functions for plotting, is also available in that repository. We note that due to the inherent randomness of some of the modules, e.g., k-means clustering and permutation-based statistical tests, results may slightly vary across multiple runs. We carefully checked that the empirical results reported below are qualitatively consistent across runs. The data The data set we use has been recorded by Kato et al. [37] and is available at https://osf.io/2395t/. We use the data subset that has been recorded without externally applied chemosensory stimulation. It consists of data from five immobilized worms with 107—131 neurons recorded in each individual worm for a period of 18 minutes at a sampling rate of approximately 2.85 Hz. We subsequently refer to a sample of the calcium traces as the neuronal state vector X[t], which represents the state of all neurons recorded in one animal at time t. Even though all five worms were immobilized during the recordings, established equivalences between the activity of individual neurons and the worms’ behavior were used to label each neuronal state vector with the behavior that would have been concurrently observed in a non-immobilized worm; these encompass motor commands for forward crawling, forward slowing, backward crawling, turning dorsally and turning ventrally [37]. Based on distinct activity patterns, the neuronal state corresponding to backward crawling was further subdivided into reversal 1, reversal 2, and sustained reversal [37]. To date, the behavioral correlates of these sub-divisions are not known. The behavioral labels assigned to the neuronal states by [37] are . Neuronal states that appeared ambiguous, i.e., for which the behavioral label could not be clearly identified, are labeled as nostate, corresponding to a small fraction (∼1%) of the data frames. Learning a cognitive model of C. elegans Learning a cognitive model proceeds in three steps: Predicting the probabilities of all behaviors for every neuronal state vector, clustering the predicted probabilities to obtain the cognitive states, and testing the induced cognitive state transitions for Markovianity. To learn the mapping τ, we employed k-means clustering in the 28-dimensional space of probability estimates to assign neuronal states to clusters with (approximately) constant conditional behavioral probabilities (cf. Definition 4). We varied the number of clusters for k-means between two and 20 and re-ran k-means 100 times for each k with random initial seeds. For each k and run, we thereby obtained an assignment of every neuronal state to one out of k cognitive states, which we subsequently refer to as a cognitive state trajectory C[t]. We then tested each of the 5 (worms) × 20 (range of the number of cognitive states) × 100 (clustering runs) cognitive state trajectories for Markovianity by, first, estimating the first- and second-order cognitive state transition probability matrices P(C[t]|C[t − 1]) and P(C[t]|C[t − 1], C[t − 2]), second, computing the total variance of P(C[t]|C[t−1]) across all states of C[t−2] (whose expected value for a Markov process is zero), and third, computing the same total variance for one thousand simulated Markovian cognitive state trajectories with state transition probability matrix P(C[t]|C[t − 1]). This enabled us to estimate the p-value under the null hypothesis of a Markovian cognitive state trajectory as the frequency at which the simulated total variance exceeded the observed one (this test for Markovianity is implemented as the function markovian() in the nilab toolbox). Because higher p-values signify a better clustering result, in the sense that the learned cognitive state trajectory exhibits no evidence against Markovianity, we then picked, for each worm and number of cognitive states, the cognitive state trajectory with the highest p-value. We remark that we could also split the data into a training- and a test set, learn and select the best clustering on the training set, and then test for Markovianity on the held-out test set. Because the number of samples available in this setting is limited, and splitting the data would further reduce the probability of finding evidence against Markovianity on the test set, using all data for clustering is the more conservative approach. This issue is relevant when invoking the DCC property, which we revisit in the section on decision making. Fig 3 shows the p-values of the selected cognitive state trajectories as a function of the number of cognitive states for each worm (dots) as well as averaged across all worms (line). When choosing α = 0.05 as a lower threshold for accepting the null hypothesis, we obtain Markovian cognitive state trajectories for any number of cognitive states in the range from three to 19, with a maximum average p-value for seven states. As such, we infer that all cognitive state trajectories with three to 19 states are dynamically causally consistent. As discussed below, we can then control the degree of granularity at which we study the dynamics of C. elegans by varying the number of cognitive states in this range. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. p-values for rejecting the null-hypothesis of Markovianity for each worm (and averaged across worms) as a function of the number of cognitive states. https://doi.org/10.1371/journal.pcbi.1012674.g003 To build a better intuition for the procedure of learning a cognitive state trajectory, Fig 4 illustrates the relevant steps on the data of the first worm with five cognitive states (all trajectories are projected to their first two principal components for visualization): The neuronal state trajectories together with the behavioral labels (A) are used to estimate the behavioral probability trajectories (B). These trajectories are then clustered to assign each element of the behavioral probability trajectories to a cognitive state (C). We remark that Fig 4B and 4C are projections from a 28- to a two-dimensional space via PCA, which does not adequately represent the spatial separation of neuronal states that belong to distinct cognitive clusters. Re-projecting the cognitive labels onto the neuronal state trajectories gives the neuro-cognitive state trajectories (D), from which we can compute the cognitive state transition model P(C[t]|C[t − 1]). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Illustration of the procedure of learning a cognitive state trajectory (see main text for details). https://doi.org/10.1371/journal.pcbi.1012674.g004 We remark that the cognitive dSCM of each animal is behaviorally observationally consistent by construction and dynamically causally consistent if the cognitive dSCM is Markovian (we emphasize again that Markovianity can not be verified; we can only fail to find evidence against Markovianity and thus not reject the null hypothesis). Because we built the cognitive dSCM on the observational distribution P(B|X) rather than on the interventional distribution P(B|do(X)), which is not available in the present setting, behavioral causal consistency is not guaranteed. In its present form, the cognitive dSCM thus only supports causal statements about the dynamics of the cognitive dSCM but not about its relation to behavior. However, its observational behavioral consistency can be used to derive causal hypotheses that guide the design of experimental studies to establish causal relations between neuronal states and behaviors via interventions, a topic we discuss below in more detail. Interpreting the cognitive model of C. elegans In this section, we demonstrate the ability of the learned cognitive models to reveal behavioral motifs of C. elegans, show that from a data-visualization perspective the NC-MCM framework can be understood as a neuronal manifold learning technique that abstracts the essential features of a manifold into a directed graph, and discuss how the cognitive model can provide insights into decision-making processes. Behavioral motifs of C. elegans. In the following, we juxtapose the behavioral state transition diagram with the state diagram that is obtained by expanding the behavioral- by the cognitive states, i.e., we compare P(B[t]|B[t − 1]) and P(C[t], B[t]|C[t − 1], B[t − 1]). In particular, we show that the behavioral state transition diagram conflates distinct behavioral motifs that are revealed when considering the behavior commands in relation to the cognitive states in which they appear. We first illustrate the results on the third worm in the data set because this worm exhibits the most simple state transitions of all the five worms. We then show that the behavioral motifs discussed below are present with small variations in every worm. Fig 5 shows the behavioral- (A, left column) and the cognitive-behavioral state diagram with three cognitive states (B, right column) of the third worm in the data set (we have chosen three cognitive states because this is the lowest number of cognitive states for which we do not find evidence against Markovianity; the behavioral motifs discussed below are also apparent when considering models with more cognitive states). In the left diagram, each node represents one behavioral state. The size of each node represents the probability of the worm being in the corresponding behavioral state at any point in time. The width of an outgoing edge represents the probability of the worm transitioning from a particular state to another state (to reduce spurious edges that result from jitters of the neuronal trajectories across the decision boundaries in the probability space, we only plot state transition if the length of a state exceeds two samples). Edges indicating the probabilities of staying in a particular behavioral state, i.e., edges from one node to itself, are not drawn to reduce clutter. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Behavioral- and cognitive-behavioral state diagrams of the third worm (see text for details). Arrows that account for less than 0.1% of outgoing transitions of each node have been removed to reduce clutter. https://doi.org/10.1371/journal.pcbi.1012674.g005 Fig 5A reveals a clear structure in the worm’s behavioral command (or states) sequences. The worm is most often found executing a sustained reversal state (revsus). A sustained reversal is either followed by a ventral- (vt) or dorsal (dt) turn state, both of which segue into a slowing state (slow). The slowing state alternates with a forward state (fwd) before returning to a sustained reversal state via either of two types of reversal states (rev1 or rev2). We have chosen observational language to describe this behavioral state transition diagram because the behavioral dynamics are not Markovian (the statistical test for Markovianity described earlier rejects the null hypothesis of Markovianity at α = 0.05 (p = 0.015)). This non-Markovianity indicates that past behavioral states provide information about the future state that is not contained in the current state, as was suggested by [37]. This fact becomes intuitively plausible when considering the cognitive-behavioral state transition diagram in Fig 5B. This diagram expands the behavioral by the cognitive states, i.e., we represent the eight behaviors separately for each cognitive state in which they occur (note that the cognitive states are drawn counter clockwise with the eight behaviors of each cognitive state also arranged in circles). This Markovian representation reveals two distinct behavioral motifs: A revsus → vt → slow → rev1 → revsus and a revsus → dt → fwd ↔ slow → rev2 → revsus loop. These two loops that occur in cognitive states C3 and C2, respectively, are connected via cognitive state transitions between C1 and C3 during a sustained reversal (which we discuss in more detail below). The first motif represents a repeated execution of a state sequence composed of sustained reversal followed by a ventral turn, slowing and reversal 1. The second motif is composed mostly of switching between slowing and forward crawling states. In the third worm, this motif always starts with a dorsal turn state and terminates with a reversal 2. These behavioral motifs are well known in C. elegans [38, 39]. The first motif corresponds to a pirouette, where animals execute multiple reverse-turning events in close succession. The second motif corresponds to a forward run episode, where animals persistently head in a forward direction. This motif is terminated by a rev2 state, corresponding to the neuronal signature that terminates a forward run episode (cf. [37]). Notably, the NC-MCM framework discovered these motifs without any prior knowledge of the underlying behavioral dynamics. Rather, the behavioral motifs are revealed by the NC-MCM framework because they are embedded in distinct neuronal dynamics. Importantly, this entails the ability of the NC-MCM framework to assign time frames that were manually annotated with the same behavioral label to distinct cognitive states if the executed behaviors differ in their neuronal realization. In Fig 5, this is apparent in two distinct slowing behaviors, one in each behavioral motif, that are supported by cognitive states C2 and C3, respectively. In C2, slowing could correspond to brief episodes of reducing locomotion speed while the animals remain in a forward run [37]. In C3, slowing could correspond to intermittent pausing or brief forward crawling during successive re-orientations. In the behavioral state diagram in the left column of Fig 5, these two different types of slowing movements are conflated. This conflation leads to a non-Markovian representation because the probabilities of entering a rev1 and rev2 state are higher when the slowing movement is preceded by a ventral and a dorsal turn, respectively. Conversely, the Markovian representation of the cognitive-behavioral state diagram ensures that it correctly distinguishes seemingly similar behaviors that are supported by distinct neuronal dynamics. While the third worm exhibits the most distinct separation of the two behavioral motifs, both motifs are present in every worm (Fig 6). Because the worms slightly vary in the complexity of their state transitions, with the other worms also showing infrequent state transitions between the two motifs, the number of cognitive states at which the two behavioral motifs first emerge varies across worms (four cognitive states for the first two worms and three cognitive states for the remaining three worms). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Behavioral- and cognitive-behavioral state diagrams of all worms. Arrows that account for less than 0.075% of outgoing transitions of each node have been removed to reduce clutter. https://doi.org/10.1371/journal.pcbi.1012674.g006 Representing C. elegans’ neuronal manifold as a directed graph. Individual neurons are embedded in brain networks that collectively organize their high-dimensional neuronal activity patterns into lower-dimensional neuronal manifolds [40, 41]. The goal of neuronal manifold learning techniques is to find low-dimensional representations of neuronal data that enable insights into the structure of neuronal dynamics and their relation to behavior [9]. In neuroscience, classic dimensionality reduction techniques, such as principal component analysis (PCA), Laplacian eigenmaps (LEM), and t-SNE, are complemented by modern techniques such as UMAP [10], MIND [11], CEBRA [12], and BundDLe-Net [13]. In the following, we show that the cognitive-behavioral state diagrams introduced in the previous section can be interpreted as a neuronal manifold learning technique that represents the essential aspects of the manifold as a directed graph. Fig 7 shows a side-by-side comparison of the neuronal manifold of the third worm as inferred by BundDLe-Net [13] (left column) and the cognitive-behavioral state diagram of the NC-MCM framework (right column). Looking at the left column, it is immediately apparent that the neuronal manifold exhibits two main cycles, a revsus (gray) → vt (turquoise) → slow (red) → rev1 (green) → revsus (gray) and a revsus (gray) → dt (orange) → slow (red) → fwd (yellow) → rev2 (blue) → revsus (gray) cycle, that correspond to the behavioral motifs revealed by the cognitive-behavioral state diagram in the right column of Fig 7 and discussed in the previous section. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 7. Side-by-side comparison of the neuronal manifold of the third worm as learned by BunDLe-Net and its representation as a directed graph in the NC-MCM framework. https://doi.org/10.1371/journal.pcbi.1012674.g007 To better understand the structure of the neuronal manifold, we now consider the branching and convergence points of the neuronal trajectories. We identify the dense cluster of neuronal states that correspond to a sustained reversal (revsus shown in gray at the bottom of the left plot in Fig 7) as the root node of the manifold that acts both as a branching and a convergence point. In particular, the neuronal state trajectories branch out from here into one of the two behavioral motifs via a ventral- (vt—turquoise) or a dorsal turn (dt—orange). Once a ventral turn is initiated, the neuronal trajectories are highly consistent; all trajectories execute a loop (accompanied by a ventral turn, slowing, and reversal 1) before converging again into the root node during a sustained reversal. If a dorsal turn is initiated, on the other hand, the neuronal trajectories enter a second branching point during the subsequent slowing movement (red), from which they either return directly during a rev2 behavior (blue) into the root node (gray) or take the longer route into the extended branch of the persistent alternation between slowing (red) and forward (yellow) behavior. Both of these branches converge again in the rev2 behavior before returning to the sustained reversal root node. To summarize, the dynamics on the neuronal manifold are determined by a small number of branching and convergence points with highly consistent trajectories between these points. In the cognitive-behavioral state diagram in the right column of Fig 7, the branching and convergence points of the neuronal manifold are represented by nodes with an out- and in-degree greater than one, respectively. The trajectories between the branching and convergence points are represented by nodes with an out- and in-degree of one. Based on this correspondence, it is easy to verify visually that the cognitive-behavioral state diagram constitutes a representation of the structure of the neuronal manifold as a directed graph in the sense that the nodes of the graph represent bundles of trajectories on the neuronal manifold and the edges between the nodes represent the possible paths that the neuronal trajectories can take on the manifold. This correspondence also holds for the other four worms shown in Figs 8–11 (three-dimensional rotating GIF visualizations of the embeddings are available in the BunDLe-Net repository at https://github.com/akshey-kumar/BunDLe-Net). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 8. Side-by-side comparison of the neuronal manifold and the cognitive-behavioral state diagram of worm 1. https://doi.org/10.1371/journal.pcbi.1012674.g008 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 9. Side-by-side comparison of the neuronal manifold and the cognitive-behavioral state diagram of worm 2. https://doi.org/10.1371/journal.pcbi.1012674.g009 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 10. Side-by-side comparison of the neuronal manifold and the cognitive-behavioral state diagram of worm 4. https://doi.org/10.1371/journal.pcbi.1012674.g010 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 11. Side-by-side comparison of the neuronal manifold and the cognitive-behavioral state diagram of worm 5. https://doi.org/10.1371/journal.pcbi.1012674.g011 As such, the cognitive state transition models of the NC-MCM abstract away variability of the the neuronal dynamics to reveal the essential structure of the neuronal manifold in a directed graph. In this way, the NC-MCM reduces the study of the structure of neuronal manifolds to a graph theoretic problem, a mature scientific field for which a plethora of computationally efficient algorithms are available [42]. Towards understanding decision making in C. elegans. As we have seen in the previous section, the neuronal trajectories alternate between segments with highly consistent dynamics, e.g., when executing the behavioral motif initiated by a ventral turn, and segments with high uncertainty about the future dynamics and behavior, e.g., when performing a sustained reversal movement. In the following, we show how the cognitive-behavioral state transition diagrams of the NC-MCM framework provide insights into the neuronal basis of future neuronal dynamics during segments with high uncertainty, i.e., we show how to use the NC-MCM framework to study decision-making in C. elegans. In particular, we study how a ventral vs. a dorsal turn is initiated during a sustained reversal and how the alternating forward and slowing behavior is terminated by a rev2-reversal. We begin again by considering the cognitive-behavioral state transition diagram of the third worm in Fig 7. In particular, we note that the sustained reversal movement (revsus) is distributed across the two cognitive states C1 and C3, with frequent, cyclical transitions between the two states. Importantly, the probability that the worm transitions from a sustained reversal to a particular turning behavior differs between these cognitive states. When the worm is in cognitive state C3, it transitions from a sustained reversal into a ventral- and a dorsal turn at frequencies of 3.04% and 0.72%, respectively. In cognitive state C, on the other hand, the frequencies of transitions from a sustained reversal into a ventral- and a dorsal turn are 0.17% and 0.84%, respectively. Ventral turns are thus roughly 18 times more frequently initiated in state C3 than in state C1 while dorsal turn initiations occur with similar frequencies in both cognitive states (they are 1.17 times more likely in C1 than in C3). We can thus interpret a cognitive state transition from C1 into C3 during a sustained reversal movement as a decision process that makes a ventral turn more likely than a dorsal turn and vice versa. We now turn to the question of how these cognitive state transitions are realized on the neuronal level. To do so, we analyze which global perturbation of neuronal states during a particular type of movement the model predicts to induce a cognitive state that increases the probability of another type of behavior. To illustrate this idea, consider again the two states C1:revsus and C3:revsus of the third worm. By first computing the mean activation of every neuron in each of the two states, i.e., the mean of all neuronal states that map to C1:revsus and the mean of all neuronal states that map to C3:revsus, and then subtracting the two means, we obtain the neuronal perturbation that the cognitive-behavioral model predicts to induce a state transition from C1:revsus to C3:revsus; a state transition that increases the probability of transitioning into a ventral turn from 0.17% to 3.04%. To generalize from this example, we can pick a source behavior, e.g., a sustained reversal, and a target behavior, e.g., a ventral turn, determine all cognitive states in which the source behavior occurs, for each pair of cognitive states compute the neuronal perturbation that the NC-MCM model predicts to induce a transition from one to the other, weigh this neuronal perturbation by the difference in probability in the target behavior between the two cognitive states, and finally average the neuronal perturbations across all possible state transitions and worms. This results in a neuronal perturbation vector, which we subsequently denote by , that the NC-MCM model predicts, on average across all cognitive states and worms, to induce a state transition that increases the probability of the target behavior while maintaining the current source behavior. This procedure necessitates that the cognitive-behavioral state diagram of each worm represents the source behavior in multiple cognitive states. As can be checked in Figs 7–11, out of the cognitive-behavioral state diagrams with three cognitive states only the state diagram of the third worm represents the uncertainty during the sustained reversal movement by more than one cognitive state. This limitation can be easily remedied by considering more than three cognitive states. In the following, we consider cognitive-behavioral state diagrams with seven cognitive states, because we found seven to be the optimal number of cognitive states with respect to (the absence of evidence against) Markovianity across all worms, cf. Fig 3 (we remark that among all Markovian cognitive models there is no correct or incorrect number of cognitive states—we can vary the number of cognitive states and thus choose the level of granularity at which we analyze the cognitive dynamics). Fig 12 displays the corresponding cognitive-behavioral state diagram for the third worm. Increasing the number of cognitive states from three to seven preserves the overall structure as well as the two primary behavioral motifs while splitting up the cognitive states into a more fine-grained representation of the neuronal dynamics and their relation to behavior. In particular, the pirouette motif that is initiated by a ventral turn and that was represented by cognitive state C3 in Fig 7 is represented in Fig 12 by the two cognitive states C1 and C2. The forward-run motif which is initiated by a dorsal turn and that was represented in Fig 7 by C2 is represented in Fig 12 by the two cognitive states C4 and C5. The uncertainty in the transitions between these two motifs, which was characterized in Fig 7 by the cyclical alterations between cognitive states C1 and C3 during a sustained reversal movement, is represented in the seven-states model by transitions between the cognitive states C2, C6, and C7. Moving to cognitive-behavioral state diagrams with seven cognitive states in each worm, we can thus compute the neuronal perturbation vector between a source and a target behavior across a broad range of cognitive states across all worms. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 12. Cognitive-behavioral state transition diagram of the third worm with seven cognitive states. https://doi.org/10.1371/journal.pcbi.1012674.g012 Fig 13 displays the neuronal perturbation vectors , , and for all neurons that are shared across the recordings of all five worms, i.e., the three neuronal perturbation vectors that during a sustained reversal render the maintenance of the sustained reversal, a transition to a ventral turn, and a transition to a dorsal turn more likely. Double and single markers indicate the rejection of the null hypothesis of equal means across conditions at significance levels α = 0.01 and α = 0.05, respectively. Plus signs and asterisks indicate significance tests with and without Bonferroni correction, respectively. Statistical tests were carried out with an ANOVA based on 10.000 random permutations with Bonferroni correction for multiple comparisons according to the number of neurons. Not considering those neurons used for labeling the behaviors (AVAL, AVAR, SMDDR, SMDDL, SMDVR, SMDVL, RIBR, and RIBL), the neuronal perturbation vector exhibits highly statistically significant differences across specific sets of descending interneurons. In particular, the NC-MCM model predicts that increasing the activation of AVBL and AVBR while reducing the activity of AVEL and AVER increases the probability of a turning behavior, with weaker and stronger changes linked to dorsal- and ventral turns, respectively. Conversely, applying the opposite pattern makes it more likely that worms maintain a sustained reversal. We remark that RIVL and RIVR may play a further role in differentiating between dorsal and ventral turns. Because these two neurons show no significant differences after Bonferroni correction for multiple comparisons, we refrain from further interpreting their effects. These neuronal perturbation vectors for maintaining or terminating a sustained reversal are markedly different from those obtained for changes between cognitive states that either maintain a slowing-forward motif or interrupt this motif through a rev2-type reversal, as shown in Fig 14. Here, an increase in activation of AIBL, AIBR, ASKR, RIVL, and RIVR and a decrease in activation of RMED, RMEL, and RMER are predicted to maintain a slowing behavior, with the converse pattern inducing a cognitive state that interrupts the slowing-forward motif by a rev2-type reversal. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 13. Neuronal perturbations that, when applied during a sustained reversal, the NC-MCM framework predicts to induce a cognitive state transition that increases the probability of maintaining a sustained reversal movement (blue), initiating a ventral turn (orange), or initiating a dorsal turn (green). Double and single markers indicate the rejection of the null hypothesis of equal means across conditions at significance levels α = 0.01 and α = 0.05, respectively. Plus signs and asterisks indicate significance tests with and without Bonferroni correction, respectively. https://doi.org/10.1371/journal.pcbi.1012674.g013 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 14. Neuronal perturbations that, when applied during a slowing movement, the NC-MCM framework predicts to induce a cognitive state transition that increases the probability of maintaining a slowing movement (blue), initiating a forward movement (orange), or initiating a rev2 reversal (green). Double and single markers indicate the rejection of the null hypothesis of equal means across conditions at significance levels α = 0.01 and α = 0.05, respectively. Plus signs and asterisks indicate significance tests with and without Bonferroni correction, respectively. https://doi.org/10.1371/journal.pcbi.1012674.g014 Due to the DCC property of the cognitive models discussed above, we may interpret the neuronal perturbation vectors as empirically testable predictions which perturbations of neurons via external stimulation induce cognitive states that render certain behaviors more or less likely (because the learned cognitive models are only observationally but not causally behaviorally consistent, we may not apply the same causal reasoning to the relations between cognitive state transitions and behaviors). However, it remains an open question how these perturbations come about naturally, i.e., how the worm decides which behavioral motif to initiate and when. A priori, there are two potential explanations. In the first explanation, the worm’s neuronal dynamics could be deterministic with the apparent randomness during sustained reversals and the slowing-forward motifs being due to some unobserved (latent) neurons. In this interpretation, observing all behaviorally relevant neurons would eliminate the randomness and lead to non-overlapping trajectory bundles in Figs 7–11. In the second interpretation, the indeterminacy may be due to inherent randomness in neuronal activation. Due to the DCC property, our results are consistent with the second but not with the first interpretation: A latent neuron that affects at least one observed neuron would induce a dependence between the current- and past states of the observed neuron, rendering the neuronal dynamics non-Markovian. Because we found no evidence against Markovianity in the cognitive state transitions, which constitute an abstraction of all behaviorally relevant information in the neuronal dynamics, our results are consistent with the interpretation that decision-making in C. elegans has an intrinsically random component, as was suggested for switches between forward and backward movement [43]. However, we remark that in the present setting the worms have been immobilized and received no time-varying sensory input. In more ecologically realistic settings, sensory stimuli may override inherent randomness. Discussion of the experimental results in C. elegans The NC-MCM framework, applied to brain wide Ca2+-imaging data of C. elegans, revealed cognitive states that correspond to neuronal representations of behavioral hierarchies that sit on top of a previously described motor hierarchy [44]. A pirouette is a long time scale behavioral motif that encompasses the repetitive execution of reversal-turn command sequences, which can be triggered spontaneously or by sensory inputs [38, 45]. In our data, a pirouette manifests as a motif of switching dynamics between neuronal activity patterns representing these actions; notably the NC-MCM framework was able to find these patterns in the absence of neuronal activities that specifically mark the initiation, duration or termination of a pirouette, i.e., which we would have interpreted as explicit upper-hierarchy neuronal representations of a pirouette. We cannot exclude that our recording technique is insensitive to such activation patterns. But alternatively, and in analogy to the pressure of a gas (see Introduction), a pirouette could be seen as a useful causally consistent description of a higher-level navigational strategy of C. elegans [38, 45] that emerges from the ‘microscopic’ dynamical switching behavior of neuronal circuits without any explicit neuronal representation. This has implication on potential control mechanisms, which, thereby could act directly at the level of synapses and neuronal excitability within these circuits, and without the need of upper-hierarchy command-circuits. In the same vein, the learned NC-MCM suggests that forward run is a dynamical motif that encompasses continuous forward crawling with intermittent slowing events. It further suggests that slowing during forward runs is neuronally distinct from slowing during pirouettes, which were, however, lumped together into one state in our previous work [37]. When allowing for a finer granularity of many cognitive states, the NC-MCM framework can be a useful discovery tool to identify circuit elements that are potentially implicated in decision making. How C. elegans decides between a dorsal or ventral turn following a reversal is not known [37]. The learned NC-MCM suggests circuit elements that terminate sustained reversals and influence the subsequent binary choice via expected motor neuron classes (RIV, SMDV or RMED) [46], but also unexpected neuronal cell types like the descending interneuron class AVE or the neuromodulatory neuron class ALA. Likewise, the NC-MCM makes concrete suggestion for the circuitry that controls slowing, like the AIB neurons that show Ca2+-transients during discrete slowing events in freely moving worms [37], or RIB neurons that have been implicated in the control of locomotion speed [37, 44, 47]. Interestingly, ASK sensory neurons, which exhibit spontaneous Ca2+-fluctuations under the experimental conditions used here [48, 49] are further suggested to control slowing. The NC-MCM framework, thereby, can effectively guide future interrogation strategies, such as optogenetics, to confirm the role of these classes as decison making neurons. The data The data set we use has been recorded by Kato et al. [37] and is available at https://osf.io/2395t/. We use the data subset that has been recorded without externally applied chemosensory stimulation. It consists of data from five immobilized worms with 107—131 neurons recorded in each individual worm for a period of 18 minutes at a sampling rate of approximately 2.85 Hz. We subsequently refer to a sample of the calcium traces as the neuronal state vector X[t], which represents the state of all neurons recorded in one animal at time t. Even though all five worms were immobilized during the recordings, established equivalences between the activity of individual neurons and the worms’ behavior were used to label each neuronal state vector with the behavior that would have been concurrently observed in a non-immobilized worm; these encompass motor commands for forward crawling, forward slowing, backward crawling, turning dorsally and turning ventrally [37]. Based on distinct activity patterns, the neuronal state corresponding to backward crawling was further subdivided into reversal 1, reversal 2, and sustained reversal [37]. To date, the behavioral correlates of these sub-divisions are not known. The behavioral labels assigned to the neuronal states by [37] are . Neuronal states that appeared ambiguous, i.e., for which the behavioral label could not be clearly identified, are labeled as nostate, corresponding to a small fraction (∼1%) of the data frames. Learning a cognitive model of C. elegans Learning a cognitive model proceeds in three steps: Predicting the probabilities of all behaviors for every neuronal state vector, clustering the predicted probabilities to obtain the cognitive states, and testing the induced cognitive state transitions for Markovianity. To learn the mapping τ, we employed k-means clustering in the 28-dimensional space of probability estimates to assign neuronal states to clusters with (approximately) constant conditional behavioral probabilities (cf. Definition 4). We varied the number of clusters for k-means between two and 20 and re-ran k-means 100 times for each k with random initial seeds. For each k and run, we thereby obtained an assignment of every neuronal state to one out of k cognitive states, which we subsequently refer to as a cognitive state trajectory C[t]. We then tested each of the 5 (worms) × 20 (range of the number of cognitive states) × 100 (clustering runs) cognitive state trajectories for Markovianity by, first, estimating the first- and second-order cognitive state transition probability matrices P(C[t]|C[t − 1]) and P(C[t]|C[t − 1], C[t − 2]), second, computing the total variance of P(C[t]|C[t−1]) across all states of C[t−2] (whose expected value for a Markov process is zero), and third, computing the same total variance for one thousand simulated Markovian cognitive state trajectories with state transition probability matrix P(C[t]|C[t − 1]). This enabled us to estimate the p-value under the null hypothesis of a Markovian cognitive state trajectory as the frequency at which the simulated total variance exceeded the observed one (this test for Markovianity is implemented as the function markovian() in the nilab toolbox). Because higher p-values signify a better clustering result, in the sense that the learned cognitive state trajectory exhibits no evidence against Markovianity, we then picked, for each worm and number of cognitive states, the cognitive state trajectory with the highest p-value. We remark that we could also split the data into a training- and a test set, learn and select the best clustering on the training set, and then test for Markovianity on the held-out test set. Because the number of samples available in this setting is limited, and splitting the data would further reduce the probability of finding evidence against Markovianity on the test set, using all data for clustering is the more conservative approach. This issue is relevant when invoking the DCC property, which we revisit in the section on decision making. Fig 3 shows the p-values of the selected cognitive state trajectories as a function of the number of cognitive states for each worm (dots) as well as averaged across all worms (line). When choosing α = 0.05 as a lower threshold for accepting the null hypothesis, we obtain Markovian cognitive state trajectories for any number of cognitive states in the range from three to 19, with a maximum average p-value for seven states. As such, we infer that all cognitive state trajectories with three to 19 states are dynamically causally consistent. As discussed below, we can then control the degree of granularity at which we study the dynamics of C. elegans by varying the number of cognitive states in this range. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. p-values for rejecting the null-hypothesis of Markovianity for each worm (and averaged across worms) as a function of the number of cognitive states. https://doi.org/10.1371/journal.pcbi.1012674.g003 To build a better intuition for the procedure of learning a cognitive state trajectory, Fig 4 illustrates the relevant steps on the data of the first worm with five cognitive states (all trajectories are projected to their first two principal components for visualization): The neuronal state trajectories together with the behavioral labels (A) are used to estimate the behavioral probability trajectories (B). These trajectories are then clustered to assign each element of the behavioral probability trajectories to a cognitive state (C). We remark that Fig 4B and 4C are projections from a 28- to a two-dimensional space via PCA, which does not adequately represent the spatial separation of neuronal states that belong to distinct cognitive clusters. Re-projecting the cognitive labels onto the neuronal state trajectories gives the neuro-cognitive state trajectories (D), from which we can compute the cognitive state transition model P(C[t]|C[t − 1]). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Illustration of the procedure of learning a cognitive state trajectory (see main text for details). https://doi.org/10.1371/journal.pcbi.1012674.g004 We remark that the cognitive dSCM of each animal is behaviorally observationally consistent by construction and dynamically causally consistent if the cognitive dSCM is Markovian (we emphasize again that Markovianity can not be verified; we can only fail to find evidence against Markovianity and thus not reject the null hypothesis). Because we built the cognitive dSCM on the observational distribution P(B|X) rather than on the interventional distribution P(B|do(X)), which is not available in the present setting, behavioral causal consistency is not guaranteed. In its present form, the cognitive dSCM thus only supports causal statements about the dynamics of the cognitive dSCM but not about its relation to behavior. However, its observational behavioral consistency can be used to derive causal hypotheses that guide the design of experimental studies to establish causal relations between neuronal states and behaviors via interventions, a topic we discuss below in more detail. Interpreting the cognitive model of C. elegans In this section, we demonstrate the ability of the learned cognitive models to reveal behavioral motifs of C. elegans, show that from a data-visualization perspective the NC-MCM framework can be understood as a neuronal manifold learning technique that abstracts the essential features of a manifold into a directed graph, and discuss how the cognitive model can provide insights into decision-making processes. Behavioral motifs of C. elegans. In the following, we juxtapose the behavioral state transition diagram with the state diagram that is obtained by expanding the behavioral- by the cognitive states, i.e., we compare P(B[t]|B[t − 1]) and P(C[t], B[t]|C[t − 1], B[t − 1]). In particular, we show that the behavioral state transition diagram conflates distinct behavioral motifs that are revealed when considering the behavior commands in relation to the cognitive states in which they appear. We first illustrate the results on the third worm in the data set because this worm exhibits the most simple state transitions of all the five worms. We then show that the behavioral motifs discussed below are present with small variations in every worm. Fig 5 shows the behavioral- (A, left column) and the cognitive-behavioral state diagram with three cognitive states (B, right column) of the third worm in the data set (we have chosen three cognitive states because this is the lowest number of cognitive states for which we do not find evidence against Markovianity; the behavioral motifs discussed below are also apparent when considering models with more cognitive states). In the left diagram, each node represents one behavioral state. The size of each node represents the probability of the worm being in the corresponding behavioral state at any point in time. The width of an outgoing edge represents the probability of the worm transitioning from a particular state to another state (to reduce spurious edges that result from jitters of the neuronal trajectories across the decision boundaries in the probability space, we only plot state transition if the length of a state exceeds two samples). Edges indicating the probabilities of staying in a particular behavioral state, i.e., edges from one node to itself, are not drawn to reduce clutter. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Behavioral- and cognitive-behavioral state diagrams of the third worm (see text for details). Arrows that account for less than 0.1% of outgoing transitions of each node have been removed to reduce clutter. https://doi.org/10.1371/journal.pcbi.1012674.g005 Fig 5A reveals a clear structure in the worm’s behavioral command (or states) sequences. The worm is most often found executing a sustained reversal state (revsus). A sustained reversal is either followed by a ventral- (vt) or dorsal (dt) turn state, both of which segue into a slowing state (slow). The slowing state alternates with a forward state (fwd) before returning to a sustained reversal state via either of two types of reversal states (rev1 or rev2). We have chosen observational language to describe this behavioral state transition diagram because the behavioral dynamics are not Markovian (the statistical test for Markovianity described earlier rejects the null hypothesis of Markovianity at α = 0.05 (p = 0.015)). This non-Markovianity indicates that past behavioral states provide information about the future state that is not contained in the current state, as was suggested by [37]. This fact becomes intuitively plausible when considering the cognitive-behavioral state transition diagram in Fig 5B. This diagram expands the behavioral by the cognitive states, i.e., we represent the eight behaviors separately for each cognitive state in which they occur (note that the cognitive states are drawn counter clockwise with the eight behaviors of each cognitive state also arranged in circles). This Markovian representation reveals two distinct behavioral motifs: A revsus → vt → slow → rev1 → revsus and a revsus → dt → fwd ↔ slow → rev2 → revsus loop. These two loops that occur in cognitive states C3 and C2, respectively, are connected via cognitive state transitions between C1 and C3 during a sustained reversal (which we discuss in more detail below). The first motif represents a repeated execution of a state sequence composed of sustained reversal followed by a ventral turn, slowing and reversal 1. The second motif is composed mostly of switching between slowing and forward crawling states. In the third worm, this motif always starts with a dorsal turn state and terminates with a reversal 2. These behavioral motifs are well known in C. elegans [38, 39]. The first motif corresponds to a pirouette, where animals execute multiple reverse-turning events in close succession. The second motif corresponds to a forward run episode, where animals persistently head in a forward direction. This motif is terminated by a rev2 state, corresponding to the neuronal signature that terminates a forward run episode (cf. [37]). Notably, the NC-MCM framework discovered these motifs without any prior knowledge of the underlying behavioral dynamics. Rather, the behavioral motifs are revealed by the NC-MCM framework because they are embedded in distinct neuronal dynamics. Importantly, this entails the ability of the NC-MCM framework to assign time frames that were manually annotated with the same behavioral label to distinct cognitive states if the executed behaviors differ in their neuronal realization. In Fig 5, this is apparent in two distinct slowing behaviors, one in each behavioral motif, that are supported by cognitive states C2 and C3, respectively. In C2, slowing could correspond to brief episodes of reducing locomotion speed while the animals remain in a forward run [37]. In C3, slowing could correspond to intermittent pausing or brief forward crawling during successive re-orientations. In the behavioral state diagram in the left column of Fig 5, these two different types of slowing movements are conflated. This conflation leads to a non-Markovian representation because the probabilities of entering a rev1 and rev2 state are higher when the slowing movement is preceded by a ventral and a dorsal turn, respectively. Conversely, the Markovian representation of the cognitive-behavioral state diagram ensures that it correctly distinguishes seemingly similar behaviors that are supported by distinct neuronal dynamics. While the third worm exhibits the most distinct separation of the two behavioral motifs, both motifs are present in every worm (Fig 6). Because the worms slightly vary in the complexity of their state transitions, with the other worms also showing infrequent state transitions between the two motifs, the number of cognitive states at which the two behavioral motifs first emerge varies across worms (four cognitive states for the first two worms and three cognitive states for the remaining three worms). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Behavioral- and cognitive-behavioral state diagrams of all worms. Arrows that account for less than 0.075% of outgoing transitions of each node have been removed to reduce clutter. https://doi.org/10.1371/journal.pcbi.1012674.g006 Representing C. elegans’ neuronal manifold as a directed graph. Individual neurons are embedded in brain networks that collectively organize their high-dimensional neuronal activity patterns into lower-dimensional neuronal manifolds [40, 41]. The goal of neuronal manifold learning techniques is to find low-dimensional representations of neuronal data that enable insights into the structure of neuronal dynamics and their relation to behavior [9]. In neuroscience, classic dimensionality reduction techniques, such as principal component analysis (PCA), Laplacian eigenmaps (LEM), and t-SNE, are complemented by modern techniques such as UMAP [10], MIND [11], CEBRA [12], and BundDLe-Net [13]. In the following, we show that the cognitive-behavioral state diagrams introduced in the previous section can be interpreted as a neuronal manifold learning technique that represents the essential aspects of the manifold as a directed graph. Fig 7 shows a side-by-side comparison of the neuronal manifold of the third worm as inferred by BundDLe-Net [13] (left column) and the cognitive-behavioral state diagram of the NC-MCM framework (right column). Looking at the left column, it is immediately apparent that the neuronal manifold exhibits two main cycles, a revsus (gray) → vt (turquoise) → slow (red) → rev1 (green) → revsus (gray) and a revsus (gray) → dt (orange) → slow (red) → fwd (yellow) → rev2 (blue) → revsus (gray) cycle, that correspond to the behavioral motifs revealed by the cognitive-behavioral state diagram in the right column of Fig 7 and discussed in the previous section. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 7. Side-by-side comparison of the neuronal manifold of the third worm as learned by BunDLe-Net and its representation as a directed graph in the NC-MCM framework. https://doi.org/10.1371/journal.pcbi.1012674.g007 To better understand the structure of the neuronal manifold, we now consider the branching and convergence points of the neuronal trajectories. We identify the dense cluster of neuronal states that correspond to a sustained reversal (revsus shown in gray at the bottom of the left plot in Fig 7) as the root node of the manifold that acts both as a branching and a convergence point. In particular, the neuronal state trajectories branch out from here into one of the two behavioral motifs via a ventral- (vt—turquoise) or a dorsal turn (dt—orange). Once a ventral turn is initiated, the neuronal trajectories are highly consistent; all trajectories execute a loop (accompanied by a ventral turn, slowing, and reversal 1) before converging again into the root node during a sustained reversal. If a dorsal turn is initiated, on the other hand, the neuronal trajectories enter a second branching point during the subsequent slowing movement (red), from which they either return directly during a rev2 behavior (blue) into the root node (gray) or take the longer route into the extended branch of the persistent alternation between slowing (red) and forward (yellow) behavior. Both of these branches converge again in the rev2 behavior before returning to the sustained reversal root node. To summarize, the dynamics on the neuronal manifold are determined by a small number of branching and convergence points with highly consistent trajectories between these points. In the cognitive-behavioral state diagram in the right column of Fig 7, the branching and convergence points of the neuronal manifold are represented by nodes with an out- and in-degree greater than one, respectively. The trajectories between the branching and convergence points are represented by nodes with an out- and in-degree of one. Based on this correspondence, it is easy to verify visually that the cognitive-behavioral state diagram constitutes a representation of the structure of the neuronal manifold as a directed graph in the sense that the nodes of the graph represent bundles of trajectories on the neuronal manifold and the edges between the nodes represent the possible paths that the neuronal trajectories can take on the manifold. This correspondence also holds for the other four worms shown in Figs 8–11 (three-dimensional rotating GIF visualizations of the embeddings are available in the BunDLe-Net repository at https://github.com/akshey-kumar/BunDLe-Net). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 8. Side-by-side comparison of the neuronal manifold and the cognitive-behavioral state diagram of worm 1. https://doi.org/10.1371/journal.pcbi.1012674.g008 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 9. Side-by-side comparison of the neuronal manifold and the cognitive-behavioral state diagram of worm 2. https://doi.org/10.1371/journal.pcbi.1012674.g009 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 10. Side-by-side comparison of the neuronal manifold and the cognitive-behavioral state diagram of worm 4. https://doi.org/10.1371/journal.pcbi.1012674.g010 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 11. Side-by-side comparison of the neuronal manifold and the cognitive-behavioral state diagram of worm 5. https://doi.org/10.1371/journal.pcbi.1012674.g011 As such, the cognitive state transition models of the NC-MCM abstract away variability of the the neuronal dynamics to reveal the essential structure of the neuronal manifold in a directed graph. In this way, the NC-MCM reduces the study of the structure of neuronal manifolds to a graph theoretic problem, a mature scientific field for which a plethora of computationally efficient algorithms are available [42]. Towards understanding decision making in C. elegans. As we have seen in the previous section, the neuronal trajectories alternate between segments with highly consistent dynamics, e.g., when executing the behavioral motif initiated by a ventral turn, and segments with high uncertainty about the future dynamics and behavior, e.g., when performing a sustained reversal movement. In the following, we show how the cognitive-behavioral state transition diagrams of the NC-MCM framework provide insights into the neuronal basis of future neuronal dynamics during segments with high uncertainty, i.e., we show how to use the NC-MCM framework to study decision-making in C. elegans. In particular, we study how a ventral vs. a dorsal turn is initiated during a sustained reversal and how the alternating forward and slowing behavior is terminated by a rev2-reversal. We begin again by considering the cognitive-behavioral state transition diagram of the third worm in Fig 7. In particular, we note that the sustained reversal movement (revsus) is distributed across the two cognitive states C1 and C3, with frequent, cyclical transitions between the two states. Importantly, the probability that the worm transitions from a sustained reversal to a particular turning behavior differs between these cognitive states. When the worm is in cognitive state C3, it transitions from a sustained reversal into a ventral- and a dorsal turn at frequencies of 3.04% and 0.72%, respectively. In cognitive state C, on the other hand, the frequencies of transitions from a sustained reversal into a ventral- and a dorsal turn are 0.17% and 0.84%, respectively. Ventral turns are thus roughly 18 times more frequently initiated in state C3 than in state C1 while dorsal turn initiations occur with similar frequencies in both cognitive states (they are 1.17 times more likely in C1 than in C3). We can thus interpret a cognitive state transition from C1 into C3 during a sustained reversal movement as a decision process that makes a ventral turn more likely than a dorsal turn and vice versa. We now turn to the question of how these cognitive state transitions are realized on the neuronal level. To do so, we analyze which global perturbation of neuronal states during a particular type of movement the model predicts to induce a cognitive state that increases the probability of another type of behavior. To illustrate this idea, consider again the two states C1:revsus and C3:revsus of the third worm. By first computing the mean activation of every neuron in each of the two states, i.e., the mean of all neuronal states that map to C1:revsus and the mean of all neuronal states that map to C3:revsus, and then subtracting the two means, we obtain the neuronal perturbation that the cognitive-behavioral model predicts to induce a state transition from C1:revsus to C3:revsus; a state transition that increases the probability of transitioning into a ventral turn from 0.17% to 3.04%. To generalize from this example, we can pick a source behavior, e.g., a sustained reversal, and a target behavior, e.g., a ventral turn, determine all cognitive states in which the source behavior occurs, for each pair of cognitive states compute the neuronal perturbation that the NC-MCM model predicts to induce a transition from one to the other, weigh this neuronal perturbation by the difference in probability in the target behavior between the two cognitive states, and finally average the neuronal perturbations across all possible state transitions and worms. This results in a neuronal perturbation vector, which we subsequently denote by , that the NC-MCM model predicts, on average across all cognitive states and worms, to induce a state transition that increases the probability of the target behavior while maintaining the current source behavior. This procedure necessitates that the cognitive-behavioral state diagram of each worm represents the source behavior in multiple cognitive states. As can be checked in Figs 7–11, out of the cognitive-behavioral state diagrams with three cognitive states only the state diagram of the third worm represents the uncertainty during the sustained reversal movement by more than one cognitive state. This limitation can be easily remedied by considering more than three cognitive states. In the following, we consider cognitive-behavioral state diagrams with seven cognitive states, because we found seven to be the optimal number of cognitive states with respect to (the absence of evidence against) Markovianity across all worms, cf. Fig 3 (we remark that among all Markovian cognitive models there is no correct or incorrect number of cognitive states—we can vary the number of cognitive states and thus choose the level of granularity at which we analyze the cognitive dynamics). Fig 12 displays the corresponding cognitive-behavioral state diagram for the third worm. Increasing the number of cognitive states from three to seven preserves the overall structure as well as the two primary behavioral motifs while splitting up the cognitive states into a more fine-grained representation of the neuronal dynamics and their relation to behavior. In particular, the pirouette motif that is initiated by a ventral turn and that was represented by cognitive state C3 in Fig 7 is represented in Fig 12 by the two cognitive states C1 and C2. The forward-run motif which is initiated by a dorsal turn and that was represented in Fig 7 by C2 is represented in Fig 12 by the two cognitive states C4 and C5. The uncertainty in the transitions between these two motifs, which was characterized in Fig 7 by the cyclical alterations between cognitive states C1 and C3 during a sustained reversal movement, is represented in the seven-states model by transitions between the cognitive states C2, C6, and C7. Moving to cognitive-behavioral state diagrams with seven cognitive states in each worm, we can thus compute the neuronal perturbation vector between a source and a target behavior across a broad range of cognitive states across all worms. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 12. Cognitive-behavioral state transition diagram of the third worm with seven cognitive states. https://doi.org/10.1371/journal.pcbi.1012674.g012 Fig 13 displays the neuronal perturbation vectors , , and for all neurons that are shared across the recordings of all five worms, i.e., the three neuronal perturbation vectors that during a sustained reversal render the maintenance of the sustained reversal, a transition to a ventral turn, and a transition to a dorsal turn more likely. Double and single markers indicate the rejection of the null hypothesis of equal means across conditions at significance levels α = 0.01 and α = 0.05, respectively. Plus signs and asterisks indicate significance tests with and without Bonferroni correction, respectively. Statistical tests were carried out with an ANOVA based on 10.000 random permutations with Bonferroni correction for multiple comparisons according to the number of neurons. Not considering those neurons used for labeling the behaviors (AVAL, AVAR, SMDDR, SMDDL, SMDVR, SMDVL, RIBR, and RIBL), the neuronal perturbation vector exhibits highly statistically significant differences across specific sets of descending interneurons. In particular, the NC-MCM model predicts that increasing the activation of AVBL and AVBR while reducing the activity of AVEL and AVER increases the probability of a turning behavior, with weaker and stronger changes linked to dorsal- and ventral turns, respectively. Conversely, applying the opposite pattern makes it more likely that worms maintain a sustained reversal. We remark that RIVL and RIVR may play a further role in differentiating between dorsal and ventral turns. Because these two neurons show no significant differences after Bonferroni correction for multiple comparisons, we refrain from further interpreting their effects. These neuronal perturbation vectors for maintaining or terminating a sustained reversal are markedly different from those obtained for changes between cognitive states that either maintain a slowing-forward motif or interrupt this motif through a rev2-type reversal, as shown in Fig 14. Here, an increase in activation of AIBL, AIBR, ASKR, RIVL, and RIVR and a decrease in activation of RMED, RMEL, and RMER are predicted to maintain a slowing behavior, with the converse pattern inducing a cognitive state that interrupts the slowing-forward motif by a rev2-type reversal. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 13. Neuronal perturbations that, when applied during a sustained reversal, the NC-MCM framework predicts to induce a cognitive state transition that increases the probability of maintaining a sustained reversal movement (blue), initiating a ventral turn (orange), or initiating a dorsal turn (green). Double and single markers indicate the rejection of the null hypothesis of equal means across conditions at significance levels α = 0.01 and α = 0.05, respectively. Plus signs and asterisks indicate significance tests with and without Bonferroni correction, respectively. https://doi.org/10.1371/journal.pcbi.1012674.g013 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 14. Neuronal perturbations that, when applied during a slowing movement, the NC-MCM framework predicts to induce a cognitive state transition that increases the probability of maintaining a slowing movement (blue), initiating a forward movement (orange), or initiating a rev2 reversal (green). Double and single markers indicate the rejection of the null hypothesis of equal means across conditions at significance levels α = 0.01 and α = 0.05, respectively. Plus signs and asterisks indicate significance tests with and without Bonferroni correction, respectively. https://doi.org/10.1371/journal.pcbi.1012674.g014 Due to the DCC property of the cognitive models discussed above, we may interpret the neuronal perturbation vectors as empirically testable predictions which perturbations of neurons via external stimulation induce cognitive states that render certain behaviors more or less likely (because the learned cognitive models are only observationally but not causally behaviorally consistent, we may not apply the same causal reasoning to the relations between cognitive state transitions and behaviors). However, it remains an open question how these perturbations come about naturally, i.e., how the worm decides which behavioral motif to initiate and when. A priori, there are two potential explanations. In the first explanation, the worm’s neuronal dynamics could be deterministic with the apparent randomness during sustained reversals and the slowing-forward motifs being due to some unobserved (latent) neurons. In this interpretation, observing all behaviorally relevant neurons would eliminate the randomness and lead to non-overlapping trajectory bundles in Figs 7–11. In the second interpretation, the indeterminacy may be due to inherent randomness in neuronal activation. Due to the DCC property, our results are consistent with the second but not with the first interpretation: A latent neuron that affects at least one observed neuron would induce a dependence between the current- and past states of the observed neuron, rendering the neuronal dynamics non-Markovian. Because we found no evidence against Markovianity in the cognitive state transitions, which constitute an abstraction of all behaviorally relevant information in the neuronal dynamics, our results are consistent with the interpretation that decision-making in C. elegans has an intrinsically random component, as was suggested for switches between forward and backward movement [43]. However, we remark that in the present setting the worms have been immobilized and received no time-varying sensory input. In more ecologically realistic settings, sensory stimuli may override inherent randomness. Behavioral motifs of C. elegans. In the following, we juxtapose the behavioral state transition diagram with the state diagram that is obtained by expanding the behavioral- by the cognitive states, i.e., we compare P(B[t]|B[t − 1]) and P(C[t], B[t]|C[t − 1], B[t − 1]). In particular, we show that the behavioral state transition diagram conflates distinct behavioral motifs that are revealed when considering the behavior commands in relation to the cognitive states in which they appear. We first illustrate the results on the third worm in the data set because this worm exhibits the most simple state transitions of all the five worms. We then show that the behavioral motifs discussed below are present with small variations in every worm. Fig 5 shows the behavioral- (A, left column) and the cognitive-behavioral state diagram with three cognitive states (B, right column) of the third worm in the data set (we have chosen three cognitive states because this is the lowest number of cognitive states for which we do not find evidence against Markovianity; the behavioral motifs discussed below are also apparent when considering models with more cognitive states). In the left diagram, each node represents one behavioral state. The size of each node represents the probability of the worm being in the corresponding behavioral state at any point in time. The width of an outgoing edge represents the probability of the worm transitioning from a particular state to another state (to reduce spurious edges that result from jitters of the neuronal trajectories across the decision boundaries in the probability space, we only plot state transition if the length of a state exceeds two samples). Edges indicating the probabilities of staying in a particular behavioral state, i.e., edges from one node to itself, are not drawn to reduce clutter. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Behavioral- and cognitive-behavioral state diagrams of the third worm (see text for details). Arrows that account for less than 0.1% of outgoing transitions of each node have been removed to reduce clutter. https://doi.org/10.1371/journal.pcbi.1012674.g005 Fig 5A reveals a clear structure in the worm’s behavioral command (or states) sequences. The worm is most often found executing a sustained reversal state (revsus). A sustained reversal is either followed by a ventral- (vt) or dorsal (dt) turn state, both of which segue into a slowing state (slow). The slowing state alternates with a forward state (fwd) before returning to a sustained reversal state via either of two types of reversal states (rev1 or rev2). We have chosen observational language to describe this behavioral state transition diagram because the behavioral dynamics are not Markovian (the statistical test for Markovianity described earlier rejects the null hypothesis of Markovianity at α = 0.05 (p = 0.015)). This non-Markovianity indicates that past behavioral states provide information about the future state that is not contained in the current state, as was suggested by [37]. This fact becomes intuitively plausible when considering the cognitive-behavioral state transition diagram in Fig 5B. This diagram expands the behavioral by the cognitive states, i.e., we represent the eight behaviors separately for each cognitive state in which they occur (note that the cognitive states are drawn counter clockwise with the eight behaviors of each cognitive state also arranged in circles). This Markovian representation reveals two distinct behavioral motifs: A revsus → vt → slow → rev1 → revsus and a revsus → dt → fwd ↔ slow → rev2 → revsus loop. These two loops that occur in cognitive states C3 and C2, respectively, are connected via cognitive state transitions between C1 and C3 during a sustained reversal (which we discuss in more detail below). The first motif represents a repeated execution of a state sequence composed of sustained reversal followed by a ventral turn, slowing and reversal 1. The second motif is composed mostly of switching between slowing and forward crawling states. In the third worm, this motif always starts with a dorsal turn state and terminates with a reversal 2. These behavioral motifs are well known in C. elegans [38, 39]. The first motif corresponds to a pirouette, where animals execute multiple reverse-turning events in close succession. The second motif corresponds to a forward run episode, where animals persistently head in a forward direction. This motif is terminated by a rev2 state, corresponding to the neuronal signature that terminates a forward run episode (cf. [37]). Notably, the NC-MCM framework discovered these motifs without any prior knowledge of the underlying behavioral dynamics. Rather, the behavioral motifs are revealed by the NC-MCM framework because they are embedded in distinct neuronal dynamics. Importantly, this entails the ability of the NC-MCM framework to assign time frames that were manually annotated with the same behavioral label to distinct cognitive states if the executed behaviors differ in their neuronal realization. In Fig 5, this is apparent in two distinct slowing behaviors, one in each behavioral motif, that are supported by cognitive states C2 and C3, respectively. In C2, slowing could correspond to brief episodes of reducing locomotion speed while the animals remain in a forward run [37]. In C3, slowing could correspond to intermittent pausing or brief forward crawling during successive re-orientations. In the behavioral state diagram in the left column of Fig 5, these two different types of slowing movements are conflated. This conflation leads to a non-Markovian representation because the probabilities of entering a rev1 and rev2 state are higher when the slowing movement is preceded by a ventral and a dorsal turn, respectively. Conversely, the Markovian representation of the cognitive-behavioral state diagram ensures that it correctly distinguishes seemingly similar behaviors that are supported by distinct neuronal dynamics. While the third worm exhibits the most distinct separation of the two behavioral motifs, both motifs are present in every worm (Fig 6). Because the worms slightly vary in the complexity of their state transitions, with the other worms also showing infrequent state transitions between the two motifs, the number of cognitive states at which the two behavioral motifs first emerge varies across worms (four cognitive states for the first two worms and three cognitive states for the remaining three worms). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Behavioral- and cognitive-behavioral state diagrams of all worms. Arrows that account for less than 0.075% of outgoing transitions of each node have been removed to reduce clutter. https://doi.org/10.1371/journal.pcbi.1012674.g006 Representing C. elegans’ neuronal manifold as a directed graph. Individual neurons are embedded in brain networks that collectively organize their high-dimensional neuronal activity patterns into lower-dimensional neuronal manifolds [40, 41]. The goal of neuronal manifold learning techniques is to find low-dimensional representations of neuronal data that enable insights into the structure of neuronal dynamics and their relation to behavior [9]. In neuroscience, classic dimensionality reduction techniques, such as principal component analysis (PCA), Laplacian eigenmaps (LEM), and t-SNE, are complemented by modern techniques such as UMAP [10], MIND [11], CEBRA [12], and BundDLe-Net [13]. In the following, we show that the cognitive-behavioral state diagrams introduced in the previous section can be interpreted as a neuronal manifold learning technique that represents the essential aspects of the manifold as a directed graph. Fig 7 shows a side-by-side comparison of the neuronal manifold of the third worm as inferred by BundDLe-Net [13] (left column) and the cognitive-behavioral state diagram of the NC-MCM framework (right column). Looking at the left column, it is immediately apparent that the neuronal manifold exhibits two main cycles, a revsus (gray) → vt (turquoise) → slow (red) → rev1 (green) → revsus (gray) and a revsus (gray) → dt (orange) → slow (red) → fwd (yellow) → rev2 (blue) → revsus (gray) cycle, that correspond to the behavioral motifs revealed by the cognitive-behavioral state diagram in the right column of Fig 7 and discussed in the previous section. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 7. Side-by-side comparison of the neuronal manifold of the third worm as learned by BunDLe-Net and its representation as a directed graph in the NC-MCM framework. https://doi.org/10.1371/journal.pcbi.1012674.g007 To better understand the structure of the neuronal manifold, we now consider the branching and convergence points of the neuronal trajectories. We identify the dense cluster of neuronal states that correspond to a sustained reversal (revsus shown in gray at the bottom of the left plot in Fig 7) as the root node of the manifold that acts both as a branching and a convergence point. In particular, the neuronal state trajectories branch out from here into one of the two behavioral motifs via a ventral- (vt—turquoise) or a dorsal turn (dt—orange). Once a ventral turn is initiated, the neuronal trajectories are highly consistent; all trajectories execute a loop (accompanied by a ventral turn, slowing, and reversal 1) before converging again into the root node during a sustained reversal. If a dorsal turn is initiated, on the other hand, the neuronal trajectories enter a second branching point during the subsequent slowing movement (red), from which they either return directly during a rev2 behavior (blue) into the root node (gray) or take the longer route into the extended branch of the persistent alternation between slowing (red) and forward (yellow) behavior. Both of these branches converge again in the rev2 behavior before returning to the sustained reversal root node. To summarize, the dynamics on the neuronal manifold are determined by a small number of branching and convergence points with highly consistent trajectories between these points. In the cognitive-behavioral state diagram in the right column of Fig 7, the branching and convergence points of the neuronal manifold are represented by nodes with an out- and in-degree greater than one, respectively. The trajectories between the branching and convergence points are represented by nodes with an out- and in-degree of one. Based on this correspondence, it is easy to verify visually that the cognitive-behavioral state diagram constitutes a representation of the structure of the neuronal manifold as a directed graph in the sense that the nodes of the graph represent bundles of trajectories on the neuronal manifold and the edges between the nodes represent the possible paths that the neuronal trajectories can take on the manifold. This correspondence also holds for the other four worms shown in Figs 8–11 (three-dimensional rotating GIF visualizations of the embeddings are available in the BunDLe-Net repository at https://github.com/akshey-kumar/BunDLe-Net). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 8. Side-by-side comparison of the neuronal manifold and the cognitive-behavioral state diagram of worm 1. https://doi.org/10.1371/journal.pcbi.1012674.g008 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 9. Side-by-side comparison of the neuronal manifold and the cognitive-behavioral state diagram of worm 2. https://doi.org/10.1371/journal.pcbi.1012674.g009 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 10. Side-by-side comparison of the neuronal manifold and the cognitive-behavioral state diagram of worm 4. https://doi.org/10.1371/journal.pcbi.1012674.g010 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 11. Side-by-side comparison of the neuronal manifold and the cognitive-behavioral state diagram of worm 5. https://doi.org/10.1371/journal.pcbi.1012674.g011 As such, the cognitive state transition models of the NC-MCM abstract away variability of the the neuronal dynamics to reveal the essential structure of the neuronal manifold in a directed graph. In this way, the NC-MCM reduces the study of the structure of neuronal manifolds to a graph theoretic problem, a mature scientific field for which a plethora of computationally efficient algorithms are available [42]. Towards understanding decision making in C. elegans. As we have seen in the previous section, the neuronal trajectories alternate between segments with highly consistent dynamics, e.g., when executing the behavioral motif initiated by a ventral turn, and segments with high uncertainty about the future dynamics and behavior, e.g., when performing a sustained reversal movement. In the following, we show how the cognitive-behavioral state transition diagrams of the NC-MCM framework provide insights into the neuronal basis of future neuronal dynamics during segments with high uncertainty, i.e., we show how to use the NC-MCM framework to study decision-making in C. elegans. In particular, we study how a ventral vs. a dorsal turn is initiated during a sustained reversal and how the alternating forward and slowing behavior is terminated by a rev2-reversal. We begin again by considering the cognitive-behavioral state transition diagram of the third worm in Fig 7. In particular, we note that the sustained reversal movement (revsus) is distributed across the two cognitive states C1 and C3, with frequent, cyclical transitions between the two states. Importantly, the probability that the worm transitions from a sustained reversal to a particular turning behavior differs between these cognitive states. When the worm is in cognitive state C3, it transitions from a sustained reversal into a ventral- and a dorsal turn at frequencies of 3.04% and 0.72%, respectively. In cognitive state C, on the other hand, the frequencies of transitions from a sustained reversal into a ventral- and a dorsal turn are 0.17% and 0.84%, respectively. Ventral turns are thus roughly 18 times more frequently initiated in state C3 than in state C1 while dorsal turn initiations occur with similar frequencies in both cognitive states (they are 1.17 times more likely in C1 than in C3). We can thus interpret a cognitive state transition from C1 into C3 during a sustained reversal movement as a decision process that makes a ventral turn more likely than a dorsal turn and vice versa. We now turn to the question of how these cognitive state transitions are realized on the neuronal level. To do so, we analyze which global perturbation of neuronal states during a particular type of movement the model predicts to induce a cognitive state that increases the probability of another type of behavior. To illustrate this idea, consider again the two states C1:revsus and C3:revsus of the third worm. By first computing the mean activation of every neuron in each of the two states, i.e., the mean of all neuronal states that map to C1:revsus and the mean of all neuronal states that map to C3:revsus, and then subtracting the two means, we obtain the neuronal perturbation that the cognitive-behavioral model predicts to induce a state transition from C1:revsus to C3:revsus; a state transition that increases the probability of transitioning into a ventral turn from 0.17% to 3.04%. To generalize from this example, we can pick a source behavior, e.g., a sustained reversal, and a target behavior, e.g., a ventral turn, determine all cognitive states in which the source behavior occurs, for each pair of cognitive states compute the neuronal perturbation that the NC-MCM model predicts to induce a transition from one to the other, weigh this neuronal perturbation by the difference in probability in the target behavior between the two cognitive states, and finally average the neuronal perturbations across all possible state transitions and worms. This results in a neuronal perturbation vector, which we subsequently denote by , that the NC-MCM model predicts, on average across all cognitive states and worms, to induce a state transition that increases the probability of the target behavior while maintaining the current source behavior. This procedure necessitates that the cognitive-behavioral state diagram of each worm represents the source behavior in multiple cognitive states. As can be checked in Figs 7–11, out of the cognitive-behavioral state diagrams with three cognitive states only the state diagram of the third worm represents the uncertainty during the sustained reversal movement by more than one cognitive state. This limitation can be easily remedied by considering more than three cognitive states. In the following, we consider cognitive-behavioral state diagrams with seven cognitive states, because we found seven to be the optimal number of cognitive states with respect to (the absence of evidence against) Markovianity across all worms, cf. Fig 3 (we remark that among all Markovian cognitive models there is no correct or incorrect number of cognitive states—we can vary the number of cognitive states and thus choose the level of granularity at which we analyze the cognitive dynamics). Fig 12 displays the corresponding cognitive-behavioral state diagram for the third worm. Increasing the number of cognitive states from three to seven preserves the overall structure as well as the two primary behavioral motifs while splitting up the cognitive states into a more fine-grained representation of the neuronal dynamics and their relation to behavior. In particular, the pirouette motif that is initiated by a ventral turn and that was represented by cognitive state C3 in Fig 7 is represented in Fig 12 by the two cognitive states C1 and C2. The forward-run motif which is initiated by a dorsal turn and that was represented in Fig 7 by C2 is represented in Fig 12 by the two cognitive states C4 and C5. The uncertainty in the transitions between these two motifs, which was characterized in Fig 7 by the cyclical alterations between cognitive states C1 and C3 during a sustained reversal movement, is represented in the seven-states model by transitions between the cognitive states C2, C6, and C7. Moving to cognitive-behavioral state diagrams with seven cognitive states in each worm, we can thus compute the neuronal perturbation vector between a source and a target behavior across a broad range of cognitive states across all worms. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 12. Cognitive-behavioral state transition diagram of the third worm with seven cognitive states. https://doi.org/10.1371/journal.pcbi.1012674.g012 Fig 13 displays the neuronal perturbation vectors , , and for all neurons that are shared across the recordings of all five worms, i.e., the three neuronal perturbation vectors that during a sustained reversal render the maintenance of the sustained reversal, a transition to a ventral turn, and a transition to a dorsal turn more likely. Double and single markers indicate the rejection of the null hypothesis of equal means across conditions at significance levels α = 0.01 and α = 0.05, respectively. Plus signs and asterisks indicate significance tests with and without Bonferroni correction, respectively. Statistical tests were carried out with an ANOVA based on 10.000 random permutations with Bonferroni correction for multiple comparisons according to the number of neurons. Not considering those neurons used for labeling the behaviors (AVAL, AVAR, SMDDR, SMDDL, SMDVR, SMDVL, RIBR, and RIBL), the neuronal perturbation vector exhibits highly statistically significant differences across specific sets of descending interneurons. In particular, the NC-MCM model predicts that increasing the activation of AVBL and AVBR while reducing the activity of AVEL and AVER increases the probability of a turning behavior, with weaker and stronger changes linked to dorsal- and ventral turns, respectively. Conversely, applying the opposite pattern makes it more likely that worms maintain a sustained reversal. We remark that RIVL and RIVR may play a further role in differentiating between dorsal and ventral turns. Because these two neurons show no significant differences after Bonferroni correction for multiple comparisons, we refrain from further interpreting their effects. These neuronal perturbation vectors for maintaining or terminating a sustained reversal are markedly different from those obtained for changes between cognitive states that either maintain a slowing-forward motif or interrupt this motif through a rev2-type reversal, as shown in Fig 14. Here, an increase in activation of AIBL, AIBR, ASKR, RIVL, and RIVR and a decrease in activation of RMED, RMEL, and RMER are predicted to maintain a slowing behavior, with the converse pattern inducing a cognitive state that interrupts the slowing-forward motif by a rev2-type reversal. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 13. Neuronal perturbations that, when applied during a sustained reversal, the NC-MCM framework predicts to induce a cognitive state transition that increases the probability of maintaining a sustained reversal movement (blue), initiating a ventral turn (orange), or initiating a dorsal turn (green). Double and single markers indicate the rejection of the null hypothesis of equal means across conditions at significance levels α = 0.01 and α = 0.05, respectively. Plus signs and asterisks indicate significance tests with and without Bonferroni correction, respectively. https://doi.org/10.1371/journal.pcbi.1012674.g013 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 14. Neuronal perturbations that, when applied during a slowing movement, the NC-MCM framework predicts to induce a cognitive state transition that increases the probability of maintaining a slowing movement (blue), initiating a forward movement (orange), or initiating a rev2 reversal (green). Double and single markers indicate the rejection of the null hypothesis of equal means across conditions at significance levels α = 0.01 and α = 0.05, respectively. Plus signs and asterisks indicate significance tests with and without Bonferroni correction, respectively. https://doi.org/10.1371/journal.pcbi.1012674.g014 Due to the DCC property of the cognitive models discussed above, we may interpret the neuronal perturbation vectors as empirically testable predictions which perturbations of neurons via external stimulation induce cognitive states that render certain behaviors more or less likely (because the learned cognitive models are only observationally but not causally behaviorally consistent, we may not apply the same causal reasoning to the relations between cognitive state transitions and behaviors). However, it remains an open question how these perturbations come about naturally, i.e., how the worm decides which behavioral motif to initiate and when. A priori, there are two potential explanations. In the first explanation, the worm’s neuronal dynamics could be deterministic with the apparent randomness during sustained reversals and the slowing-forward motifs being due to some unobserved (latent) neurons. In this interpretation, observing all behaviorally relevant neurons would eliminate the randomness and lead to non-overlapping trajectory bundles in Figs 7–11. In the second interpretation, the indeterminacy may be due to inherent randomness in neuronal activation. Due to the DCC property, our results are consistent with the second but not with the first interpretation: A latent neuron that affects at least one observed neuron would induce a dependence between the current- and past states of the observed neuron, rendering the neuronal dynamics non-Markovian. Because we found no evidence against Markovianity in the cognitive state transitions, which constitute an abstraction of all behaviorally relevant information in the neuronal dynamics, our results are consistent with the interpretation that decision-making in C. elegans has an intrinsically random component, as was suggested for switches between forward and backward movement [43]. However, we remark that in the present setting the worms have been immobilized and received no time-varying sensory input. In more ecologically realistic settings, sensory stimuli may override inherent randomness. Discussion of the experimental results in C. elegans The NC-MCM framework, applied to brain wide Ca2+-imaging data of C. elegans, revealed cognitive states that correspond to neuronal representations of behavioral hierarchies that sit on top of a previously described motor hierarchy [44]. A pirouette is a long time scale behavioral motif that encompasses the repetitive execution of reversal-turn command sequences, which can be triggered spontaneously or by sensory inputs [38, 45]. In our data, a pirouette manifests as a motif of switching dynamics between neuronal activity patterns representing these actions; notably the NC-MCM framework was able to find these patterns in the absence of neuronal activities that specifically mark the initiation, duration or termination of a pirouette, i.e., which we would have interpreted as explicit upper-hierarchy neuronal representations of a pirouette. We cannot exclude that our recording technique is insensitive to such activation patterns. But alternatively, and in analogy to the pressure of a gas (see Introduction), a pirouette could be seen as a useful causally consistent description of a higher-level navigational strategy of C. elegans [38, 45] that emerges from the ‘microscopic’ dynamical switching behavior of neuronal circuits without any explicit neuronal representation. This has implication on potential control mechanisms, which, thereby could act directly at the level of synapses and neuronal excitability within these circuits, and without the need of upper-hierarchy command-circuits. In the same vein, the learned NC-MCM suggests that forward run is a dynamical motif that encompasses continuous forward crawling with intermittent slowing events. It further suggests that slowing during forward runs is neuronally distinct from slowing during pirouettes, which were, however, lumped together into one state in our previous work [37]. When allowing for a finer granularity of many cognitive states, the NC-MCM framework can be a useful discovery tool to identify circuit elements that are potentially implicated in decision making. How C. elegans decides between a dorsal or ventral turn following a reversal is not known [37]. The learned NC-MCM suggests circuit elements that terminate sustained reversals and influence the subsequent binary choice via expected motor neuron classes (RIV, SMDV or RMED) [46], but also unexpected neuronal cell types like the descending interneuron class AVE or the neuromodulatory neuron class ALA. Likewise, the NC-MCM makes concrete suggestion for the circuitry that controls slowing, like the AIB neurons that show Ca2+-transients during discrete slowing events in freely moving worms [37], or RIB neurons that have been implicated in the control of locomotion speed [37, 44, 47]. Interestingly, ASK sensory neurons, which exhibit spontaneous Ca2+-fluctuations under the experimental conditions used here [48, 49] are further suggested to control slowing. The NC-MCM framework, thereby, can effectively guide future interrogation strategies, such as optogenetics, to confirm the role of these classes as decison making neurons. Discussion The framework of NC-MCM provides a formal and mathematically rigorous framework to bridge the explanatory gap between neuronal activity and cognition. This bridging is achieved by construing cognitive states as abstractions of neuronal states that are causally consistent with respect to a set of behavioral patterns (behavioral causal consistency) and with respect to the system’s dynamics (dynamic causal consistency). As such, a NC-MCM enables us to causally reason about a system’s dynamics and behavior on the cognitive level while grounding all causal statements in the system’s neuronal states. When learning a NC-MCM from observational data, dynamic causal consistency is achieved by constructing a Markovian representation of the observed dynamics. Markovianity is essential here because it guarantees a causally sufficient description of the dynamics of a physical system: Consider a simple pendulum whose complete physical state at any given point in time is given by its position and momentum. As such, setting the pendulum’s position and momentum by an external intervention fully determine its future behavior. Based on observational data only, we can check that the position and momentum provide a full characterization of the pendulum’s state by testing for Markovianity. This would reveal that the past positions and momenta do not provide additional information on the pendulum’s future states if the current position and momentum are known. Conversely, only observing the position of the pendulum does not allow us to determine its future state, e.g., because it could be swinging forward or backward. In other words, the current position is not a Markovian representation of the pendulum’s dynamics and thus does not provide a causally sufficient representation of the physical system (we remark that we can attempt to reconstruct the full state space by considering the pendulum’s past positions from which we infer its momentum). Returning to the NC-MCM framework, a Markovian cognitive-level dSCM guarantees that we have found a complete, causally sufficient characterization of the neuronal dynamics that are relevant for a given behavioral context. However, there are two caveats. Firstly, we require the assumption of causal sufficiency (or, equivalently, Markovianity) of the neuronal dynamics relevant to the observed behavior in order to learn a Markovian cognitive-level dSCM. While we consider neuronal recordings that provide a full Markovian state description of a biological model organism exceeding the current state of the art in neuronal recording techniques, we emphasize that we only require the subset of observed neuronal dynamics that control the observed behaviors to form a Markov process. This is a much weaker assumption, as it allows us to adjust the complexity of the behavioral labels to find a level of granularity at which the observed neuronal dynamics that govern the chosen behaviors are Markovian. Further, more advanced algorithms can be designed that address the problem of causal sufficiency by reconstructing a diffeomorphism of the complete state of a neuronal dynamical system based on Taken’s Time-Delay Embedding Theorem [13]. Secondly, we reiterate that concluding that a cognitive-level dSCM is dynamically causally consistent is based on the absence of sufficient evidence to reject the null hypothesis of Markovianity. Since absence of evidence is not the same as evidence of absence, this conclusion effectively operates under the null hypothesis until additional (potentially interventional) data can falsify it. Therefore, a NC-MCM that has been learned from purely observational data should be regarded as expressing experimentally testable predictions of dynamically and behaviorally causally consistent abstractions, rather than as direct proof thereof. In contrast to other frameworks that model cognition and its relation to neuronal dynamics, e.g., ACT-R [50, 51] and atlases of cognition [52, 53], a cognitive model in the NC-MCM framework does not incorporate prior assumptions on the structure of cognition, e.g., by postulating information-processing modules (cf. [54]) or by assuming a cognitive ontology [53]. Rather, a cognitive model in the NC-MCM framework is learned bottom-up from the system’s neuronal dynamics in combination with a behavioral context. The NC-MCM framework has been motivated by our earlier work on causal relations between neuronal states and behaviors [15–17], which led us to recognize that causal frameworks are suitable for describing relations between neuronal states and their effect on behavior but are not appropriate to model cognitive processes. Our work on causally consistent transformations then provided the theoretical foundation to model cognitive states as causally consistent abstractions of neuronal states [21]. However, the work in [21] was purely theoretical, in so far that it formalized the notion of causally consistent transformations but did not demonstrate its power on a particular problem in another scientific field. As such, one of the primary keystones of the present work was the realization that the framework of causally consistent transformations can be adapted to the domain of cognitive neuroscience by learning cognitive states as consistent abstractions of causal models on the level of neuronal dynamics and behaviors. In the following, we discuss in which sense such a model provides an understanding of the neuronal system it models. On a practical level, the NC-MCM framework can be interpreted as a data compression and visualization method. By eliminating redundant information in the neuronal states with respect to the behavioral context, irrelevant information is discarded and meaningful relationships emerge in an intuitively interpretable fashion, e.g., by representing the branching and convergence points of a neuronal manifold as a directed graph. This reduction of the neuronal manifold to a directed graph opens up the rich field of graph algorithms for the study of neuronal dynamics [42]. In this view, the NC-MCM framework is a tool for cognitive neuroscientists to analyze complex neuronal data. On a more fundamental level, we first note that the term understanding is not well defined and as such subjective, e.g., what qualifies as an explanation that leads to understanding by one person may be considered an insufficient explanation by another person. We adopt the viewpoint of [55] that the deepest level of understanding is that of identities, e.g., if we learn that the morning star and the evening star are in fact the same star, i.e., Venus, it does not make sense anymore to ask why the morning star is the evening star. In analogy, the NC-MCM framework, first, describes an equivalence relation between neuronal states with respect to a set of behaviors and, second, establishes identities between the neuronal equivalence classes and cognitive states. This raises the question what a cognitive state is or what it should be. We have defined a cognitive state as a causally meaningful abstraction of a neuronal state, implying that a cognitive model captures all relevant causal relations between neuronal states and behaviors (behavioral causal consistency) and enables us to causally reason about its dynamics (dynamic causal consistency). We argue that this definition of a cognitive state is intuitively plausible because it is in line with how we use cognitive states in everyday life. Faced with the daunting task of understanding the complex neuronal systems that support our own as well as other people’s behavior, we have developed a (sometimes more and sometimes less adequate) set of concepts which we employ to reason about our own and other people’s mental processes in a given behavioral context. Importantly, the extent to which this definition can reflect internal cognitive processes depends on how the behavioral context is operationalized. If behaviors are operationalized such that their probabilities are affected by (postulated) internal cognitive processes, e.g., by memory retention in a delayed match to sample task, these internal states would have to be represented by the NC-MCM framework to obtain a dynamically and behaviorally causally consistent abstraction. In this context, we consider it important to point out that when studying the behavior of a biological system there are two biological systems that interact: The observer of the system, e.g., the scientist, and the system that is being studied, e.g., the worm. By describing the behavior of the worm, the scientist imposes their own lens on the worm’s behavior. Because in the NC-MCM framework different behavioral representations result in different cognitive states, the cognitive processes of the system under study are partially determined by the cognitive processes of the scientist through their selection of the behavioral labels. As such, the NC-MCM framework provides a rigorous framework to search for internal cognitive states through careful construction of behavioral labels that are hypothesized to be affected by postulated cognitive processes. On a more abstract philosophical level, we remark that the NC-MCM framework offers a potential resolution to the problems of causal overdetermination and downward causation in the philosophy of mind [56]. By showing how a neuronal- and a cognitive level model can be constructed that are causally consistent, i.e., that allow us to interchangeably argue about a system’s dynamics and behavior on both levels, we hope that the NC-MCM framework will stimulate discussions on how to reconcile dualistic with physicalistic accounts of mental causation [57, 58]. Returning to more practically relevant interpretations of the NC-MCM framework, we note that (in contrast to hand-crafted mechanistic models often employed in computational neuroscience) the NC-MCM framework does not require mechanistic models of the neuronal-level dynamics to learn (observationally) behaviorally and dynamically causally consistent cognitive-level models. While a mechanistic model may be useful to derive the interventional distribution P(B[t]|do(X[t])) from which a causally behaviorally consistent model can be constructed, machine learning models in combination with a number of experimental interventions on the order of the number of cognitive states are in principle sufficient for learning P(B[t]|do(X[t])). Being able to leverage state-of-the-art machine learning algorithms for modeling the relations between neuronal activity patterns and behavior is particularly appealing when attempting to scale up NC-MCMs from small systems such as C. elegans, where hand-crafted mechanistic models may be feasible, to more complex organisms consisting of potentially hundred thousands or even millions of neurons. While detailed mechanistic models may continue to provide important insights into the computations of individual circuits, we conjecture that causally consistent NC-MCMs will be more useful in the domain of cognitive neural engineering, where we would like to know how to experimentally intervene on the neuronal level of large-scale organisms to treat cognitive disorders. To conclude the article, we discuss some extensions of NC-MCMs for future work. First, we recall that we have chosen a rather simple machine learning pipeline in this work to illustrate how to learn a NC-MCM. More complex computational models of C. elegans are available and could be leveraged for learning NC-MCMs [59]. In general, leveraging the power of state-of-the-art artificial intelligence algorithms, e.g., as being developed within the framework of causal representation learning [60], is probably required to learn NC-MCMs in more complex organisms. Second, we consider the extension to multiple behavioral contexts. In the present analyses of C. elegans, we have considered a set of behavioral states that are mutually exclusive, e.g., the worm can either crawl forward or backward but not execute both actions simultaneously. In more complex organisms, we may consider multiple behavioral contexts that are not mutually exclusive, e.g., eye movements to counteract visual motion and swimming bouts in larval zebrafish. We could then learn a NC-MCM for each behavioral context and study causal interactions between the cognitive states of each NC-MCM. Naturally, this approach could be scaled-up to an arbitrary number of behavioral contexts, potentially giving rise to a causally meaningful and neuronally grounded approach to studying cognition in complex organisms. Finally, we remark that we have focused in this work on learning cognitive states from neuronal data based on behavioral contexts. We could also consider other learning problems, e.g., settings in which sets of neuronal and cognitive states are already given and where we wish to determine whether there exists a behavioral context that gives rise to a NC-MCM. Such learning problems may be helpful to study whether already established cognitive ontologies, e.g., in psychology, are grounded in neuronal activity or, conversely, whether certain established cognitive concepts should be refined or excluded from scientific discourse. Independently of which particular learning problem we are interested in, the NC-MCM framework constitutes a rich and theoretically principled approach to bridging the explanatory gaps between neuronal activity patterns, cognitive states, and behaviors. Acknowledgments The authors would like to thank the following colleagues (in alphabetical order) for their insightful comments on an earlier version of the manuscript: Jozsef Arato, Dirk Bernhardt-Walther, Mauricio Gonzalez Soto and Kevin Reuter.

Zaaraoui, Hind;Schumer, Clarisse;Duval, Xavier;Hoen, Bruno;Opatowski, Lulla;Guedj, Jérémie

doi: 10.1371/journal.pcbi.1012573pmid: 39636821

Why was this study done? The attack rate of acute respiratory viruses is particularly high in households. Antiviral treatments can be strategically used to break transmission chains and reduce virus burden, however their use is limited by the difficulty to understand the factors that determine their effectiveness. Two distinct antiviral strategies, either treating the index individual upon diagnosis, or treating all household members upon index diagnosis, can be evaluated in terms of their reduction of viral infections and virological burden in the household. What did we do and find? We developed a theoretical framework to determine the impact of antiviral treatment in households, integrating the complex interplay between viral load dynamics, antiviral treatment initiation with respect to symptom onset and peak viral load, secondary attack rate and household size. All strategies reduce the number of infections by more than 50% when antiviral treatment can be be administered before symptom onset of the index case, but not after. Treatment initiated after symptom onset are effective in reducing the virological burden in household, in particular when both index cases and household contacts are treated and peak viral load occurs after symptom onset. The effectiveness of antiviral treatment strategies is larger for SAR ranging between 20 and 80%, and increases with the size of the household. What do these findings mean? Our work offers a theoretical framework to anticipate the effectiveness of antiviral strategies in households. It can be used to optimize prescription strategies aiming to prevent viral transmission and reduce disease burden in households. Introduction Households are the epicenter of community transmission of acute respiratory viruses, such as Influenza or SARS-CoV-2, with transmission rates resulting from close, repeated and inter-generational interactions [1, 3, 45]. For instance, in the case of pre-Omicron SARS-CoV-2 virus, the household attack rate, defined as the fraction of infected individuals after infection of an index case, is close to 40%, albeit with large variations across studies reflecting the heterogeneity in pre-existing immunity, social habits, as well as size and composition of households [4, 6, 7, 19]. In the Omicron variant era, values as large as 80% have been reported, reflecting the dominance of even more transmissible viruses that can escape immunity conferred by vaccine or previous infection [7]. Finding ways to prevent the transmission of respiratory viruses in households is therefore key to prevent community transmission and, in case of threat to public health, avoid to resort to interventions with an unsustainable economic and social, such as lockdown, curfew or school closure. Given the versatility of respiratory viruses, preventing community transmission of severe infections cannot rely on a single, albeit effective, intervention but rather requires a combination of non-pharmacological and pharmacological interventions that, together, can reduce the risk of severe disease in individuals with high-risk factors and break the chains of transmissions. One important but still under-employed layer of defense is the use of antiviral treatments, such as monoclonal antibodies or small molecules. As exemplified for SARS-CoV-2, these treatments can be highly effective in treating infected patients, reducing the risk of severe disease by 70–90% when administered within the first week of symptom onset [8, 9, 16], but they can also be used as pre- or post-exposure prophylaxis to reduce the risk of symptomatic disease [10, 11] or transmission. This approach is not novel, and large-scale treatment strategies in households have already been proposed in the past to reduce the burden of seasonal or pandemic Influenza virus [13, 14]. In the context of SARS-CoV-2, such strategies have been so far limited by a limited supply and the need for many of these treatments to be administered intravenously but the rapid development of safe, effective and orally available drugs, such as Paxlovid [2, 12], could broaden their use. The large deployment of antiviral treatment strategies remains however limited by the difficulty to measure their effectiveness and to factor in the multiple parameters that modulate it, in particular the virus transmission rate and the timing of treatment administration. While “the sooner, the better” is a long-standing paradigm of antiviral therapy [15], its implementation in real life is hampered by the challenge to identify patients before large quantities of virus have already been excreted. In the future, the experience acquired during the last pandemic in terms of contact tracing and implementation of contactless clinical studies [42] will enable more precocious strategies that can be implemented in case of threat to public health. This, therefore, makes it urgent to clearly identify the conditions required for a successive deployment of antivirals in households, and to quantify the effectiveness that can be expected. We here define a quantitative framework to predict the effectiveness of antiviral treatment strategies in households. We develop a multi-scale model integrating both the evolution of viral load within infected individuals and the risk of virus transmission. By factoring in the relative role of the epidemiological, clinical, pharmacological, virological and immunological parameters and by using the model to simulate thousands of households, we evaluate the impact of antiviral treatment under different prescription scenarios/strategies to mitigate acute respiratory virus transmission in households. Methods A multi-scale model to link viral dynamics and the risk of transmission over time Viral dynamic model. The viral dynamic (within-host) model characterizes the change in viral load levels after infection at time t = 0. It builds on previous model developed for SARS-CoV-2 and other acute viral infections [20, 21, 34]. In brief, the model includes three types of cell populations: uninfected susceptible target cells (T), infected cells in an eclipse phase (I1), and productively infected cells (I2). The model assumes that target cells are infected at a constant rate β. Once infected, cells enter an eclipse phase and become productively infected at a constant rate k. Productively infected cells produce viral particles at a constant rate π and are eliminated at a dynamic rate Δ(t) where Δ(t) = δ1 before adaptive response time τ, and Δ(t) = δ2 when t ≥ τ. A fraction μ of the viral particles is infectious, noted VI, and the remaining viral particles are non infectious, noted VNI. Viral load at time t post infection, V(t), is the sum of infectious and non-infectious viral particles, both cleared at the same rate constant c. In addition, the model accounts for a time-dependent immune response via a dimensionless compartment, F, which is stimulated by the presence of viral particles. In this model, F has an intrinsic loss rate, noted dF, and F acts by increasing the loss of infected cells, with a non-linear and saturable effect defined by where θ is the level of F required to achieve 50% of the maximum immune response. The model also incorporates an adaptive immune response with a refractory state. The innate immune response is modelled through the refractory compartment R and the effect of IFNs. The transition rate of cells from the susceptible state to refractory one is defined by the parameter ϕ. Cells within the refractory compartment can come back with rate ρ to the susceptible target compartment, providing temporary protection induced by IFNs. The model is given by the following equations: (1) The basic within-host reproduction number, , defined as the average number of secondary infected cells resulting from one infected cell in a population of fully susceptible target cells, is equal to . Transmission model. Following previous publications [24], a Power-law model is used to relate the non-linear relationship between viral load at time t, to the instantaneous risk of transmission during a high-risk contact, p(t): (2) where M quantifies the strength of the association between viral load and transmission and h reflects the stiffness of this association. Another way at looking at the model is to observe that , such that M can also be interpreted as a proxy of the intensity of the contact. To account for the variability in this parameter across individuals, due to different behavioral or biological factors, we further assumed that M follows a log-normal distribution with mean value m and standard deviation σ. As household contacts are not unique and may be repeated, we note P(∞) = limt→∞ P(t) the probability that at least on these contacts leads to an infection, given by: P(t) = 1 − ∏u∈[0,t](1 − p(u))), and we assume without loss of generality 1 contact every 12 hours (i.e., 2 contacts per day). Finally we define the Secondary Attack Rate (SAR) as the mean of P(∞) over all infected individuals. Impact of antiviral treatment on viral load. When an individual receives an antiviral treatment at time t = tx after infection, the viral dynamic model given by the equations differential system (1) is modified to reflect the effect of treatment on reducing the production of viruses by infected cells, with an efficacy noted ϵ, leading to the following model: (3) Importantly, we here assume that treatment, once initiated, is continued until virus eradication. By reducing viral replication, treatment reduces viral load levels and hence the risk of transmission as given by Eq (2) (see Figs 1 and 2). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Viral dynamics and antiviral treatment. (A) Individual viral dynamic profiles predicted by the model in 30 individuals (Eq 3), that are either untreated (I0, gray), treated within 5 days after symptom onset (Icur, orange), or treated within 4 days after infection (Ipep, green). All parameters are given in S1 Table, and the model assumes that a mean incubation period of 4 days, and a mean treatment antiviral efficacy of 99%. (B) Distribution of the peak viral load predicted by the model. (C) Distribution of the time to peak viral load predicted by the model. Note that gray and orange distributions overlap in (B) and (C) as the treatment is mostly initiated after the peak load time. https://doi.org/10.1371/journal.pcbi.1012573.g001 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Impact of antiviral treatment on the risk of virus transmission. (A) Instantaneous probability of virus transmission, p(t), in 30 individuals that are either untreated (I0, gray), treated within 5 days from symptom onset (Icur, orange), or treated within 4 days after infection (Ipep, green). All parameters are given in S1 Table. (B) Cumulative probability of transmission to another member of the household, P(t), assuming two contacts per day. (C) Distribution of the overall transmission probability, P(∞). Top right, without treatment, with SAR defined as the mean of the distribution. Similar definitions are used to defined SAR when treatment is initiated after symptom onset (SARcur, orange) or before symptom onset (SARpep, green). (D) SAR, SARcur, and SARpep values according to different values of m. (E, F) Generation time distribution for SAR = 40% and SAR = 80% respectively. A-B: Simulations conducted for m = 3.76 × 106 mL/copies (see Eq (2)), corresponding to SAR = 40%. https://doi.org/10.1371/journal.pcbi.1012573.g002 Model calibration. To calibrate our viral dynamic model, we used data from the National Basketball Association’s cohort [17, 18], which constitutes the most detailed dataset available on SARS-CoV-2 viral load so far. Overall the model was fitted to 607 individuals, that could be infected with pre-Omicron or Omicron variants. Inference procedure is detailed in S1 Text. “Viral load dynamic model and calibration” and parameter values are summarized in S1 Table. In addition, we assume that the incubation period follows a log-normal distribution with a mean of 4 days and a standard deviation of 0.125 days, i.e., 90% of patients have an incubation period ranging between 3 and 6 days as observed for SARS-CoV-2 [26]. We assume that treatment antiviral efficacy, ϵ, is equal to 99% on average (S1 Table), as observed for highly potent protease inhibitors [12]. Regarding the parameters governing the relationship between the viral load and the risk of transmission in Eq (2), we assume homogeneous mixing in the household and that all individuals have two (high risk) contacts per day to any other household member. We fix the value of h to 0.49, as previously estimated [24]. The parameter M is sampled from a log-normal distribution with mean m and standard deviation σ = 0.85 (see S1 Table). We consider m values ranging from 2.9 × 10−7 to 7.9 × 10−5 mL/cp in order to reproduce SAR ranging from 5% to 97% in absence of antiviral treatment (Fig 2D). The joint within- and between- host model can generate usual metrics of epidemiological studies, such as the SAR or the generation time, defined as the interval between the infection of an index infector and the infection time of its secondary cases. For instance, m = 3.76 × 10−6 mL/copies corresponds to SAR = 40%, as typically observed with SARS-CoV-2 pre-Omicron variants [31], leading to a mean generation interval of 4 days, consistent with values reported in the literature [47–50]. Similar calculations can be done to reproduce values observed with SARS-CoV-2 Omicron variants (Fig 2). Modeling household transmission and impact of antiviral treatment Household transmission and measure of the outbreak severity. The within- and between-host model presented until now only considers the risk of transmission from one infected individual to another individual, but it does not account for transmission chains that can occur in households of size S > 2 (Fig 3A). We therefore generalize the previous model to households of size S > 2 assuming homogeneous mixing and fixed contact rate between all household members. Therefore, transmission can occur to any other non-infected individuals in the household, and we assume that an individual can be infected only once during an outbreak (no reinfection). When all transmission chains in the household have extinguished, outbreak severity in a household h can be measured using two metrics (Fig 3A): The number of cases, noted ch, from which one can define the transmission rate, defined as the proportion of secondary infections in the household, also called the Final Attack Rate (FAR), as . The total virological burden, AUCVLh, computed as the sum of the area under the curve of the viral load of all infected individuals in the household. Because AUCVLh measures the total amount virus that has been excreted by a household during an outbreak, it can be used as a proxy of the risk of both severe infection (within the household) and of virus transmission in the community (outside the household) as, where Vk,h(u) the viral load at time u for individual k in the household h. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Transmission chains in households. (A) Top: Illustrative schematic of a transmission chain in a household; Bottom: temporal profile of the virological burden. The dashed area represents the cumulative area under the curve of the viral load in the household (these two figures were created with BioRender.com). (B) Model based prediction of the relationship between the Final Attack rate (FAR) and SAR, for household sizes ranging from 2 to 6. The yellow line represents the average FAR when sampling in the household size distribution in France. https://doi.org/10.1371/journal.pcbi.1012573.g003 Intervention strategies and measure of their effectiveness. We consider four treatment strategies. In the first scenario, noted Icur, the index individual initiates treatment within 5 days from symptom onset, assuming a uniform distribution for treatment initiation between 0 and 5 days post symptom onset. The second scenario, noted Ipep, assumes that treatment is administered before symptom onset as a post-exposure prophylaxis, assuming again a uniform distribution for treatment initiation between infection time (t = 0) and symptom onset. Then we consider the same treatment strategies, assuming now that treatment is given not only to the index individual, but also and simultaneously to all household members, regardless of their infection status, as either a post-exposure prophylaxis (when the index is already symptomatic, noted Icur + Hpep), or as a pre-exposure prophylaxis (when the index is not yet symptomatic, noted Ipep + Hprep) (Table 1). Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Main assumptions of the model. https://doi.org/10.1371/journal.pcbi.1012573.t001 Next, for each of these strategies, we simulated outbreaks in a large number of households, Nsim, using the distribution of household sizes in France (see below) and this provided us with an (expected) number of infection (), virological burden () and transmission rate: . We then calculated the effectiveness of each strategy Itx, as the relative reduction in the mean outcome compared to that obtained when no treatment is given, I0. Thus, an effectiveness of 50% in the transmission rate implies that a strategy reduces by half the mean FAR as compared to a strategy where no treatment is used (). We also calculated for each strategy Itx the number of cases avoided per number of treatments administered as the reduction in the total number of infected individuals as compared to no treatment (I0), divided by the total number treatment courses given, noted , as (4) where nh is the number of treatment given in household h which can be either equal to 1 (Icur or Ipep strategies) or Sh, the size of the household h (Icur + Hpep or Ipep + Hprep strategies) Household distribution. We simulated the outbreak in Nsim = 50, 000 households for each antiviral strategy considered. Because results depend on household size, we rely on the distribution of household sizes reported by the French National Institute of Statistics and Economic Studies (INSEE) [33] with households of size 2, 3, 4, 5 and 6 representing 50.85%, 21.62%, 18.20%, 6.69% and 2.64% of all households, respectively (see S2 Table). Designing a clinical trial to evaluate the effectiveness of an antiviral strategy. We finally evaluate the number of households that would be required to demonstrate in a clinical trial a significant effectiveness of the different antiviral strategies, considering as primary endpoints either the mean Final Attack Rate (FAR) or the mean virological burden (AUCVL). We consider a controlled study versus placebo (no treatment), with a 1:1 randomization on households, a type I error of 5% (two-sided test) and a statistical power of 90% for. AUCVL was treated as a continuous random variable normally distributed and a t-test was used to assess difference between treated and untreated, while FAR was analyzed as a discrete random variable using a binomial test. Generalization of the model and sensitivity analysis Evaluating the impact of different viral kinetic patterns and viruses. Because the effectiveness of each strategy critically depends on the delay between treatment initiation and peak viral load, we finally generalize our approach to different patterns of viral kinetics. In addition to the initial scenario where the time to peak viral load coincides with symptom onset and occurs at 4 days post infection, we now also consider more general cases where the time to peak viral load occurs at 1 or 7 days post infection, i.e., 3 days before or after symptom onset, respectively. Parameter values used to reproduce these kinetics are summarized in S3 Table. Exploring a secondary Attack Rate that depends on household size. Importantly, our model assumes that SAR is independent of the household size. As a large household size increases the risk of tertiary transmission (i.e., not originating from the index case), our model predicts that FAR increases with household size (Fig 3B). This may however overlook the existence of mechanisms that limit the spread of the virus in large households, such as the fact that contact rate are not homogeneous, as observed in many epidemiological studies [19, 27–30, 32]. We therefore also considered in a sensitivity analysis a model where SAR decreases with the size of the household size. Noting SAR the value observed for S = 2, as before, we note SARS the SAR for household size greater than 2 and we assume that (S ≥ 3). Thus, in this setting, we used a different value of m in each household size, relying on the relationship between m and SAR that was established above. Simulations show that the expected value of FAR is much less sensitive to the household sizes (see S1 Fig). Drug antiviral efficacy. In our main analysis we assume that treatment antiviral efficacy, ϵ, is equal to 99% on average (S1 Table) [12]. As a sensitivity analysis, we also consider lower antiviral efficacy of 50% and 90% (Table 1), as observed for other drugs [51]. A multi-scale model to link viral dynamics and the risk of transmission over time Viral dynamic model. The viral dynamic (within-host) model characterizes the change in viral load levels after infection at time t = 0. It builds on previous model developed for SARS-CoV-2 and other acute viral infections [20, 21, 34]. In brief, the model includes three types of cell populations: uninfected susceptible target cells (T), infected cells in an eclipse phase (I1), and productively infected cells (I2). The model assumes that target cells are infected at a constant rate β. Once infected, cells enter an eclipse phase and become productively infected at a constant rate k. Productively infected cells produce viral particles at a constant rate π and are eliminated at a dynamic rate Δ(t) where Δ(t) = δ1 before adaptive response time τ, and Δ(t) = δ2 when t ≥ τ. A fraction μ of the viral particles is infectious, noted VI, and the remaining viral particles are non infectious, noted VNI. Viral load at time t post infection, V(t), is the sum of infectious and non-infectious viral particles, both cleared at the same rate constant c. In addition, the model accounts for a time-dependent immune response via a dimensionless compartment, F, which is stimulated by the presence of viral particles. In this model, F has an intrinsic loss rate, noted dF, and F acts by increasing the loss of infected cells, with a non-linear and saturable effect defined by where θ is the level of F required to achieve 50% of the maximum immune response. The model also incorporates an adaptive immune response with a refractory state. The innate immune response is modelled through the refractory compartment R and the effect of IFNs. The transition rate of cells from the susceptible state to refractory one is defined by the parameter ϕ. Cells within the refractory compartment can come back with rate ρ to the susceptible target compartment, providing temporary protection induced by IFNs. The model is given by the following equations: (1) The basic within-host reproduction number, , defined as the average number of secondary infected cells resulting from one infected cell in a population of fully susceptible target cells, is equal to . Transmission model. Following previous publications [24], a Power-law model is used to relate the non-linear relationship between viral load at time t, to the instantaneous risk of transmission during a high-risk contact, p(t): (2) where M quantifies the strength of the association between viral load and transmission and h reflects the stiffness of this association. Another way at looking at the model is to observe that , such that M can also be interpreted as a proxy of the intensity of the contact. To account for the variability in this parameter across individuals, due to different behavioral or biological factors, we further assumed that M follows a log-normal distribution with mean value m and standard deviation σ. As household contacts are not unique and may be repeated, we note P(∞) = limt→∞ P(t) the probability that at least on these contacts leads to an infection, given by: P(t) = 1 − ∏u∈[0,t](1 − p(u))), and we assume without loss of generality 1 contact every 12 hours (i.e., 2 contacts per day). Finally we define the Secondary Attack Rate (SAR) as the mean of P(∞) over all infected individuals. Impact of antiviral treatment on viral load. When an individual receives an antiviral treatment at time t = tx after infection, the viral dynamic model given by the equations differential system (1) is modified to reflect the effect of treatment on reducing the production of viruses by infected cells, with an efficacy noted ϵ, leading to the following model: (3) Importantly, we here assume that treatment, once initiated, is continued until virus eradication. By reducing viral replication, treatment reduces viral load levels and hence the risk of transmission as given by Eq (2) (see Figs 1 and 2). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Viral dynamics and antiviral treatment. (A) Individual viral dynamic profiles predicted by the model in 30 individuals (Eq 3), that are either untreated (I0, gray), treated within 5 days after symptom onset (Icur, orange), or treated within 4 days after infection (Ipep, green). All parameters are given in S1 Table, and the model assumes that a mean incubation period of 4 days, and a mean treatment antiviral efficacy of 99%. (B) Distribution of the peak viral load predicted by the model. (C) Distribution of the time to peak viral load predicted by the model. Note that gray and orange distributions overlap in (B) and (C) as the treatment is mostly initiated after the peak load time. https://doi.org/10.1371/journal.pcbi.1012573.g001 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Impact of antiviral treatment on the risk of virus transmission. (A) Instantaneous probability of virus transmission, p(t), in 30 individuals that are either untreated (I0, gray), treated within 5 days from symptom onset (Icur, orange), or treated within 4 days after infection (Ipep, green). All parameters are given in S1 Table. (B) Cumulative probability of transmission to another member of the household, P(t), assuming two contacts per day. (C) Distribution of the overall transmission probability, P(∞). Top right, without treatment, with SAR defined as the mean of the distribution. Similar definitions are used to defined SAR when treatment is initiated after symptom onset (SARcur, orange) or before symptom onset (SARpep, green). (D) SAR, SARcur, and SARpep values according to different values of m. (E, F) Generation time distribution for SAR = 40% and SAR = 80% respectively. A-B: Simulations conducted for m = 3.76 × 106 mL/copies (see Eq (2)), corresponding to SAR = 40%. https://doi.org/10.1371/journal.pcbi.1012573.g002 Model calibration. To calibrate our viral dynamic model, we used data from the National Basketball Association’s cohort [17, 18], which constitutes the most detailed dataset available on SARS-CoV-2 viral load so far. Overall the model was fitted to 607 individuals, that could be infected with pre-Omicron or Omicron variants. Inference procedure is detailed in S1 Text. “Viral load dynamic model and calibration” and parameter values are summarized in S1 Table. In addition, we assume that the incubation period follows a log-normal distribution with a mean of 4 days and a standard deviation of 0.125 days, i.e., 90% of patients have an incubation period ranging between 3 and 6 days as observed for SARS-CoV-2 [26]. We assume that treatment antiviral efficacy, ϵ, is equal to 99% on average (S1 Table), as observed for highly potent protease inhibitors [12]. Regarding the parameters governing the relationship between the viral load and the risk of transmission in Eq (2), we assume homogeneous mixing in the household and that all individuals have two (high risk) contacts per day to any other household member. We fix the value of h to 0.49, as previously estimated [24]. The parameter M is sampled from a log-normal distribution with mean m and standard deviation σ = 0.85 (see S1 Table). We consider m values ranging from 2.9 × 10−7 to 7.9 × 10−5 mL/cp in order to reproduce SAR ranging from 5% to 97% in absence of antiviral treatment (Fig 2D). The joint within- and between- host model can generate usual metrics of epidemiological studies, such as the SAR or the generation time, defined as the interval between the infection of an index infector and the infection time of its secondary cases. For instance, m = 3.76 × 10−6 mL/copies corresponds to SAR = 40%, as typically observed with SARS-CoV-2 pre-Omicron variants [31], leading to a mean generation interval of 4 days, consistent with values reported in the literature [47–50]. Similar calculations can be done to reproduce values observed with SARS-CoV-2 Omicron variants (Fig 2). Viral dynamic model. The viral dynamic (within-host) model characterizes the change in viral load levels after infection at time t = 0. It builds on previous model developed for SARS-CoV-2 and other acute viral infections [20, 21, 34]. In brief, the model includes three types of cell populations: uninfected susceptible target cells (T), infected cells in an eclipse phase (I1), and productively infected cells (I2). The model assumes that target cells are infected at a constant rate β. Once infected, cells enter an eclipse phase and become productively infected at a constant rate k. Productively infected cells produce viral particles at a constant rate π and are eliminated at a dynamic rate Δ(t) where Δ(t) = δ1 before adaptive response time τ, and Δ(t) = δ2 when t ≥ τ. A fraction μ of the viral particles is infectious, noted VI, and the remaining viral particles are non infectious, noted VNI. Viral load at time t post infection, V(t), is the sum of infectious and non-infectious viral particles, both cleared at the same rate constant c. In addition, the model accounts for a time-dependent immune response via a dimensionless compartment, F, which is stimulated by the presence of viral particles. In this model, F has an intrinsic loss rate, noted dF, and F acts by increasing the loss of infected cells, with a non-linear and saturable effect defined by where θ is the level of F required to achieve 50% of the maximum immune response. The model also incorporates an adaptive immune response with a refractory state. The innate immune response is modelled through the refractory compartment R and the effect of IFNs. The transition rate of cells from the susceptible state to refractory one is defined by the parameter ϕ. Cells within the refractory compartment can come back with rate ρ to the susceptible target compartment, providing temporary protection induced by IFNs. The model is given by the following equations: (1) The basic within-host reproduction number, , defined as the average number of secondary infected cells resulting from one infected cell in a population of fully susceptible target cells, is equal to . Transmission model. Following previous publications [24], a Power-law model is used to relate the non-linear relationship between viral load at time t, to the instantaneous risk of transmission during a high-risk contact, p(t): (2) where M quantifies the strength of the association between viral load and transmission and h reflects the stiffness of this association. Another way at looking at the model is to observe that , such that M can also be interpreted as a proxy of the intensity of the contact. To account for the variability in this parameter across individuals, due to different behavioral or biological factors, we further assumed that M follows a log-normal distribution with mean value m and standard deviation σ. As household contacts are not unique and may be repeated, we note P(∞) = limt→∞ P(t) the probability that at least on these contacts leads to an infection, given by: P(t) = 1 − ∏u∈[0,t](1 − p(u))), and we assume without loss of generality 1 contact every 12 hours (i.e., 2 contacts per day). Finally we define the Secondary Attack Rate (SAR) as the mean of P(∞) over all infected individuals. Impact of antiviral treatment on viral load. When an individual receives an antiviral treatment at time t = tx after infection, the viral dynamic model given by the equations differential system (1) is modified to reflect the effect of treatment on reducing the production of viruses by infected cells, with an efficacy noted ϵ, leading to the following model: (3) Importantly, we here assume that treatment, once initiated, is continued until virus eradication. By reducing viral replication, treatment reduces viral load levels and hence the risk of transmission as given by Eq (2) (see Figs 1 and 2). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Viral dynamics and antiviral treatment. (A) Individual viral dynamic profiles predicted by the model in 30 individuals (Eq 3), that are either untreated (I0, gray), treated within 5 days after symptom onset (Icur, orange), or treated within 4 days after infection (Ipep, green). All parameters are given in S1 Table, and the model assumes that a mean incubation period of 4 days, and a mean treatment antiviral efficacy of 99%. (B) Distribution of the peak viral load predicted by the model. (C) Distribution of the time to peak viral load predicted by the model. Note that gray and orange distributions overlap in (B) and (C) as the treatment is mostly initiated after the peak load time. https://doi.org/10.1371/journal.pcbi.1012573.g001 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Impact of antiviral treatment on the risk of virus transmission. (A) Instantaneous probability of virus transmission, p(t), in 30 individuals that are either untreated (I0, gray), treated within 5 days from symptom onset (Icur, orange), or treated within 4 days after infection (Ipep, green). All parameters are given in S1 Table. (B) Cumulative probability of transmission to another member of the household, P(t), assuming two contacts per day. (C) Distribution of the overall transmission probability, P(∞). Top right, without treatment, with SAR defined as the mean of the distribution. Similar definitions are used to defined SAR when treatment is initiated after symptom onset (SARcur, orange) or before symptom onset (SARpep, green). (D) SAR, SARcur, and SARpep values according to different values of m. (E, F) Generation time distribution for SAR = 40% and SAR = 80% respectively. A-B: Simulations conducted for m = 3.76 × 106 mL/copies (see Eq (2)), corresponding to SAR = 40%. https://doi.org/10.1371/journal.pcbi.1012573.g002 Model calibration. To calibrate our viral dynamic model, we used data from the National Basketball Association’s cohort [17, 18], which constitutes the most detailed dataset available on SARS-CoV-2 viral load so far. Overall the model was fitted to 607 individuals, that could be infected with pre-Omicron or Omicron variants. Inference procedure is detailed in S1 Text. “Viral load dynamic model and calibration” and parameter values are summarized in S1 Table. In addition, we assume that the incubation period follows a log-normal distribution with a mean of 4 days and a standard deviation of 0.125 days, i.e., 90% of patients have an incubation period ranging between 3 and 6 days as observed for SARS-CoV-2 [26]. We assume that treatment antiviral efficacy, ϵ, is equal to 99% on average (S1 Table), as observed for highly potent protease inhibitors [12]. Regarding the parameters governing the relationship between the viral load and the risk of transmission in Eq (2), we assume homogeneous mixing in the household and that all individuals have two (high risk) contacts per day to any other household member. We fix the value of h to 0.49, as previously estimated [24]. The parameter M is sampled from a log-normal distribution with mean m and standard deviation σ = 0.85 (see S1 Table). We consider m values ranging from 2.9 × 10−7 to 7.9 × 10−5 mL/cp in order to reproduce SAR ranging from 5% to 97% in absence of antiviral treatment (Fig 2D). The joint within- and between- host model can generate usual metrics of epidemiological studies, such as the SAR or the generation time, defined as the interval between the infection of an index infector and the infection time of its secondary cases. For instance, m = 3.76 × 10−6 mL/copies corresponds to SAR = 40%, as typically observed with SARS-CoV-2 pre-Omicron variants [31], leading to a mean generation interval of 4 days, consistent with values reported in the literature [47–50]. Similar calculations can be done to reproduce values observed with SARS-CoV-2 Omicron variants (Fig 2). Modeling household transmission and impact of antiviral treatment Household transmission and measure of the outbreak severity. The within- and between-host model presented until now only considers the risk of transmission from one infected individual to another individual, but it does not account for transmission chains that can occur in households of size S > 2 (Fig 3A). We therefore generalize the previous model to households of size S > 2 assuming homogeneous mixing and fixed contact rate between all household members. Therefore, transmission can occur to any other non-infected individuals in the household, and we assume that an individual can be infected only once during an outbreak (no reinfection). When all transmission chains in the household have extinguished, outbreak severity in a household h can be measured using two metrics (Fig 3A): The number of cases, noted ch, from which one can define the transmission rate, defined as the proportion of secondary infections in the household, also called the Final Attack Rate (FAR), as . The total virological burden, AUCVLh, computed as the sum of the area under the curve of the viral load of all infected individuals in the household. Because AUCVLh measures the total amount virus that has been excreted by a household during an outbreak, it can be used as a proxy of the risk of both severe infection (within the household) and of virus transmission in the community (outside the household) as, where Vk,h(u) the viral load at time u for individual k in the household h. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Transmission chains in households. (A) Top: Illustrative schematic of a transmission chain in a household; Bottom: temporal profile of the virological burden. The dashed area represents the cumulative area under the curve of the viral load in the household (these two figures were created with BioRender.com). (B) Model based prediction of the relationship between the Final Attack rate (FAR) and SAR, for household sizes ranging from 2 to 6. The yellow line represents the average FAR when sampling in the household size distribution in France. https://doi.org/10.1371/journal.pcbi.1012573.g003 Intervention strategies and measure of their effectiveness. We consider four treatment strategies. In the first scenario, noted Icur, the index individual initiates treatment within 5 days from symptom onset, assuming a uniform distribution for treatment initiation between 0 and 5 days post symptom onset. The second scenario, noted Ipep, assumes that treatment is administered before symptom onset as a post-exposure prophylaxis, assuming again a uniform distribution for treatment initiation between infection time (t = 0) and symptom onset. Then we consider the same treatment strategies, assuming now that treatment is given not only to the index individual, but also and simultaneously to all household members, regardless of their infection status, as either a post-exposure prophylaxis (when the index is already symptomatic, noted Icur + Hpep), or as a pre-exposure prophylaxis (when the index is not yet symptomatic, noted Ipep + Hprep) (Table 1). Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Main assumptions of the model. https://doi.org/10.1371/journal.pcbi.1012573.t001 Next, for each of these strategies, we simulated outbreaks in a large number of households, Nsim, using the distribution of household sizes in France (see below) and this provided us with an (expected) number of infection (), virological burden () and transmission rate: . We then calculated the effectiveness of each strategy Itx, as the relative reduction in the mean outcome compared to that obtained when no treatment is given, I0. Thus, an effectiveness of 50% in the transmission rate implies that a strategy reduces by half the mean FAR as compared to a strategy where no treatment is used (). We also calculated for each strategy Itx the number of cases avoided per number of treatments administered as the reduction in the total number of infected individuals as compared to no treatment (I0), divided by the total number treatment courses given, noted , as (4) where nh is the number of treatment given in household h which can be either equal to 1 (Icur or Ipep strategies) or Sh, the size of the household h (Icur + Hpep or Ipep + Hprep strategies) Household distribution. We simulated the outbreak in Nsim = 50, 000 households for each antiviral strategy considered. Because results depend on household size, we rely on the distribution of household sizes reported by the French National Institute of Statistics and Economic Studies (INSEE) [33] with households of size 2, 3, 4, 5 and 6 representing 50.85%, 21.62%, 18.20%, 6.69% and 2.64% of all households, respectively (see S2 Table). Designing a clinical trial to evaluate the effectiveness of an antiviral strategy. We finally evaluate the number of households that would be required to demonstrate in a clinical trial a significant effectiveness of the different antiviral strategies, considering as primary endpoints either the mean Final Attack Rate (FAR) or the mean virological burden (AUCVL). We consider a controlled study versus placebo (no treatment), with a 1:1 randomization on households, a type I error of 5% (two-sided test) and a statistical power of 90% for. AUCVL was treated as a continuous random variable normally distributed and a t-test was used to assess difference between treated and untreated, while FAR was analyzed as a discrete random variable using a binomial test. Household transmission and measure of the outbreak severity. The within- and between-host model presented until now only considers the risk of transmission from one infected individual to another individual, but it does not account for transmission chains that can occur in households of size S > 2 (Fig 3A). We therefore generalize the previous model to households of size S > 2 assuming homogeneous mixing and fixed contact rate between all household members. Therefore, transmission can occur to any other non-infected individuals in the household, and we assume that an individual can be infected only once during an outbreak (no reinfection). When all transmission chains in the household have extinguished, outbreak severity in a household h can be measured using two metrics (Fig 3A): The number of cases, noted ch, from which one can define the transmission rate, defined as the proportion of secondary infections in the household, also called the Final Attack Rate (FAR), as . The total virological burden, AUCVLh, computed as the sum of the area under the curve of the viral load of all infected individuals in the household. Because AUCVLh measures the total amount virus that has been excreted by a household during an outbreak, it can be used as a proxy of the risk of both severe infection (within the household) and of virus transmission in the community (outside the household) as, where Vk,h(u) the viral load at time u for individual k in the household h. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Transmission chains in households. (A) Top: Illustrative schematic of a transmission chain in a household; Bottom: temporal profile of the virological burden. The dashed area represents the cumulative area under the curve of the viral load in the household (these two figures were created with BioRender.com). (B) Model based prediction of the relationship between the Final Attack rate (FAR) and SAR, for household sizes ranging from 2 to 6. The yellow line represents the average FAR when sampling in the household size distribution in France. https://doi.org/10.1371/journal.pcbi.1012573.g003 Intervention strategies and measure of their effectiveness. We consider four treatment strategies. In the first scenario, noted Icur, the index individual initiates treatment within 5 days from symptom onset, assuming a uniform distribution for treatment initiation between 0 and 5 days post symptom onset. The second scenario, noted Ipep, assumes that treatment is administered before symptom onset as a post-exposure prophylaxis, assuming again a uniform distribution for treatment initiation between infection time (t = 0) and symptom onset. Then we consider the same treatment strategies, assuming now that treatment is given not only to the index individual, but also and simultaneously to all household members, regardless of their infection status, as either a post-exposure prophylaxis (when the index is already symptomatic, noted Icur + Hpep), or as a pre-exposure prophylaxis (when the index is not yet symptomatic, noted Ipep + Hprep) (Table 1). Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Main assumptions of the model. https://doi.org/10.1371/journal.pcbi.1012573.t001 Next, for each of these strategies, we simulated outbreaks in a large number of households, Nsim, using the distribution of household sizes in France (see below) and this provided us with an (expected) number of infection (), virological burden () and transmission rate: . We then calculated the effectiveness of each strategy Itx, as the relative reduction in the mean outcome compared to that obtained when no treatment is given, I0. Thus, an effectiveness of 50% in the transmission rate implies that a strategy reduces by half the mean FAR as compared to a strategy where no treatment is used (). We also calculated for each strategy Itx the number of cases avoided per number of treatments administered as the reduction in the total number of infected individuals as compared to no treatment (I0), divided by the total number treatment courses given, noted , as (4) where nh is the number of treatment given in household h which can be either equal to 1 (Icur or Ipep strategies) or Sh, the size of the household h (Icur + Hpep or Ipep + Hprep strategies) Household distribution. We simulated the outbreak in Nsim = 50, 000 households for each antiviral strategy considered. Because results depend on household size, we rely on the distribution of household sizes reported by the French National Institute of Statistics and Economic Studies (INSEE) [33] with households of size 2, 3, 4, 5 and 6 representing 50.85%, 21.62%, 18.20%, 6.69% and 2.64% of all households, respectively (see S2 Table). Designing a clinical trial to evaluate the effectiveness of an antiviral strategy. We finally evaluate the number of households that would be required to demonstrate in a clinical trial a significant effectiveness of the different antiviral strategies, considering as primary endpoints either the mean Final Attack Rate (FAR) or the mean virological burden (AUCVL). We consider a controlled study versus placebo (no treatment), with a 1:1 randomization on households, a type I error of 5% (two-sided test) and a statistical power of 90% for. AUCVL was treated as a continuous random variable normally distributed and a t-test was used to assess difference between treated and untreated, while FAR was analyzed as a discrete random variable using a binomial test. Generalization of the model and sensitivity analysis Evaluating the impact of different viral kinetic patterns and viruses. Because the effectiveness of each strategy critically depends on the delay between treatment initiation and peak viral load, we finally generalize our approach to different patterns of viral kinetics. In addition to the initial scenario where the time to peak viral load coincides with symptom onset and occurs at 4 days post infection, we now also consider more general cases where the time to peak viral load occurs at 1 or 7 days post infection, i.e., 3 days before or after symptom onset, respectively. Parameter values used to reproduce these kinetics are summarized in S3 Table. Exploring a secondary Attack Rate that depends on household size. Importantly, our model assumes that SAR is independent of the household size. As a large household size increases the risk of tertiary transmission (i.e., not originating from the index case), our model predicts that FAR increases with household size (Fig 3B). This may however overlook the existence of mechanisms that limit the spread of the virus in large households, such as the fact that contact rate are not homogeneous, as observed in many epidemiological studies [19, 27–30, 32]. We therefore also considered in a sensitivity analysis a model where SAR decreases with the size of the household size. Noting SAR the value observed for S = 2, as before, we note SARS the SAR for household size greater than 2 and we assume that (S ≥ 3). Thus, in this setting, we used a different value of m in each household size, relying on the relationship between m and SAR that was established above. Simulations show that the expected value of FAR is much less sensitive to the household sizes (see S1 Fig). Drug antiviral efficacy. In our main analysis we assume that treatment antiviral efficacy, ϵ, is equal to 99% on average (S1 Table) [12]. As a sensitivity analysis, we also consider lower antiviral efficacy of 50% and 90% (Table 1), as observed for other drugs [51]. Evaluating the impact of different viral kinetic patterns and viruses. Because the effectiveness of each strategy critically depends on the delay between treatment initiation and peak viral load, we finally generalize our approach to different patterns of viral kinetics. In addition to the initial scenario where the time to peak viral load coincides with symptom onset and occurs at 4 days post infection, we now also consider more general cases where the time to peak viral load occurs at 1 or 7 days post infection, i.e., 3 days before or after symptom onset, respectively. Parameter values used to reproduce these kinetics are summarized in S3 Table. Exploring a secondary Attack Rate that depends on household size. Importantly, our model assumes that SAR is independent of the household size. As a large household size increases the risk of tertiary transmission (i.e., not originating from the index case), our model predicts that FAR increases with household size (Fig 3B). This may however overlook the existence of mechanisms that limit the spread of the virus in large households, such as the fact that contact rate are not homogeneous, as observed in many epidemiological studies [19, 27–30, 32]. We therefore also considered in a sensitivity analysis a model where SAR decreases with the size of the household size. Noting SAR the value observed for S = 2, as before, we note SARS the SAR for household size greater than 2 and we assume that (S ≥ 3). Thus, in this setting, we used a different value of m in each household size, relying on the relationship between m and SAR that was established above. Simulations show that the expected value of FAR is much less sensitive to the household sizes (see S1 Fig). Drug antiviral efficacy. In our main analysis we assume that treatment antiviral efficacy, ϵ, is equal to 99% on average (S1 Table) [12]. As a sensitivity analysis, we also consider lower antiviral efficacy of 50% and 90% (Table 1), as observed for other drugs [51]. Results A multiscale model to understand the interplay between treatment, viral dynamics and transmission risk The multiscale model combines a within-host model of viral dynamics and a between-host model that estimates the probability of virus transmission after a contact [22, 24]. The model reproduces the patterns of kinetics observed with SARS-CoV-2, with a time to peak viral load that coincides with the incubation duration and is equal to 4 days on average, albeit with a large inter-subject variability (Fig 1A), and a median time to viral clearance of 15 days [23]. In this context, initiating a treatment after symptom onset, hence after peak viral load in general, has only a minimal effect on viral load (Fig 1A). Conversely, initiating a treatment before symptom onset can have a dramatic effect on viral load, reducing peak viral load and the duration of viral shedding. Accordingly, treatment initiated after symptom onset is predicted to have a minimal effect on transmission, while a pre-symptomatic treatment may dramatically reduce the risk of transmission (Fig 2). Using a secondary attack rate (SAR) of 40%, as typically reported for SARS-CoV-2 pre-Omicron variants [31], we predict that treatment initiation within 5 days of symptom onset will reduce the mean secondary attack rate to 30%. Treating before symptom onset will reduce SAR to 4.5% (Fig 2C). Similar results are observed for other ranges of SAR, confirming the large benefit of early treatment on reducing virus transmission (Fig 2D). Treating index case and household members reduces transmission and virological burden The model integrates transmission chains to simulate outbreaks in households. In households of size 2 or 3, the Final Attack Rate (FAR), defined as the mean proportion of secondary cases in household (see Methods), and the virological burden, defined as the mean cumulative viral load in household (see Methods), increase linearly with SAR, consistent with the fact that most transmission event originate from the index case. However, in households of size greater than 4, the FAR increases in a non-linear fashion with SAR (Fig 3B), reflecting the multiple chains of transmission that can exist in large households, in particular when SAR has intermediate ranges between 10 and 40%. Next, we evaluate the impact of antiviral treatment on household transmission. Because antiviral drugs are most generally prescribed with the aim of treating an infected individual, we here consider the most common case where the index case is already symptomatic and is treated within 5 days of symptom onset (Icur, see Methods). Because most index cases are already in the clearance phase of the virus when treatment is initiated, this strategy has only a minimal effect in reducing both the number of infected individuals and the virological burden, regardless of SAR (Fig 4) and household size (S2 Fig). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Impact of treatment strategies on transmission and virological burden for SAR of 40% (left) and 80% (right). (A) Number of infected individuals (per 1000 households); (B) Virological burden (per household). Antiviral strategies are the following: no treatment (I0, black), treatment initiated when the index case is symptomatic (Icur and Icur + Hpep, orange), treatment initiated when the index case is pre-symptomatic (Ipep and Ipep + Hprep, green). Results are shown by household sizes, S = 2 (darkest color, top) to S = 6 (lightest color, bottom), with the same distribution of household size than observed in the French population. https://doi.org/10.1371/journal.pcbi.1012573.g004 A more aggressive strategy where all household members are treated upon diagnosis of the symptomatic index, regardless of their infection status (Icur + Hpep, see Methods), is more effective in this case. Although most transmission events originating from the index case have already occurred when treatment is initiated, this strategy acts as a post-exposure prophylaxis, reducing both the risk of further transmission events and the virological burden in secondary infected individuals. Consequently, it leads to a larger reduction in the number of infections, in particular in household of size greater than 4, where several transmission chains are more likely to occur (see S2C and S2D Fig). Because only about 25% of households are of size ≥ 4 in France, the overall effectiveness of this strategy on the number of infections is nonetheless small and remains equal to 20–38% for most cases considered (Fig 5A). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Effectiveness of treatment strategies. (A) Treatment effectiveness on transmission. (B) Treatment effectiveness on virological burden. (C) Number of cases avoided per amount of treatments deployed. All effectiveness are relative to no treatment. https://doi.org/10.1371/journal.pcbi.1012573.g005 Despite its limited effectiveness on virus transmission, treating all household members can be effective to reduce the virological burden. When baseline SAR is high, e.g. SAR > 30%, the effectiveness ranges 45–52% (Fig 5B). This average effectiveness masks important differences across households composition, with larger effectiveness in households of size ≥ 4 (see S3A and S3B Fig). Preempting symptom development in community contacts largely improves the effectiveness of antiviral treatment strategies We then investigate a strategy in which the index case is treated before symptom onset as post-exposure prophylaxis (Ipep). Because treatment is initiated before peak viral load, most transmission events are averted. Consequently, this strategy has an effectiveness greater than 75% for most SAR values, and up to 95%, on both the transmission rate and the virological burden (Fig 5), regardless of household size (S3 Fig). Of note the effectiveness of this strategy marks a net decrease for SAR ≥ 80%, as the generation time decreases with SAR (Fig 2E and 2F), thereby increasing the risk of virus transmission before treatment initiation (Fig 5A and 5B). Although the additional benefit of treating all household members on virus is low in general (Fig 5), this strategy is relevant to reduce the virological burden, with effectiveness close to 100%, regardless of SAR and household size (Fig 5A and 5B and S3 Fig). In a cost-effectiveness perspective, treating only the index case before symptom onset is highly efficient (Fig 5C), with a number of cases avoided per treatment unit deployed larger than 1 if SAR is greater than 45%, as compared to a maximum value of 0.27 when the index case is treated after symptom onset (i.e., 4 individuals are treated to avoid 1 infection). The model can be directly used to support the design of clinical trials The modeling framework can also be used to design clinical trials assessing the effectiveness of antiviral strategies (see Methods and S10 and S11 Figs). Focusing on SAR ranging between 20 and 80%, treating only the index case after symptom onset (Icur) would require 170–430 households per arm to evidence a clinical effectiveness on transmission (see Fig 6), and 340–450 households to demonstrate effectiveness on virological burden. Treating all household members (Icur + Hpep) would be more advantageous, requiring 70–240 and 30–100 households to demonstrate a reduction in the transmission rate and the virological burden, respectively. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Number of households (per arm) required to achieve 90% statistical power in 1:1 randomized clinical trial vs placebo (no treatment). (A) Number of households required to demonstrate a reduction in the Final Attack Rate (FAR). (B) Number of households (per arm) required to demonstrate a reduction in the virological burden (AUCVL). https://doi.org/10.1371/journal.pcbi.1012573.g006 Much lower numbers are required when the index case is treated in the pre-symptomatic phase (Ipep or Ipep + Hprep), with less than 25 households needed to demonstrate an effect on the number of transmissions, and less than 20 households to show an effectiveness on the virological burden. Note that these results are obtained in a context where only 25% of households have more than 4 members (French distribution). Focusing only on large households (see S11B Fig) and treating all their members would diminish the number of households required by a factor of 1.3 and 2.3 for Ipep + Hprep and Icur + Hpep strategies, respectively (S9 Fig). Results are robust to changes in SAR expressions and lower levels of antiviral activity As a sensitivity analysis we also considered the case where SAR decreases with household size (see Methods and S1, S4 and S8 Figs). The results obtained with this alternative model were largely similar to those presented above, which is due to the fact that less than 25% of households are of size 4 or more, thereby limiting the impact of large households (see S4 Fig). We also evaluated the impact of a treatment with a lower mean antiviral efficacy, noted ϵ, than what was assumed in our baseline scenario (ϵ = 99%). ϵ = 50% leads to a marked decrease of the effectiveness of all strategies (S5 Fig), while ϵ = 90% shows almost similar effectiveness in Icur strategies than our baseline scenario. However ϵ = 90% leads to a markedly lower effectiveness of Ipep strategies (S5 Fig), reducing the clinical effectiveness from more than 75% in the baseline scenario (see above) to between 40 and 80% for most SAR values. Treatment effectiveness for other patterns of viral kinetics Finally, we generalize our approach to different viral kinetics profiles, with a time to peak viral load occurring at 1, 4 and 7 days post infection (S7 Fig), respectively, and a incubation period that remains equal to 4 days [26]. This allows us to generalize our results to different acute viral infections, where the peak viral load can either precede or coincide with symptom onset, as observed for SARS-CoV-2 or RSV [5], or afterwards, as observed for Influenza and SARS-CoV-1 [34–36]. When the index case is already symptomatic (Icur strategies) and peak viral load occurs before or at symptom onset, the effectiveness on reducing transmission remains low in all cases considered (Fig 7). The effectiveness of Icur strategy increases with the time to peak viral load, with values ranging 0–25% when the peak viral load occurs 3 days before symptom onset and 25–75% when the peak viral load occurs 3 days after symptom onset. In all these scenarios, the effectiveness decreases with SAR and we observe a limited additional benefit of treating all household members (Icur + Hpep). As previously reported in the main scenario, the effectiveness on reducing virological burden is systematically larger than on transmission, in particular when all household members are treated Icur + Hpep, with effectiveness values ranging from 25–50% when peak viral coincides with symptom onset for SAR < 60%, to values up to 75% when peak viral load occurs more than 3 days after symptom onset. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 7. Effectiveness of treatment strategies according to time to peak viral load. Left: effectiveness on transmission; right: effectiveness on the virological burden. Top: index case treated after symptom onset (Icur and Icur + Hpep); bottom: index case treated before symptom onset (Ipep and Ipep + Hprep). All effectiveness are relative to no treatment. https://doi.org/10.1371/journal.pcbi.1012573.g007 The effectiveness of strategies reaching out the index case before symptom onset is undoubtedly better (Ipep strategies), with effectiveness greater than 75% in all scenarios considered, regardless of SAR, as long as peak viral load coincides or occurs after symptom onset. Notably, the effectiveness is very high even if only the index case is treated. This strategy can even be effective in unfavorable cases where peak viral load precedes symptom onset by three days, with effectiveness in the range 50–75% as long as SAR is less than 70%. Of note these results remain almost equivalent when using the alternative model of SAR according to household size (S8 Fig). A multiscale model to understand the interplay between treatment, viral dynamics and transmission risk The multiscale model combines a within-host model of viral dynamics and a between-host model that estimates the probability of virus transmission after a contact [22, 24]. The model reproduces the patterns of kinetics observed with SARS-CoV-2, with a time to peak viral load that coincides with the incubation duration and is equal to 4 days on average, albeit with a large inter-subject variability (Fig 1A), and a median time to viral clearance of 15 days [23]. In this context, initiating a treatment after symptom onset, hence after peak viral load in general, has only a minimal effect on viral load (Fig 1A). Conversely, initiating a treatment before symptom onset can have a dramatic effect on viral load, reducing peak viral load and the duration of viral shedding. Accordingly, treatment initiated after symptom onset is predicted to have a minimal effect on transmission, while a pre-symptomatic treatment may dramatically reduce the risk of transmission (Fig 2). Using a secondary attack rate (SAR) of 40%, as typically reported for SARS-CoV-2 pre-Omicron variants [31], we predict that treatment initiation within 5 days of symptom onset will reduce the mean secondary attack rate to 30%. Treating before symptom onset will reduce SAR to 4.5% (Fig 2C). Similar results are observed for other ranges of SAR, confirming the large benefit of early treatment on reducing virus transmission (Fig 2D). Treating index case and household members reduces transmission and virological burden The model integrates transmission chains to simulate outbreaks in households. In households of size 2 or 3, the Final Attack Rate (FAR), defined as the mean proportion of secondary cases in household (see Methods), and the virological burden, defined as the mean cumulative viral load in household (see Methods), increase linearly with SAR, consistent with the fact that most transmission event originate from the index case. However, in households of size greater than 4, the FAR increases in a non-linear fashion with SAR (Fig 3B), reflecting the multiple chains of transmission that can exist in large households, in particular when SAR has intermediate ranges between 10 and 40%. Next, we evaluate the impact of antiviral treatment on household transmission. Because antiviral drugs are most generally prescribed with the aim of treating an infected individual, we here consider the most common case where the index case is already symptomatic and is treated within 5 days of symptom onset (Icur, see Methods). Because most index cases are already in the clearance phase of the virus when treatment is initiated, this strategy has only a minimal effect in reducing both the number of infected individuals and the virological burden, regardless of SAR (Fig 4) and household size (S2 Fig). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Impact of treatment strategies on transmission and virological burden for SAR of 40% (left) and 80% (right). (A) Number of infected individuals (per 1000 households); (B) Virological burden (per household). Antiviral strategies are the following: no treatment (I0, black), treatment initiated when the index case is symptomatic (Icur and Icur + Hpep, orange), treatment initiated when the index case is pre-symptomatic (Ipep and Ipep + Hprep, green). Results are shown by household sizes, S = 2 (darkest color, top) to S = 6 (lightest color, bottom), with the same distribution of household size than observed in the French population. https://doi.org/10.1371/journal.pcbi.1012573.g004 A more aggressive strategy where all household members are treated upon diagnosis of the symptomatic index, regardless of their infection status (Icur + Hpep, see Methods), is more effective in this case. Although most transmission events originating from the index case have already occurred when treatment is initiated, this strategy acts as a post-exposure prophylaxis, reducing both the risk of further transmission events and the virological burden in secondary infected individuals. Consequently, it leads to a larger reduction in the number of infections, in particular in household of size greater than 4, where several transmission chains are more likely to occur (see S2C and S2D Fig). Because only about 25% of households are of size ≥ 4 in France, the overall effectiveness of this strategy on the number of infections is nonetheless small and remains equal to 20–38% for most cases considered (Fig 5A). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Effectiveness of treatment strategies. (A) Treatment effectiveness on transmission. (B) Treatment effectiveness on virological burden. (C) Number of cases avoided per amount of treatments deployed. All effectiveness are relative to no treatment. https://doi.org/10.1371/journal.pcbi.1012573.g005 Despite its limited effectiveness on virus transmission, treating all household members can be effective to reduce the virological burden. When baseline SAR is high, e.g. SAR > 30%, the effectiveness ranges 45–52% (Fig 5B). This average effectiveness masks important differences across households composition, with larger effectiveness in households of size ≥ 4 (see S3A and S3B Fig). Preempting symptom development in community contacts largely improves the effectiveness of antiviral treatment strategies We then investigate a strategy in which the index case is treated before symptom onset as post-exposure prophylaxis (Ipep). Because treatment is initiated before peak viral load, most transmission events are averted. Consequently, this strategy has an effectiveness greater than 75% for most SAR values, and up to 95%, on both the transmission rate and the virological burden (Fig 5), regardless of household size (S3 Fig). Of note the effectiveness of this strategy marks a net decrease for SAR ≥ 80%, as the generation time decreases with SAR (Fig 2E and 2F), thereby increasing the risk of virus transmission before treatment initiation (Fig 5A and 5B). Although the additional benefit of treating all household members on virus is low in general (Fig 5), this strategy is relevant to reduce the virological burden, with effectiveness close to 100%, regardless of SAR and household size (Fig 5A and 5B and S3 Fig). In a cost-effectiveness perspective, treating only the index case before symptom onset is highly efficient (Fig 5C), with a number of cases avoided per treatment unit deployed larger than 1 if SAR is greater than 45%, as compared to a maximum value of 0.27 when the index case is treated after symptom onset (i.e., 4 individuals are treated to avoid 1 infection). The model can be directly used to support the design of clinical trials The modeling framework can also be used to design clinical trials assessing the effectiveness of antiviral strategies (see Methods and S10 and S11 Figs). Focusing on SAR ranging between 20 and 80%, treating only the index case after symptom onset (Icur) would require 170–430 households per arm to evidence a clinical effectiveness on transmission (see Fig 6), and 340–450 households to demonstrate effectiveness on virological burden. Treating all household members (Icur + Hpep) would be more advantageous, requiring 70–240 and 30–100 households to demonstrate a reduction in the transmission rate and the virological burden, respectively. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Number of households (per arm) required to achieve 90% statistical power in 1:1 randomized clinical trial vs placebo (no treatment). (A) Number of households required to demonstrate a reduction in the Final Attack Rate (FAR). (B) Number of households (per arm) required to demonstrate a reduction in the virological burden (AUCVL). https://doi.org/10.1371/journal.pcbi.1012573.g006 Much lower numbers are required when the index case is treated in the pre-symptomatic phase (Ipep or Ipep + Hprep), with less than 25 households needed to demonstrate an effect on the number of transmissions, and less than 20 households to show an effectiveness on the virological burden. Note that these results are obtained in a context where only 25% of households have more than 4 members (French distribution). Focusing only on large households (see S11B Fig) and treating all their members would diminish the number of households required by a factor of 1.3 and 2.3 for Ipep + Hprep and Icur + Hpep strategies, respectively (S9 Fig). Results are robust to changes in SAR expressions and lower levels of antiviral activity As a sensitivity analysis we also considered the case where SAR decreases with household size (see Methods and S1, S4 and S8 Figs). The results obtained with this alternative model were largely similar to those presented above, which is due to the fact that less than 25% of households are of size 4 or more, thereby limiting the impact of large households (see S4 Fig). We also evaluated the impact of a treatment with a lower mean antiviral efficacy, noted ϵ, than what was assumed in our baseline scenario (ϵ = 99%). ϵ = 50% leads to a marked decrease of the effectiveness of all strategies (S5 Fig), while ϵ = 90% shows almost similar effectiveness in Icur strategies than our baseline scenario. However ϵ = 90% leads to a markedly lower effectiveness of Ipep strategies (S5 Fig), reducing the clinical effectiveness from more than 75% in the baseline scenario (see above) to between 40 and 80% for most SAR values. Treatment effectiveness for other patterns of viral kinetics Finally, we generalize our approach to different viral kinetics profiles, with a time to peak viral load occurring at 1, 4 and 7 days post infection (S7 Fig), respectively, and a incubation period that remains equal to 4 days [26]. This allows us to generalize our results to different acute viral infections, where the peak viral load can either precede or coincide with symptom onset, as observed for SARS-CoV-2 or RSV [5], or afterwards, as observed for Influenza and SARS-CoV-1 [34–36]. When the index case is already symptomatic (Icur strategies) and peak viral load occurs before or at symptom onset, the effectiveness on reducing transmission remains low in all cases considered (Fig 7). The effectiveness of Icur strategy increases with the time to peak viral load, with values ranging 0–25% when the peak viral load occurs 3 days before symptom onset and 25–75% when the peak viral load occurs 3 days after symptom onset. In all these scenarios, the effectiveness decreases with SAR and we observe a limited additional benefit of treating all household members (Icur + Hpep). As previously reported in the main scenario, the effectiveness on reducing virological burden is systematically larger than on transmission, in particular when all household members are treated Icur + Hpep, with effectiveness values ranging from 25–50% when peak viral coincides with symptom onset for SAR < 60%, to values up to 75% when peak viral load occurs more than 3 days after symptom onset. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 7. Effectiveness of treatment strategies according to time to peak viral load. Left: effectiveness on transmission; right: effectiveness on the virological burden. Top: index case treated after symptom onset (Icur and Icur + Hpep); bottom: index case treated before symptom onset (Ipep and Ipep + Hprep). All effectiveness are relative to no treatment. https://doi.org/10.1371/journal.pcbi.1012573.g007 The effectiveness of strategies reaching out the index case before symptom onset is undoubtedly better (Ipep strategies), with effectiveness greater than 75% in all scenarios considered, regardless of SAR, as long as peak viral load coincides or occurs after symptom onset. Notably, the effectiveness is very high even if only the index case is treated. This strategy can even be effective in unfavorable cases where peak viral load precedes symptom onset by three days, with effectiveness in the range 50–75% as long as SAR is less than 70%. Of note these results remain almost equivalent when using the alternative model of SAR according to household size (S8 Fig). Discussion Evaluating the global benefit of an antiviral treatment requires not only the quantification of its benefit at the individual level, using virological or clinical endpoints, but also at the population level, in reducing the risk of virus transmission. In the case of acute respiratory viruses, this evaluation is challenging because the risk of transmission is not constant and changes very rapidly over a short period of time, reflecting the dynamic evolution of viral replication. Here, we developed a multi-scale modeling approach that follows viral dynamics at the individual level and relates it to the risk of transmission. Using this framework and applying it to household setting, we could predict the effectiveness of antiviral strategies according to key factors, in particular the virus contagiousness level or what we called the Secondary Attack Rate (SAR), household size, incubation period, time to peak viral load, antiviral efficacy and timing of treatment initiation. Our results show that in the typical conditions of SARS-CoV-2 pre-Omicron and Omicron variants, with a peak viral load that coincides on average with symptom onset, treatment strategies targeting a symptomatic index case will be poorly effective at reducing both the number of secondary infections and the virological burden. Of note, this result is obtained at the population level, where most households are of size 2 or 3, and masks the role of treatment in large households, with four or more individuals where treating all household members is effective to break transmission chains and reduce virological burden (S3 Fig). In general our results show that when the index case is symptomatic, there is a benefit in treating all household members, regardless of their infection status. This strategy can achieve effectiveness greater than 50% when SAR is less than 80% and peak viral load occurs after symptom onset, as possibly observed for Influenza [34]. The effectiveness of strategies targeting individuals when they are exposed contact cases, i.e. the index case is treated before symptom onset, is unambiguously better, with effectiveness mostly larger than 50% in all cases considered. In that framework, the additional benefit of treating all household members (of the exposed contact case) is limited, even though a virtually complete abrogation of the virological burden can theoretically be achieved, regardless of SAR. Identifying and treating index cases before symptom onset is challenging, but is not out of reach. During the last pandemic, most clinical trials focused on outpatients within 5 or 7 days of symptom onset [12], or on pre-exposure prophylaxis, but some studies enrolled high risk contact patients [11, 44]. In future outbreaks, the experience acquired on contact tracing, rapid drug distribution, and implementation of contactless clinical studies [42] will improve the capability to run reactive clinical trials in the community, provided that affordable, safe and oral drugs are available. In that perspective, our results show that strategies targeting the index case are key to achieve a high effectiveness, and could participate to the arsenal of public health measures to reduce the burden of disease, protect most at risk individuals and accordingly reduce the duration of isolation of high risk contacts. This modeling framework can also be used to design clinical trials. As an illustration, we calculated the number of households required to assess effectiveness on either transmission or virological burden, assuming a 1:1 household randomized study between treatment and placebo. Because of their relatively low effectiveness, strategies enrolling only symptomatic index would require about 170–430 households (per arm) to assess effectiveness on transmission. Although these numbers may appear to be high, they remain doable, in particular if considering contactless trial [42]. More sophisticated designs could also be proposed, where the randomization could be at the individual rather than at the household level, or where different treatment strategies could be proposed in index and contact cases. To keep our results easily interpretable we did not consider this possibility here. Such policy would require to address ethical and practical challenges, in particular the risk of pill sharing. Also, our results show that study endpoints focusing on virological burden are systematically more powerful than those focusing on transmission, reducing the number of needed household to less than 100 for SAR greater than 20%. However, a virological endpoint has also practical drawbacks, in particular the requirement to draw repeated PCR tests in all household members. Note that, here as well, focusing on households of size of 4 and larger may be more powerful. Our model makes a number of important assumptions and simplifications that we discuss below. First, while our central scenario relied on typical parameters of SARS-CoV-2 viral dynamics, our approach aims to provide a general understanding of the factors governing treatment efficacy against ARI and is not tailored for one specific virus. Indeed, we explored a variety of viruses, as illustrated by the large range of SAR values covering different possible ranges of virus transmissibility, and the different times to peak viral load, close to other ARI such as Influenza or RSV. Although vaccination or new variants can affect the within-host viral dynamics, this question was not in the scope of this study but the model can easily be adapted to any host-pathogen characteristics by changing within-host parameters. Further, a power law model was used to relate the viral load at time t and the risk of transmission (see Eq (2)). Although this model was successfully used to reproduce the risk of transmission in SARS-CoV-2 [24], and that we here explored a large range of parameter values to encompass different levels of virus transmissibility, this model may not universally apply to all ARI. Such differences could for instance reflect the importance of the types of droplets causing the infection and/or different symptom patterns leading to different functional forms between viral load and transmission. Our model assumes that SAR is independent of household size. Although the results of epidemiological studies are heterogeneous, they generally report that SAR decreases with household size [19, 27–30], possibly reflecting the increased heterogeneity of contacts between members of different ages in large households. We therefore investigated an alternative model in which larger households leads to lower SAR. This assumption did not affect highly our results (see S8 Fig), as less than a third of our households has four or more members in our population. More generally we assumed homogeneous mixing the household, and did not consider more realistic contact patterns. In the future, model refinement relying directly on contact data [37] could be used to modulate the risk of virus acquisition according to individual demographic factors, such as age, type of relationship (siblings, parents) [38–41], or viral dynamic factors [21]. Taking into account individual heterogeneity would also be relevant to evaluate the benefit of antiviral treatments on other endpoints, such as the risk of severe disease, and to adjust for pre-existing immunity and vaccine intake. Another important assumption of the model is that infectiousness is directly related to viral load levels in the nasopharynx, albeit in a non-linear fashion and with large inter-individual variability. Although we and others have shown that higher viral load at the time of a high risk contact is associated with a higher risk of transmission [24, 25], it should be recognized that a formal quantification of the impact of nasopharyngeal viral load on the risk of transmission is still lacking. Likewise, fomites, or simply viral dynamics in other compartments, such as saliva, could be relevant when it comes to household transmission, even though its relationship with infectiousness has not been established [43]. Second, our results emphasize the benefit of strategies focusing on pre-symptomatic index case (Ipep). Treating invididuals in post exposure prophylaxis however increases the risk to treat uninfected individuals, and in turn increases the risk of over-treatment. While we recognize that this may lead to overestimate the cost-effectiveness of Ipep strategies, we note that this could be mitigated by including only individuals with a positive PCR test, as the time between infection and first positive nasal swab PCR test is very short and is between 1 and 3 days for Influenza or SARS-CoV-2 viruses [34, 36, 46]. To conclude our model can be used to quantify and anticipate the clinical effectiveness of antiviral treatment strategies against acute respiratory viruses in households. It provides a novel understanding on the conditions that need to be met, at the pharmacological, virological and behavioural levels, for an antiviral treatment to be effective, and can guide interventions aiming to reduce disease burden during a viral pandemics. Supporting information S1 Text. Viral load dynamic model and calibration. https://doi.org/10.1371/journal.pcbi.1012573.s001 (PDF) S1 Fig. Impact of modified SAR, SARS, on the attack rate. Left figure is the FAR according to SAR in the modified SAR model for the different household sizes (darkest gray for S = 2 to lightest S = 6) and averaged over France household statistics with the non-modified SAR model (yellow) and with the modified one (red). The right figure represents Mean estimate of the household reproductive number, R0, (computed over the French distribution of household sizes) as a function of SAR. R0 represents the mean number of individuals infected directly by the index case in a household. The red line represents the value obtained using the modified SARS model; the yellow line represents the value obtained using the non-modified SAR model. https://doi.org/10.1371/journal.pcbi.1012573.s002 (EPS) S2 Fig. Transmission and virological burden according to household size for different treatment timings. (A,C,E,G) Number of infected individuals per 1000 households. (B,D,F,H) Virological burden. Gray: I0; orange dashed line: Icur; orange line: Icur + Hpep; green dashed line: Ipep; green line: Ipep + Hprep. Results are shown by household sizes, S = 2 (darkest color) to S = 6 (lightest color). https://doi.org/10.1371/journal.pcbi.1012573.s003 (PDF) S3 Fig. Treatment Effectiveness according to household size. (A,D) Treatment effectiveness on transmission. (B,E) Treatment effectiveness on virological burden. (C,F) Number of cases avoided per amount of treatments deployed. Gray: I0; orange dashed line: Icur; orange continuous line: Icur + Hpep; green dashed line: Ipep; green continuous line: Ipep + Hprep. Results are shown by household sizes, S = 2 (darkest color) to S = 6 (lightest color). https://doi.org/10.1371/journal.pcbi.1012573.s004 (EPS) S4 Fig. Treatment Effectiveness according to the different scenarios for the alternative SAR definition SARS. (A) Treatment effectiveness on transmission. (B) Treatment effectiveness on virological burden. https://doi.org/10.1371/journal.pcbi.1012573.s005 (EPS) S5 Fig. Viral dynamics for different antiviral efficacies. Top: ϵ = 50%; bottom: ϵ = 90%. (A,D) Individual viral dynamic profiles predicted by the model in 30 individuals (see Eq (3)), that are either left untreated (I0, gray), treated within 5 days after symptom onset (Icur, orange), or treated symptom onset (Ipep, green). All parameters are given in S1 Table. (B,E) Distribution of the peak viral load predicted by the model. (C,F) Distribution of the time to peak viral load predicted by the model. https://doi.org/10.1371/journal.pcbi.1012573.s006 (EPS) S6 Fig. Effectiveness of treatment strategies depending on the antiviral efficacy. (A,D) Treatment effectiveness on transmission. (B,E) Treatment effectiveness on virological burden. (C,F) Number of cases avoided per amount of treatments deployed. Plain line: ϵ = 99%; Diamond line: ϵ = 90%; Square line: ϵ = 50%; Orange dashed line: Icur; Orange continuous line: Icur + Hpep; Green dashed line: Ipep; Green continuous line: Ipep + Hprep. All effectiveness are relative to no treatment. https://doi.org/10.1371/journal.pcbi.1012573.s007 (EPS) S7 Fig. Viral dynamics and antiviral treatment for mean time to viral load peak 1d and 7d. Individual viral dynamic profiles predicted by the model in 30 individuals (see Eq (1) and S3 Table), that are either left untreated I0 (gray), treated within 5 days after symptom onset Icur (orange), or treated before symptoms onset Ipep (green). All parameters are given in S1 and S3 Tables, and the model assumes that a mean incubation period of 4 days, and a mean treatment antiviral efficacy of 99%. Top: Mean time to viral load peak = 1d. Bottom: Mean time to viral load peak = 7d. https://doi.org/10.1371/journal.pcbi.1012573.s008 (EPS) S8 Fig. Effectiveness of treatment strategies according to time to peak viral load with modified SARS according to household sizes S. Left: effectiveness on transmission; right: effectiveness on the virological burden. Top: index case treated after symptom onset (Icur and Icur + Hpep); bottom: index case treated before symptom onset (Ipep and Ipep + Hprep). All effectiveness are relative to no treatment. https://doi.org/10.1371/journal.pcbi.1012573.s009 (EPS) S9 Fig. Number of households of size 4 and larger required to achieve 90% statistical power in 1:1 randomized clinical trial vs placebo (no treatment). (A) Number of households required to demonstrate a reduction in the Final Attack Rate (FAR). (B) Number of households required to demonstrate a reduction in the virological burden (AUCVL). https://doi.org/10.1371/journal.pcbi.1012573.s010 (EPS) S10 Fig. Distribution of transmission rate (FARh) and virological burden (AUCVLh) for SAR = 40% and different household sizes S. Gray: I0; Orange: Icur + Hpep; Green: Ipep + Hprep. https://doi.org/10.1371/journal.pcbi.1012573.s011 (EPS) S11 Fig. Mean value and standard deviation for AUCVLh. These values were calculated for all household size statistics (A, upper figures), and for households of size 4 and larger (B, bottom figures). Gray: untreated households, and for treated households, Orange dashed line: Icur; Orange continuous line: Icur + Hpep; Green dashed line: Ipep; Green continuous line: Ipep + Hprep. https://doi.org/10.1371/journal.pcbi.1012573.s012 (EPS) S1 Table. Parameter values used to generate viral dynamic profiles and transmission probabilities. https://doi.org/10.1371/journal.pcbi.1012573.s013 (PDF) S2 Table. Proportion of household sizes in France [33]. https://doi.org/10.1371/journal.pcbi.1012573.s014 (PDF) S3 Table. Modified parameters in the viral load model (see Eq (1)) to reproduce time to peak viral load equal to 1 and 7 days. https://doi.org/10.1371/journal.pcbi.1012573.s015 (PDF) S1 Code. Matlab code and code description in README file.pdf. The code is protected by Agence de Protection des Programmes (APP) in France, and we have the permission to share it. https://doi.org/10.1371/journal.pcbi.1012573.s016 (ZIP) Acknowledgments The authors thank France Mentré, Evelina Tacconelli and Fabrice Carrat for providing helpful comments on a previous version of the manuscript.