PLoS Computational Biology

Mittler, John E.;Murphy, James T.;Stansfield, Sarah E.;Peebles, Kathryn;Gottlieb, Geoffrey S.;Abernethy, Neil F.;Reid, Molly C.;Goodreau, Steven M.;Herbeck, Joshua T.

doi: 10.1371/journal.pcbi.1007561pmid: 31846456

Introduction Despite the scale-up of antiretroviral treatment, HIV continues to be a major source of mortality in people ages 15–24 in low- and middle-income (LMIC) countries [1]. This can be traced to high rates of infection [2], low testing rates [3], difficulties linking young people to care [4], and low treatment adherence rates compared to older people [5]. HIV infection is distributed unevenly between the sexes at these ages, with young women having 3- to 5-fold higher rates of infection than young men [2, 6–10]. Higher rates of infection in young women are in part due to higher per-act probabilities of infection [11] and age-disparate relationships with older men [12–14] who are more likely to be HIV positive and less likely to use condoms [15–18]. These gaps in coverage have led to numerous calls to ramp-up testing and treatment of young people [19–20]. Two computational models suggest that youth-focused treatment as prevention (TasP) could reduce HIV incidence and/or increase quality-adjusted life years (QALYs) [21, 22]. These models assume that young people are more likely than older people to have short-term relationships [22–26]. They also include, either implicitly [21] or explicitly [22], the potential for treatment of young people to protect adolescents entering the sexually active population. Neither of these studies, however, makes a particularly strong case for youth-focused TasP. Alsallaq et al. [21] show a small increase in cost-effectiveness of youth-focused treatment parameterized for a specific country (Kenya) and a specific set of cost values. Bershteyn et al. [22] propose a slightly more general model that predicts stronger advantages to age-based TasP; however, they conclude with the pessimistic note that age-targeted TasP is "unlikely to eliminate HIV epidemics.” These models, however, do not account for two age-related risks: age-related declines in coital frequency within relationships [27–30] and higher per-act rates of infection in young people [31]. Also, since people infected with high setpoint viral load (SPVLs) viruses (i.e., more virulent viruses) die earlier in the absence of therapy and since older HIV+ people will, on average, have been infected for a longer time, older HIV+ people may be infected with fewer high SPVL viruses before the start of a TasP campaign. This opens up the possibility that youth-focused TasP, by treating individuals infected with higher SPVLs, could select for lower SPVL viruses, a factor that could reduce incidence and HIV-related mortality over time. Factors that increase risk to youth need to be weighed against factors that protect youth, namely higher rates of condom usage [15–18] and slower progression to AIDS in the absence of therapy compared to older people [32]. Also, the tendency for youth not to have been infected a long time means that youth-focused treatment may treat fewer people who are in the AIDS stage of infection, though this will be offset by a slightly greater number who are in the acute phase of infection. Weighing the importance of these age-related risks and benefits is difficult because they occur in the context of a demographic network with age-dependent relationship durations and complex patterns of age-related homophily (i.e., the tendency of people to partner with people their own age) in which women also tend to partner with older men. Under these conditions, network models predict that the mean degree of the population will depend on the age distribution–a distribution that can, in turn, be profoundly influenced by HIV-related mortality. To address these complexities, we used an agent-based model for HIV epidemiology that accounts for SPVL variation to test the long-term effects of targeting young people for linkage to effective care in a generalized heterosexual epidemic. To keep the focus on age effects, we model a population with risk behaviors characteristic of the general population; that is, we do not explicitly model high-risk groups such as men who have sex with men (MSM), sex workers, and people who inject drugs (PWIDs). The model, which utilizes established routines for the underlying social network dynamics, was parameterized to reproduce broad features of sub-Saharan HIV-1 epidemics. To create a basis for comparison, we compared age-based targeting strategies to untargeted treatment (similar to currently recommended Test and Treat) and to targeting strategies based on CD4 count (historically recommended by the UNAIDS/WHO), viral load, and combinations of CD4 and age. Because of age-disparate relationships, we also tested strategies with sex-specific age targets, hypothesizing that a higher age target for men could improve age-based TasP by protecting younger female partners. For any given strategy and coverage level (i.e., percent of HIV+ people who are virally suppressed after the TasP campaign), we quantified total and time-discounted AIDS deaths over 25 years, person-years of therapy over 25 years, and incidence 20–25 years after the start of the TasP campaign. We also measured the percentage of people who receive therapy specifically as a result of being a member of a target group (i.e., the relative inclusivity/exclusivity of the strategy). Finally, we performed sensitivity analyses in which we removed each of the age-related risk factors and altered parameters for testing rates, dropout rates, background incidence, and the percentage of people who could potentially be linked to care under a vigorous TasP campaign. Methods Software package We used the Evonet_HIV package (https://github.com/EvoNetHIV), a stochastic, agent-based HIV epidemic model that accounts for a broad set of virological, immunological, behavioral, and epidemiological phenomena [33–35]. Each agent has attributes such as age, sex, HIV status, viral load, CD4 count (discretized into 5 bins), and HIV diagnosis status. Partnership lists and agent attributes (such as viral load and CD4 counts) were updated sequentially each day. The formation and dissolution of sexual partnerships were modeled using separable temporal exponential random graph model (STERGM) terms [36]. Since the model accounts for changes in behavior with age (e.g., young people having shorter relationships and preferentially forming attachments with other young people) and infection status (e.g., AIDS patients dying and/or having lower probabilities of sex) the model can be classified as a time-evolving adaptive network. The R package includes modules for HIV testing; viral load changes within hosts; age-dependent relationship durations; age- and viral load-dependent CD4 decline; CD4-dependent probabilities of sex; the effect of treatment on viral load; age- and diagnosis-dependent condom use; and age-, sex-, condom-, circumcision-, and viral load-dependent transmission rates. Different modules (sexual relational formation, viral load updates, etc.) were updated sequentially each time step. Within-host viral dynamics, CD4 progression patterns, probabilities of dying of AIDS, testing procedures, and transmission probabilities were, for the most part, identical to those in previous studies using Evonet_HIV [33–35]. In the sections below, we summarize features that differ from these previous studies and/or that relate specifically to age (i.e., the key assumptions of this study). Additional details about the software are given in S1 Supplementary Methods and at github.com/EvoNetHIV. Data sources We have derived most of the parameters from studies of sub-Saharan countries (mainly South Africa); however, we have used parameters from other regions when they were of significantly better quality and/or more applicable to our model. We give more details about data sources, algorithms, and network estimation procedures in the Supplemental Methods. Age-specific relationship durations To approximate the tendency of young people to have short-term relationships [22–26], we divided the population into two groups: agents in group 1 (primarily composed of young people) tend to form short-term partnerships (default 2 years), agents in group 2 (primarily composed older people) tend to form long-term partnerships (default 10 years). We assumed that 90% of agents entering the sexually active population at age 16 belong to group 1. Agents in group 1 (median age 27) were assumed to have a small probability (default 0.00011 per day = ~5.5% per year) of transitioning to group 2 (median age 44). That is, agents exhibit a tendency to have longer partnerships as they age. These values were chosen so that the mean duration for people under 25 will approximate the midpoint of estimates in refs [23, 37, 38] and so that the ratio of the number partners per unit time for people under 25 to those over 40 is consistent with refs 23–26 (however, see below for caveats). Relationships between people from different groups were assumed to break up each day with probability 1/(365*AveDur), where AveDur is the geometric mean of each partner’s duration tendency. Use of the geometric mean (i.e., the square root of the product) means that relationship durations are determined more strongly by the partner with the shorter relationship tendency. With the parameters given above, relationships will last an average of ~6 years in a typical simulation, with ~40% and ~60% of agents, respectively, belonging to groups 1 and 2. We are aware that our age-specific relationship terms provide but a loose approximation to a broad, but heterogeneous (and potentially unreliable) swath of data. Nguyen et al. [37], for example, show that estimates can vary 3-fold depending on methods used to analyze the data. Fortunately, we are able to show below that our main results do not depend on the exact values. Rather than trying to create a more complex model or trying to fit of this model to additional data, we have elected to stick with these approximate values and then examine the effect of changing the relationship duration terms in sensitivity tests (including tests in relationship durations are assumed to be completely independent of age). Age- and sex-specific relationship formation terms To represent age-disparate relationships, we added a network term that pushes the average age difference between men and women to equal a specified value (default: women partner with men who are, on average, 4 years older [12, 13, 39]). To mimic the tendency of people to partner with others who are about the same age, we added an additional term that forces the average age difference in the population (after adjusting for the average male-female age difference) to equal a specified value (default 4 years). With these default values, 20-year-old women will typically partner with men between 20 and 28. In Fig 1, we give an example age-age plot from one of our simulations. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Ages of sexually active males plotted against the age of their sexually active female partners. While the model allows for partnerships between people of different ages, the majority of people partner with someone who is about the same age after accounting for age-disparate relationships. The red line is a lowess fit to the data. The blue line is the y = x line that would occur if women partnered only with men who were exactly the same age. This graph shows data from a simulation in which treatment had reduced incidence to zero. https://doi.org/10.1371/journal.pcbi.1007561.g001 To increase prevalence in young women relative to young men [26], we also added network terms for the sex-specific cross-sectional prevalence of relational concurrency [40, 41] (defaults: Men 0.24, Women 0.04). Age-dependent probabilities of coitus Following data in refs [25–28], we modeled declining probabilities of coitus, Psex, with age as follows: where AveAge is the average age of the partners and Psex19 (default 0.2 per day) is the probability of having coitus if AveAge = 19. Age- and sex-dependent probabilities of transmission We assumed that women have a higher (default 2-fold) per act probability of infection [11]. Following an adaptation of data from Hughes et al. [31] used in our previous studies [33–35], we also assumed that the relative per-act risk of transmission is higher when the susceptible partner is young (default: RR 1.492 for each decade under 46). Age-dependent progression rates We followed the general strategy used in our previous studies [33–35] in which the initial placement into a CD4 category (1 = CD4>500 units, 2 = 350<CD4<500, 3 = 200<CD4<350, 4 = CD4<200, 5 = died of AIDS) and the daily probability of transitioning to a different CD4 category depend on set point viral load (SPVL) and treatment status. However, we modified the SPVL-dependent probability of progression to a higher CD4 category, in the absence of ART, to reflect the slower disease progression observed in people infected at younger ages [35] (Table 1). Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Effect of age on progression in the absence of therapy. https://doi.org/10.1371/journal.pcbi.1007561.t001 Age-specific condom usage Based on reports of condom usage in developing countries [15–18], we modeled decreasing per act probabilities of using a condom, Pcondom, with age as follows: where Pcondom16 (default 0.8) is the probability that two 16-year-olds will use a condom, Age50 (default 35) is the age at which condom usage drops to 0.5 * Pcondom16, and AveAge is the average age of the partners. Demographic setup HIV-negative people were assumed to enter the sexually active population in our model at age 16 and die of natural causes according to age-specific mortality tables; the arrival rate was set so that the population will increase by ~1% per year in the absence of HIV-induced mortality. We set the initial age distribution so that the age distribution will be stable in the absence of HIV-induced mortality. Because our TERGM-based network model is computationally time-consuming, we set the initial population size (N) to 2,000. Simulations yielding interesting or important results were then re-run with N = 10,000 or 20,000. Since key transmission rate parameters, such as the average number of partners per person, are independent of population size for sexually transmitted diseases [42, 43], our results should generalize to larger populations. Treatment cascade We assume a certain percentage of the population (default 95%) is linkable; i.e., could become virally suppressed in response to a TasP campaign. We assumed that 100% of the linkable population is tested annually but only a subset of diagnosed people, the size of which we vary in our simulations, were linked to effective care, resulting in a suppressed viral load. In our default runs, individuals remained in care indefinitely even if they were originally linked to care due to a factor (e.g., youth) that no longer applies. Rollout of ART prior to targeted TasP campaigns To establish the presence of suppressive ART in the population prior to our targeted TasP campaign, we modeled both gradual and sudden rollout schemes where the number of people receiving suppressive ART increases linearly from zero treated, starting either 9 years (gradual) or 1 year (sudden) prior to the targeted TasP campaign, to where Starg is the percentage of people who will be targeted in response to the subsequent targeted TasP campaign, Ipre is the number of infected people at the start of the pre-TasP rollout. The “/2” in the equation above signifies that the subsequent TasP campaign will roughly double the number of people receiving suppressive therapy in our model. Under both the gradual and sudden pre-TasP rollout schemes, random subsets of agents received suppressive ART. While gradual rollouts are more realistic, sudden rollouts allow for more precise statements about the percentage of people receiving ART at the start of the TasP campaign. That is, under a gradual rollout, the percent of people receiving ART prior to the TasP campaign is a complex function of other dynamic variables (e.g., the age distribution and the percentage of people in the two relationship-length groups). We focused, therefore, on sudden pre-TasP rollouts for summary figures with Starg in the x-axis. Parameter tuning While most of the parameters are based directly on data, we adjusted the maximum probability of infection (default 0.0025), mean degree (default 0.75), and the male- and female-specific concurrency parameters (defaults 0.25 and 0.04) to give prevalence and incidence values and male-to-female infection ratios that fall within ranges seen in sub-Saharan epidemics. [1, 2, 26, 44]. Simulated TasP campaigns Following the pre-TasP ART rollout, we introduced a targeted TasP campaign that increases the number of HIV+ people receiving suppressive ART ~2-fold to StargI0, where I0 is the number of infected people at the start of the TasP campaign. (This will be an exact doubling if the number of infected people does not change between the start of the pre-TasP rollout and the start of the TasP campaign). To allow us to make more precise statements about the percentage of people treated, we stipulated for our default runs that the TasP campaign was implemented instantly (as we did for the pre-TasP rollout). After the TasP target has been hit, the absolute number of individuals receiving suppressive ART was assumed to grow annually at rate r (default 2% per year) to account for population growth and general increases in public health expenditures. That is, we capped the number of people receiving ART at StargI0(1+r)t, where t is the number of years since the start of the TasP campaign. Under the targeted TasP campaigns, different categories of individuals were targeted for linkage to effective care in different scenarios. Targeting strategies included: age (e.g. “under age 25”), immunological status (e.g., “CD4<500”), combinations of age and immunological status (e.g., “Under 25, CD4 < 500”), viral load (e.g., “SPVL”), and no targeting at all (“random”) (Table 2). Several of the strategies include a targeting hierarchy within the primary target range. “Under age 30”, for example, targets those under age 25 first, and then those between ages 25 and 30. In these cases, agents are linked to care at random within each successive target group until the overall treatment limit is reached. To ensure that equal numbers are treated under all strategies (prior to one strategy linking 100% of people to effective care), all strategies included a final “random” (untargeted) component that is applied once all of the people in the target groups have been linked to effective care. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 2. Key TasP strategies modeled in this paper. https://doi.org/10.1371/journal.pcbi.1007561.t002 In the era of test-and-treat, the treatment strategies employed in Sub-Saharan African countries now most closely reflect the randomized treatment model we use in the United States. We included two additional non-random strategies, SPVL- and CD4-based targeting in order to understand how age-based strategies perform against other strategies that have the potential to outperform random (untargeted) TasP. Doing so allows us to separate the general effect of having a structured, targeted campaign from the specific effects of focusing on younger age groups. We are aware that SPVL, as defined in Table 2, has never been used and that CD4-based targeting, while once used, is no longer recommended. We are, furthermore, aware that “CD4<500” and “SPVL”, respectively, assume access to information that will be inconsistently or rarely available. For the simulations with “sudden” pre-TasP rollouts, we quantified impacts as a function of Starg, the percent of infected people linked to effective care in response to the TasP campaign. We note that Starg gives the percentage of HIV+ people receiving suppressive therapy at a single point in time (i.e., just after the TasP target has been hit). For a TasP campaign that succeeds in reducing the number of untreated people, the percentage of HIV+ people receiving ART will exceed Starg as the treatment expands at the annual post-TasP rate, r. For a TasP campaign that fails to reduce the number of untreated people, the percentage of HIV+ people receiving ART will remain at or fall below Starg depending on how fast the epidemic expands compared to the post-TasP expansion rate r. For each value of Starg, we quantified the impact of TasP campaigns using: mean incidence 20–25 years after campaign initiation; the total number of AIDS deaths and person-years of therapy during the first 25 years of the campaign; the percentage of the population initiating suppressive ART that was not a part of a target group; and the percentage of untargeted infected people receiving suppressive ART. Since strategies that do not suppress viral load in a significant percentage of people outside the target group could be considered unethical, we used the two last measures as screening tools to help us decide which strategies to investigate in more detail. For strategies for which we noted a conflict between short- and long-term AIDS death rates, we calculated time-discounted AIDS deaths—a measure that quantifies the tendency of people to prefer strategies that minimize deaths in current years over strategies that minimize deaths in later years. We also considered a gradual version of the TasP rollout, in which the absolute number of people being treated increased linearly (without any cap) beginning 9 years before the start of the targeted TasP campaign. To prevent 100% of newly treated people belonging to a target group in the first few years of the TasP campaign, we modified these gradual simulations so that 40% of newly treated people were automatically treated at random and the other 60% were treated according to the targeting rules in Table 2 after the start of the TasP campaign. This capped the percentage of newly treated people who were treated specifically because they belonged to the target group at 60% of HIV+ people receiving ART. Sensitivity analyses To test the dependence of our results to parameter values, we conducted a series of sensitivity analyses in which we altered or removed parameters. For these sensitivity analyses, we focused on the TasP target (Starg value) needed to reduce long-term incidence (i.e., incidence 20–25 years after the start of the TasP campaign) 20-fold compared to a no-ART control. In preliminary work, we found that this 20-fold reduction allows for more meaningful comparisons than an absolute cutoff (e.g., incidence less than 0.1 infections/100 person-years) because parameter changes can dramatically change the baseline incidence rate. Given our 2% annual post-TasP increase in viral suppression, we found in preliminary work that a 20-fold reduction in incidence led to eventual viral eradication in all of the scenarios studied here. For sensitivity tests in which the parameter perturbation resulted in a substantially weakened epidemic, we increased an unrelated parameter (baseline transmission rate) to create a more realistic epidemic and reduce run-to-run variation. As a part of these sensitivity analyses, we explored the effect of allowing 10% of treated individuals to discontinue treatment each year [46–49]. We did not include this in the default model because members of a target group who discontinue ART are more likely to be relinked to care than those who are not. This effect could, in principle, cycle an increasing percentage of people into the target group into ART as people outside the target group drop out. Statistics and replication Error bars in the figures represent standard deviations using n-1 degrees of freedom where n is the number of replicates. Experiments shown in the figures and tables were replicated either 16 (default) or 32 (marked by ++ signs in the summary table below) times. For some of the experiments described in the text, we replicated the simulation 120 times. Software package We used the Evonet_HIV package (https://github.com/EvoNetHIV), a stochastic, agent-based HIV epidemic model that accounts for a broad set of virological, immunological, behavioral, and epidemiological phenomena [33–35]. Each agent has attributes such as age, sex, HIV status, viral load, CD4 count (discretized into 5 bins), and HIV diagnosis status. Partnership lists and agent attributes (such as viral load and CD4 counts) were updated sequentially each day. The formation and dissolution of sexual partnerships were modeled using separable temporal exponential random graph model (STERGM) terms [36]. Since the model accounts for changes in behavior with age (e.g., young people having shorter relationships and preferentially forming attachments with other young people) and infection status (e.g., AIDS patients dying and/or having lower probabilities of sex) the model can be classified as a time-evolving adaptive network. The R package includes modules for HIV testing; viral load changes within hosts; age-dependent relationship durations; age- and viral load-dependent CD4 decline; CD4-dependent probabilities of sex; the effect of treatment on viral load; age- and diagnosis-dependent condom use; and age-, sex-, condom-, circumcision-, and viral load-dependent transmission rates. Different modules (sexual relational formation, viral load updates, etc.) were updated sequentially each time step. Within-host viral dynamics, CD4 progression patterns, probabilities of dying of AIDS, testing procedures, and transmission probabilities were, for the most part, identical to those in previous studies using Evonet_HIV [33–35]. In the sections below, we summarize features that differ from these previous studies and/or that relate specifically to age (i.e., the key assumptions of this study). Additional details about the software are given in S1 Supplementary Methods and at github.com/EvoNetHIV. Data sources We have derived most of the parameters from studies of sub-Saharan countries (mainly South Africa); however, we have used parameters from other regions when they were of significantly better quality and/or more applicable to our model. We give more details about data sources, algorithms, and network estimation procedures in the Supplemental Methods. Age-specific relationship durations To approximate the tendency of young people to have short-term relationships [22–26], we divided the population into two groups: agents in group 1 (primarily composed of young people) tend to form short-term partnerships (default 2 years), agents in group 2 (primarily composed older people) tend to form long-term partnerships (default 10 years). We assumed that 90% of agents entering the sexually active population at age 16 belong to group 1. Agents in group 1 (median age 27) were assumed to have a small probability (default 0.00011 per day = ~5.5% per year) of transitioning to group 2 (median age 44). That is, agents exhibit a tendency to have longer partnerships as they age. These values were chosen so that the mean duration for people under 25 will approximate the midpoint of estimates in refs [23, 37, 38] and so that the ratio of the number partners per unit time for people under 25 to those over 40 is consistent with refs 23–26 (however, see below for caveats). Relationships between people from different groups were assumed to break up each day with probability 1/(365*AveDur), where AveDur is the geometric mean of each partner’s duration tendency. Use of the geometric mean (i.e., the square root of the product) means that relationship durations are determined more strongly by the partner with the shorter relationship tendency. With the parameters given above, relationships will last an average of ~6 years in a typical simulation, with ~40% and ~60% of agents, respectively, belonging to groups 1 and 2. We are aware that our age-specific relationship terms provide but a loose approximation to a broad, but heterogeneous (and potentially unreliable) swath of data. Nguyen et al. [37], for example, show that estimates can vary 3-fold depending on methods used to analyze the data. Fortunately, we are able to show below that our main results do not depend on the exact values. Rather than trying to create a more complex model or trying to fit of this model to additional data, we have elected to stick with these approximate values and then examine the effect of changing the relationship duration terms in sensitivity tests (including tests in relationship durations are assumed to be completely independent of age). Age- and sex-specific relationship formation terms To represent age-disparate relationships, we added a network term that pushes the average age difference between men and women to equal a specified value (default: women partner with men who are, on average, 4 years older [12, 13, 39]). To mimic the tendency of people to partner with others who are about the same age, we added an additional term that forces the average age difference in the population (after adjusting for the average male-female age difference) to equal a specified value (default 4 years). With these default values, 20-year-old women will typically partner with men between 20 and 28. In Fig 1, we give an example age-age plot from one of our simulations. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Ages of sexually active males plotted against the age of their sexually active female partners. While the model allows for partnerships between people of different ages, the majority of people partner with someone who is about the same age after accounting for age-disparate relationships. The red line is a lowess fit to the data. The blue line is the y = x line that would occur if women partnered only with men who were exactly the same age. This graph shows data from a simulation in which treatment had reduced incidence to zero. https://doi.org/10.1371/journal.pcbi.1007561.g001 To increase prevalence in young women relative to young men [26], we also added network terms for the sex-specific cross-sectional prevalence of relational concurrency [40, 41] (defaults: Men 0.24, Women 0.04). Age-dependent probabilities of coitus Following data in refs [25–28], we modeled declining probabilities of coitus, Psex, with age as follows: where AveAge is the average age of the partners and Psex19 (default 0.2 per day) is the probability of having coitus if AveAge = 19. Age- and sex-dependent probabilities of transmission We assumed that women have a higher (default 2-fold) per act probability of infection [11]. Following an adaptation of data from Hughes et al. [31] used in our previous studies [33–35], we also assumed that the relative per-act risk of transmission is higher when the susceptible partner is young (default: RR 1.492 for each decade under 46). Age-dependent progression rates We followed the general strategy used in our previous studies [33–35] in which the initial placement into a CD4 category (1 = CD4>500 units, 2 = 350<CD4<500, 3 = 200<CD4<350, 4 = CD4<200, 5 = died of AIDS) and the daily probability of transitioning to a different CD4 category depend on set point viral load (SPVL) and treatment status. However, we modified the SPVL-dependent probability of progression to a higher CD4 category, in the absence of ART, to reflect the slower disease progression observed in people infected at younger ages [35] (Table 1). Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Effect of age on progression in the absence of therapy. https://doi.org/10.1371/journal.pcbi.1007561.t001 Age-specific condom usage Based on reports of condom usage in developing countries [15–18], we modeled decreasing per act probabilities of using a condom, Pcondom, with age as follows: where Pcondom16 (default 0.8) is the probability that two 16-year-olds will use a condom, Age50 (default 35) is the age at which condom usage drops to 0.5 * Pcondom16, and AveAge is the average age of the partners. Demographic setup HIV-negative people were assumed to enter the sexually active population in our model at age 16 and die of natural causes according to age-specific mortality tables; the arrival rate was set so that the population will increase by ~1% per year in the absence of HIV-induced mortality. We set the initial age distribution so that the age distribution will be stable in the absence of HIV-induced mortality. Because our TERGM-based network model is computationally time-consuming, we set the initial population size (N) to 2,000. Simulations yielding interesting or important results were then re-run with N = 10,000 or 20,000. Since key transmission rate parameters, such as the average number of partners per person, are independent of population size for sexually transmitted diseases [42, 43], our results should generalize to larger populations. Treatment cascade We assume a certain percentage of the population (default 95%) is linkable; i.e., could become virally suppressed in response to a TasP campaign. We assumed that 100% of the linkable population is tested annually but only a subset of diagnosed people, the size of which we vary in our simulations, were linked to effective care, resulting in a suppressed viral load. In our default runs, individuals remained in care indefinitely even if they were originally linked to care due to a factor (e.g., youth) that no longer applies. Rollout of ART prior to targeted TasP campaigns To establish the presence of suppressive ART in the population prior to our targeted TasP campaign, we modeled both gradual and sudden rollout schemes where the number of people receiving suppressive ART increases linearly from zero treated, starting either 9 years (gradual) or 1 year (sudden) prior to the targeted TasP campaign, to where Starg is the percentage of people who will be targeted in response to the subsequent targeted TasP campaign, Ipre is the number of infected people at the start of the pre-TasP rollout. The “/2” in the equation above signifies that the subsequent TasP campaign will roughly double the number of people receiving suppressive therapy in our model. Under both the gradual and sudden pre-TasP rollout schemes, random subsets of agents received suppressive ART. While gradual rollouts are more realistic, sudden rollouts allow for more precise statements about the percentage of people receiving ART at the start of the TasP campaign. That is, under a gradual rollout, the percent of people receiving ART prior to the TasP campaign is a complex function of other dynamic variables (e.g., the age distribution and the percentage of people in the two relationship-length groups). We focused, therefore, on sudden pre-TasP rollouts for summary figures with Starg in the x-axis. Parameter tuning While most of the parameters are based directly on data, we adjusted the maximum probability of infection (default 0.0025), mean degree (default 0.75), and the male- and female-specific concurrency parameters (defaults 0.25 and 0.04) to give prevalence and incidence values and male-to-female infection ratios that fall within ranges seen in sub-Saharan epidemics. [1, 2, 26, 44]. Simulated TasP campaigns Following the pre-TasP ART rollout, we introduced a targeted TasP campaign that increases the number of HIV+ people receiving suppressive ART ~2-fold to StargI0, where I0 is the number of infected people at the start of the TasP campaign. (This will be an exact doubling if the number of infected people does not change between the start of the pre-TasP rollout and the start of the TasP campaign). To allow us to make more precise statements about the percentage of people treated, we stipulated for our default runs that the TasP campaign was implemented instantly (as we did for the pre-TasP rollout). After the TasP target has been hit, the absolute number of individuals receiving suppressive ART was assumed to grow annually at rate r (default 2% per year) to account for population growth and general increases in public health expenditures. That is, we capped the number of people receiving ART at StargI0(1+r)t, where t is the number of years since the start of the TasP campaign. Under the targeted TasP campaigns, different categories of individuals were targeted for linkage to effective care in different scenarios. Targeting strategies included: age (e.g. “under age 25”), immunological status (e.g., “CD4<500”), combinations of age and immunological status (e.g., “Under 25, CD4 < 500”), viral load (e.g., “SPVL”), and no targeting at all (“random”) (Table 2). Several of the strategies include a targeting hierarchy within the primary target range. “Under age 30”, for example, targets those under age 25 first, and then those between ages 25 and 30. In these cases, agents are linked to care at random within each successive target group until the overall treatment limit is reached. To ensure that equal numbers are treated under all strategies (prior to one strategy linking 100% of people to effective care), all strategies included a final “random” (untargeted) component that is applied once all of the people in the target groups have been linked to effective care. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 2. Key TasP strategies modeled in this paper. https://doi.org/10.1371/journal.pcbi.1007561.t002 In the era of test-and-treat, the treatment strategies employed in Sub-Saharan African countries now most closely reflect the randomized treatment model we use in the United States. We included two additional non-random strategies, SPVL- and CD4-based targeting in order to understand how age-based strategies perform against other strategies that have the potential to outperform random (untargeted) TasP. Doing so allows us to separate the general effect of having a structured, targeted campaign from the specific effects of focusing on younger age groups. We are aware that SPVL, as defined in Table 2, has never been used and that CD4-based targeting, while once used, is no longer recommended. We are, furthermore, aware that “CD4<500” and “SPVL”, respectively, assume access to information that will be inconsistently or rarely available. For the simulations with “sudden” pre-TasP rollouts, we quantified impacts as a function of Starg, the percent of infected people linked to effective care in response to the TasP campaign. We note that Starg gives the percentage of HIV+ people receiving suppressive therapy at a single point in time (i.e., just after the TasP target has been hit). For a TasP campaign that succeeds in reducing the number of untreated people, the percentage of HIV+ people receiving ART will exceed Starg as the treatment expands at the annual post-TasP rate, r. For a TasP campaign that fails to reduce the number of untreated people, the percentage of HIV+ people receiving ART will remain at or fall below Starg depending on how fast the epidemic expands compared to the post-TasP expansion rate r. For each value of Starg, we quantified the impact of TasP campaigns using: mean incidence 20–25 years after campaign initiation; the total number of AIDS deaths and person-years of therapy during the first 25 years of the campaign; the percentage of the population initiating suppressive ART that was not a part of a target group; and the percentage of untargeted infected people receiving suppressive ART. Since strategies that do not suppress viral load in a significant percentage of people outside the target group could be considered unethical, we used the two last measures as screening tools to help us decide which strategies to investigate in more detail. For strategies for which we noted a conflict between short- and long-term AIDS death rates, we calculated time-discounted AIDS deaths—a measure that quantifies the tendency of people to prefer strategies that minimize deaths in current years over strategies that minimize deaths in later years. We also considered a gradual version of the TasP rollout, in which the absolute number of people being treated increased linearly (without any cap) beginning 9 years before the start of the targeted TasP campaign. To prevent 100% of newly treated people belonging to a target group in the first few years of the TasP campaign, we modified these gradual simulations so that 40% of newly treated people were automatically treated at random and the other 60% were treated according to the targeting rules in Table 2 after the start of the TasP campaign. This capped the percentage of newly treated people who were treated specifically because they belonged to the target group at 60% of HIV+ people receiving ART. Sensitivity analyses To test the dependence of our results to parameter values, we conducted a series of sensitivity analyses in which we altered or removed parameters. For these sensitivity analyses, we focused on the TasP target (Starg value) needed to reduce long-term incidence (i.e., incidence 20–25 years after the start of the TasP campaign) 20-fold compared to a no-ART control. In preliminary work, we found that this 20-fold reduction allows for more meaningful comparisons than an absolute cutoff (e.g., incidence less than 0.1 infections/100 person-years) because parameter changes can dramatically change the baseline incidence rate. Given our 2% annual post-TasP increase in viral suppression, we found in preliminary work that a 20-fold reduction in incidence led to eventual viral eradication in all of the scenarios studied here. For sensitivity tests in which the parameter perturbation resulted in a substantially weakened epidemic, we increased an unrelated parameter (baseline transmission rate) to create a more realistic epidemic and reduce run-to-run variation. As a part of these sensitivity analyses, we explored the effect of allowing 10% of treated individuals to discontinue treatment each year [46–49]. We did not include this in the default model because members of a target group who discontinue ART are more likely to be relinked to care than those who are not. This effect could, in principle, cycle an increasing percentage of people into the target group into ART as people outside the target group drop out. Statistics and replication Error bars in the figures represent standard deviations using n-1 degrees of freedom where n is the number of replicates. Experiments shown in the figures and tables were replicated either 16 (default) or 32 (marked by ++ signs in the summary table below) times. For some of the experiments described in the text, we replicated the simulation 120 times. Results Epidemic Our base model gives a pre-TasP epidemic that reproduces broad features of sub-Saharan African epidemics [1, 44], namely: an initial prevalence of ~10% and a decline in incidence, though not necessarily prevalence, following a ramp-up in treatment (Fig 2A and 2B, years -0 to 10). Prior to the TasP campaign reducing incidence, the model predicts 2- to 3-fold higher prevalence in young women than young men and higher prevalence in people between 25 and 50 than in people under 25 or over 50, as reported by Shisana et al. [26]. In our model, women under 30 tend to get infected by older men (S1 Fig), while men over 30 tend to get infected by younger women (S2 Fig). Importantly, our model predicts that adolescents will typically get infected by partners who are under 30 (S1 Fig). This suggests that treating people under 30 could help to protect uninfected adolescents. Consistent with data in ref [50], our base model predicts a median inter-infection time of ~2.4 years (S3 Fig). Although considerably lower than it is in the presence of a very-high-risk group (see the section on sensitivity analyses below), our baseline model has a greater-than-random percentage of agents with many partners (S4 Fig). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Simulations showing the percent of the population that is HIV+ (panels a-c, red lines), HIV+ and receiving treatment (panels a-c, blue lines), and HIV+ and receiving treatment specifically due to being part of a target group (panels a-c, purple lines); incidence (panels d-f); and AIDS death rates (panels g-h) following our “CD4<500” (left-side), “random” (middle), and “under age 30” (right side) treatment-as-prevention (TasP) strategies. To account for existing treatment, we assume a linear increase in the number of people receiving suppressive therapy beginning nine years before the TasP campaign. The TasP campaign immediately increases the percentage of HIV+ people receiving therapy to ~60%. Once the TasP campaign starts, the model uses the CD4-, random- or age-based strategies to link a subset of unsuppressed diagnosed people to care. The 2%/yr increase in the number treated after the campaign reflects population growth and generalized increases in health care efficiency or expenditures. Thick lines give the mean of 16 independent replicates; thin dashed lines show individual runs. The decline in prevalence of treated people after year 15 in panel c (blue lines) occurs because prevalence decreases once all infected agents are treated. The black lines in panels h and i give the means from the CD4<500 simulations in panel g (to highlight differences between short- versus long-term effects of “CD4<500” and “Under age 30”). For these simulations, we set the initial population size to 20,000 to reduce run-to-run variation. https://doi.org/10.1371/journal.pcbi.1007561.g002 Advantages to youth-focused TasP We compared age-based TasP strategies to untargeted TasP (random), a theoretical strategy (SPVL-based) and a historical (CD4-based) strategy (see Table 2 for definitions). In Fig 2 we show the number of infected, the number treated, incidence, and AIDS death rates under the “CD4<500”, “random”, and “under age 30” targeting strategies in a simulation in which ~60% of infected people received suppressive therapy as a result of the TasP campaign. In contrast to the “CD4<500” strategy and random (untargeted) TasP, where the final mean incidences were ~0.65 and ~0.3, respectively, incidence in the “under age 30” strategy dropped to less than 0.1 infections/100 person-years in all (16/16) replicate simulations ~15 years after the start of the TasP campaign. The increase in prevalence under the "CD4<500" targeting strategy in Fig 2A is due primarily to HIV-infected people living longer and the failure of this strategy to reduce incidence to zero. However, we note that demographic shifts (AIDS deaths reducing the number of people between ages 30 and 50) also contribute to increasing prevalence in this and other simulations. This attrition increases the proportion of people who have a higher risk of getting infected (i.e., young adults) in the first 20 years of the TasP campaign. Also, because our baseline model includes comparatively favorable conditions for the spread of virus, the average SPVL increases by about ~0.1 logs between years 0 and 20 (due to a well-known evolutionary tradeoff verified in earlier versions of our software [34]). This small increase in the average SPVL contributes somewhat to the increasing prevalence during this period. The purple lines in Fig 2A and 2C give the percent of the population that was treated specifically due to their having CD4 counts below 500 and being under age 30, respectively. For the “Under age 30” strategy, the percent of treated people who were treated specifically due to being <30 peaked at ~38% after ~6 years, then declined slowly to ~32% at year 15 (after which ~100% of people were virally suppressed). For the “CD4<500” strategy, the percent of treated people who were treated specifically due to having a low CD4 count increased monotonically from ~44% at year 1 to ~83% at year 25. This indicates that the “under age 30” strategy succeeded in reducing incidence despite being significantly less exclusionary that the historical “CD4<500” strategy. Fig 3 presents results for a broader range of strategies and for multiple values of Starg (percent infected people linked to effective care in response to the TasP campaign) demonstrating the range of scenarios over which age-based strategies convey an advantage. Fig 3A (average incidence 20–25 years after implementing the TasP campaign) shows that Starg had to be set to 70% and 80%, respectively, to reduce incidence rates to negligible levels for the “random” and “CD4<500” targeting strategies. Under the age-based targeting strategies, by contrast, incidence could be reduced to negligible levels using Starg values of 40% to 60%. Age-based strategies also resulted in fewer AIDS deaths over 25 years than random or “CD4<500” (Fig 3B). The SPVL strategy gave results comparable to the age-based strategies for Starg > = 50%, but somewhat worse results for Starg < 50%. With the exception of the “CD4<500” strategy, which resulted in more person-years of therapy for Starg between 60 and 80%, the total number of person-years of treatment over the course of the simulation was similar for all strategies (Fig 3C). This shows that the success of the age-based strategies is not due to these strategies treating more people. In particular, it shows that this success is not due to the longer lifespan of younger people starting therapy (relative to old people starting therapy) increasing the number of person years of therapy. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Effect of targeting strategy and the TasP target (Starg) on: (a) incidence 20–25 years after the TasP campaign, (b) AIDS deaths between years 0 and 25, (c) person-years of therapy (included to demonstrate that the age-based strategies did not inadvertently result in more people being treated), and (d) the percentage of HIV+ people initiating treatment at the start of the TasP campaign who were not a member of a target group. For random (untargeted) TasP, the values in panel d will always be 100% (data omitted from graph). The apparent decline in panel d between 90% and 100%, a decline not seen in other experiments, reflects statistical noise accentuated by the fact that only 95% of the population is linkable in this simulation (i.e., Starg = 100% translating to 95% suppression). Each point is the mean of 16 replicates. Bars give standard deviations (SDs). For normally distributed data, 95% confidence intervals would be ~55% the width [since t0.025,15 *SD/sqrt(15) = ~0.55*SD]. For this simulation, we set the initial population size to 10,000. In this and subsequent figures we assumed sudden pre-TasP rollouts so that the TasP campaign will roughly double the number of virally suppressed people. https://doi.org/10.1371/journal.pcbi.1007561.g003 Fig 3A and 3B also show that the benefit of age-based targeting depends on the percentage of the overall population that can be virally suppressed. For high levels of suppression (i.e., Starg > = 80%), the advantage to age-based targeting largely disappears. A similar effect can be seen using our alternative (gradual) rollout strategy in which the number of people being treated increases linearly with time: age-based targeting is highly effective when rollouts are slow or moderate (S5 Fig, left and middle columns); however, the advantage to age-based targeting disappears when treatment ramps up quickly (S5 Fig, right column). We note that Figs 2 and 3 assume that 95% of any given target group could be reached during the treatment campaign. We kept this value high in order to demonstrate the underlying potential of age-based TasP. In the Sensitivity Analysis section below, we report on what happens when this percentage is altered. Inclusivity / exclusivity of strategies A targeting strategy that focuses the majority of resources on a single group is not an acceptable strategy due to the ethical issues of potentially leaving a proportion of the population without access to care. Of the five “non-random” strategies in Fig 3, the percentage of the HIV+ population initiating suppressive therapy at the start of the TasP campaign that was not a member of a target group was highest under the “under age 25” and “under age 30” strategies (Fig 3D). For Starg = 60% and the “under age 25” strategy, for example, ~87% of people initiating suppressive therapy at the start of TasP campaign fell outside the target group—a result that can be attributed to there being relatively few HIV+ people under the age of 25. For the “under age 30”, “under age 35”, “SPVL”, and “CD4<500” strategies, the percentages outside the targeted group were 58%, 35%, 7% and 0%, respectively (Fig 3D). If we were to include people who were virally suppressed prior to the TasP campaign, the percentage of treated people who were not a member of the target group for the “under age 25”, “under age 30” “under age 35”, “SPVL”, and “CD4<500” strategies, would jump slightly (to ~90%, ~70%, ~50%, ~25%, and ~10%, respectively). In subsequent simulations, we focus mainly on comparing the “under age 25” and “under age 30” strategies and variants thereof with the “random” and/or “CD4<500” strategies. The y-axis in Fig 3D is similar to Starg in that it quantifies “inclusivity” at a single point in time (i.e., just after the TasP target has been hit). For TasP campaigns that reduced prevalence, the percentage of untargeted infected individuals receiving suppressive therapy increased over time (as can be inferred from the narrowing difference between the red and blue lines in Fig 2C). However, for the TasP campaigns that failed to reduce prevalence, the percentage of untargeted individuals receiving suppressive therapy either decreased or remained roughly constant (Fig 2C before prevalence flattened around year 20) or in subsequent years. In other words, the successful TasP campaigns became more inclusionary over time, while unsuccessful TasP campaigns either became more exclusionary or retained similar levels of inclusivity over time. Fortunately, our simulations suggest that one will not have to wait long to find out whether a campaign will succeed. For the successful “under age 30” strategy shown in Fig 2, for example, we observed a ~2-fold drop in incidence within 2 years of starting the TasP campaign. While drops in incidence are unlikely to be as rapid under more realistic scale-ups, incidence is a highly sensitive measure that can, with the aid of an epidemiological model, predict whether an in-progress TasP campaign is likely to reduce prevalence. In cases where incidence cannot easily be calculated, age-based TasP can be assessed from HIV prevalence in cohorts of young people who become sexually active after starting the TasP campaign. For our “under age 30” strategy in Fig 2, for example, we observed a 4.8-fold drop in prevalence in those under 25 six years into the TasP campaign. Increases in Starg above a specific threshold (given by points in Fig 3D in which the lines intersect the x-axis) result in treatment being directed almost entirely to people outside the target group. For the “under age 25” and “under age 30” strategies we observed continued benefits (e.g., reductions in AIDS deaths) as Starg increased beyond this threshold (Fig 3A and 3B). For the “under age 25” strategy in Fig 3A, for example, we observed lower incidence and fewer AIDS deaths for Starg = 20% than for Starg = 10% (the value of Starg at which all reachable agents under the age of 25 have been linked to care). While not surprising, this is important for broader discussions about age-based TasP (see last paragraph of discussion). Time-discounted AIDS death rates While age-based targeting greatly reduces the number of deaths over the long-term, it comes at the cost of slightly more AIDS deaths relative to CD4-based targeting in the years immediately following the TasP campaign (as illustrated in Fig 2I). This tradeoff can be quantified using time-discounted AIDS deaths; i.e., a measure of AIDS deaths that gives less weight to future deaths than current deaths. In S6 Fig, we show that, for discount rates between 0 and 7% per year, age-based targeting reduces time-discounted AIDS deaths compared to the other strategies in Fig 3. Sensitivity analyses To test the extent to which our results depend on parameter values and functional forms, we conducted a series of sensitivity analyses in which we altered parameters identified a priori as being likely to contribute to the success of age-based TasP targeting (see methods for details). In Table 3, we demonstrate continued benefits of age-based targeting when: (i) the probability of transmission was increased; (ii) all agents have the same average relationship duration; (iii) young and old people have the same coital frequency; (iv) young people no longer have a higher per-act risk of infection; (v) the average age difference between male and female partners increased to 8 years; (vi) the average difference in ages after accounting for the male-female age difference has increased to 8 years; (vii) the age-related homophily term was removed in its entirety so that young people no longer preferentially form relationships with other young people; (viii) the percentage of people who could potentially be linked to HIV-1 treatment services was decreased; (viii) testing rates were varied; (ix), treated people have a small per-day probability of discontinuing treatment; (x) all agents have the same SPVL, and (xi) in a triple-perturbation experiment in which relationship durations, transmission rates, and probabilities of coitus were all independent of age (Fig 4 and Table 3, perturbation 14) (though in this case only "Under age 30" and "Under age 35" performed substantially better than untargeted TasP). We also observed benefits to age-based TasP when condom use was independent of age, though this is not surprising since our baseline model assumes that young people use condoms more often. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Performance of age-based TasP in sensitivity experiment in simulations [Table 3, perturbation 14] in which all of the primary age-related risk factors except for age-related homophily were removed (i.e., in a model in which relationship durations, transmission rates, and probabilities were all independent of age). (a) Incidence 20–25 years after the TasP campaign. (b) AIDS deaths between years 0 and 25. (c) Person-years of therapy. (d) Percentage of HIV+ people initiating treatment at the start of the TasP campaign who were not a member of a target group. Error bars present standard deviations based on 32 replicates. https://doi.org/10.1371/journal.pcbi.1007561.g004 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 3. Sensitivity experiments in which we varied key age-related and epidemiological parameters. https://doi.org/10.1371/journal.pcbi.1007561.t003 To keep our focus on normal age-related risks, we did not include a very-high-risk group in our base model. To explore the effect of adding a "very-high-risk" group to our model, we did a series of perturbation experiments in which we manipulated relationship durations for groups 1 and 2, made these groups age-independent, and/or changed the proportion of agents in the two groups. Table 3 (perturbations 3–6) gives some representative results. We found that age-based TasP greatly outperformed random (untargeted) TasP when members of group 1 have very high risks and do not transition to group 2 (Table 3, perturbation 3), as well as when a greater percentage of people belong to group 1 and in which a group 1 has moderately or greatly elevated risks (Table 3, perturbations 4 and 5, respectively). (We note that the coloring in Table 1 understates effects when percentages are high: in perturbations 4 and 5 age-based TasP doubled the number of people who did not have to treated in order to drive incidence to zero.) These perturbations, therefore, show that age-based TasP can be highly effective in a model with a risk group that results in a heavy-tailed partnership-distribution curve (S4 Fig). However, when we pushed incidence even higher by reducing relationship durations in both groups and increasing the proportion of agents in group 1 (Table 3, perturbation 6), we found that none of the strategies could reduce incidence to zero despite 100% of eligible agents (i.e., 95% of all agents) receiving suppressive therapy. While incidence was unrealistically high, this establishes that TasP (age-based or not) is not assured of controlling the epidemic. Age-based TasP also failed to provide an advantage over untargeted TasP in a “zero age risks” perturbation in which we eliminated age-related homophily and made relationship durations, transmission rates, and probabilities of coitus independent of age (Table 3, perturbation 15). While not surprising, this provides re-assurance that age-based TasP did not outperform the other strategies due to some unappreciated aspect of our model. Contrary to our initial expectations, SPVL variation had little effect on the efficacy of age-based TasP (Table 3, perturbation 20). We found that untreated HIV+ people under the age of 30 have SPVLs that are roughly double those of untreated HIV+ people over 50, an increase that should translate to ~25% increase in the infectivity of untreated young people. However, this is proved to be fairly small compared to other age-based risks shown to have modest effects in Table 3. The probability of sex, for example, will be roughly halved by age 50, and yet we continued to a large advantage to age-based TasP when the term for age-based coital frequency was removed from the model (Table 3, perturbations 7 and 8). The largest and most informative reductions in the efficacy of age-based TasP relative to random (untargeted) TasP occurred in perturbations in which we removed the age-related homophily term (Table 3, perturbations 13 and 15) and in which we reduced the maximum percentage of people who could be linked to care to 60–70% (Table 3, perturbations 17 and 18 and Fig 5). We note that neither age-based homophily nor overall linkage to care directly affects the susceptibility of a young person. Instead, they affect the probability that a young person’s partners will be infectious. Together these results point to a substantial contribution of age-specific herd immunity (ASHI) to the success of age-based TasP: in order for ASHI to work, a sufficient percentage of the partners of HIV- adolescents entering the sexually active population need to be virally suppressed. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Performance of age-based TasP in simulations [Table 3, perturbation 17] in which only 70% of HIV+ people could be linked to care (i.e., only 70% would get tested and treated under a vigorous treatment campaign). (a) Incidence 20–25 years after the TasP campaign. (b) AIDS deaths between years 0 and 25. (c) Person-years of therapy. (d) Percentage of HIV+ people initiating treatment at the start of the TasP campaign who were not a member of a target group. Error bars present standard deviations based on 32 replicates. https://doi.org/10.1371/journal.pcbi.1007561.g005 While decreasing the maximum percentage of people who could be linked to care reduced the efficacy of age-based TasP, we found it remarkable that age-based TasP could succeed (albeit not as dramatically) in a model in which only 60% to 70% of agents could be linked to care (Fig 5 and Table 3, perturbations 17 and 18). Three factors that contributed to this success. First, the percentage of people who are not linkable decreases over time in the face of HIV-induced mortality. Second, and more interestingly, surviving HIV+ people who are not linkable became less infectious over time due to increases in their average age (since age-based TasP reduces incidence in young people) and reductions in their average viral load (since people infected with high SPVL viruses die sooner in the absence of therapy). In most of the simulations in Table 3 where the value of Starg needed to drive incidence to zero following age-based TasP was the same or only slightly lower than random (untargeted) TasP, we continued to see advantages to age-based TasP for lower values Starg. For Starg < 55 in Fig 5, final incidence and total AIDS deaths continued to be lower for the age-based strategies than for our “random” and “CD4<500” strategies. For some of these simulations, “SPVL” (which doesn’t depend on ASHI) reduced final incidence and AIDS deaths more than "under age 25” for selected Starg values. However, we have not considered “SPVL” further because it is, as defined in Table 2, a theoretical strategy introduced for purposes of comparison. “SPVL” furthermore did not perform as well as “under age 30” in our baseline simulations (Fig 3A and 3B), and is considerably more restrictive (Figs 3D, 4D, and 5D). Hybrid strategies combining age with CD4 status or sex Our analyses have focused so far on strategies that consider age, sex, and CD4 status in isolation. We also studied strategies that integrated these in various combinations. Although we hypothesized that combining age and CD4 status for targeting might yield even better results than either factor in isolation, we did not see any clear advantages to strategies combining “under age 25” and either “CD4<500” or “CD4<200.” (S7 and S8 Figs). [For these simulations we first considered combinations of “Under age 25” and “CD4<500”, but these strategies were so exclusive (S7 Fig) that we switched to “CD4<200”-age hybrids.] CD4-based strategies that utilized age as a secondary criterion performed about the same as those that used only CD4 status, while age-based strategies that utilized CD4 status as a secondary strategy performed about as well as strategies that used only age (S7 and S8 Figs). Although there was a weak trend for “Under 25, CD4<200” to reduce time-discounted AIDS deaths relative to “Under age 25” for Starg between 50 and 60% using a 7% discount rate, CD4-age hybrids did not reduce time-discounted AIDS deaths using the standard 3% discount rate (S9 Fig). These tests, in other words, failed to support our hypothesis that combining age and CD4 status would yield significantly better outcomes compared to straight age-based or CD4-based targeting. Since young women tend to get infected by older men, we hypothesized that including a higher target age for men (e.g., men < 30, women < 25) might enhance the effectiveness of age-based TasP relative to sex-independent cutoffs; however, we did not see any significant advantages of “men <30, women<25” compared to “men <25, women < 30 (S10 Fig). The “men <30, women<25” and “men <25, women < 30” strategies, while considerably better than random, generated incidence rates and AIDS deaths that fell between the “under age 25” or “under age 30” strategies. Epidemic Our base model gives a pre-TasP epidemic that reproduces broad features of sub-Saharan African epidemics [1, 44], namely: an initial prevalence of ~10% and a decline in incidence, though not necessarily prevalence, following a ramp-up in treatment (Fig 2A and 2B, years -0 to 10). Prior to the TasP campaign reducing incidence, the model predicts 2- to 3-fold higher prevalence in young women than young men and higher prevalence in people between 25 and 50 than in people under 25 or over 50, as reported by Shisana et al. [26]. In our model, women under 30 tend to get infected by older men (S1 Fig), while men over 30 tend to get infected by younger women (S2 Fig). Importantly, our model predicts that adolescents will typically get infected by partners who are under 30 (S1 Fig). This suggests that treating people under 30 could help to protect uninfected adolescents. Consistent with data in ref [50], our base model predicts a median inter-infection time of ~2.4 years (S3 Fig). Although considerably lower than it is in the presence of a very-high-risk group (see the section on sensitivity analyses below), our baseline model has a greater-than-random percentage of agents with many partners (S4 Fig). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Simulations showing the percent of the population that is HIV+ (panels a-c, red lines), HIV+ and receiving treatment (panels a-c, blue lines), and HIV+ and receiving treatment specifically due to being part of a target group (panels a-c, purple lines); incidence (panels d-f); and AIDS death rates (panels g-h) following our “CD4<500” (left-side), “random” (middle), and “under age 30” (right side) treatment-as-prevention (TasP) strategies. To account for existing treatment, we assume a linear increase in the number of people receiving suppressive therapy beginning nine years before the TasP campaign. The TasP campaign immediately increases the percentage of HIV+ people receiving therapy to ~60%. Once the TasP campaign starts, the model uses the CD4-, random- or age-based strategies to link a subset of unsuppressed diagnosed people to care. The 2%/yr increase in the number treated after the campaign reflects population growth and generalized increases in health care efficiency or expenditures. Thick lines give the mean of 16 independent replicates; thin dashed lines show individual runs. The decline in prevalence of treated people after year 15 in panel c (blue lines) occurs because prevalence decreases once all infected agents are treated. The black lines in panels h and i give the means from the CD4<500 simulations in panel g (to highlight differences between short- versus long-term effects of “CD4<500” and “Under age 30”). For these simulations, we set the initial population size to 20,000 to reduce run-to-run variation. https://doi.org/10.1371/journal.pcbi.1007561.g002 Advantages to youth-focused TasP We compared age-based TasP strategies to untargeted TasP (random), a theoretical strategy (SPVL-based) and a historical (CD4-based) strategy (see Table 2 for definitions). In Fig 2 we show the number of infected, the number treated, incidence, and AIDS death rates under the “CD4<500”, “random”, and “under age 30” targeting strategies in a simulation in which ~60% of infected people received suppressive therapy as a result of the TasP campaign. In contrast to the “CD4<500” strategy and random (untargeted) TasP, where the final mean incidences were ~0.65 and ~0.3, respectively, incidence in the “under age 30” strategy dropped to less than 0.1 infections/100 person-years in all (16/16) replicate simulations ~15 years after the start of the TasP campaign. The increase in prevalence under the "CD4<500" targeting strategy in Fig 2A is due primarily to HIV-infected people living longer and the failure of this strategy to reduce incidence to zero. However, we note that demographic shifts (AIDS deaths reducing the number of people between ages 30 and 50) also contribute to increasing prevalence in this and other simulations. This attrition increases the proportion of people who have a higher risk of getting infected (i.e., young adults) in the first 20 years of the TasP campaign. Also, because our baseline model includes comparatively favorable conditions for the spread of virus, the average SPVL increases by about ~0.1 logs between years 0 and 20 (due to a well-known evolutionary tradeoff verified in earlier versions of our software [34]). This small increase in the average SPVL contributes somewhat to the increasing prevalence during this period. The purple lines in Fig 2A and 2C give the percent of the population that was treated specifically due to their having CD4 counts below 500 and being under age 30, respectively. For the “Under age 30” strategy, the percent of treated people who were treated specifically due to being <30 peaked at ~38% after ~6 years, then declined slowly to ~32% at year 15 (after which ~100% of people were virally suppressed). For the “CD4<500” strategy, the percent of treated people who were treated specifically due to having a low CD4 count increased monotonically from ~44% at year 1 to ~83% at year 25. This indicates that the “under age 30” strategy succeeded in reducing incidence despite being significantly less exclusionary that the historical “CD4<500” strategy. Fig 3 presents results for a broader range of strategies and for multiple values of Starg (percent infected people linked to effective care in response to the TasP campaign) demonstrating the range of scenarios over which age-based strategies convey an advantage. Fig 3A (average incidence 20–25 years after implementing the TasP campaign) shows that Starg had to be set to 70% and 80%, respectively, to reduce incidence rates to negligible levels for the “random” and “CD4<500” targeting strategies. Under the age-based targeting strategies, by contrast, incidence could be reduced to negligible levels using Starg values of 40% to 60%. Age-based strategies also resulted in fewer AIDS deaths over 25 years than random or “CD4<500” (Fig 3B). The SPVL strategy gave results comparable to the age-based strategies for Starg > = 50%, but somewhat worse results for Starg < 50%. With the exception of the “CD4<500” strategy, which resulted in more person-years of therapy for Starg between 60 and 80%, the total number of person-years of treatment over the course of the simulation was similar for all strategies (Fig 3C). This shows that the success of the age-based strategies is not due to these strategies treating more people. In particular, it shows that this success is not due to the longer lifespan of younger people starting therapy (relative to old people starting therapy) increasing the number of person years of therapy. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Effect of targeting strategy and the TasP target (Starg) on: (a) incidence 20–25 years after the TasP campaign, (b) AIDS deaths between years 0 and 25, (c) person-years of therapy (included to demonstrate that the age-based strategies did not inadvertently result in more people being treated), and (d) the percentage of HIV+ people initiating treatment at the start of the TasP campaign who were not a member of a target group. For random (untargeted) TasP, the values in panel d will always be 100% (data omitted from graph). The apparent decline in panel d between 90% and 100%, a decline not seen in other experiments, reflects statistical noise accentuated by the fact that only 95% of the population is linkable in this simulation (i.e., Starg = 100% translating to 95% suppression). Each point is the mean of 16 replicates. Bars give standard deviations (SDs). For normally distributed data, 95% confidence intervals would be ~55% the width [since t0.025,15 *SD/sqrt(15) = ~0.55*SD]. For this simulation, we set the initial population size to 10,000. In this and subsequent figures we assumed sudden pre-TasP rollouts so that the TasP campaign will roughly double the number of virally suppressed people. https://doi.org/10.1371/journal.pcbi.1007561.g003 Fig 3A and 3B also show that the benefit of age-based targeting depends on the percentage of the overall population that can be virally suppressed. For high levels of suppression (i.e., Starg > = 80%), the advantage to age-based targeting largely disappears. A similar effect can be seen using our alternative (gradual) rollout strategy in which the number of people being treated increases linearly with time: age-based targeting is highly effective when rollouts are slow or moderate (S5 Fig, left and middle columns); however, the advantage to age-based targeting disappears when treatment ramps up quickly (S5 Fig, right column). We note that Figs 2 and 3 assume that 95% of any given target group could be reached during the treatment campaign. We kept this value high in order to demonstrate the underlying potential of age-based TasP. In the Sensitivity Analysis section below, we report on what happens when this percentage is altered. Inclusivity / exclusivity of strategies A targeting strategy that focuses the majority of resources on a single group is not an acceptable strategy due to the ethical issues of potentially leaving a proportion of the population without access to care. Of the five “non-random” strategies in Fig 3, the percentage of the HIV+ population initiating suppressive therapy at the start of the TasP campaign that was not a member of a target group was highest under the “under age 25” and “under age 30” strategies (Fig 3D). For Starg = 60% and the “under age 25” strategy, for example, ~87% of people initiating suppressive therapy at the start of TasP campaign fell outside the target group—a result that can be attributed to there being relatively few HIV+ people under the age of 25. For the “under age 30”, “under age 35”, “SPVL”, and “CD4<500” strategies, the percentages outside the targeted group were 58%, 35%, 7% and 0%, respectively (Fig 3D). If we were to include people who were virally suppressed prior to the TasP campaign, the percentage of treated people who were not a member of the target group for the “under age 25”, “under age 30” “under age 35”, “SPVL”, and “CD4<500” strategies, would jump slightly (to ~90%, ~70%, ~50%, ~25%, and ~10%, respectively). In subsequent simulations, we focus mainly on comparing the “under age 25” and “under age 30” strategies and variants thereof with the “random” and/or “CD4<500” strategies. The y-axis in Fig 3D is similar to Starg in that it quantifies “inclusivity” at a single point in time (i.e., just after the TasP target has been hit). For TasP campaigns that reduced prevalence, the percentage of untargeted infected individuals receiving suppressive therapy increased over time (as can be inferred from the narrowing difference between the red and blue lines in Fig 2C). However, for the TasP campaigns that failed to reduce prevalence, the percentage of untargeted individuals receiving suppressive therapy either decreased or remained roughly constant (Fig 2C before prevalence flattened around year 20) or in subsequent years. In other words, the successful TasP campaigns became more inclusionary over time, while unsuccessful TasP campaigns either became more exclusionary or retained similar levels of inclusivity over time. Fortunately, our simulations suggest that one will not have to wait long to find out whether a campaign will succeed. For the successful “under age 30” strategy shown in Fig 2, for example, we observed a ~2-fold drop in incidence within 2 years of starting the TasP campaign. While drops in incidence are unlikely to be as rapid under more realistic scale-ups, incidence is a highly sensitive measure that can, with the aid of an epidemiological model, predict whether an in-progress TasP campaign is likely to reduce prevalence. In cases where incidence cannot easily be calculated, age-based TasP can be assessed from HIV prevalence in cohorts of young people who become sexually active after starting the TasP campaign. For our “under age 30” strategy in Fig 2, for example, we observed a 4.8-fold drop in prevalence in those under 25 six years into the TasP campaign. Increases in Starg above a specific threshold (given by points in Fig 3D in which the lines intersect the x-axis) result in treatment being directed almost entirely to people outside the target group. For the “under age 25” and “under age 30” strategies we observed continued benefits (e.g., reductions in AIDS deaths) as Starg increased beyond this threshold (Fig 3A and 3B). For the “under age 25” strategy in Fig 3A, for example, we observed lower incidence and fewer AIDS deaths for Starg = 20% than for Starg = 10% (the value of Starg at which all reachable agents under the age of 25 have been linked to care). While not surprising, this is important for broader discussions about age-based TasP (see last paragraph of discussion). Time-discounted AIDS death rates While age-based targeting greatly reduces the number of deaths over the long-term, it comes at the cost of slightly more AIDS deaths relative to CD4-based targeting in the years immediately following the TasP campaign (as illustrated in Fig 2I). This tradeoff can be quantified using time-discounted AIDS deaths; i.e., a measure of AIDS deaths that gives less weight to future deaths than current deaths. In S6 Fig, we show that, for discount rates between 0 and 7% per year, age-based targeting reduces time-discounted AIDS deaths compared to the other strategies in Fig 3. Sensitivity analyses To test the extent to which our results depend on parameter values and functional forms, we conducted a series of sensitivity analyses in which we altered parameters identified a priori as being likely to contribute to the success of age-based TasP targeting (see methods for details). In Table 3, we demonstrate continued benefits of age-based targeting when: (i) the probability of transmission was increased; (ii) all agents have the same average relationship duration; (iii) young and old people have the same coital frequency; (iv) young people no longer have a higher per-act risk of infection; (v) the average age difference between male and female partners increased to 8 years; (vi) the average difference in ages after accounting for the male-female age difference has increased to 8 years; (vii) the age-related homophily term was removed in its entirety so that young people no longer preferentially form relationships with other young people; (viii) the percentage of people who could potentially be linked to HIV-1 treatment services was decreased; (viii) testing rates were varied; (ix), treated people have a small per-day probability of discontinuing treatment; (x) all agents have the same SPVL, and (xi) in a triple-perturbation experiment in which relationship durations, transmission rates, and probabilities of coitus were all independent of age (Fig 4 and Table 3, perturbation 14) (though in this case only "Under age 30" and "Under age 35" performed substantially better than untargeted TasP). We also observed benefits to age-based TasP when condom use was independent of age, though this is not surprising since our baseline model assumes that young people use condoms more often. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Performance of age-based TasP in sensitivity experiment in simulations [Table 3, perturbation 14] in which all of the primary age-related risk factors except for age-related homophily were removed (i.e., in a model in which relationship durations, transmission rates, and probabilities were all independent of age). (a) Incidence 20–25 years after the TasP campaign. (b) AIDS deaths between years 0 and 25. (c) Person-years of therapy. (d) Percentage of HIV+ people initiating treatment at the start of the TasP campaign who were not a member of a target group. Error bars present standard deviations based on 32 replicates. https://doi.org/10.1371/journal.pcbi.1007561.g004 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 3. Sensitivity experiments in which we varied key age-related and epidemiological parameters. https://doi.org/10.1371/journal.pcbi.1007561.t003 To keep our focus on normal age-related risks, we did not include a very-high-risk group in our base model. To explore the effect of adding a "very-high-risk" group to our model, we did a series of perturbation experiments in which we manipulated relationship durations for groups 1 and 2, made these groups age-independent, and/or changed the proportion of agents in the two groups. Table 3 (perturbations 3–6) gives some representative results. We found that age-based TasP greatly outperformed random (untargeted) TasP when members of group 1 have very high risks and do not transition to group 2 (Table 3, perturbation 3), as well as when a greater percentage of people belong to group 1 and in which a group 1 has moderately or greatly elevated risks (Table 3, perturbations 4 and 5, respectively). (We note that the coloring in Table 1 understates effects when percentages are high: in perturbations 4 and 5 age-based TasP doubled the number of people who did not have to treated in order to drive incidence to zero.) These perturbations, therefore, show that age-based TasP can be highly effective in a model with a risk group that results in a heavy-tailed partnership-distribution curve (S4 Fig). However, when we pushed incidence even higher by reducing relationship durations in both groups and increasing the proportion of agents in group 1 (Table 3, perturbation 6), we found that none of the strategies could reduce incidence to zero despite 100% of eligible agents (i.e., 95% of all agents) receiving suppressive therapy. While incidence was unrealistically high, this establishes that TasP (age-based or not) is not assured of controlling the epidemic. Age-based TasP also failed to provide an advantage over untargeted TasP in a “zero age risks” perturbation in which we eliminated age-related homophily and made relationship durations, transmission rates, and probabilities of coitus independent of age (Table 3, perturbation 15). While not surprising, this provides re-assurance that age-based TasP did not outperform the other strategies due to some unappreciated aspect of our model. Contrary to our initial expectations, SPVL variation had little effect on the efficacy of age-based TasP (Table 3, perturbation 20). We found that untreated HIV+ people under the age of 30 have SPVLs that are roughly double those of untreated HIV+ people over 50, an increase that should translate to ~25% increase in the infectivity of untreated young people. However, this is proved to be fairly small compared to other age-based risks shown to have modest effects in Table 3. The probability of sex, for example, will be roughly halved by age 50, and yet we continued to a large advantage to age-based TasP when the term for age-based coital frequency was removed from the model (Table 3, perturbations 7 and 8). The largest and most informative reductions in the efficacy of age-based TasP relative to random (untargeted) TasP occurred in perturbations in which we removed the age-related homophily term (Table 3, perturbations 13 and 15) and in which we reduced the maximum percentage of people who could be linked to care to 60–70% (Table 3, perturbations 17 and 18 and Fig 5). We note that neither age-based homophily nor overall linkage to care directly affects the susceptibility of a young person. Instead, they affect the probability that a young person’s partners will be infectious. Together these results point to a substantial contribution of age-specific herd immunity (ASHI) to the success of age-based TasP: in order for ASHI to work, a sufficient percentage of the partners of HIV- adolescents entering the sexually active population need to be virally suppressed. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Performance of age-based TasP in simulations [Table 3, perturbation 17] in which only 70% of HIV+ people could be linked to care (i.e., only 70% would get tested and treated under a vigorous treatment campaign). (a) Incidence 20–25 years after the TasP campaign. (b) AIDS deaths between years 0 and 25. (c) Person-years of therapy. (d) Percentage of HIV+ people initiating treatment at the start of the TasP campaign who were not a member of a target group. Error bars present standard deviations based on 32 replicates. https://doi.org/10.1371/journal.pcbi.1007561.g005 While decreasing the maximum percentage of people who could be linked to care reduced the efficacy of age-based TasP, we found it remarkable that age-based TasP could succeed (albeit not as dramatically) in a model in which only 60% to 70% of agents could be linked to care (Fig 5 and Table 3, perturbations 17 and 18). Three factors that contributed to this success. First, the percentage of people who are not linkable decreases over time in the face of HIV-induced mortality. Second, and more interestingly, surviving HIV+ people who are not linkable became less infectious over time due to increases in their average age (since age-based TasP reduces incidence in young people) and reductions in their average viral load (since people infected with high SPVL viruses die sooner in the absence of therapy). In most of the simulations in Table 3 where the value of Starg needed to drive incidence to zero following age-based TasP was the same or only slightly lower than random (untargeted) TasP, we continued to see advantages to age-based TasP for lower values Starg. For Starg < 55 in Fig 5, final incidence and total AIDS deaths continued to be lower for the age-based strategies than for our “random” and “CD4<500” strategies. For some of these simulations, “SPVL” (which doesn’t depend on ASHI) reduced final incidence and AIDS deaths more than "under age 25” for selected Starg values. However, we have not considered “SPVL” further because it is, as defined in Table 2, a theoretical strategy introduced for purposes of comparison. “SPVL” furthermore did not perform as well as “under age 30” in our baseline simulations (Fig 3A and 3B), and is considerably more restrictive (Figs 3D, 4D, and 5D). Hybrid strategies combining age with CD4 status or sex Our analyses have focused so far on strategies that consider age, sex, and CD4 status in isolation. We also studied strategies that integrated these in various combinations. Although we hypothesized that combining age and CD4 status for targeting might yield even better results than either factor in isolation, we did not see any clear advantages to strategies combining “under age 25” and either “CD4<500” or “CD4<200.” (S7 and S8 Figs). [For these simulations we first considered combinations of “Under age 25” and “CD4<500”, but these strategies were so exclusive (S7 Fig) that we switched to “CD4<200”-age hybrids.] CD4-based strategies that utilized age as a secondary criterion performed about the same as those that used only CD4 status, while age-based strategies that utilized CD4 status as a secondary strategy performed about as well as strategies that used only age (S7 and S8 Figs). Although there was a weak trend for “Under 25, CD4<200” to reduce time-discounted AIDS deaths relative to “Under age 25” for Starg between 50 and 60% using a 7% discount rate, CD4-age hybrids did not reduce time-discounted AIDS deaths using the standard 3% discount rate (S9 Fig). These tests, in other words, failed to support our hypothesis that combining age and CD4 status would yield significantly better outcomes compared to straight age-based or CD4-based targeting. Since young women tend to get infected by older men, we hypothesized that including a higher target age for men (e.g., men < 30, women < 25) might enhance the effectiveness of age-based TasP relative to sex-independent cutoffs; however, we did not see any significant advantages of “men <30, women<25” compared to “men <25, women < 30 (S10 Fig). The “men <30, women<25” and “men <25, women < 30” strategies, while considerably better than random, generated incidence rates and AIDS deaths that fell between the “under age 25” or “under age 30” strategies. Discussion Within a heterosexual epidemic with ordinary risk behaviors, our model shows that age-based TasP can halve the percentage of people that need to be virally suppressed in order to reduce HIV incidence to negligible levels. Benefits to youth-focused TasP were obtained in scenarios in which each of the age-related risk factors (e.g. shorter relationship durations in young people) were removed, in scenarios where the male-female age difference was doubled, and in scenarios in which only ~60% of people could be linked to care given a sufficiently vigorous TasP campaign. These benefits were not due to age-based TasP treating more people over time. In fact, for TasP campaigns treating more than ~50% of infected people, age-based TasP typically reduced the total number of person-years of therapy relative to untargeted TasP (Figs 3B, 4B and 5B). Sensitivity analyses revealed age-related homophily to be the biggest single driver of the success of youth-focused TasP. Age-related homophily is important because, in its presence, treatment of young people provides age-specific herd immunity (ASHI) that protects adolescents entering the sexually active population (a concept referred to as “ring immunity” by Bershteyn et al. [22]). Protection of these adolescents will, in turn, translate to protection for subsequent cohorts of adolescents entering the sexually active population. Over time, ASHI gives rise to an ever-expanding “AIDS-free generation” that drives HIV to extinction. Our model does not include MSM, PWID, people with very high mean degrees (e.g. sex workers), entry of HIV+ people from other regions, and people infected with drug resistant viruses. We decided against including these groups for two reasons. First, for the sub-Saharan countries with the highest HIV-1 prevalence, HIV is predominately spread via heterosexual contact (thus reducing the need to include MSM and PWID). Second, we wanted to establish the effects of youth-focused TasP in a population with a minimum number of non-age-specific complications. Had we created a comprehensive model that included all of these groups, we would likely have ended with a complex multi-part recommendation that could have obscured the significant and potentially unique role that untreated young people play in sustaining transmission within heterosexual populations with risk characteristics of the general population. That said, we did perform supplemental sensitivity analyses showing that age-based TasP can significantly outperform untargeted TasP in the presence of both an age-independent high-risk group (Table 3, perturbation 3) and an age-dependent high-risk group (Table 3, perturbations 4 and 5). While these perturbation analyses were limited, they show that the model is not overtly sensitive to the presence of high-risk groups. The model also includes sex-specific concurrency terms that cause a higher proportion of men to have concurrent partners than women. In combination with our term for age-disparate male-female relationships, this parameter increases the proportion of men having concurrent relationships with younger women. We note that predicted inter-infection times, furthermore, tend to be shorter (mean < 22 months) in MSM-dominated cohorts [51] than in our heterosexual model (mean ~2.4 years). Indeed, Yousef et al. [52] estimated an inter-infection time of ~6 months in an MSM-dominated transmission cluster; i.e., an inter-infection time that is considerably shorter than the typical time from infection to diagnosis. In populations like this, it is conceivable that no amount of TasP (regardless of the strategy) will be enough control the epidemic. In such cases, TasP would need to be combined with other strategies (e.g., PrEP). Within the context of our general population heterosexual model and our goal of reducing incidence (but not prevalence) to zero, youth-focused TasP yielded the largest benefits in simulations with treatment coverage under ~70% (i.e., at levels below those that would be attained under UNAIDS 90-90-90 goals for 2020). At higher levels, incidence could be driven to negligible levels under all strategies considered here. However, there are reasons to expect that youth-focused TasP could be beneficial even in regions that have met or are on target to meet 90-90-90 goals. For populations in which virus spreads rapidly, the 73% viral suppression conferred by 90-90-90 may not be enough to drive incidence to zero. In fact, our simulations show that in the presence of a subpopulation of people with very short relationship durations that no amount of youth- or adult-focused treatment could end the epidemic despite annual testing (Table 3, perturbation 6). Even in populations in which 90-90-90 is sufficient, it may be hard to sustain treatment services for the full 50–60 years needed to eradicate the virus from the population (i.e., to get to the point where there are no sexually active infected people left in the population). At any time during this period, R0 could increase if people perceive less risk due to the declining epidemic and “let their guard down” with respect to preventative measures like condom and PrEP use. These and other considerations (e.g., uncertainties in funding; lack of an effective vaccine) dictate that public health officials should consider additional efficacies that modelers identify and demonstrate as robust in the context of broader ethical and logistical considerations identified by other stakeholders. We had expected that setting a higher age target for men might benefit the population by protecting young women partnered with older men. However, our “men under 30, women under 25” strategy was not measurably better at reducing long term incidence than our “under age 25” and “under age 30” strategies, or one that included a higher target for women (“men under 25, women under 30"). Sex-stratified age targeting conferred few long-term benefits because treatment of young people extends to older people over time. A strategy that treated all infected women under 25, for example, would, in the absence of drop outs, cover a significant percentage of women under 30 after 5 years. While we leave open the possibility that more extensive investigations with a more detailed model will show a benefit to sex-specific age targeting, we expect any such benefits to be small compared to the large benefits that accrue to straightforward age-based targeting. Given the inverse correlation between HIV viral load and CD4 counts we had also expected that previously used CD4-based targeting would reduce incidence compared to random (untargeted) TasP. Our simulations, however, showed, if anything, slightly worse long-term outcomes under CD4-based targeting. This can be explained, in part, by delays between infection and CD4 decline: people with low CD4 counts will, on average, be older than people with high CD4 counts. Another factor is that our model accounts for reductions in sexual activity in late-stage AIDS. CD4-based targeting, therefore, channels a greater percentage of "resources" to people who would otherwise be non-infectious due to illness. However, we severely doubt that treatment of end-stage AIDS patients (i.e., patients who are most likely to seek out care) would negatively affect the percentage of non-AIDS patients who receive therapy in real-life. The slightly worse performance of the CD4-based strategies, therefore, is, in part, an artifact of the way we set up the comparisons in our model. Our model suggests a significantly stronger advantage to youth-focused TasP than what one might infer from previous modeling studies by Alsallaq et al. [21] and Bershteyn et al. [22]. This is due to several factors. First, we included age-related risk factors (declines in coital frequency and per-act transmission rates with age) that were not included in these previous models. It may also be explained, in part, by differences in the groups being modeled. Alsallaq et al. modeled the effects of prioritizing people under the age of 25. In the absence of other age-related risks, our "under age 30" and "under age 35" strategies outperform our "under age 25" strategy (Table 3, perturbation 14). We speculate Alsallaq et al. would have seen larger benefits had they extended their target age to 30. Bershteyn et al. focused on a strategy that targets those between ages 20 and 30. Although this strategy avoids the costs of testing a group with low prevalence (i.e., adolescents), it does not protect adolescents from getting infected by other adolescents. While the “ring of protection” imposed by their “ages 20 to 30” strategy is very likely to reduce the number of adolescents who get infected in the first place, the age-based strategies considered here are more robust to the risk of HIV spreading between adolescents. In addition, Bershteyn et al. compared their “ages 20–30” strategy to a cohort-based strategy in which a defined cohort (i.e., people entering the sexually active population during the first 10 years of the treatment campaign) was prioritized for treatment for the rest of their lives to the exclusion of others. While useful for determining what kinds of age-based strategies are likely to be the most effective, this cohort-based comparator is not so helpful for our purposes because it enhances treatment of young people during the critical early years of the treatment campaign (though, admittedly, not during the later years). Third, we included more optimistic treatment scenarios. Bershteyn et al. and Alsallaq et al. proposed their models when public health officials had just introduced the 90-90-90 concept. In the time that has elapsed since their papers, public health officials have since introduced the 95-95-95 concept, a level of suppression what would be well within the ~80% linkage to care that optimizes age-based TasP in our model. In other words, we have taken an aspirational approach that asks what it would take to end the epidemic rather than accept the status quo. Finally, although the effect proved to be rather small (Table 3, perturbation 20) our model accounts for the potential for youth-focused TasP to dampen spread by selecting for less virulent viruses (since young people are less likely to have died from high SPVL viruses prior to the TasP campaign). Because of the way we set up our comparisons, we believe that our model provides a clearer illustration of the potential of youth-focused treatment and prevention services than previous studies. Our model is also backed by extensive sensitivity analyses designed to identify the key factors responsible for the success of age-based TasP. Our model, furthermore, takes advantages of an established and well-reviewed set of social network routines that preserve key features of the network over time. Although somewhat tangential to our main hypothesis, the ability to preserve network features contributes interesting and potentially explanatory details; for example, we noted that the increase in prevalence in Fig 2A could be tied, in part, to shifts in the age distribution induced by HIV-related mortality. This shift is an example of a kind of network factor that the underlying statnet routines are well-equipped to handle. Bershteyn et al.’s and Alsallaq et al.’s models, however, have other strengths. Bershteyn et al., for example, include a greater variety of relationships, while Alsallaq et al. include cost and quality life-year calculations that we have not incorporated in our model. Although we differ in the magnitude of the benefit, the fact that independently derived models that include different levels of detail and methods of analysis all show advantages to age-based HIV treatment and prevention services lends support to our findings. There are, of course, many additional details concerning age-related infection risks that could potentially impact the effectiveness of age-based treatment. Brewis and Meyer [30], for example, suggest the overall reduction in coital frequency with age is a result of strong reductions in coital frequency with age in males masking an underlying tendency for coital frequency to peak around age 29 in females. A more detailed coital frequency function could, in principle, capture this and other complexities (e.g., the effect of partnership duration on the probability of coitus) described in the sexual behavior literature. Similar complexities exist with parameters for age-dependent relationship durations, and per-act probabilities of transmissions. Our finding, however, that youth-focused TasP continues to provide large benefits in variants of the model in which key age-related parameters were completely removed indicates robustness of the model to these potential complications. While we used time-dependent treatment limits (i.e., we assumed a “zero-sum” game) to ensure a fair comparison of strategies in the simulations, this should in no way be taken to argue for redistributing HIV/AIDS resources from “old” to “young” PLHIV (or from AIDS patients to non-AIDS patients) in order to “more efficiently” end the epidemic. We argue instead for expanded funding and efforts to increase care and treatment for all ages and CD4 counts. Our model, of course, comes at a time when testing and treatment rates in youth significantly lag behind those in older people [19]. Giving extra attention to youth-centered testing and treatment services could therefore be an equitable means of bringing the epidemic to an end in the context of an overall expansion of care. Attention to HIV infection in youth may take on a particular urgency in low- and middle- income countries where a looming demographic 'youth bulge’ [53] threatens to further strain treatment and prevention programs. Supporting information S1 Text. Supplementary methods. Gives an overview of the Evonet_HIV package. https://doi.org/10.1371/journal.pcbi.1007561.s001 (DOCX) S1 Fig. Ages of infectors of women and men under the age of 35. Each histogram shows the ages of the infectors (i.e., the ages of people who infected someone in the group shown in each histogram) in the absence of treatment. The bars show all infection events between years 0 and 5 involving infectors and recipients who were in the relevant age range at the time of infection. Each bar gives the average of 16 replicates. Red bars indicate infectors who were in the same age range as the recipients. https://doi.org/10.1371/journal.pcbi.1007561.s002 (TIF) S2 Fig. Ages of infectors of women and men over the age of 35. This is a continuation of S1 Fig. Axes, colors, and conditions are described in S1 Fig. https://doi.org/10.1371/journal.pcbi.1007561.s003 (TIF) S3 Fig. Inter-infection times. This graph shows the time elapsed between the time of infection and the time that their infector got infected during the last 25 years of a 45-year simulation without treatment. https://doi.org/10.1371/journal.pcbi.1007561.s004 (TIF) S4 Fig. Cumulative distribution of partnership numbers for our baseline model and selected perturbation experiments from Table 3. The x-axes give natural logarithm of the total number of partners per person for people. The y-axes give the natural logarithms of cumulative distribution starting with the agent with the most partners. To reduce variation caused by young people not having as much time to form relationships as older people, we plotted distributions for people within narrow age ranges. The red circles and blue squares, respectively, show distributions for two representative age ranges: 30–32 and 50–52. The red and blue lines, respectively, show Poisson distributions with means equal to the 30–32 and 50–52 age ranges (i.e., the distributions that would result if partnership numbers were distributed at random). The five panels all show distributions from year 25 after a simulated TasP campaign with Starg = 100% (i.e., from simulations with a minimal number of AIDS deaths). https://doi.org/10.1371/journal.pcbi.1007561.s005 (TIF) S5 Fig. Rollouts in which the number of people being treated increases linearly until everyone has been treated following untargeted (black and blue lines) and “under age 30” (red and green lines) strategies. The left hand, center, and right-hand column gives outcomes following slow, moderate, and fast increases in the number of people being treated over time. The x-and y-axes are same as in Fig 2 in the main text. https://doi.org/10.1371/journal.pcbi.1007561.s006 (TIF) S6 Fig. Time-discounted AIDS deaths for the experiment shown in Fig 3. The x-axis and other strategies are described in Fig 3. Data in the top-left panel is identical to the top-right panel in Fig 3. Error bars have been left out for clarity. https://doi.org/10.1371/journal.pcbi.1007561.s007 (TIF) S7 Fig. Performance of strategies that consider both CD4 and age (grey and blue lines) relative to untargeted “random” (black lines) and age-based TasP (brown and red lines). The x- and y-axes and other strategies are described in Fig 3. https://doi.org/10.1371/journal.pcbi.1007561.s008 (TIF) S8 Fig. Repeat of S7 Fig with a lower CD4 cutoff. The x- and y-axes and other strategies are described in Fig 3. https://doi.org/10.1371/journal.pcbi.1007561.s009 (TIF) S9 Fig. Time-discounted AIDS deaths for the experiment shown in S8 Fig. The x-axis and other strategies are described in Fig 3. Data in the top-left panel is identical to the top-right panel in S8 Fig. Error bars left out for clarity. https://doi.org/10.1371/journal.pcbi.1007561.s010 (TIF) S10 Fig. Performance of strategies with sex-specific age targets (blue and purple lines) compared to untargeted “random” (black lines) and age-based TasP with ordinary, sex-independent, age targets (brown, red, and orange lines). The x- and y-axis and other strategies are described in Fig 3. https://doi.org/10.1371/journal.pcbi.1007561.s011 (TIF) Acknowledgments We thank Vitaly Ganusov for comments. This work was facilitated though the use of advanced computational, storage, and networking infrastructure provided by the UW Hyak supercomputer system.

Wang, Tong;Goyal, Akshit;Dubinkina, Veronika;Maslov, Sergei

doi: 10.1371/journal.pcbi.1007524pmid: 31856158

Introduction The human gut microbiome is a complex ecosystem with several hundreds of microbial species [1, 2] consuming, producing and exchanging hundreds of metabolites [3, 4, 5, 6, 7]. With the advent of high-throughput genomics and metabolomics techniques, it is now possible to simultaneously measure the levels of individual metabolites (the fecal metabolome), as well as the abundances of individual microbial species [8]. Quantitatively connecting these levels with each other, requires knowledge of the relationships between microbes and metabolites in their shared environment: who produces what, and who consumes what? [9, 10] In recent studies, information about these relationships for all of the common species and metabolites in the human gut has been gathered using both manual curation from published studies [6] and automated genome reconstruction methods [3]. This has laid the foundation for mechanistic models which would allow one to relate metabolome composition to microbiome composition [11, 12]. More generally, the construction of mechanistic models has been hindered by the complexity of dynamical processes taking place in the human gut, which in addition to cross-feeding and competition, includes differential spatial distribution and species motility, interactions of microbes with host immune system and bacteriophages, changes in activity of metabolic pathways in individual species in response to environmental parameters, etc. This complexity can be tackled on several distinct levels. For 2-3 species it is possible to construct a detailed dynamical model taking into account the spatial organization and flow of microbes and nutrients within the lower gut [13, 14], or optimizing the intracellular metabolic flows as well as competition for extracellular nutrients using dynamic flux balance analysis (dFBA) models [15, 4]. For around 10 microbial species, and a comparable number of metabolites, it is possible to construct a consumer-resource model (CRM) taking into account microbial competition for nutrients [16], the generation of metabolic byproducts [17], and the different tolerance of species to various environmental factors like pH [14, 18]. Using the existing experimental data on consumption and production kinetics of different metabolites, it is possible to fit some (but not all) of around 80 parameters in such a model [19]. These models are also capable of incorporating cross-feeding interactions between microbial species, as well as community assembly processes [19, 20] However, modeling 100s of species and metabolites, typically present in an individual’s gut microbiome, requires thousands of parameters, which cannot be estimated from the current experimental data. Therefore, any such model must instead resort to a few “global parameters” that appropriately coarse-grain the relevant ecosystem dynamics. Here, we propose such a coarse-grained model of the human gut microbiome, hierarchically organized into several distinct trophic levels. In each level, metabolites are consumed by a subset of microbial species in the microbiome, and partially converted to microbial biomass. A remainder of these metabolites is excreted as metabolic byproducts, which then form the next level of metabolites. The metabolites in this level can then be consumed as nutrients by another subset of microbial species. Our model needs two global parameters: (1) the fraction of nutrients converted to metabolic byproducts by any microbial species, and (2) the number of trophic levels into which the ecosystem is hierarchically organized. While previous studies have suggested that such cross-feeding of metabolic byproducts is common in the microbiome, the extent to which this ecosystem is hierarchically organized has not been quantified. Our model suggests that both, the gut microbiome, and its relevant metabolites, are organized into roughly 4 trophic levels, which interconnect these microbes and metabolites in quantitative agreement with their experimentally measured levels. We also show that this model generates new predictions for the distribution of biomass and flow of metabolites through these trophic levels, quantify the relative contribution of the observed microbes and metabolites to these levels, and thereby describe the effective diversity at each level. Results Multi-level trophic model of the human gut microbiome Our model aims to approximate the metabolic flow through the intricate cross-feeding network of microbes in the lower intestine (hereafter, “gut”) human individuals (Fig 1A). This flow begins with metabolites entering the gut, which are subsequently consumed and processed by multiple microbial species. We assume that each microbial species grows by converting a certain fraction of its metabolic inputs (nutrients) to its biomass and secretes the rest as metabolic byproducts (Fig 1B). We define the byproduct fraction, f, one of the two key parameters of our model, as the fraction of nutrients secreted as byproducts. The complementary biomass fraction, 1 − f, is the fraction of nutrient inputs converted to microbial biomass. The metabolic byproducts produced from the nutrients entering the gut, can be further consumed by some species in the microbiome, in turn generating a set of secondary metabolic byproducts. We call each step of this process of metabolite consumption and byproduct generation, a trophic level. Due to factors such as limited gut motility, and a finite length of the lower gut, we assume that this process only continues for a finite number of levels, Nℓ, the second key parameter of our model. At the end of this process, metabolites left unconsumed after passing through Nℓ trophic levels are assumed to leave the gut as a part of the feces (Fig 1B). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Overview of the trophic model, its calibration and predictions. (A) Schematic diagram showing the various steps in the trophic model, which uses fits the gut nutrient intake profile best approximating the measured metagenome, and outputs a predicted metagenome (microbial abundances) and metabolome. The experimentally measured metabolome is used to calibrate the number of trophic levels, Nℓ and byproduct fraction, f of the model. (B) “Zoomed-in” view of the trophic model from (A), with different microbial species (red) and metabolites (blue) spread across the four trophic levels suggested by the model. At each level, metabolites are consumed by microbial species, and converted partially to their biomass, while the remainder is secreted as metabolic byproducts, which are nutrients for the next trophic level. Metabolites that are left unconsumed across each level are assumed to eventually exit the gut as part of the fecal metabolome, while the biomass accumulated by each species across all levels contributes to the metagenome. https://doi.org/10.1371/journal.pcbi.1007524.g001 In order to quantitatively describe all the steps of this process, our model requires the following information: The metabolic capabilities of different microbial species in the gut, i.e., which microbes can consume which metabolites, and secrete which others. For this, we used a manually curated database connecting 567 common human gut microbes to 235 gut-relevant metabolites they are capable of either consuming or producing as byproducts [6] (see Methods for details). The nutrient intake to the gut, which is the first set of metabolites that are consumed by the microbiome. Since the levels of these metabolites in a given individual are generally unknown, we first curated a list of 19 metabolites likely to constitute the bulk of this nutrient intake, and subsequently fitted their levels to best describe the observed microbial abundances in the gut of each individual (see Methods). We collected such microbial abundance data from various sources, in particular: 380 samples from the large-scale whole-genome sequencing (WGS) studies of healthy individuals (Human Microbiome Project (HMP) [1] and the MetaHIT consortium [2, 21]), 41 samples from a recent 16S rRNA study of 10 year old children in Thailand [22]. The kinetics of nutrient uptake and byproduct release, i.e., the rates we refer to as λ’s, at which different microbial species obtain and secrete different metabolites in the gut environment (see Methods for details of how we defined λ’s). Since this information is unknown for most of our microbes and metabolites, we made some simplifying assumptions. We assumed that, in a given level, when species consume the same metabolite, they receive it in proportion to their abundance in the microbiome. When secreting metabolic byproducts, we assumed equal splitting, such that every metabolite secreted by a given species was released in the same fraction. However, we later verified that the predictions of our model was relatively insensitive to the exact values of these parameters, by repeating our simulations with randomized values of these parameters (see S1 Fig). Simulating the trophic model Our model describes the transit of nutrients from the lower gut to the feces of a specific human individual. As the nutrients transit through the gut, the microbial species in the gut consume, digest and convert them to microbial biomass and metabolic byproducts. For a specific individual, our model comprises multiple iterative steps of metabolite consumption by microbes and the subsequent generation of metabolic byproducts, with each step constituting a trophic level. At each level, all metabolites produced in the previous level could be consumed by all microbial species detected in the specific individual’s gut. Note that at the first level, these metabolites were given by the nutrient intake to the gut, as described above. Any metabolite that could be consumed by multiple microbial species, was split across those species in proportion to their experimentally measured relative abundances (see Methods for details). Those metabolites that could not be consumed at any level were assumed to eventually exit the gut, and form part of the individual’s fecal metabolome. Upon metabolite consumption in any trophic level, we assumed that all microbial species that consumed these metabolites and converted a fraction (1 − f) of the total consumed metabolites to their biomass. The remaining fraction, f (assumed fixed for all species) was converted to byproducts for the next level. Here, we assumed that each of the species produced all the byproducts it was capable of in equal amounts. After Nℓ such iterative rounds (calibrated separately, see the next section), we assumed that this process ends. We added up all the biomass accumulated by each microbial species across all trophic levels as their total biomass, and added up all the unconsumed metabolite levels as the total fecal metabolome. Finally, we normalized, both the microbial biomass and metabolite amounts separately, to obtain the relative microbial abundances and relative metabolome profiles, respectively. Calibrating the key parameters of the model To calibrate the two key parameters of our model, f and Nℓ, we used data from the 41 individuals from a recent 16S rRNA sequencing study of Thai children [22] for which both, 16S rRNA metagenomic profiles, as well as quantitative levels of 214 metabolites in the fecal metabolome, were available. We used these data specifically because they had simultaneously measured the metagenomes and fecal metabolomes with high accuracy, i.e., at the level of individual species and metabolites, which we required for calibration. In each individual we fitted the nutrient intakes of the 19 metabolites to best agree with experimental microbial abundances. A representative example comparing the predicted and measured bacterial abundances is shown in Fig 2B. The Pearson correlation coefficient for data shown in this plot is 0.94, while in individual participants it ranged between 0.81 ± 0.17. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Calibration of the model. (A) Heatmap of the Pearson correlation between experimentally measured and predicted metabolomes for different combinations of parameters f and Nℓ. The plotted value is the correlation coefficient averaged over 41 individuals in Ref. [22] (B) Comparison between the experimentally observed bacterial abundances in a representative individual (y-axis) and their best fits from our model (x-axis) with f = 0.9 and Nℓ = 4. (C) Comparison between the experimentally observed fecal metabolome (y-axis) and the predictions of our model (x-axis) with f = 0.9 and Nℓ = 4 in the same individual shown in panel (B) (Pearson correlation coefficient 0.68; P-value < 10−5). https://doi.org/10.1371/journal.pcbi.1007524.g002 We carried out these fits of microbial abundances for each of the 41 individuals studied in Ref. [22] for a broad range of two parameters of our model—the byproduct fraction f ranging between 0.1 and 0.99 and the number of trophic levels Nℓ between 2 and 10. For each individual and each pair of parameters f and Nℓ we used our model to predict the fecal metabolome profile. This predicted metabolome was subsequently compared to the experimental data of Ref. [22] measured in the same individual. Around 19 of our predicted metabolites (variable across individuals) were actually among the ones experimentally measured in Ref. [22]. We quantified the quality of our predictions using the Pearson correlation coefficient between the predicted and experimentally measured metabolomes, and it’s associated P-value. The model with parameters f = 0.9 and Nℓ = 4 best agreed with the experimental metabolome data, among all the values we tried (Pearson correlation 0.7 ± 0.2; median P-value 8 × 10−4; see Fig 2A). To account for the fact that we used two adjustable parameters in our model (f and Nℓ, we have corrected the P-values appropriately (see Methods for details). We found that even after this correction the median P-value ∼ 10−3 is well below the commonly used significance threshold of 0.05. To ensure that our calibration was not sensitive to this specific measure of fit quality, we also calculated an alternative measure—that of a logarithmic accuracy—which quantifies the average order-of-magnitude error in our predicted fecal metabolome, when compared with the experimentally measured one (see Methods for details). We found that the best logarithmic accuracy was still achieved in a model with f = 0.9 and Nℓ = 4 (the mean error is 0.8 orders of magnitude; see S3 Fig). Hence, we used this combination of parameters in all subsequent simulations of our model. An example of the agreement between predicted and experimentally observed fecal metabolome in a single individual (the same one as in Fig 2B) is shown in Fig 2C (Pearson correlation coefficient 0.89; the adjusted P-value < 10−6). Note that, while the agreement between the experimentally observed and predicted microbial abundances shown in Fig 2B is the outcome of our fitting the levels of 19 intake metabolites, the fecal metabolome is an independent prediction of our model. It naturally emerges from the trophic organization of the metabolic flow and agrees well with the experimentally observed metabolome. To test the quality of this independent prediction, and to show its dependence on metabolic interactions, we repeated our simulations using a randomly shuffled set of microbial metabolic capabilities (i.e., we independently shuffled consumption and secretion abilities of individual microbial species; see Methods for details). We found that the model now generated a much worse correlation coefficient, and more importantly, a non-significant median P-value 0.05 which did not clear the commonly used threshold of P < 0.05 (for example, the individual in Fig 2B and 2C has Pearson correlation 0.32; P-value = 0.19). For all individuals, the Pearson correlation is 0.44 ± 0.2 and the median of their corresponded P-value 0.046. Taken together, our simplified model supports the organization of the human gut microbiome into roughly four trophic levels with byproduct fraction around 0.9. To apply our model to broader, more representative and better-studied samples of the human gut microbiome, we carried over the results of this calibration to another dataset. This dataset (discussed in the next section) consisted of a cohort of 380 human individuals from the Human Microbiome Project (HMP) and the MetaHIT study. We carried over this calibration for three reasons: (1) the lack of availability of simultaneous metabolome measurements for the latter dataset; (2) the fact that both datasets are derived from the human gut; and (3) the similarity in the level of metagenome variability in both datasets. Predictions of the multi-level trophic model Metabolite and biomass flow through trophic levels. With a well-calibrated and tested model we are now in a position to apply it to a broader set of human microbiome data. To this end we chose data for 380 healthy adult individuals from several countries (Europe [2], USA [1], and China [21]). For each individual, we used our model to predict its metabolome (that has not been measured experimentally) and quantified the flow of nutrients (or metabolic activity) through 4 trophic levels in our model averaged over these individuals. Fig 3A shows the cascading nature of this flow: metabolites enter the gut as nutrient intake shown as the leftmost turquoise bar in Fig 3A. Roughly, a fraction 1 − f = 0.1 of this nutrient intake is converted into microbial biomass (red bar), while the remaining fraction f = 0.9 is excreted as metabolic byproducts. Some fraction of these metabolic byproducts (blue bar) cannot be consumed by any of the microbes in individuals microbiome and hence ultimately it leaves the individual as part of their fecal metabolome. The metabolic byproducts that can be consumed by the microbiome (turquoise bar) serve as the nutrient intake for microbes in the next level (i.e., level 3). This scenario repeats itself over the next levels until the level 4, beyond which we assume all the byproducts enter the fecal metabolome. Note that, even though some of these byproducts can be consumed by gut microbes, our previous calibration (Fig 2A) suggests that this does not happen. We believe this may be due to the finite time of flow of nutrients through the gut. Fig 3B shows the normalized contributions of the nutrient intake to microbial biomass (red) and fecal metabolome (blue) split across trophic levels. We observe a contrasting pattern across levels, with the contribution to microbial biomass decreasing along levels, whereas the fraction of unused metabolites (contribution to the fecal metabolome) increases. It is also worth noting that the same microbial and metabolic species get contributions from multiple trophic levels, i.e., the same microbes that consume nutrients and excrete byproducts in earlier levels can also grow on metabolites generated in later levels. Thus, even though the dominant contribution to a species’ biomass is typically derived from a specific trophic level, species can grow by consuming metabolites from multiple levels. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Metabolite and biomass flow through the levels. (A) Cascading nature of nutrient flow across trophic levels: nutrient intake to the gut (the leftmost turquoise bar) is gradually converted into microbial biomass (red bars in each level) and metabolic byproducts (turquoise bars in each level). Some fraction of these byproducts (blue bars in each level) cannot be consumed by the microbiome and hence remains further unprocessed until it leaves an individual as their fecal metabolome. The metabolic byproducts of each level (turquoise bars) serve as the nutrient intake for microbes in the next level. The process ends at level 4 where all byproducts remain unconsumed thereby enter the fecal metabolome. (B) Normalized contribution of of the nutrient intake to microbial biomass (red) and fecal metabolome (blue) split across levels 2 to 4. Dashed lines show that consumable metabolites generated at a previous level serve as metabolic inputs to the next level. https://doi.org/10.1371/journal.pcbi.1007524.g003 Quantifying diversity across trophic levels. The diversity of microbial communities can be separately defined both phylogenetically and functionally. Phylogenetic diversity counts the number of abundant microbial species inferred from the metagenomic profile. On the other hand, functional diversity quantifies the variety of collective metabolic activities of these species, which in our case could be inferred from the metabolome profile. Our model allows to quantify both types of diversity on a level-by-level basis. Instead of just calculating the presence or absence of microbial species or metabolites at each level, we weighed each microbe or metabolite by their relative contribution to the metabolic activity at that trophic level. At each level, we calculated the effective α-, β- and γ-diversity, separately for microbes and metabolites (see Methods for details). Fig 4 shows the effective α-, β- and γ- diversity for microbes (grouped at the species and genus levels) and metabolites, averaged over our 380 healthy individuals. The microbes first appear in the second trophic level feeding off the nutrient intake metabolites in the first level. We found that the α-diversity (the average number of abundant entities weighted by their contribution to each level) systematically increases with the level number for both microbes and metabolites. There is no clear trend in the γ-diversity of microbes grouped at the species level (the “pan-microbiome” diversity, i.e., the number of abundant species in the combined metagenomes of 380 individuals). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Metabolite and microbial diversity at different levels. Effective (A–C) α-diversity, (D–F) β-diversity, (G–I) γ-diversity in microbial species (A, D, G), microbial genera (B, E, H), and metabolites (C, F, I) plotted as a function of trophic level (1–4) and averaged across 380 individuals. https://doi.org/10.1371/journal.pcbi.1007524.g004 Finally the beta-diversity of microbial species, defined as the ratio between γ− and α-diversity is the highest (∼ 4) in the first level, while being considerably lower (∼ 2.5) in the next two levels. The β-diversity addresses the following important question: how variable are the abundant species between individuals? While we found that the β-diversity of microbial species could be as large as 4 (Fig 4), when we grouped organisms by their genus, β diversity decreased down to ∼ 2 across all levels (Fig 4E). This drop in β-diversity was the most pronounced in the uppermost trophic level. The overall reduction of β-diversity shown in Fig 4E relative to Fig 4D suggests that the chief driver of species variability in the gut microbiome is within-genus competition. Such a pattern has previously been explained by a “lottery-like” process of microbial competition within the gut [23]. We also quantified the diversity of metabolites across 4 trophic levels. We found that the β diversity of metabolites was the highest in the uppermost level of nutrients (∼ 2) and lower in the next three levels (∼ 1). While this declining trend was similar to that observed for microbial diversity, surprisingly, the value of β diversity for nutrients was much smaller than for microbes (about 2.5 times lower across all levels). This suggests the picture of functional stability—in spite of taxonomic variability—in all trophic levels of the human gut microbiome, namely that even though the species composition of the microbiome can be quite different for different individuals, their metabolic function is quite similar. These results supplement similar findings of the HMP project [1] by breaking them up into trophic levels and by using metabolome diversity instead of metabolic pathways diversity to quantify the extent of functional similarity. Multi-level trophic model of the human gut microbiome Our model aims to approximate the metabolic flow through the intricate cross-feeding network of microbes in the lower intestine (hereafter, “gut”) human individuals (Fig 1A). This flow begins with metabolites entering the gut, which are subsequently consumed and processed by multiple microbial species. We assume that each microbial species grows by converting a certain fraction of its metabolic inputs (nutrients) to its biomass and secretes the rest as metabolic byproducts (Fig 1B). We define the byproduct fraction, f, one of the two key parameters of our model, as the fraction of nutrients secreted as byproducts. The complementary biomass fraction, 1 − f, is the fraction of nutrient inputs converted to microbial biomass. The metabolic byproducts produced from the nutrients entering the gut, can be further consumed by some species in the microbiome, in turn generating a set of secondary metabolic byproducts. We call each step of this process of metabolite consumption and byproduct generation, a trophic level. Due to factors such as limited gut motility, and a finite length of the lower gut, we assume that this process only continues for a finite number of levels, Nℓ, the second key parameter of our model. At the end of this process, metabolites left unconsumed after passing through Nℓ trophic levels are assumed to leave the gut as a part of the feces (Fig 1B). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Overview of the trophic model, its calibration and predictions. (A) Schematic diagram showing the various steps in the trophic model, which uses fits the gut nutrient intake profile best approximating the measured metagenome, and outputs a predicted metagenome (microbial abundances) and metabolome. The experimentally measured metabolome is used to calibrate the number of trophic levels, Nℓ and byproduct fraction, f of the model. (B) “Zoomed-in” view of the trophic model from (A), with different microbial species (red) and metabolites (blue) spread across the four trophic levels suggested by the model. At each level, metabolites are consumed by microbial species, and converted partially to their biomass, while the remainder is secreted as metabolic byproducts, which are nutrients for the next trophic level. Metabolites that are left unconsumed across each level are assumed to eventually exit the gut as part of the fecal metabolome, while the biomass accumulated by each species across all levels contributes to the metagenome. https://doi.org/10.1371/journal.pcbi.1007524.g001 In order to quantitatively describe all the steps of this process, our model requires the following information: The metabolic capabilities of different microbial species in the gut, i.e., which microbes can consume which metabolites, and secrete which others. For this, we used a manually curated database connecting 567 common human gut microbes to 235 gut-relevant metabolites they are capable of either consuming or producing as byproducts [6] (see Methods for details). The nutrient intake to the gut, which is the first set of metabolites that are consumed by the microbiome. Since the levels of these metabolites in a given individual are generally unknown, we first curated a list of 19 metabolites likely to constitute the bulk of this nutrient intake, and subsequently fitted their levels to best describe the observed microbial abundances in the gut of each individual (see Methods). We collected such microbial abundance data from various sources, in particular: 380 samples from the large-scale whole-genome sequencing (WGS) studies of healthy individuals (Human Microbiome Project (HMP) [1] and the MetaHIT consortium [2, 21]), 41 samples from a recent 16S rRNA study of 10 year old children in Thailand [22]. The kinetics of nutrient uptake and byproduct release, i.e., the rates we refer to as λ’s, at which different microbial species obtain and secrete different metabolites in the gut environment (see Methods for details of how we defined λ’s). Since this information is unknown for most of our microbes and metabolites, we made some simplifying assumptions. We assumed that, in a given level, when species consume the same metabolite, they receive it in proportion to their abundance in the microbiome. When secreting metabolic byproducts, we assumed equal splitting, such that every metabolite secreted by a given species was released in the same fraction. However, we later verified that the predictions of our model was relatively insensitive to the exact values of these parameters, by repeating our simulations with randomized values of these parameters (see S1 Fig). Simulating the trophic model Our model describes the transit of nutrients from the lower gut to the feces of a specific human individual. As the nutrients transit through the gut, the microbial species in the gut consume, digest and convert them to microbial biomass and metabolic byproducts. For a specific individual, our model comprises multiple iterative steps of metabolite consumption by microbes and the subsequent generation of metabolic byproducts, with each step constituting a trophic level. At each level, all metabolites produced in the previous level could be consumed by all microbial species detected in the specific individual’s gut. Note that at the first level, these metabolites were given by the nutrient intake to the gut, as described above. Any metabolite that could be consumed by multiple microbial species, was split across those species in proportion to their experimentally measured relative abundances (see Methods for details). Those metabolites that could not be consumed at any level were assumed to eventually exit the gut, and form part of the individual’s fecal metabolome. Upon metabolite consumption in any trophic level, we assumed that all microbial species that consumed these metabolites and converted a fraction (1 − f) of the total consumed metabolites to their biomass. The remaining fraction, f (assumed fixed for all species) was converted to byproducts for the next level. Here, we assumed that each of the species produced all the byproducts it was capable of in equal amounts. After Nℓ such iterative rounds (calibrated separately, see the next section), we assumed that this process ends. We added up all the biomass accumulated by each microbial species across all trophic levels as their total biomass, and added up all the unconsumed metabolite levels as the total fecal metabolome. Finally, we normalized, both the microbial biomass and metabolite amounts separately, to obtain the relative microbial abundances and relative metabolome profiles, respectively. Calibrating the key parameters of the model To calibrate the two key parameters of our model, f and Nℓ, we used data from the 41 individuals from a recent 16S rRNA sequencing study of Thai children [22] for which both, 16S rRNA metagenomic profiles, as well as quantitative levels of 214 metabolites in the fecal metabolome, were available. We used these data specifically because they had simultaneously measured the metagenomes and fecal metabolomes with high accuracy, i.e., at the level of individual species and metabolites, which we required for calibration. In each individual we fitted the nutrient intakes of the 19 metabolites to best agree with experimental microbial abundances. A representative example comparing the predicted and measured bacterial abundances is shown in Fig 2B. The Pearson correlation coefficient for data shown in this plot is 0.94, while in individual participants it ranged between 0.81 ± 0.17. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Calibration of the model. (A) Heatmap of the Pearson correlation between experimentally measured and predicted metabolomes for different combinations of parameters f and Nℓ. The plotted value is the correlation coefficient averaged over 41 individuals in Ref. [22] (B) Comparison between the experimentally observed bacterial abundances in a representative individual (y-axis) and their best fits from our model (x-axis) with f = 0.9 and Nℓ = 4. (C) Comparison between the experimentally observed fecal metabolome (y-axis) and the predictions of our model (x-axis) with f = 0.9 and Nℓ = 4 in the same individual shown in panel (B) (Pearson correlation coefficient 0.68; P-value < 10−5). https://doi.org/10.1371/journal.pcbi.1007524.g002 We carried out these fits of microbial abundances for each of the 41 individuals studied in Ref. [22] for a broad range of two parameters of our model—the byproduct fraction f ranging between 0.1 and 0.99 and the number of trophic levels Nℓ between 2 and 10. For each individual and each pair of parameters f and Nℓ we used our model to predict the fecal metabolome profile. This predicted metabolome was subsequently compared to the experimental data of Ref. [22] measured in the same individual. Around 19 of our predicted metabolites (variable across individuals) were actually among the ones experimentally measured in Ref. [22]. We quantified the quality of our predictions using the Pearson correlation coefficient between the predicted and experimentally measured metabolomes, and it’s associated P-value. The model with parameters f = 0.9 and Nℓ = 4 best agreed with the experimental metabolome data, among all the values we tried (Pearson correlation 0.7 ± 0.2; median P-value 8 × 10−4; see Fig 2A). To account for the fact that we used two adjustable parameters in our model (f and Nℓ, we have corrected the P-values appropriately (see Methods for details). We found that even after this correction the median P-value ∼ 10−3 is well below the commonly used significance threshold of 0.05. To ensure that our calibration was not sensitive to this specific measure of fit quality, we also calculated an alternative measure—that of a logarithmic accuracy—which quantifies the average order-of-magnitude error in our predicted fecal metabolome, when compared with the experimentally measured one (see Methods for details). We found that the best logarithmic accuracy was still achieved in a model with f = 0.9 and Nℓ = 4 (the mean error is 0.8 orders of magnitude; see S3 Fig). Hence, we used this combination of parameters in all subsequent simulations of our model. An example of the agreement between predicted and experimentally observed fecal metabolome in a single individual (the same one as in Fig 2B) is shown in Fig 2C (Pearson correlation coefficient 0.89; the adjusted P-value < 10−6). Note that, while the agreement between the experimentally observed and predicted microbial abundances shown in Fig 2B is the outcome of our fitting the levels of 19 intake metabolites, the fecal metabolome is an independent prediction of our model. It naturally emerges from the trophic organization of the metabolic flow and agrees well with the experimentally observed metabolome. To test the quality of this independent prediction, and to show its dependence on metabolic interactions, we repeated our simulations using a randomly shuffled set of microbial metabolic capabilities (i.e., we independently shuffled consumption and secretion abilities of individual microbial species; see Methods for details). We found that the model now generated a much worse correlation coefficient, and more importantly, a non-significant median P-value 0.05 which did not clear the commonly used threshold of P < 0.05 (for example, the individual in Fig 2B and 2C has Pearson correlation 0.32; P-value = 0.19). For all individuals, the Pearson correlation is 0.44 ± 0.2 and the median of their corresponded P-value 0.046. Taken together, our simplified model supports the organization of the human gut microbiome into roughly four trophic levels with byproduct fraction around 0.9. To apply our model to broader, more representative and better-studied samples of the human gut microbiome, we carried over the results of this calibration to another dataset. This dataset (discussed in the next section) consisted of a cohort of 380 human individuals from the Human Microbiome Project (HMP) and the MetaHIT study. We carried over this calibration for three reasons: (1) the lack of availability of simultaneous metabolome measurements for the latter dataset; (2) the fact that both datasets are derived from the human gut; and (3) the similarity in the level of metagenome variability in both datasets. Predictions of the multi-level trophic model Metabolite and biomass flow through trophic levels. With a well-calibrated and tested model we are now in a position to apply it to a broader set of human microbiome data. To this end we chose data for 380 healthy adult individuals from several countries (Europe [2], USA [1], and China [21]). For each individual, we used our model to predict its metabolome (that has not been measured experimentally) and quantified the flow of nutrients (or metabolic activity) through 4 trophic levels in our model averaged over these individuals. Fig 3A shows the cascading nature of this flow: metabolites enter the gut as nutrient intake shown as the leftmost turquoise bar in Fig 3A. Roughly, a fraction 1 − f = 0.1 of this nutrient intake is converted into microbial biomass (red bar), while the remaining fraction f = 0.9 is excreted as metabolic byproducts. Some fraction of these metabolic byproducts (blue bar) cannot be consumed by any of the microbes in individuals microbiome and hence ultimately it leaves the individual as part of their fecal metabolome. The metabolic byproducts that can be consumed by the microbiome (turquoise bar) serve as the nutrient intake for microbes in the next level (i.e., level 3). This scenario repeats itself over the next levels until the level 4, beyond which we assume all the byproducts enter the fecal metabolome. Note that, even though some of these byproducts can be consumed by gut microbes, our previous calibration (Fig 2A) suggests that this does not happen. We believe this may be due to the finite time of flow of nutrients through the gut. Fig 3B shows the normalized contributions of the nutrient intake to microbial biomass (red) and fecal metabolome (blue) split across trophic levels. We observe a contrasting pattern across levels, with the contribution to microbial biomass decreasing along levels, whereas the fraction of unused metabolites (contribution to the fecal metabolome) increases. It is also worth noting that the same microbial and metabolic species get contributions from multiple trophic levels, i.e., the same microbes that consume nutrients and excrete byproducts in earlier levels can also grow on metabolites generated in later levels. Thus, even though the dominant contribution to a species’ biomass is typically derived from a specific trophic level, species can grow by consuming metabolites from multiple levels. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Metabolite and biomass flow through the levels. (A) Cascading nature of nutrient flow across trophic levels: nutrient intake to the gut (the leftmost turquoise bar) is gradually converted into microbial biomass (red bars in each level) and metabolic byproducts (turquoise bars in each level). Some fraction of these byproducts (blue bars in each level) cannot be consumed by the microbiome and hence remains further unprocessed until it leaves an individual as their fecal metabolome. The metabolic byproducts of each level (turquoise bars) serve as the nutrient intake for microbes in the next level. The process ends at level 4 where all byproducts remain unconsumed thereby enter the fecal metabolome. (B) Normalized contribution of of the nutrient intake to microbial biomass (red) and fecal metabolome (blue) split across levels 2 to 4. Dashed lines show that consumable metabolites generated at a previous level serve as metabolic inputs to the next level. https://doi.org/10.1371/journal.pcbi.1007524.g003 Quantifying diversity across trophic levels. The diversity of microbial communities can be separately defined both phylogenetically and functionally. Phylogenetic diversity counts the number of abundant microbial species inferred from the metagenomic profile. On the other hand, functional diversity quantifies the variety of collective metabolic activities of these species, which in our case could be inferred from the metabolome profile. Our model allows to quantify both types of diversity on a level-by-level basis. Instead of just calculating the presence or absence of microbial species or metabolites at each level, we weighed each microbe or metabolite by their relative contribution to the metabolic activity at that trophic level. At each level, we calculated the effective α-, β- and γ-diversity, separately for microbes and metabolites (see Methods for details). Fig 4 shows the effective α-, β- and γ- diversity for microbes (grouped at the species and genus levels) and metabolites, averaged over our 380 healthy individuals. The microbes first appear in the second trophic level feeding off the nutrient intake metabolites in the first level. We found that the α-diversity (the average number of abundant entities weighted by their contribution to each level) systematically increases with the level number for both microbes and metabolites. There is no clear trend in the γ-diversity of microbes grouped at the species level (the “pan-microbiome” diversity, i.e., the number of abundant species in the combined metagenomes of 380 individuals). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Metabolite and microbial diversity at different levels. Effective (A–C) α-diversity, (D–F) β-diversity, (G–I) γ-diversity in microbial species (A, D, G), microbial genera (B, E, H), and metabolites (C, F, I) plotted as a function of trophic level (1–4) and averaged across 380 individuals. https://doi.org/10.1371/journal.pcbi.1007524.g004 Finally the beta-diversity of microbial species, defined as the ratio between γ− and α-diversity is the highest (∼ 4) in the first level, while being considerably lower (∼ 2.5) in the next two levels. The β-diversity addresses the following important question: how variable are the abundant species between individuals? While we found that the β-diversity of microbial species could be as large as 4 (Fig 4), when we grouped organisms by their genus, β diversity decreased down to ∼ 2 across all levels (Fig 4E). This drop in β-diversity was the most pronounced in the uppermost trophic level. The overall reduction of β-diversity shown in Fig 4E relative to Fig 4D suggests that the chief driver of species variability in the gut microbiome is within-genus competition. Such a pattern has previously been explained by a “lottery-like” process of microbial competition within the gut [23]. We also quantified the diversity of metabolites across 4 trophic levels. We found that the β diversity of metabolites was the highest in the uppermost level of nutrients (∼ 2) and lower in the next three levels (∼ 1). While this declining trend was similar to that observed for microbial diversity, surprisingly, the value of β diversity for nutrients was much smaller than for microbes (about 2.5 times lower across all levels). This suggests the picture of functional stability—in spite of taxonomic variability—in all trophic levels of the human gut microbiome, namely that even though the species composition of the microbiome can be quite different for different individuals, their metabolic function is quite similar. These results supplement similar findings of the HMP project [1] by breaking them up into trophic levels and by using metabolome diversity instead of metabolic pathways diversity to quantify the extent of functional similarity. Metabolite and biomass flow through trophic levels. With a well-calibrated and tested model we are now in a position to apply it to a broader set of human microbiome data. To this end we chose data for 380 healthy adult individuals from several countries (Europe [2], USA [1], and China [21]). For each individual, we used our model to predict its metabolome (that has not been measured experimentally) and quantified the flow of nutrients (or metabolic activity) through 4 trophic levels in our model averaged over these individuals. Fig 3A shows the cascading nature of this flow: metabolites enter the gut as nutrient intake shown as the leftmost turquoise bar in Fig 3A. Roughly, a fraction 1 − f = 0.1 of this nutrient intake is converted into microbial biomass (red bar), while the remaining fraction f = 0.9 is excreted as metabolic byproducts. Some fraction of these metabolic byproducts (blue bar) cannot be consumed by any of the microbes in individuals microbiome and hence ultimately it leaves the individual as part of their fecal metabolome. The metabolic byproducts that can be consumed by the microbiome (turquoise bar) serve as the nutrient intake for microbes in the next level (i.e., level 3). This scenario repeats itself over the next levels until the level 4, beyond which we assume all the byproducts enter the fecal metabolome. Note that, even though some of these byproducts can be consumed by gut microbes, our previous calibration (Fig 2A) suggests that this does not happen. We believe this may be due to the finite time of flow of nutrients through the gut. Fig 3B shows the normalized contributions of the nutrient intake to microbial biomass (red) and fecal metabolome (blue) split across trophic levels. We observe a contrasting pattern across levels, with the contribution to microbial biomass decreasing along levels, whereas the fraction of unused metabolites (contribution to the fecal metabolome) increases. It is also worth noting that the same microbial and metabolic species get contributions from multiple trophic levels, i.e., the same microbes that consume nutrients and excrete byproducts in earlier levels can also grow on metabolites generated in later levels. Thus, even though the dominant contribution to a species’ biomass is typically derived from a specific trophic level, species can grow by consuming metabolites from multiple levels. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Metabolite and biomass flow through the levels. (A) Cascading nature of nutrient flow across trophic levels: nutrient intake to the gut (the leftmost turquoise bar) is gradually converted into microbial biomass (red bars in each level) and metabolic byproducts (turquoise bars in each level). Some fraction of these byproducts (blue bars in each level) cannot be consumed by the microbiome and hence remains further unprocessed until it leaves an individual as their fecal metabolome. The metabolic byproducts of each level (turquoise bars) serve as the nutrient intake for microbes in the next level. The process ends at level 4 where all byproducts remain unconsumed thereby enter the fecal metabolome. (B) Normalized contribution of of the nutrient intake to microbial biomass (red) and fecal metabolome (blue) split across levels 2 to 4. Dashed lines show that consumable metabolites generated at a previous level serve as metabolic inputs to the next level. https://doi.org/10.1371/journal.pcbi.1007524.g003 Quantifying diversity across trophic levels. The diversity of microbial communities can be separately defined both phylogenetically and functionally. Phylogenetic diversity counts the number of abundant microbial species inferred from the metagenomic profile. On the other hand, functional diversity quantifies the variety of collective metabolic activities of these species, which in our case could be inferred from the metabolome profile. Our model allows to quantify both types of diversity on a level-by-level basis. Instead of just calculating the presence or absence of microbial species or metabolites at each level, we weighed each microbe or metabolite by their relative contribution to the metabolic activity at that trophic level. At each level, we calculated the effective α-, β- and γ-diversity, separately for microbes and metabolites (see Methods for details). Fig 4 shows the effective α-, β- and γ- diversity for microbes (grouped at the species and genus levels) and metabolites, averaged over our 380 healthy individuals. The microbes first appear in the second trophic level feeding off the nutrient intake metabolites in the first level. We found that the α-diversity (the average number of abundant entities weighted by their contribution to each level) systematically increases with the level number for both microbes and metabolites. There is no clear trend in the γ-diversity of microbes grouped at the species level (the “pan-microbiome” diversity, i.e., the number of abundant species in the combined metagenomes of 380 individuals). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Metabolite and microbial diversity at different levels. Effective (A–C) α-diversity, (D–F) β-diversity, (G–I) γ-diversity in microbial species (A, D, G), microbial genera (B, E, H), and metabolites (C, F, I) plotted as a function of trophic level (1–4) and averaged across 380 individuals. https://doi.org/10.1371/journal.pcbi.1007524.g004 Finally the beta-diversity of microbial species, defined as the ratio between γ− and α-diversity is the highest (∼ 4) in the first level, while being considerably lower (∼ 2.5) in the next two levels. The β-diversity addresses the following important question: how variable are the abundant species between individuals? While we found that the β-diversity of microbial species could be as large as 4 (Fig 4), when we grouped organisms by their genus, β diversity decreased down to ∼ 2 across all levels (Fig 4E). This drop in β-diversity was the most pronounced in the uppermost trophic level. The overall reduction of β-diversity shown in Fig 4E relative to Fig 4D suggests that the chief driver of species variability in the gut microbiome is within-genus competition. Such a pattern has previously been explained by a “lottery-like” process of microbial competition within the gut [23]. We also quantified the diversity of metabolites across 4 trophic levels. We found that the β diversity of metabolites was the highest in the uppermost level of nutrients (∼ 2) and lower in the next three levels (∼ 1). While this declining trend was similar to that observed for microbial diversity, surprisingly, the value of β diversity for nutrients was much smaller than for microbes (about 2.5 times lower across all levels). This suggests the picture of functional stability—in spite of taxonomic variability—in all trophic levels of the human gut microbiome, namely that even though the species composition of the microbiome can be quite different for different individuals, their metabolic function is quite similar. These results supplement similar findings of the HMP project [1] by breaking them up into trophic levels and by using metabolome diversity instead of metabolic pathways diversity to quantify the extent of functional similarity. Discussion Above we introduced and studied a mechanistic, consumer-resource model of the human gut microbiome quantifying the flow of metabolites and the gradual building up of microbial biomass across several trophic levels. What distinguishes our model is its ability to simultaneously capture the metabolic activities of hundreds of species consuming and producing hundreds of metabolites. Using only the metabolic capabilities—who eats what, and makes what—of different species in the microbiome, we uncovered roughly four trophic levels in the human gut microbiome. At each of these levels, some microbes consume nutrients, and convert them partially to their biomass, while the remainder gets secreted as metabolic byproducts. These metabolic byproducts can then serve as nutrients for microbes in the next trophic level. Understanding such a trophic organization of microbial ecosystems is important because it helps identify causal relationships between microbes and metabolites at two consecutive trophic levels and helps to separate them from purely correlative connections, either at the same or at more distant levels. Thus it extends the previously introduced concept of a “microbial metabolic influence network” [6] by highlighting its hierarchical structure in which species/metabolites assigned to higher trophic levels could affect a large number of species/metabolites located downstream from them. The concept of trophic levels has been widely used in macro-ecosystems to make sense of flow of nutrients and energy in large food webs [24, 25, 26], but it has only received limited attention in microbial ecosystems, one example being ref. [27]. There is no absolute consensus definition of a trophic level with several interpretations discussed in Refs. [28, 29, 30]. However, all of these definitions agree with each other on the following two criteria that the trophic structure of an ecosystem typically satisfies: (1) there is explicit level-to-level conversion and flow of energy (and biomass), taking place in several discrete steps; and (2) these steps are temporally staged, because the conversion process at every level takes a finite amount of time. Here we define a trophic level as a discrete step in the metabolic conversion of nutrients after it enters the lower gut. Each such step involves multiple microbial species generating byproducts for the next conversion step. Thus, according to our definition, the same species and metabolite can be present at more than one trophic level. Furthermore, because of the finite motility in the human gut, the metabolic activity at each of our trophic levels would tend to be spatially separated with that in level 1 taking place near the entrance to the lower gut and that in level 4, near the end of the gut. This definition of trophic levels also results in an imperfect hierarchical structure of the food web in which some species or metabolites linking non-consecutive trophic levels (see Ref. [29] for similar processes in macroscopic ecosystems). Also note that spatially separated microbial compositions, corresponding to the trophic levels in our model, could in principle be tested in artificial gut systems (such as in Refs. [31, 32]). Further, there are several well-known ecological factors that constrain the number of trophic levels in an ecosystem, such as ecological energetics and population dynamics (see ref. [28] for a Discussion). Our work introduces additional factors that can limit the number of trophic levels in the human gut microbiome—namely the limited length and finite motility of the gut. The human gut microbiome is notorious for several complex and interlinked metabolic cross-feeding interactions between its resident microbial species [6, 33]. Even though we exploit this aspect of the gut’s microbial ecology to study its trophic organization, we wish to highlight that we do not confine a metabolite or microbial species to participate strictly at one trophic level. We can nevertheless tentatively assign metabolites and microbial species to the level to which they contribute the most. We find that doing so results in trophic level assignments that are consistent with the expectations of the rest of the gut microbial literature [34]; see S5 Fig for a representative example of a trophic network. Specifically, we find that various polysaccharide-degrading species from the genera Prevotella and Bacteroides tend to be assigned to the first microbial layer, leading to the production of acetate [34]. This acetate is, in turn, the major substrate for butyrate-producing bacteria such as various species of Eubacterium and Roseburia, as well as the well-known Faecalibacterium prausnitzii; our tentative assignment procedure places these species in the subsequent layers of the trophic network. The butyrate and valerate secreted by these species consequently end up, and are assigned to, metabolite trophic levels 3 and 4. Similarly, various sulfate-reducing species (e.g., Desulfovibrio piger, Bilophila wadsworthia) and acetogenic bacteria (e.g., Blautia hansenii), as well as their byproducts, are typically assigned to the lower trophic levels by our model. One can also see that, towards the lower trophic levels metabolites are either very simple and energy-poor, like CO2, H2, H2S, or are those that cannot be consumed by any gut microbial species, such as various amines, short-chain fatty acids (SCFAs), and secondary bile acids. We expect these latter set of metabolites to therefore be present in an individual’s fecal metabolome. By assuming such a fluid multi-level trophic organization, our model is able to independently predict the fecal metabolome of individual humans, in quantitative agreement with experimental measurements, comparable to or better than the state of the art. For example, Ref. [12] used intra-cellular metabolic flux balance analysis (FBA) to achieve a Pearson correlation coefficient 0.4 between the predicted and a representative experimentally measured fecal metabolome. In contrast, our model achieved the Pearson correlation of 0.7 in individualized predictions using only two ecologically meaningful parameters. Note that though the datasets in this comparison were not the same, they were both contemporary datasets of the human gut microbiome. This suggests that incorporating ecological information about the human gut microbiome can generate mechanistically-grounded and internally consistent fecal metabolome predictions given information about an individual’s metagenome (species abundance profile). Our model also allows us to quantify the diversity of both species and metabolites contributing to different trophic levels. One conclusion we made was that the functional convergence of the microbiome holds roughly equally across all trophic levels. Indeed, at each level we observed the microbial diversity across different individuals was considerably higher than their metabolic diversity. Our model also provides additional support to the “lottery” scenario described in Ref. [23], especially in the first trophic level. According to this scenario, there are multiple species nearly equally capable of occupying a certain ecological niche, which in our model corresponds to the set of nutrients they consume and secrete as byproducts. The first species to occupy this niche prevents equivalent microbes from entering it. This is reflected in a high β-diversity of microbial species combined with a low to moderate β-diversity of microbial genera to which they belong and low β-diversity of their metabolic byproducts. Our model is focused on studying the effects of cross-feeding and competition of different microbes for their nutrients. Thereby it ignores a number of important factors known to impact the composition of the human gut microbiome. These include interactions with host and its immune system [35] as well as with viruses [36], and environmental parameters other than nutrients, such as pH [14], spatial organization [37], etc. Instead, our model uses only two adjustable parameters: the byproduct fraction f and the number of trophic levels Nℓ, assumed to be common to all species. This very small number of parameters has been a conscious choice on our part. We are perfectly aware that species differ from each other in their byproduct ratios, and that the metabolic flows are not equally split among multiple byproducts. This can be easily captured by a variant of our model in which different nutrient inputs and and byproduct outputs of a given microbial species are characterized by different kinetic rates. However, this would immediately increase the number of parameters from 2 to more than 3, 600. To calibrate a model with such a huge number of parameters one needs many more experimental data than we have access to right now. However, we tested the sensitivity of our model to variation in these parameters by repeating our simulations for 100 random sets of nutrient kinetic uptake and byproduct release rates (λ’s in our model), and found that this did not qualitatively change our central result (i.e., that the human gut microbiome is composed of roughly Nℓ = 4 trophic levels with a byproduct fraction f = 0.9). Surprisingly, our metabolome predictions were also relatively insensitive with respect to varying these parameters (S1 Fig). The exact nature of the robustness of these metabolome predictions is beyond the scope of this paper, and the subject of future work. Finally, note that our model makes several simplifying assumptions. For instance, we use metabolite concentrations in the same units of measurement as the experiment, and do not normalize them by each metabolite’s molecular weight. Additionally, we use the experimentally measured microbial abundances as an input to the model, to avoid fitting or randomizing the initial microbial abundances. We also assume that microbial species secrete metabolites in equal amounts (see S7 Fig for model performance when we relax this assumption). Finally, we do not penalize species for being generalists, i.e., for using many nutrients (see S6 Fig for model performance when this assumption is relaxed). Methods Obtaining data for microbial metabolic capabilities For information about the metabolic capabilities of human gut microbes, we adopted a recently published manually-curated database, NJS16, which includes such data for 570 common gut microbial species and 244 relevant metabolites from Ref. [6]. This database recorded, for each microbial species, which metabolites each of the species could consume, and which they secreted as byproducts. Since we were interested in those metabolites that could be used for microbial growth, we removed metabolites such as ions (e.g., Na+, Ca+) from NJS16. Moreover, we constrained our analyses to microbes only, and therefore removed the 3 types of human cells from NJS16. This left us with a database with 567 microbes, 235 metabolites and 4,248 interactions connecting these microbes with corresponding metabolites (see S1 Table for the complete table of interactions). Obtaining metagenomic and metabolomic data To calibrate the key parameters of our model, we used a previously published dataset, namely a 16S rRNA sequencing study of 41 human individuals from rural and urban areas in Thailand [22]. From these data, we collected the reported 16S rRNA OTU abundances as well as their corresponding taxonomy. We explicitly removed all OTUs that did not have an assigned species-level taxonomy. The remaining OTUs explained roughly 71%(±15%) of the bacterial abundances per sample. We then mapped these species names to those listed in the NJS16 database. We found an exact match for 110 species out of 208 in this table. In order to improve the species coverage from the abundance data, we manually mapped the remaining species in the following manner. For those genera in NJS16, whose member species had identical metabolic capabilities, we assumed that the capabilities of other, unmapped species from these genera were the same as these species. For several well-studied bacterial genera, such as Bacteroides, we determined a “core” set of metabolic capabilities (i.e., those metabolites that could either be consumed or secreted by all species in that genus), and assigned them to all unmapped species in that genus (i.e., those with known abundances, but otherwise understudied metabolic capabilities in NJS16). This allowed us to map an additional 20 microbial species from the abundance data, and incorporate into our model. Note that we did this additional mapping, only for those genera, where species metabolic capabilities were identical. To quantify the metabolome levels in each individual, we used the available quantitative metabolome profiles (obtained via from CE-TOF MS) corresponding to the 41 individuals whose metagenomic samples we had. Here, we mapped the reported metabolites to our database of metabolic capabilities using KEGG identifiers, which revealed 84 such measured metabolites. To make predictions about metabolic flow and effective diversity from our model, we used additional metagenomic datasets, namely those from the Human Microbiome Project (HMP) [1] and MetaHIT [2, 21], for which we had microbial abundances, but no fecal metabolome. This resulted in an additional 380 human individuals, for which we obtained tables of MetaPhlAn2 microbial abundances, and mapped species names to those in NJS16 using the same procedure described above. Here, out of a total of 532 microbial species detected over these data, we could map and incorporate 316 species. Of these, 207 were mapped through an exact taxonomic match, and 109 by a genus-capability match. These incorporated species covered, on average, 90% of the total microbial abundance in each individual sample. Determining the components of the nutrient intake to the gut The inputs of our model are the experimentally measured relative abundances of microbial species in each individual, which are known (and described above), and the levels of various nutrients reaching their lower gut, which we fit using the model. Note that we always used the experimentally measured relative microbial abundances, which simplified calculations and made the model easy to run. This also removed the model’s dependence on the initial relative abundances, and the need for a new set of parameters to represent them. Moreover, this assumption is valid; our model’s calculated abundances are very close to the experimentally observed abundances (see Fig 2B). This is discussed in greater detail in the next section. For simplicity, we did not explicitly include the various polysaccharides (dietary fibers, starch, etc.) known to constitute the bulk of an individual’s diet. Instead, we chose not to include the polysaccharides themselves, but instead use their breakdown products as the direct nutrient intake to the gut. The reason for this is our limited quantitative understanding of the processes by which these polysachharides are converted to these breakdown products, e.g., the levels of extracellular enzymes, variability in their composition (their lability), etc. This curated nutrient intake consisted of 19 metabolites, such as arabinose, raffinose, and xylose (see S2 Table for the complete list of metabolites). Constructing and validating the trophic model Our model incorporates a set of observed microbial species abundances and the known metabolic cross-feeding interactions between these species, to calculate and predict both the step-wise metabolic flow through the lower gut, and the resulting fecal metabolome. The model does this on an individual-to-individual basis. We started simulating the model with the various levels of nutrients entering the lower gut, represented by the 19-dimensional vector . Each element of , say cnut,i represents the amount of one of the 19 metabolites entering the lower gut of that individual. We inferred these amounts through a fitting procedure described in the next section. Throughout this description, we use the subscript i to refer to metabolites, and α to refer to microbial species. In the first trophic level, we calculated how these nutrients entering the gut were consumed by the gut microbiome and converted to microbial biomass, and metabolic byproducts, . For this, we calculated the relative increase in microbial biomass for each species, α, as follows: (1) where (1 − f) represents the fraction of consumed metabolites converted to biomass, and f represents the fraction of input nutrients converted to metabolic byproducts. Ain is a matrix which represents how each species takes up and consumes the nutrients it is capable of. Each term of this matrix, (Ain)α,i was set to zero if species α was incapable of consuming metabolite i as a nutrient (using the set of microbial metabolic capabilities in S1 Table). If species α was instead capable of consuming metabolite i as a nutrient, then (Ain)α,i was set as follows: (2) Here, λα,i represents the rate at which species α takes up nutrient i, is the experimentally measured abundance of strain α, and κi is a normalization constant such that, for any nutrient i, the (Ain)α,i for all microbial species α sum up to 1, i.e., . Throughout the manuscript, we set λα,i = 1 for all values of α and i; this is because we lacked knowledge of the precise rates at which each species takes up different nutrients, and had insufficient data about microbial growth to fit them using our model. To verify that this assumption did not significantly affect the predictions of our model, we repeated our metabolome predictions 100 times by assigning each value of λα,i randomly, chosen from a uniform distribution between 0 and 1 (see S1 Fig). After calculating the contribution of nutrient consumption to microbial biomass, we computed the relative levels of the first level of metabolic byproducts produced by them, as follows: (3) where the 1 indicates that we were calculating the first layer of byproducts, and i, each metabolite which could be secreted as a byproduct. Aout is matrix which represents which byproduct each species could secrete, and in what amount. Each term of this matrix (Aout)i,α was set to zero if species α could not secrete metabolite i as a byproduct (using the interactions in NJS16 described previously; see S1 Table). If species α was instead capable of secreting metabolite i as a byproduct, then (Aout)i,α was set as follows: (4) where is the number of byproducts that species α was capable of secreting. In the second trophic level (and all subsequent levels), we calculated how the byproducts secreted by the microbes in the previous step were consumed by the gut microbiome and converted to further biomass and byproducts. After Nℓ such steps, we calculated the final microbial abundances, , and the accumulated metabolic byproducts, . We wil later compare these with the individual’s experimentally measured metagenome and fecal metabolome, respectively. The final microbial abundances, , were calculated as follows: (5) Here, we chose the appropriate number of trophic levels, Nℓ and the byproduct fraction, f, by comparing the model’s predicted fecal metabolome with the individual’s experimentally measured metabolome. The number of levels and byproduct fraction that best matched the experimentally observed metabolomes, averaged over all individuals, were the ones that were considered to best represent their gut microbiome. To measure the best match, we used two different measures: (1) the Pearson correlation coefficient between the predicted and experimentally measured fecal metabolomes (see Fig 2A), and (2) a logarithmic accuracy, i.e., the average difference between the log-transformed predicted and observed metabolome levels (see S2 Fig), i.e., , where mi is the experimentally measured metabolome level of metabolite i, and pi is the predicted metabolome level of metabolite i, calculated by summing up the levels of all unused metabolites. Specifically, at each level, we calculated the byproducts similar to the first level (see Eq (3)), as follows: (6) We split the byproducts at each level, , into two parts: a consumable part, and an unconsumable part, . While the consumable part of the byproducts was available to the next trophic level of microbial species, the unconsumable part was composed of all the byproducts which could not be consumed by any microbial species in the individual’s gut microbiome (i.e., it satisfied ). The former, consumable part was obtained by subtracting the unconsumable part from the generated byproducts at each level, i.e., . Finally, we calculated the predicted metabolome, , by adding up the unconsumable byproducts from all previous levels with all the byproducts from the final trophic level, as follows: (7) Note that while the Pearson correlation (and its associated P-value) gives an indication of the similarity in the trends predicted by our model with the experimentally observed metabolome, the logarithmic accuracy actually calculates the average error (measured in orders of magnitude) between the predicted and experimentally observed metabolomes. In both cases, we used the log-transformed values because we were interested in comparing the quality of our predictions with the experimental measurements at the level of resolution of an order of magnitude. This avoided overfitting in the model. Moreover, note that the nutrient input to the model (which we fit; see next section) resulted in a predicted set of microbial abundances, (obtained from Eq (5)) that were very close to the experimentally observed abundances. This allowed us to simplify our calculation; we used the experimentally measured microbial abundances instead of a more complicated, step-wise calculation in the sum of Eq (5). For each metabolome correlation coefficient that we calculated, we also corrected its associated P-value, in order to account for the two adjustable parameters in our model. We did this by adjusting (1) t-statistic: the adjusted t-statistic was obtained by dividing the original t-statistic by , where n was the number of metabolites (or points) that were used to measure the correlation, and p was the number of adjustable model parameters (in our case, p = 2); (2) t-test: typically the one-tailed t-test with degree of freedom n − 2 is used to compute of P-value for the Pearson correlation coefficient. Here the one-tailed t-test with degree of freedom n − 2 − p was used to account for adjustable model parameters. Fitting and inferring the nutrient intake to the gut Simulating the model required us to know the nutrient intake to the gut, for which there are no available experimental measurements. Therefore, we inferred the amounts of these 19 intake metabolites by fitting the microbial abundances predicted by our model with those measured from each individual’s microbiome. We used a nonlinear optimization technique for this (implemented as lsqnonlin in MATLAB R2018a, Mathworks Inc.). We initially chose a random set of nutrient inputs, each chosen randomly from a uniform distribution between 0 and 1, and normalized so that all nutrient inputs summed up to one. For this random set of nutrient inputs, we calculated the predicted microbial abundances using Eq (5). We then calculated the error in this prediction, by using the log-transformed differences between the predicted and experimentally measured microbial abundances, i.e., , where S is the number of microbial species with non-zero abundances in the individual, pα is the predicted relative abundance of species α, and mα is the experimentally measured abdunace of species α. We then let the nonlinear optimization routine vary and choose that set of nutrient inputs, which minimized this error. We assumed that this set of nutrient inputs, which best explained the observed microbial abundances, given the microbial cross-feeding interactions, as the nutrient intake to the lower gut, or first trophic level, of that individual. Note that this is only step where we perform fitting in the model. All other subsequent steps, especially the prediction of the fecal metabolome, is an independent prediction from the model. Typically, we fit 19 metabolite amounts for each human individual, who had roughly 80 microbial species. Shuffling microbial metabolic capabilities to test model predictions To test how good our model’s gut metagenome and fecal metabolome predictions were against a null, or random, expectation, we repeated our simulations using a randomly shuffled set of microbial metabolic capabilities. For each individual microbial species, we picked one metabolite that they either could consume randomly, and swapped it with a metabolite that could be consumed by another microbial species. We also did this separately and independently with metabolites that they could secrete. Such swaps ensured that each microbial species could still consume and secrete the same number of metabolites as in the original dataset, but shuffled all the ecologically relevant metabolic relationships between species and metabolites. The swapping is performed three times as many the number of edges in the network to guarantee enough randomness. At the end of several rounds of swapping such relationships, we repeated our model’s simulations exactly as described above, except with this shuffled set of microbial metabolic capabilities. Calculating level-by-level diversity To quantify the diversity of microbes and metabolites at each trophic level across the 380 individuals we studied, we used three measures popular in the ecosystems literature: namely the α-, β- and γ- diversity [38, 39, 40]. For each individual, we calculated the α-diversity of microbes and metabolites on each of the trophic levels. For this we first quantified the relative contributions of a given level to microbial abundances, and separately to the fecal metabolome profile. The contribution of a given trophic level ℓ to the relative abundance of a species (microbial or, separately, metabolic) i in a specific individual j is given by pi(ℓ, j) normalized by . The α-diversity where 〈⋅〉j represents taking the average across 380 individuals used in our analysis. Across all individuals, we calculated the γ-diversity of microbes and metabolites in their gut, which quantified the “global” diversity across all individuals, as: where pi(ℓ) = 〈pi(ℓ, j)〉j is the mean relative abundance of species (or metabolite) i at the trophic level ℓ across all individuals used in our analysis. Finally, to quantify the between-individual variability in microbial and metabolite diversity, we calculated the overall β-diversity, which is the ratio of the global to local diversity, as: Code availability All computer code and extracted data files used in this study are available at the following URL: https://github.com/eltanin4/trophic_gut. Obtaining data for microbial metabolic capabilities For information about the metabolic capabilities of human gut microbes, we adopted a recently published manually-curated database, NJS16, which includes such data for 570 common gut microbial species and 244 relevant metabolites from Ref. [6]. This database recorded, for each microbial species, which metabolites each of the species could consume, and which they secreted as byproducts. Since we were interested in those metabolites that could be used for microbial growth, we removed metabolites such as ions (e.g., Na+, Ca+) from NJS16. Moreover, we constrained our analyses to microbes only, and therefore removed the 3 types of human cells from NJS16. This left us with a database with 567 microbes, 235 metabolites and 4,248 interactions connecting these microbes with corresponding metabolites (see S1 Table for the complete table of interactions). Obtaining metagenomic and metabolomic data To calibrate the key parameters of our model, we used a previously published dataset, namely a 16S rRNA sequencing study of 41 human individuals from rural and urban areas in Thailand [22]. From these data, we collected the reported 16S rRNA OTU abundances as well as their corresponding taxonomy. We explicitly removed all OTUs that did not have an assigned species-level taxonomy. The remaining OTUs explained roughly 71%(±15%) of the bacterial abundances per sample. We then mapped these species names to those listed in the NJS16 database. We found an exact match for 110 species out of 208 in this table. In order to improve the species coverage from the abundance data, we manually mapped the remaining species in the following manner. For those genera in NJS16, whose member species had identical metabolic capabilities, we assumed that the capabilities of other, unmapped species from these genera were the same as these species. For several well-studied bacterial genera, such as Bacteroides, we determined a “core” set of metabolic capabilities (i.e., those metabolites that could either be consumed or secreted by all species in that genus), and assigned them to all unmapped species in that genus (i.e., those with known abundances, but otherwise understudied metabolic capabilities in NJS16). This allowed us to map an additional 20 microbial species from the abundance data, and incorporate into our model. Note that we did this additional mapping, only for those genera, where species metabolic capabilities were identical. To quantify the metabolome levels in each individual, we used the available quantitative metabolome profiles (obtained via from CE-TOF MS) corresponding to the 41 individuals whose metagenomic samples we had. Here, we mapped the reported metabolites to our database of metabolic capabilities using KEGG identifiers, which revealed 84 such measured metabolites. To make predictions about metabolic flow and effective diversity from our model, we used additional metagenomic datasets, namely those from the Human Microbiome Project (HMP) [1] and MetaHIT [2, 21], for which we had microbial abundances, but no fecal metabolome. This resulted in an additional 380 human individuals, for which we obtained tables of MetaPhlAn2 microbial abundances, and mapped species names to those in NJS16 using the same procedure described above. Here, out of a total of 532 microbial species detected over these data, we could map and incorporate 316 species. Of these, 207 were mapped through an exact taxonomic match, and 109 by a genus-capability match. These incorporated species covered, on average, 90% of the total microbial abundance in each individual sample. Determining the components of the nutrient intake to the gut The inputs of our model are the experimentally measured relative abundances of microbial species in each individual, which are known (and described above), and the levels of various nutrients reaching their lower gut, which we fit using the model. Note that we always used the experimentally measured relative microbial abundances, which simplified calculations and made the model easy to run. This also removed the model’s dependence on the initial relative abundances, and the need for a new set of parameters to represent them. Moreover, this assumption is valid; our model’s calculated abundances are very close to the experimentally observed abundances (see Fig 2B). This is discussed in greater detail in the next section. For simplicity, we did not explicitly include the various polysaccharides (dietary fibers, starch, etc.) known to constitute the bulk of an individual’s diet. Instead, we chose not to include the polysaccharides themselves, but instead use their breakdown products as the direct nutrient intake to the gut. The reason for this is our limited quantitative understanding of the processes by which these polysachharides are converted to these breakdown products, e.g., the levels of extracellular enzymes, variability in their composition (their lability), etc. This curated nutrient intake consisted of 19 metabolites, such as arabinose, raffinose, and xylose (see S2 Table for the complete list of metabolites). Constructing and validating the trophic model Our model incorporates a set of observed microbial species abundances and the known metabolic cross-feeding interactions between these species, to calculate and predict both the step-wise metabolic flow through the lower gut, and the resulting fecal metabolome. The model does this on an individual-to-individual basis. We started simulating the model with the various levels of nutrients entering the lower gut, represented by the 19-dimensional vector . Each element of , say cnut,i represents the amount of one of the 19 metabolites entering the lower gut of that individual. We inferred these amounts through a fitting procedure described in the next section. Throughout this description, we use the subscript i to refer to metabolites, and α to refer to microbial species. In the first trophic level, we calculated how these nutrients entering the gut were consumed by the gut microbiome and converted to microbial biomass, and metabolic byproducts, . For this, we calculated the relative increase in microbial biomass for each species, α, as follows: (1) where (1 − f) represents the fraction of consumed metabolites converted to biomass, and f represents the fraction of input nutrients converted to metabolic byproducts. Ain is a matrix which represents how each species takes up and consumes the nutrients it is capable of. Each term of this matrix, (Ain)α,i was set to zero if species α was incapable of consuming metabolite i as a nutrient (using the set of microbial metabolic capabilities in S1 Table). If species α was instead capable of consuming metabolite i as a nutrient, then (Ain)α,i was set as follows: (2) Here, λα,i represents the rate at which species α takes up nutrient i, is the experimentally measured abundance of strain α, and κi is a normalization constant such that, for any nutrient i, the (Ain)α,i for all microbial species α sum up to 1, i.e., . Throughout the manuscript, we set λα,i = 1 for all values of α and i; this is because we lacked knowledge of the precise rates at which each species takes up different nutrients, and had insufficient data about microbial growth to fit them using our model. To verify that this assumption did not significantly affect the predictions of our model, we repeated our metabolome predictions 100 times by assigning each value of λα,i randomly, chosen from a uniform distribution between 0 and 1 (see S1 Fig). After calculating the contribution of nutrient consumption to microbial biomass, we computed the relative levels of the first level of metabolic byproducts produced by them, as follows: (3) where the 1 indicates that we were calculating the first layer of byproducts, and i, each metabolite which could be secreted as a byproduct. Aout is matrix which represents which byproduct each species could secrete, and in what amount. Each term of this matrix (Aout)i,α was set to zero if species α could not secrete metabolite i as a byproduct (using the interactions in NJS16 described previously; see S1 Table). If species α was instead capable of secreting metabolite i as a byproduct, then (Aout)i,α was set as follows: (4) where is the number of byproducts that species α was capable of secreting. In the second trophic level (and all subsequent levels), we calculated how the byproducts secreted by the microbes in the previous step were consumed by the gut microbiome and converted to further biomass and byproducts. After Nℓ such steps, we calculated the final microbial abundances, , and the accumulated metabolic byproducts, . We wil later compare these with the individual’s experimentally measured metagenome and fecal metabolome, respectively. The final microbial abundances, , were calculated as follows: (5) Here, we chose the appropriate number of trophic levels, Nℓ and the byproduct fraction, f, by comparing the model’s predicted fecal metabolome with the individual’s experimentally measured metabolome. The number of levels and byproduct fraction that best matched the experimentally observed metabolomes, averaged over all individuals, were the ones that were considered to best represent their gut microbiome. To measure the best match, we used two different measures: (1) the Pearson correlation coefficient between the predicted and experimentally measured fecal metabolomes (see Fig 2A), and (2) a logarithmic accuracy, i.e., the average difference between the log-transformed predicted and observed metabolome levels (see S2 Fig), i.e., , where mi is the experimentally measured metabolome level of metabolite i, and pi is the predicted metabolome level of metabolite i, calculated by summing up the levels of all unused metabolites. Specifically, at each level, we calculated the byproducts similar to the first level (see Eq (3)), as follows: (6) We split the byproducts at each level, , into two parts: a consumable part, and an unconsumable part, . While the consumable part of the byproducts was available to the next trophic level of microbial species, the unconsumable part was composed of all the byproducts which could not be consumed by any microbial species in the individual’s gut microbiome (i.e., it satisfied ). The former, consumable part was obtained by subtracting the unconsumable part from the generated byproducts at each level, i.e., . Finally, we calculated the predicted metabolome, , by adding up the unconsumable byproducts from all previous levels with all the byproducts from the final trophic level, as follows: (7) Note that while the Pearson correlation (and its associated P-value) gives an indication of the similarity in the trends predicted by our model with the experimentally observed metabolome, the logarithmic accuracy actually calculates the average error (measured in orders of magnitude) between the predicted and experimentally observed metabolomes. In both cases, we used the log-transformed values because we were interested in comparing the quality of our predictions with the experimental measurements at the level of resolution of an order of magnitude. This avoided overfitting in the model. Moreover, note that the nutrient input to the model (which we fit; see next section) resulted in a predicted set of microbial abundances, (obtained from Eq (5)) that were very close to the experimentally observed abundances. This allowed us to simplify our calculation; we used the experimentally measured microbial abundances instead of a more complicated, step-wise calculation in the sum of Eq (5). For each metabolome correlation coefficient that we calculated, we also corrected its associated P-value, in order to account for the two adjustable parameters in our model. We did this by adjusting (1) t-statistic: the adjusted t-statistic was obtained by dividing the original t-statistic by , where n was the number of metabolites (or points) that were used to measure the correlation, and p was the number of adjustable model parameters (in our case, p = 2); (2) t-test: typically the one-tailed t-test with degree of freedom n − 2 is used to compute of P-value for the Pearson correlation coefficient. Here the one-tailed t-test with degree of freedom n − 2 − p was used to account for adjustable model parameters. Fitting and inferring the nutrient intake to the gut Simulating the model required us to know the nutrient intake to the gut, for which there are no available experimental measurements. Therefore, we inferred the amounts of these 19 intake metabolites by fitting the microbial abundances predicted by our model with those measured from each individual’s microbiome. We used a nonlinear optimization technique for this (implemented as lsqnonlin in MATLAB R2018a, Mathworks Inc.). We initially chose a random set of nutrient inputs, each chosen randomly from a uniform distribution between 0 and 1, and normalized so that all nutrient inputs summed up to one. For this random set of nutrient inputs, we calculated the predicted microbial abundances using Eq (5). We then calculated the error in this prediction, by using the log-transformed differences between the predicted and experimentally measured microbial abundances, i.e., , where S is the number of microbial species with non-zero abundances in the individual, pα is the predicted relative abundance of species α, and mα is the experimentally measured abdunace of species α. We then let the nonlinear optimization routine vary and choose that set of nutrient inputs, which minimized this error. We assumed that this set of nutrient inputs, which best explained the observed microbial abundances, given the microbial cross-feeding interactions, as the nutrient intake to the lower gut, or first trophic level, of that individual. Note that this is only step where we perform fitting in the model. All other subsequent steps, especially the prediction of the fecal metabolome, is an independent prediction from the model. Typically, we fit 19 metabolite amounts for each human individual, who had roughly 80 microbial species. Shuffling microbial metabolic capabilities to test model predictions To test how good our model’s gut metagenome and fecal metabolome predictions were against a null, or random, expectation, we repeated our simulations using a randomly shuffled set of microbial metabolic capabilities. For each individual microbial species, we picked one metabolite that they either could consume randomly, and swapped it with a metabolite that could be consumed by another microbial species. We also did this separately and independently with metabolites that they could secrete. Such swaps ensured that each microbial species could still consume and secrete the same number of metabolites as in the original dataset, but shuffled all the ecologically relevant metabolic relationships between species and metabolites. The swapping is performed three times as many the number of edges in the network to guarantee enough randomness. At the end of several rounds of swapping such relationships, we repeated our model’s simulations exactly as described above, except with this shuffled set of microbial metabolic capabilities. Calculating level-by-level diversity To quantify the diversity of microbes and metabolites at each trophic level across the 380 individuals we studied, we used three measures popular in the ecosystems literature: namely the α-, β- and γ- diversity [38, 39, 40]. For each individual, we calculated the α-diversity of microbes and metabolites on each of the trophic levels. For this we first quantified the relative contributions of a given level to microbial abundances, and separately to the fecal metabolome profile. The contribution of a given trophic level ℓ to the relative abundance of a species (microbial or, separately, metabolic) i in a specific individual j is given by pi(ℓ, j) normalized by . The α-diversity where 〈⋅〉j represents taking the average across 380 individuals used in our analysis. Across all individuals, we calculated the γ-diversity of microbes and metabolites in their gut, which quantified the “global” diversity across all individuals, as: where pi(ℓ) = 〈pi(ℓ, j)〉j is the mean relative abundance of species (or metabolite) i at the trophic level ℓ across all individuals used in our analysis. Finally, to quantify the between-individual variability in microbial and metabolite diversity, we calculated the overall β-diversity, which is the ratio of the global to local diversity, as: Code availability All computer code and extracted data files used in this study are available at the following URL: https://github.com/eltanin4/trophic_gut. Supporting information S1 Fig. Effect of changing kinetic parameters on model prediction. Scatter plot of the measured and predicted metabolome where, instead of considering equal specific nutrient uptake and byproduct release rates, λ’s in our model, we take several random sets (in black). Error bars (in black) indicate standard deviation in the predicted levels of specific metabolites for different sets of λ’s. The solid line represents x = y. Red squares indicate the predicted metabolome for the default set of kinetic parameters used, i.e., when all of λ’s were set equal to 1. https://doi.org/10.1371/journal.pcbi.1007524.s001 (EPS) S2 Fig. Calibrating the model parameters using logarithmic accuracy. Heatmap of the logarithmic accuracy between experimentally measured and predicted fecal metabolomes for different combinations of parameters f and Nℓ. The logarithmic accuracy quantifies the average order-of-magnitude error in our predicted fecal metabolome, when compared with the experimentally measured one (see Methods for details). The plotted value is the logarithmic accuracy averaged over 41 individuals in Ref. [22]. https://doi.org/10.1371/journal.pcbi.1007524.s002 (EPS) S3 Fig. Testing model predictions for byproduct fractions beyond 0.9. Logarithmic accuracy of the model’s predictions between the metabolomes (see Methods for details) for byproduct fraction, f, values 0.95 and 0.99. Higher values on the y-axis means worse predictions. This suggests that at 4 trophic levels, f = 0.9 gives the best calibration. https://doi.org/10.1371/journal.pcbi.1007524.s003 (EPS) S4 Fig. Adjusted P-values for the model predictions. Adjusted P-values (see Methods) at different byproduct fractions, f’s, at trophic levels 2 (blue), 3 (orange), 4 (yellow), 5 (purple) and 10 (green). https://doi.org/10.1371/journal.pcbi.1007524.s004 (EPS) S5 Fig. Layer-wise network for one of the individuals from the calibrated dataset. Blue nodes correspond to metabolites, red nodes to the microbes. Blue edges show metabolite consumption, red—production. https://doi.org/10.1371/journal.pcbi.1007524.s005 (EPS) S6 Fig. Results for model calibration assuming a constant enzyme budget for each microbe. (A) Heatmap of the average Pearson correlation between experimentally measured and predicted metabolomes for different combinations of parameters f and Nℓ. The correlation coefficient is averaged over 41 individuals in Ref. [22]. For this case with a constant enzyme budget, the best average Pearson correlation coefficient 0.69 is given by f = 0.95 and Nℓ = 4. (B) Heatmap of the logarithmic accuracy between experimentally measured and predicted metabolomes for different combinations of parameters f and Nℓ. For this case, the best average logarithmic accuracy 0.69 is given by f = 0.1 and Nℓ = 10. https://doi.org/10.1371/journal.pcbi.1007524.s006 (EPS) S7 Fig. The process of model calibration is simulated 100 times assuming random byproduct production abilities for each microbe. The byproduct production abilities of a microbial strain α are drawn from a Dirichlet distribution. (A) A frequency distribution of the highest average Pearson correlation coefficients between the predicted metabolome and the experimentally measured metabolome. The dashed line corresponds to the best mean Pearson correlation coefficients 0.678 (Nℓ = 4 and f = 0.9) achieved in the heatmap shown in Fig 2. The probability of such a simulation generating a Pearson correlation coefficient between is higher than 0.678 is 0.0. https://doi.org/10.1371/journal.pcbi.1007524.s007 (EPS) S8 Fig. The process of model calibration is simulated 100 times using shuffled microbial metabolic capabilities. (A) A frequency distribution of the highest average Pearson correlation coefficients between the predicted metabolome and the experimentally measured metabolome. The dashed line corresponds to the best mean Pearson correlation coefficients 0.678 (Nℓ = 4 and f = 0.9) achieved in the heatmap shown in Fig 2. The probability of such a simulation generating a Pearson correlation coefficient between is higher than 0.678 is 0.0. https://doi.org/10.1371/journal.pcbi.1007524.s008 (EPS) S9 Fig. Results for model performance after shuffled microbial names. (A) A frequency distribution of the highest average Pearson correlation coefficients between the predicted metabolome and the experimentally measured metabolome, when the microbe names are shuffled in the microbial metabolic capabilities. We show here results from 100 such shuffling attempts, where each correlation coefficient in the distribution corresponds to one shuffling attempt. The dashed red line corresponds to the best mean Pearson correlation coefficients 0.678 (Nℓ = 4 and f = 0.9) without any shuffling, the same as in in Fig 2. The fraction of shuffling attempts resulting in a correlation coefficient better than 0.678 is 0.13. https://doi.org/10.1371/journal.pcbi.1007524.s009 (EPS) S10 Fig. Measuring the statistical uncertainty in calibrated model parameters by bootstrapping. Distribution of resulting calibrated model parameters (A) f, the byproduct fraction, and (B) Nℓ, the number of trophic levels, using the same procedure as described in Methods, but on 100 synthetic datasets, each with 41 individuals, constructed by sampling the 41 original individuals used in the main text with replacement. The resulting distributions provide a measure of statistical uncertainty in the calibrated model parameters, with the level of uncertainty in f being approximately 0.05 (most datasets suggest a value of f = 0.95 instead of 0.90), and that in Nℓ being negligible (most datasets suggest a value of Nℓ, consistent with the real dataset). https://doi.org/10.1371/journal.pcbi.1007524.s010 (EPS) S11 Fig. Results of model fitted metagenomes for all 41 individuals from the calibrated dataset in Fig 2. Results from all 41 model fitted metagenomes, an example of which was Fig 2B. https://doi.org/10.1371/journal.pcbi.1007524.s011 (EPS) S12 Fig. Results of model predicted fecal metabolomes for all 41 individuals from the calibrated dataset in Fig 2. Results from all 41 model fitted metagenomes, an example of which was Fig 2C. https://doi.org/10.1371/journal.pcbi.1007524.s012 (EPS) S1 Table. Microbial and metabolite interactions used in the model. Table of all 4,248 interactions between microbes and metabolites used in the model, from Ref. [6]. https://doi.org/10.1371/journal.pcbi.1007524.s013 (XLSX) S2 Table. Components of the nutrient intake to the gut. List of all 19 metabolites used to fit the gut nutrient intake in the model. https://doi.org/10.1371/journal.pcbi.1007524.s014 (XLSX) S3 Table. Metabolome predictions of the model for 380 individuals from the Human Microbiome Project (HMP) and the MetaHIT study. All metabolites in metabolome predicted by the model with global parameters f = 0.9 and Nℓ = 4 for all 380 individuals are listed. https://doi.org/10.1371/journal.pcbi.1007524.s015 (XLSX) Acknowledgments We thank Parth Pratim Pandey for useful discussions.

Chasapi, Anastasia;Aivaliotis, Michalis;Angelis, Lefteris;Chanalaris, Anastasios;Iliopoulos, Ioannis;Kappas, Ilias;Karapiperis, Christos;Kyrpides, Nikos C.;Pafilis, Evangelos;Panteris, Eleftherios;Topalis, Pantelis;Tsiamis, George;Vizirianakis, Ioannis S.;Vlassi, Metaxia;Promponas, Vasilis J.;Ouzounis, Christos A.

Pierella, Camilla;Casadio, Maura;Mussa-Ivaldi, Ferdinando A.;Solla, Sara A.

doi: 10.1371/journal.pcbi.1007118pmid: 31860655

Introduction A distinct feature of the neuromotor system is the large number of muscles and degrees of freedom that allow it to attain a specific motor goal in a number of different ways [1]. This is both a resource and a computational challenge: while this motor redundancy provides the brain with a multitude of options, an enabling feature of motor dexterity, it also results in a family of ill-posed problems characterized by a lack of uniqueness in their solutions [2, 3]. Here, we consider the challenge posed by redundancy from the perspective of learning. How does the central nervous system learn to perform a novel task when multiple alternative solutions are available? This question acquires clinical relevance when a person suffering from loss of limb or some form of paralysis must reorganize the still available mobility to recover quality of life and independence through the operation of assistive devices–such as wheelchairs or robotic assistants–and dedicated human-machine interfaces. Are some possible solutions more easily learned than others? Could learning be facilitated by adapting the interface to the solution that the subject seems to be acquiring? In the last two decades, studies of motor learning [4–8] have established that the adaptation of limb movements to external perturbing forces takes place through the gradual formation of an internal representation, or "internal model" of these forces. To be predictable, the forces cannot be random disturbances, but must have a deterministic structure expressed in relation to the motion of the body and to the brain’s commands [6–9]. Donchin et al. [10] and others [11–13] have proposed to represent the development of such an internal model as the evolution of a dynamical system. Internal models are of two types: forward and inverse. Forward models owe their name to their predictive representation of the process that transforms action commands into their sensory consequences. Inverse models reverse the direction of this process by deriving action commands from desired sensory outcomes. Earlier theoretical work by Jordan and Rumelhart [14] considered how the learning of actions can be viewed as the concurrent learning of forward and inverse models of actions. They introduced the concept of distal learning, where the learner has to find a mapping from desired outcomes to actions in order to achieve a desired outcome. To do so, the subject begins by forming a predictive forward model of the transformation from actions to distal outcomes. Such transformations are often not known a priori, thus the forward model must generally be learned by exploring the outcomes associated with particular choices of action. Once the forward model has been at least partially learned, it can be used to guide the learning of an inverse model that predicts the action needed to achieve the distal outcome. Here, we extend the distal learning approach to the learning of a novel map established by a body machine interface (BoMI) that translates movements of the upper body (shoulders and arm) into movements of an external object that users must guide to a set of target locations. This BoMI has been shown to be an assistive tool for people that have lost the use of their hands after injury to the cervical spinal cord [15–19]. However, the field still lacks a mathematical description of the process that takes place while subjects are learning to proficiently use this BoMI. An analysis of if and how people form an internal model of the interface, how this representation evolves with time and depends upon the initial state, will allow us to characterize the efficiency of the interaction with assistive interfaces, both for healthy subjects and for subjects with different types and levels of disability. This fundamental knowledge will facilitate the development of an advanced coadaptive interface, capable of handling changing motor abilities as well as changing operational demands. For this purpose, we investigate how unimpaired subjects become skilled at controlling an external object via the BoMI. Our findings are consistent with the hypothesis that learning proceeds through the concurrent evolution of coupled forward and inverse models of the body-to-object mapping established by the BoMI. The validity of this description is tested by comparing the evolution of motor performance predicted by the model with the actual learning performance observed in a group of human subjects. In addition, we compare the forward and inverse models derived from simulated learning dynamics with forward and inverse models estimated from motion data at different stages of learning. Results A model of learning while practicing control via a body-machine interface We investigated how users of a body machine interface learn to reorganize or "remap" their body motions as they practice controlling an external object through the BoMI. The controlled object could be a wheelchair, a robotic assistant, or a drone [16, 17, 20]. Here we focus on the control of a computer cursor whose two-dimensional coordinates determine its location on a computer screen. Effectiveness in cursor control is the first and most common benchmark for interfaces based on neural activity [21–23], as the ability to control two-dimensional position is readily applied to a variety of tasks (e.g., an action performed via a joystick, entering computer text, etc.). We consider interfaces in which a linear mapping associates the body motion signals to the coordinates of the external object. Importantly, there is an imbalance between the dimensionality of the task space and that of the body signals, the latter being larger. Thus, any position of the controlled object corresponds to many, potentially infinite, different body configuration signals. The BoMI matrix H establishes a linear map between these two spaces; H has as many rows K as signals are needed to control the external object, and as many columns S as there are body signals. Not being square, the matrix H does not have a unique inverse. But there exist infinite “right inverses” that combined with H yield the K x K identity matrix in the task space of external control signals. Each such right inverse transforms a desired position of the controlled object into one particular set of values for the body signals. We consider users to be competent when they are able to move their body successfully in response to a presented target for the controlled object. Mathematically, we consider this as finding one right inverse G of the mapping H, out of a multitude of possible and equally valid choices. Current theories and experimental observations [10] suggest that learning is a dynamical process in which the learners modify their behavior based on the errors observed at each iteration of a task. In the kinematically redundant conditions considered here, learning is problematic because a given low-dimensional task error signal has multiple representations in the high-dimensional body space. Here, we considered an error surface defined by the squared task error in the space of the elements of the target-to-body map G adopted by a learner, where G is the learner’s inverse model of the body-to-cursor mapping H established by the BoMI. We implemented our learning model based on the hypothesis that the learners update the map G by moving along this error surface, following the line of steepest descent determined by the gradient of the squared error with respect to the elements of G. This error gradient depends on several variables; some can be directly observed by the learner, such as the error made in attempting to reach a given target position of the external device. However, the error gradient also depends upon the elements of the interface map H, which the learner cannot be assumed to know. Therefore, gradient descent learning of the inverse model requires a concurrent learning of the forward model. The latter requires a different error surface, since the forward map relates body configurations to the consequent position of the controlled object. Forward model learning does not require a target position for the controlled object, as the relevant error in this case is the difference between predicted and observed position of the controlled object. The squared prediction error defines an error surface in the space of the elements of the estimated forward map . Learning is thus described through two first-order dynamical processes determined by two state equations. A forward learning process: (1) and an inverse learning process (2) where H(n) is a KxS matrix, G(n) is an SxK matrix, p(n) and q(n) are column vectors of respectively Kx1 and Sx1 elements, ε and η are scalars. In the experiments reported here, S = 8 and K = 2. The reaching error e(n) = (p(n) − u(n)) that guides this process is the difference between actual and desired positions of the controlled object. For details on the derivation of these equations, see Methods. The forward and inverse models are effectively the states of the respective learning processes, whose n-th iteration results in state variables and G(n). Eq (1) updates the subject’s estimate of the forward model H that transforms S-dimensional body configurations q into K-dimensional positions p of the controlled object. The term in parentheses is the prediction error between actual and predicted positions of the controlled object. The concurrent process described by Eq (2) is the learning of the inverse model G that the subjects use to map the target object position u onto body signals q. Two possibly different learning rates, ε and η, provide inverse time constants for the respective learning processes. Since we focus on the case in which forward and inverse learning are carried out concurrently, naïve users are immediately presented with the reaching task, and as they practice they observe both the reaching error and the prediction error. Eqs (1) and (2) are coupled through and through a third equation that describes the body signals currently adopted to reach the target position: (3) This apparently innocuous interaction has potentially harmful effects on the convergence of the coupled learning dynamics, as the second term in the gradient contribution to Eq (1) includes a quadratic factor in q(n) and thus in G(n). This contribution may result in local minima, a problem avoided by adding noise to Eq (3), to obtain: (4) We validated our approach with six healthy subjects that learned to control the two-dimensional movement of a cursor on a monitor using eight signals from their upper body motions (shoulders and upper arms on both sides, see Methods for details). In these experiments, S = 8 and K = 2, and the state space of the combined forward-inverse learning was 2x16 = 32-dimensional. Dynamics of learning in human subjects We monitored the learning process through two scalar metrics: RE, the L2 norm of the reaching error (the difference between actual p(n) and target u(n) location of the cursor at the end of each reaching movement), and IME, the spectral norm of the inverse model error (the difference between the identity matrix IK and the product between the interface map H and the current estimate G(n) of the inverse model). The spectral norm of a matrix, indicated here by ∥·∥ to emphasize its analogy with the L2 norm of a vector, is the maximum singular value of the matrix. We estimated G(n) from target and body signal data by least squares fit on Eq (3). The elements of G(n) were estimated using data from 12 trials: trial n and its 11 preceding trials. Overlapping moving windows that included 12 trials were shifted by one trial at each iteration. In the experiment, each subject practiced with a personalized body-to-cursor map, chosen to reflect the statistics of its own freely produced upper body motions (see calibration procedure in Methods). With increasing number of practice trials, their RE decreased to values closer to the target radius (1 cm, Fig 1a). The learning process took over one hundred steps (172 ± 32 trials, mean ± SEM) before reaching asymptotic performance (Fig 1a), identified as the time when the norm of the reaching error was smaller than the radius of the target. Similarly, the matrix G(n) converged to a generalized inverse of the body-to-cursor map (Fig 1b). Although the subjects explored a number of different body configurations while learning how to control the cursor, in the end they found a stable movement pattern, and built an inverse model G. Note the asymptotically small variations in the acquired G, with ∥ΔG∥ about 10% of ∥G∥ (Fig 2). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Subjects learn to use the body-machine interface. Data for the six subjects enrolled in the study (S1- S6). (a) Temporal evolution of the norm RE of the reaching error, calculated over a moving window that includes the current and the 11 preceding trials. (b) Temporal evolution of the norm IME of the inverse model error. The inverse model G(n) was obtained by a least squares fit on Eq (3) from target and body signal data for the current and the 11 preceding trials. https://doi.org/10.1371/journal.pcbi.1007118.g001 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Temporal evolution of the changes in the inverse map G(n). Changes in the inverse map as a function of trial number n, quantified by ∥ΔG(n)∥ = ∥G(n) − G(n−1)∥⁄∥G(n)∥ (see Eq (19) in Methods), for the six subjects enrolled in the study (S1-S6). https://doi.org/10.1371/journal.pcbi.1007118.g002 For each subject, both RE and IME errors decreased with time following a trend captured by an exponential curve (Eq (22)). The learning rates, each given by the inverse of the time constant of the corresponding exponential fit, are shown in Table 1 for each subject. Note the great similarity of these two rates for any given subject. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Exponential rate used to best approximate the decay of RE (λRE) and IME (λIME) with n, respectively. The R2 values quantify the goodness-of-fit of the exponential model for each subject to the corresponding experimental data. https://doi.org/10.1371/journal.pcbi.1007118.t001 Learning dynamics: Model vs. human subjects To build a model for each subject, we used the subject-specific map H and the subject-specific sequence of targets used for training. The learning rate η for the inverse model (Eq (2)) was taken to be equal to the subject-specific rates λRE reported in Table 1 (see Methods), obtained for each subject by fitting with standard least squares an exponential decay curve to the corresponding experimental time series of reaching errors (Fig 1a) to extract the decay rate λRE. The learning rate ε for the estimation of the forward model (Eq (1)), and the amplitude σ of the noise added to the inference of body motions (Eq (3)) followed from an optimization procedure (see Methods). Table 2 reports the values of these three parameters for each subject. We let the model evolve until the norm of the reaching error was smaller than 1 cm, as the subject’s performance reached a plateau once the cursor reached the target. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 2. Subject-specific model parameters. The learning rates η and ε correspond to the inverse and the forward model, respectively; σ is the amplitude of the Gaussian noise added to the inference of target-specific body configurations. https://doi.org/10.1371/journal.pcbi.1007118.t002 We then tested how well our model captured the learning dynamics of each subject. As shown in Fig 3, these subject specific models were able to predict quite well the individual experimental learning curves. Both the RE and the IME estimated from the model follow the time evolution extracted from the real experimental data. Correlation coefficients (Table 3) quantify the similarity between the simulated and actual temporal evolution of RE and IME during learning. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Modeling human learning. (a) Temporal evolution of the norm RE of the reaching error as a function of trial number n, calculated from the data of six subjects (black) and from the respective models (blue). (b) Temporal evolution of the norm IME of the inverse model error as a function of trial number n, calculated from the experimental data (black) and from the model simulation (blue). Both metrics are calculated over a moving window that encompasses the current and its 11 preceding trials. https://doi.org/10.1371/journal.pcbi.1007118.g003 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 3. Correlation coefficients (R2) between the temporal evolution recorded during the experiment and the temporal evolution predicted by the model, for both the norm RE of the reaching error and the norm IME of the inverse model error, for each subject (S1-S6). https://doi.org/10.1371/journal.pcbi.1007118.t003 We investigated the stability of the simulated RE and IME curves against changes in the noise parameter σ. Simulations for each subject using the learning rates η and ε listed in Table 2 revealed that increasing σ by up to 0.1 and decreasing it by up to 0.2 around the optimal values listed in Table 2 did not affect the good fit of the simulated curves to the experimental ones. Further decreases in σ affect model performance because the amplitude of the noise becomes insufficient to free the gradient descent algorithm from getting trapped in local minima. Further increases in σ affect model performance because the exploration of the state space becomes dominated by noise instead of being guided by gradient descent. In principle, learning of the forward map and its inverse could take place in two separate phases. First, a phase guided by a comparison between actual object position and expected object position; their difference results in prediction errors that drive the estimation of the forward model. Second, a phase guided by a comparison between actual object position and target; their difference results in reaching errors that drive the acquisition of the inverse model. This two-phase mechanism leads to the model learning curves shown in S1 Fig (blue curves). The two-phase model does not fit the experimental data as well as a model of forward and inverse learning as concurrent processes, shown in Fig 3 (blue curves). These results support our approach of modeling forward and inverse learning as concurrent processes. The model also allows us to compute a current estimate of the forward map as acquired by the subjects while practicing. We quantified the similarity between the estimated and the BoMI map H at each iteration of the learning dynamics by using the forward model error (FME), defined as the spectral norm of the difference between H and , normalized by the spectral norm of H (see Eq (20) in Methods). This error in the estimate of the map that transforms body configurations into cursor position leads to the cursor prediction error, the difference between the actual and the expected position of the cursor. We monitored the norm PE of the prediction error and the FME as a function of trial number n (Fig 4). The estimate converged toward the actual forward map H, resulting in near zero asymptotic values for both PE and FME (Fig 4). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Subjects learn the forward model: Data from model based simulations of the temporal evolution of the forward map estimate . (a) Temporal evolution of the norm PE of the prediction error as a function of trial number n. (b) Temporal evolution of the estimate of the forward map , quantified by , as a function of trial number n. Data are shown for each subject-specific model. Only the first 100 iterations of the dynamics are presented, since the asymptotic regime has by then been reached for both metrics. https://doi.org/10.1371/journal.pcbi.1007118.g004 The dynamics of the learning model captured the errors in the low-dimensional task space of the controlled cursor as well as the history of body signals generated by the subjects in response to the successive targets (Fig 5 and Table 4); the sole exception was Subject 4, whose accuracy in reaching the target position was smaller and characterized by a higher level of variability (Fig 3a). The body and cursor signals recorded during the experiment and predicted by the model were not very similar at the beginning of training, but they quickly converged and tended to overlap by the time RE and IME reached their asymptotic condition (Fig 5, Table 4). These results support the conclusion that a model of learning based on simple gradient descent over the two quadratic error surfaces defined here captures the formation of forward and inverse representations of the map established by the body-machine interface. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Reconstruction of body and cursor signals. Comparison between real (black) and simulated (blue) data for subject 6. The top panel presents the values of the eight body signals (q1, …, q8), i.e., the (x,y) coordinates of four markers (shoulders and upper arms on both sides) in the image plane of the associated cameras; the bottom panel shows the (x, y) coordinates of the cursor (p1, p2) in the reference frame of the computer monitor. https://doi.org/10.1371/journal.pcbi.1007118.g005 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 4. Correlation coefficients (R2) between the temporal evolution of the actual signals recorded during the experiment and the temporal evolution of these quantities as predicted by the model, for both body signals (q1, …, q8) and cursor coordinates (p1, p2). Only the last 100 trials were used to compute the R2 values. https://doi.org/10.1371/journal.pcbi.1007118.t004 A model of learning while practicing control via a body-machine interface We investigated how users of a body machine interface learn to reorganize or "remap" their body motions as they practice controlling an external object through the BoMI. The controlled object could be a wheelchair, a robotic assistant, or a drone [16, 17, 20]. Here we focus on the control of a computer cursor whose two-dimensional coordinates determine its location on a computer screen. Effectiveness in cursor control is the first and most common benchmark for interfaces based on neural activity [21–23], as the ability to control two-dimensional position is readily applied to a variety of tasks (e.g., an action performed via a joystick, entering computer text, etc.). We consider interfaces in which a linear mapping associates the body motion signals to the coordinates of the external object. Importantly, there is an imbalance between the dimensionality of the task space and that of the body signals, the latter being larger. Thus, any position of the controlled object corresponds to many, potentially infinite, different body configuration signals. The BoMI matrix H establishes a linear map between these two spaces; H has as many rows K as signals are needed to control the external object, and as many columns S as there are body signals. Not being square, the matrix H does not have a unique inverse. But there exist infinite “right inverses” that combined with H yield the K x K identity matrix in the task space of external control signals. Each such right inverse transforms a desired position of the controlled object into one particular set of values for the body signals. We consider users to be competent when they are able to move their body successfully in response to a presented target for the controlled object. Mathematically, we consider this as finding one right inverse G of the mapping H, out of a multitude of possible and equally valid choices. Current theories and experimental observations [10] suggest that learning is a dynamical process in which the learners modify their behavior based on the errors observed at each iteration of a task. In the kinematically redundant conditions considered here, learning is problematic because a given low-dimensional task error signal has multiple representations in the high-dimensional body space. Here, we considered an error surface defined by the squared task error in the space of the elements of the target-to-body map G adopted by a learner, where G is the learner’s inverse model of the body-to-cursor mapping H established by the BoMI. We implemented our learning model based on the hypothesis that the learners update the map G by moving along this error surface, following the line of steepest descent determined by the gradient of the squared error with respect to the elements of G. This error gradient depends on several variables; some can be directly observed by the learner, such as the error made in attempting to reach a given target position of the external device. However, the error gradient also depends upon the elements of the interface map H, which the learner cannot be assumed to know. Therefore, gradient descent learning of the inverse model requires a concurrent learning of the forward model. The latter requires a different error surface, since the forward map relates body configurations to the consequent position of the controlled object. Forward model learning does not require a target position for the controlled object, as the relevant error in this case is the difference between predicted and observed position of the controlled object. The squared prediction error defines an error surface in the space of the elements of the estimated forward map . Learning is thus described through two first-order dynamical processes determined by two state equations. A forward learning process: (1) and an inverse learning process (2) where H(n) is a KxS matrix, G(n) is an SxK matrix, p(n) and q(n) are column vectors of respectively Kx1 and Sx1 elements, ε and η are scalars. In the experiments reported here, S = 8 and K = 2. The reaching error e(n) = (p(n) − u(n)) that guides this process is the difference between actual and desired positions of the controlled object. For details on the derivation of these equations, see Methods. The forward and inverse models are effectively the states of the respective learning processes, whose n-th iteration results in state variables and G(n). Eq (1) updates the subject’s estimate of the forward model H that transforms S-dimensional body configurations q into K-dimensional positions p of the controlled object. The term in parentheses is the prediction error between actual and predicted positions of the controlled object. The concurrent process described by Eq (2) is the learning of the inverse model G that the subjects use to map the target object position u onto body signals q. Two possibly different learning rates, ε and η, provide inverse time constants for the respective learning processes. Since we focus on the case in which forward and inverse learning are carried out concurrently, naïve users are immediately presented with the reaching task, and as they practice they observe both the reaching error and the prediction error. Eqs (1) and (2) are coupled through and through a third equation that describes the body signals currently adopted to reach the target position: (3) This apparently innocuous interaction has potentially harmful effects on the convergence of the coupled learning dynamics, as the second term in the gradient contribution to Eq (1) includes a quadratic factor in q(n) and thus in G(n). This contribution may result in local minima, a problem avoided by adding noise to Eq (3), to obtain: (4) We validated our approach with six healthy subjects that learned to control the two-dimensional movement of a cursor on a monitor using eight signals from their upper body motions (shoulders and upper arms on both sides, see Methods for details). In these experiments, S = 8 and K = 2, and the state space of the combined forward-inverse learning was 2x16 = 32-dimensional. Dynamics of learning in human subjects We monitored the learning process through two scalar metrics: RE, the L2 norm of the reaching error (the difference between actual p(n) and target u(n) location of the cursor at the end of each reaching movement), and IME, the spectral norm of the inverse model error (the difference between the identity matrix IK and the product between the interface map H and the current estimate G(n) of the inverse model). The spectral norm of a matrix, indicated here by ∥·∥ to emphasize its analogy with the L2 norm of a vector, is the maximum singular value of the matrix. We estimated G(n) from target and body signal data by least squares fit on Eq (3). The elements of G(n) were estimated using data from 12 trials: trial n and its 11 preceding trials. Overlapping moving windows that included 12 trials were shifted by one trial at each iteration. In the experiment, each subject practiced with a personalized body-to-cursor map, chosen to reflect the statistics of its own freely produced upper body motions (see calibration procedure in Methods). With increasing number of practice trials, their RE decreased to values closer to the target radius (1 cm, Fig 1a). The learning process took over one hundred steps (172 ± 32 trials, mean ± SEM) before reaching asymptotic performance (Fig 1a), identified as the time when the norm of the reaching error was smaller than the radius of the target. Similarly, the matrix G(n) converged to a generalized inverse of the body-to-cursor map (Fig 1b). Although the subjects explored a number of different body configurations while learning how to control the cursor, in the end they found a stable movement pattern, and built an inverse model G. Note the asymptotically small variations in the acquired G, with ∥ΔG∥ about 10% of ∥G∥ (Fig 2). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Subjects learn to use the body-machine interface. Data for the six subjects enrolled in the study (S1- S6). (a) Temporal evolution of the norm RE of the reaching error, calculated over a moving window that includes the current and the 11 preceding trials. (b) Temporal evolution of the norm IME of the inverse model error. The inverse model G(n) was obtained by a least squares fit on Eq (3) from target and body signal data for the current and the 11 preceding trials. https://doi.org/10.1371/journal.pcbi.1007118.g001 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Temporal evolution of the changes in the inverse map G(n). Changes in the inverse map as a function of trial number n, quantified by ∥ΔG(n)∥ = ∥G(n) − G(n−1)∥⁄∥G(n)∥ (see Eq (19) in Methods), for the six subjects enrolled in the study (S1-S6). https://doi.org/10.1371/journal.pcbi.1007118.g002 For each subject, both RE and IME errors decreased with time following a trend captured by an exponential curve (Eq (22)). The learning rates, each given by the inverse of the time constant of the corresponding exponential fit, are shown in Table 1 for each subject. Note the great similarity of these two rates for any given subject. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Exponential rate used to best approximate the decay of RE (λRE) and IME (λIME) with n, respectively. The R2 values quantify the goodness-of-fit of the exponential model for each subject to the corresponding experimental data. https://doi.org/10.1371/journal.pcbi.1007118.t001 Learning dynamics: Model vs. human subjects To build a model for each subject, we used the subject-specific map H and the subject-specific sequence of targets used for training. The learning rate η for the inverse model (Eq (2)) was taken to be equal to the subject-specific rates λRE reported in Table 1 (see Methods), obtained for each subject by fitting with standard least squares an exponential decay curve to the corresponding experimental time series of reaching errors (Fig 1a) to extract the decay rate λRE. The learning rate ε for the estimation of the forward model (Eq (1)), and the amplitude σ of the noise added to the inference of body motions (Eq (3)) followed from an optimization procedure (see Methods). Table 2 reports the values of these three parameters for each subject. We let the model evolve until the norm of the reaching error was smaller than 1 cm, as the subject’s performance reached a plateau once the cursor reached the target. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 2. Subject-specific model parameters. The learning rates η and ε correspond to the inverse and the forward model, respectively; σ is the amplitude of the Gaussian noise added to the inference of target-specific body configurations. https://doi.org/10.1371/journal.pcbi.1007118.t002 We then tested how well our model captured the learning dynamics of each subject. As shown in Fig 3, these subject specific models were able to predict quite well the individual experimental learning curves. Both the RE and the IME estimated from the model follow the time evolution extracted from the real experimental data. Correlation coefficients (Table 3) quantify the similarity between the simulated and actual temporal evolution of RE and IME during learning. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Modeling human learning. (a) Temporal evolution of the norm RE of the reaching error as a function of trial number n, calculated from the data of six subjects (black) and from the respective models (blue). (b) Temporal evolution of the norm IME of the inverse model error as a function of trial number n, calculated from the experimental data (black) and from the model simulation (blue). Both metrics are calculated over a moving window that encompasses the current and its 11 preceding trials. https://doi.org/10.1371/journal.pcbi.1007118.g003 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 3. Correlation coefficients (R2) between the temporal evolution recorded during the experiment and the temporal evolution predicted by the model, for both the norm RE of the reaching error and the norm IME of the inverse model error, for each subject (S1-S6). https://doi.org/10.1371/journal.pcbi.1007118.t003 We investigated the stability of the simulated RE and IME curves against changes in the noise parameter σ. Simulations for each subject using the learning rates η and ε listed in Table 2 revealed that increasing σ by up to 0.1 and decreasing it by up to 0.2 around the optimal values listed in Table 2 did not affect the good fit of the simulated curves to the experimental ones. Further decreases in σ affect model performance because the amplitude of the noise becomes insufficient to free the gradient descent algorithm from getting trapped in local minima. Further increases in σ affect model performance because the exploration of the state space becomes dominated by noise instead of being guided by gradient descent. In principle, learning of the forward map and its inverse could take place in two separate phases. First, a phase guided by a comparison between actual object position and expected object position; their difference results in prediction errors that drive the estimation of the forward model. Second, a phase guided by a comparison between actual object position and target; their difference results in reaching errors that drive the acquisition of the inverse model. This two-phase mechanism leads to the model learning curves shown in S1 Fig (blue curves). The two-phase model does not fit the experimental data as well as a model of forward and inverse learning as concurrent processes, shown in Fig 3 (blue curves). These results support our approach of modeling forward and inverse learning as concurrent processes. The model also allows us to compute a current estimate of the forward map as acquired by the subjects while practicing. We quantified the similarity between the estimated and the BoMI map H at each iteration of the learning dynamics by using the forward model error (FME), defined as the spectral norm of the difference between H and , normalized by the spectral norm of H (see Eq (20) in Methods). This error in the estimate of the map that transforms body configurations into cursor position leads to the cursor prediction error, the difference between the actual and the expected position of the cursor. We monitored the norm PE of the prediction error and the FME as a function of trial number n (Fig 4). The estimate converged toward the actual forward map H, resulting in near zero asymptotic values for both PE and FME (Fig 4). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Subjects learn the forward model: Data from model based simulations of the temporal evolution of the forward map estimate . (a) Temporal evolution of the norm PE of the prediction error as a function of trial number n. (b) Temporal evolution of the estimate of the forward map , quantified by , as a function of trial number n. Data are shown for each subject-specific model. Only the first 100 iterations of the dynamics are presented, since the asymptotic regime has by then been reached for both metrics. https://doi.org/10.1371/journal.pcbi.1007118.g004 The dynamics of the learning model captured the errors in the low-dimensional task space of the controlled cursor as well as the history of body signals generated by the subjects in response to the successive targets (Fig 5 and Table 4); the sole exception was Subject 4, whose accuracy in reaching the target position was smaller and characterized by a higher level of variability (Fig 3a). The body and cursor signals recorded during the experiment and predicted by the model were not very similar at the beginning of training, but they quickly converged and tended to overlap by the time RE and IME reached their asymptotic condition (Fig 5, Table 4). These results support the conclusion that a model of learning based on simple gradient descent over the two quadratic error surfaces defined here captures the formation of forward and inverse representations of the map established by the body-machine interface. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Reconstruction of body and cursor signals. Comparison between real (black) and simulated (blue) data for subject 6. The top panel presents the values of the eight body signals (q1, …, q8), i.e., the (x,y) coordinates of four markers (shoulders and upper arms on both sides) in the image plane of the associated cameras; the bottom panel shows the (x, y) coordinates of the cursor (p1, p2) in the reference frame of the computer monitor. https://doi.org/10.1371/journal.pcbi.1007118.g005 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 4. Correlation coefficients (R2) between the temporal evolution of the actual signals recorded during the experiment and the temporal evolution of these quantities as predicted by the model, for both body signals (q1, …, q8) and cursor coordinates (p1, p2). Only the last 100 trials were used to compute the R2 values. https://doi.org/10.1371/journal.pcbi.1007118.t004 Discussion We investigated the learning process that occurs when subjects reorganize or "remap" their body motions as they learn to perform a task that involves a novel relation between body motions and their observable consequences. For a patient who suffers from severe paralysis and is obliged to reorganize the available mobility to operate an assistive device such as a powered wheelchair through a human-machine interface, the ability to engage in such remapping becomes a necessity of life. Here we investigated the process of learning to perform reaching movements via a body-machine interface; we used a group of unimpaired subjects under the preliminary assumption that similar learning mechanisms are present in people suffering from injuries to the spinal cord. We considered a body-machine interface that harnesses signals generated by body configurations in order to control an external object, in this case the position of a cursor on a computer monitor. The interface establishes a many-to-one body-to-object map that the user must learn to master starting from a naïve state. The map is in fact not intuitive, as there is no obvious correspondence between the motions of the body and the motion of the controlled object. The goal of our experiment was to test the hypothesis that motor learning while using a BoMI is a state based process with first-order deterministic dynamics, and that such computational model is suitable to describe both the estimation of the forward body-to-object map and the construction of its inverse by the unimpaired subjects involved in the experiment. Subjects trained to control a computer cursor through a BoMI whose linear body-to-object map was customized to each subject through a calibration procedure based on the statistics of the subject’s own body motions. Through practice with a fixed BoMI, all subjects demonstrated exponential convergence toward an inverse of the BoMI mapping, with subject-specific learning rates. The parameters of this inverse model define a state space in which learning is modeled as a first-order dynamical process that evolves based on the specific sequence of target positions for the external object, and of two types of error: (i) the prediction error that is the difference between the actual position of the object and the position predicted by the subjects based on their internal representation of the interface map, and (ii) the task error or reaching error that is the difference between the position reached by the object and the actual target position. While the prediction error depends on the current estimate of the forward model, the reaching error depends on the current state of the inverse model, which determines the chosen body configuration. We studied the evolution of the learning system using different interface maps H, learning rates η, and target sequences U for each subject. The empirical observations of human adaptation to the BoMI were compared to predictions from a subject-specific learning model that used initial conditions inferred from each subject’s initial performance, the same H and U as used by the subject, and a learning rate η obtained by temporal regression of the experimental data. We demonstrated that a model based on first-order dynamics was sufficient to capture the evolution of learning as described through both the observed errors and the accuracy of the estimated internal models. There was however a notable difference between the models’ and the subjects’ learning: the actual decay of the norm of the reaching error computed from experimental data was not as smooth as the decay predicted by the model. This might be due to the main simplifications adopted in our model, namely, i) the learning dynamics (Eqs (1) and (2)) were assumed to be deterministic, and ii) the learning of the inverse model (Eq (2)) was assumed to be linear. The comparison between model predictions and actual data in Fig 3 indicates that our proposed model of learning is sufficient to explain the data. However, the mechanism we propose is not necessary; we cannot rule out other possibilities, such as reinforcement learning. The approach adopted here, in which learning occurs via a gradient descent search to decrease the reaching error over the space of parameters of the inverse model G, requires knowledge of the forward map H to compute the gradient (see Methods, Eqs (9), (10) and (11)). We considered it not plausible that naïve learners would possess such knowledge. Thus, we hypothesized that subjects learn an estimate of the forward map H concurrently with their acquisition of an inverse map G. In earlier work Sanger (2004) [24], proposed a computational model for estimating the user’s inverse model G that avoided the need for information about the forward map H through a direct approximation of the user’s response q to a presented target u. Our work develops these ideas further, with the goal of estimating not just the user’s inverse model but the learning process that leads to the formation of the inverse model. To estimate the inverse model, Sanger proposes an algorithm based on an error δ = q* − G(u), where q* is one of the many responses that satisfy H(q*) = u and would allow the user to achieve the target. Note that q* is not observable by the user. In contrast, the learning process that we model is based on quantities that the user can observe: body actions, the outcome of those actions, and their targets. In our model of the user’s learning process, the user must have information about the forward map to learn its inverse. Our calculations of the two metrics that quantify the learning of , the prediction error PE and the forward model error FME, are shown in Fig 4. These metrics have been obtained from the model and are not being compared to results based on experimental data. However, the estimator gets folded into the learning of G. The agreement between the results from the model and those obtained from experimental data are shown in Fig 3 for the two metrics that quantify the learning of G, the reaching error RE and the inverse model error IME. Since the model is based on the concurrent learning of G and , we argue that this agreement is not only an explicit validation of the acquisition of a valid inverse model, but also an implicit validation of the concurrent and successful estimation of the forward model. This agreement between model and experimental results does not exclude the possibility of alternative learning mechanisms, such as a direct learning of the inverse model [24] or the use of reinforcement learning [25] to acquire an action policy that would play the role of the inverse model. Although the interface forward map is linear (Methods, Eq (5)), this is a many-to-one map admitting a multitude of inverses. This “redundancy” opens the possibility of successful linear and nonlinear inverse maps. Redundancy also leads to an important consideration about gradient descent learning. The reaching error surface in the space of the inverse model elements does not have a unique minimum, but a continuously connected set of minima corresponding to the null space of the forward map. In the metaphor of a skier descending from a mountain following the gradient, this space of equivalent inverse models corresponds to a flat elongated valley at the bottom of the mountain. Anywhere along the valley is a valid end to the ride, as it corresponds to a valid inverse model. The inverse model on which the steepest descent ends depends on the initial conditions, as predicted by the dynamical model (see Fig 3b–evolution of the norm of the inverse model error), as well as on the realization of the noise employed in any given simulation of the learning model. On the other hand, there is only one valid solution for the estimate of the forward model. Our simulations of the model reach the correct solution for , as demonstrated by the convergence to zero of the prediction error and of the norm of the forward model error in Fig 4. The analysis of the learning dynamics of the users of a body-machine interface is essential for the effective development of coadaptive approaches, where the interface parameters are themselves updated based on the user’s state of learning [26–28]. Coadaptation requires a seamless integration between machine learning and human learning. Both learning processes are dynamic, evolving as functions of their own internal state and of inputs that reflect the state of their counterpart. A mismatch between the timing of the interface updates relative to the subject’s learning dynamics would likely lead to hindering human learning (if the interface update rate is too fast) or to ineffectiveness (if the interface update rate is too slow). In our current understanding, motor learning is not only a way to acquire or improve a skill, but is perhaps most importantly a biological mechanism to gain knowledge about the physical properties of the environment [9]. Through the practice of movements, the brain learns to separate unpredictable from predictable features of the world in which the body is immersed; through the formation of representations or "internal models" of the predictable features, the brain acquires the ability to anticipate the sensory consequences of its commands. The human operator of a BoMI must develop an inverse model of the forward map to transform a desired goal into an action of the body, as demonstrated by previous studies where subjects operating the BoMI were able to compensate for the effects of visuomotor rotations, changes of scale, and noise perturbations applied to the cursor in the task space [29, 30]. Further evidence from the literature indicates that adaptive, error-driven, internal model formation is a general feature of motor learning, observed during arm reaching and drawing, and during pointing with the legs and walking [31]. Theoretical studies of motor learning have focused on control policies and internal models to understand how the brain generates action commands [32]. Control policies allow the brain to select goals and plan actions, while internal models generate motor commands that are appropriate for those plans in the context established by sensory feedback. For example, when the goal is to reach a target, the brain must first evaluate the current position of the limb with respect to the target and use a control policy to plan a movement of the hand [3, 33]. An internal model of the limb dynamics, called an inverse model, then converts that plan into motor commands [34–36], while a forward model of the same limb dynamics predicts the sensory consequences of these motor commands [14, 37, 38]. The comparison of this prediction with actual sensory feedback [39] allows for a re-estimation of the current hand position with respect to the target, and for an update of the motor plan [40] by issuing an error-dependent motor command aimed at correcting the ongoing movement. Forward and inverse internal models thus play a fundamental role in movement planning and execution. Recent studies have considered the formation of these internal models as dynamical processes [10, 41, 42]. For example, to account for findings observed when reaching arm movements are perturbed by external force fields, Donchin and coworkers [10] argued that the forces generated by the subjects to compensate for the external field are the output of an internal model of the field, developed through experience. Their theory, successful at predicting the time history of adaptation, was based on two key assumptions that we also adopted here, namely i) that the movement outcomes and the ensuing errors result from a deterministic process, and ii) that the parameters of the internal model define the state space of the learning dynamics. The transformation from the movements of the BoMI user to the movements of the controlled objects establishes a new geometrical relation between body motions and their consequences. The BoMI thus essentially creates a novel geometry that the user must learn to operate. Here, we have implemented a linear transformation from body signals to a cursor; this allowed us to work under the assumption that the users would develop a linear inverse model of this map. However, linearity of the inverse map is not a necessary consequence of operating through a linear forward map, because a linear forward map that is not bijective may also admit nonlinear inverses. Therefore, our approach will not result in the most general solution to the problem of finding an inverse map. Nevertheless, our analysis demonstrated that the linear inverse model derived by coupled gradient descent on both the prediction error and the reaching error is capable of reproducing with high fidelity the entire history of a subject’s responses to a sequence of targets (Fig 5). While we are unable to exclude more complex processes that could lead to an equally effective nonlinear inverse model of the linear BoMI map, the linearity assumption not only leads to results that agree with the experimental data but also fulfills Occam’s razor criterion for simplicity. Issues of cognitive strategies that are involved in motor learning are important topics, in particular the decision processes that establish the balance of exploration and exploitation. However, our contribution to these topics can only be speculative. In our model there are two elements that drive the dynamical process: the errors {e(n)} and the inputs {u(n)}; in the reaching task, these correspond to reaching errors and to the targets presented to the subject. One might take the view of Taylor et al. [43] that “the contribution of explicit learning would be modulated by instruction and the contribution of implicit learning would be modulated by the form of error feedback”, and be tempted to conclude that these two terms are represented in our model by the inputs and the errors, respectively. However, our model prevents such a clear separation, since these two variables are closely correlated; even when the choice of body action in response to an input is distorted by noise, the error is directly connected to the input by the forward map H and the inverse model G(n). Our model provides a mechanistic view of learning as a process in which the formation of the internal model is driven by the inputs through exposure to a target sequence. The separation of the effects associated with implicit and explicit learning might only be investigated by perturbing the connection between error and input established by the evolving inverse model. We conclude with some comments on the clinical relevance of this study. Damage to the spinal cord, stroke, and other neurological disorders often cause long-lasting and devastating loss of motion and coordination, as well as weakness and altered reflexes. In most cases, some residual motor and sensory capacities remain available to the disabled survivor, and can be harnessed to provide control signals to assistive devices such as robotic systems, computers, and wheelchairs. A first challenge for the disabled [44] is to learn how to interact with the assistive devices and how these respond to the user’s actions. A broad spectrum of sensors, such as inertial measurement units (IMUs) placed on the head [45] or electroencephalography (EEG) systems [46], are available for detecting and decoding movement intentions. A body machine interface (BoMI) captures residual body motions by optical [18–20], accelerometric [15, 17], or electromyographic sensors [47], and maps the sensor signals onto commands for external devices such as powered wheelchairs [17] or drones [20], or onto computer inputs. At the other end of the spectrum, brain-machine interfaces decode motor intention from neural activity recorded in motor or premotor cortical areas [22, 23, 48]. Both brain- and body-machine interfaces take advantage of the vast number of neural signals and degrees of freedom of the human body [1, 49, 50], and of the natural ability of the motor system to reorganize the control of movement [4, 9, 51, 52]. These interfaces establish a map—most often linear—from the space of neural or motion signals to the lower dimensional space of control signals for the external device [15, 53]. The user’s ability to operate the interface is expected to change over time; either a positive change associated with the acquisition of greater control skills, or a negative change due to the worsening of the user’s medical condition. In either case, the interface map also needs to change, to coadapt with its user. This coadaptation is a critical challenge in the development of both brain- and body-machine interfaces [26, 28]; harmonizing the interface update with the processes that guide the improvement or decay of the user’s skill is of obvious importance. Understanding the dynamics of human learning through the interaction with the interface carries the promise of creating truly intelligent systems, capable of compensating for the changing abilities of their users [22, 27]. Methods Ethics statement All experimental procedures were approved by the Northwestern University Institutional Review Board. All subjects signed a consent form prior participating to the study. Computational model Inverse kinematics is a well-known and well-explored computational problem in robotics [54, 55] and human motor control [34, 56]; it refers to finding the configuration of joint angles that results in a desired position of an end effector or of the hand in the operational space [57]. Inverse kinematics problems become ill-posed [58] when there are multiple valid solutions as a consequence of the many-to-one nature of the forward kinematic map. This is the situation considered here, in which the kinematics that the subjects are controlling may be partitioned in a sequence of two maps. In a first map, the subjects control the motions of their bodies by acting on a multitude of muscles and joints. In a second map, the signals triggered by these body motions determine the lower dimensional motion of an external object such as a wheelchair [17], a cursor on a computer screen [16], or a drone [20]. Here we make the critical but reasonable assumption that the subjects have already acquired in a stable form the expertise needed to control the motion of their body, or at least portions of it that were unaffected by injury or disease. Therefore, they only need to acquire the second component. This is the component we focus on, limited here to a body-machine interface whose linear map H transforms, at any given trial n, an S-dimensional vector of body signals q into a K-dimensional control vector p as follows: (5) Here, H is a KxS matrix. Since K<S, this interface map is many-to-one and there is a "null space" of inputs for each value of the output, encompassing all different patterns of body signals that result in the same control signal. This is an important characteristic of the map; earlier work has shown that subjects learn through practice to separate the null space from its orthogonal “potent space” complement [53]. Learning dynamics as first-order state-based model. In a learning experiment where the goal is to reach targets in the control space, the superscript n labels the trials or successive repetitions of a single action; for instance, each trial is a reaching movement in a sequence of such movements. At the end of a trial, the learner observes an error e(n). This error drives the updating of the internal inverse model, which we assume to be a linear map G(n) transforming a goal u(n) into its corresponding body vector (previously Eq (3)): (6) Since the forward map H is linear, the linearity of G is a sufficient but not necessary condition. More complex, nonlinear forms of the inverse model would in principle be admissible. Here, we assume the simplest general form for a linear inverse model of the forward BoMI map; this assumption makes the investigation of learning dynamics tractable. For the n-th reaching trial, u(n) is the position of the target and the reaching error is the K-dimensional vector from the target position to the actual position of the controlled object at the end of the trial: (7) The internal model thus becomes a right-inverse model of the interface map: (8) As learning reaches a steady state, participants are expected to have eliminated this error. At this point, e(n) = 0, which requires H G(n) = IK. Note that the reaching error, a Kx1 vector, can be interpreted as the projection of the KxK matrix (HG(n) − IK) onto the Kx1 vector u(n). The metrics introduced earlier to monitor the acquisition of the inverse model are RE, the L2 norm of the Kx1 reaching error (HG(n) − IK) u(n), and IME, the spectral norm of the KxK matrix (HG(n) − IK). Learning is represented as a dynamical process whose state is the internal inverse model G(n); this state changes after the observation of each reaching error. The targets presented to the learner constitute the external input to this process. To ensure that the change in state leads to a reduction of the error, the learning process drives the state along the gradient of the quadratic error surface in the state space defined by the components of G. The gradient of the squared reaching error with respect to the components of the inverse model is (9) which leads to the update equation (10) or, equivalently (11) Here, η is a learning rate parameter that we model as a scalar. Although in principle there could be a different rate for learning every element of the forward and inverse models, we found that only two learning rates, ε for the forward model and η for the inverse model, sufficed to account for the observed learning behavior. If the interface map H is known, the update Eq (11) provides an estimate of the right inverse of H solely on the basis of control space data, without performing an explicit matrix inversion. Given the targets u(n), the variables of interest are the observed reaching errors e(n) and the estimated inverse model or "state of learning" G(n). As e(n) → 0, G(n) becomes stationary. The gradient of the error involves the actual value of the interface map H. It is not plausible to assume that our subjects had any initial notion of the interface map, let alone an exact representation. In a realistic model of learning, the value of H must be replaced with an evolving estimate . In this scenario, the current state of learning is represented in a higher dimensional space that includes the components of both G(n) and . In the case of S = 8 body signals controlling the location of an object in K = 2 dimensions, the state space for learning is 2x8x2 = 32-dimensional. We follow a concept introduced by Jordan and Rumelhart [14], and represent learning as the parallel development a of forward-inverse model. The forward model leads to a prediction of the controlled device position given the current set of body signals, whereas the inverse model generates the body signals needed to achieve a given target position of the device. In our study, we distinguish between the exploration that takes place when subjects perform tentative movements as they are seeking a goal, and the performance of aimless limb motions, which we refer to as “motor babbling” in accordance with the literature on human and robotic development [59, 60]. In our experiments, babbling takes place during the dance calibration used to establish the forward map. Motor babbling has also been considered as an exploratory behavior leading to the formation of internal dynamical models. However, here we use the concept of “exploration” to refer to the search for alternative solutions to the problem of generating a body configuration q that makes it possible to reach the target u. In principle, motor babbling and goal directed learning could take place in two separate phases. First, a motor babbling phase where freely produced, aimless body motions result in actual object motions that are compared to expected object motions to obtain prediction errors that drive the estimation of the forward model. Second, a phase where the subjects reach for specific targets, and the resulting reaching errors drive the estimation of the inverse model (see S1 Fig). This two-phase mechanism has been suggested as a possible model of motor development in infants [61], who acquire a model of the dynamical properties of limbs by driving them with erratic neuromuscular patterns of activity. However, this is not the case in experiments where subjects are presented with reaching targets from the onset; in this scenario, no aimless motor babbling was observed. It is thus more plausible to model forward and inverse learning as concurrent processes. The prediction error quantifies the difference between actual and predicted positions of the controlled object, without reference to a target. The gradient of the squared prediction error with respect to the components of the forward model H is (12) The update equations for the coupled learning process then are (13) (14) reported earlier as Eqs (1) and (2). Note that Eq (13) contains a term that is quadratic in q; this implies a quadratic dependence on the elements of G(n), since at each step q(n) is derived by applying G(n) to the corresponding target. The update equation for the estimate of the forward model is thus not linear, and the learning process is prone to getting stuck in local minima. The simplest way to avoid this is to add a small amount of noise to the body signals derived from the inverse model, (15) where the noise ξ has the same dimension S as q, and each component of ξ is independently drawn from a Gaussian distribution at each trial. Two important parameters of the combined learning model of Eqs (13) and (14) are the learning rates for the forward (ε) and the inverse (η) models. In the case of eight body signals mapped into a two-dimensional control space, these learning rates apply to the evolution of 16 elements each; their most general form would be a 2x8 matrix for ε and an 8x2 matrix for η. Here, for simplicity and to avoid overfitting, we assume both learning rates to be scalar. A similar assumption is made for the noise amplitude σ, a somewhat less critical parameter whose sole purpose is to add sufficient noise to the learning algorithm so as to avoid getting trapped in local minima. Validation of the model with experimental data To validate the outcomes of the model we recruited six unimpaired subjects (age range 21–40 years old, 3 males and 3 females) in the preliminary study. The subjects practiced the execution of reaching movements via an interface (Fig 6) that mapped an eight-dimensional signal space associated with upper body motions to the two-dimensional task space of a computer cursor. An array of four video cameras (V100, Naturalpoint Inc., OR, USA) was used to track active infrared light sources attached to the subject’s upper-body garments (two for each side of the body, one on the shoulder and one on the upper arm, as shown in Fig 6). Each camera pointed at a single marker, providing two signals defining the coordinates of the marker in the camera’s frame. Collectively, the four sensors provided an eight-dimensional body vector q that was transformed into a two-dimensional command vector p for controlling the position of a cursor on a computer screen, p = Hq. We show in S2 Fig the trajectories described by the each of the body sensors in the two-dimensional plane of its corresponding camera. Data is shown at different stages of learning, for each of the presented targets. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Experimental setup and protocol. a) Schematic overview of the setup of the Body Machine Interface. The subject sat on a chair in front of the computer. Active optical markers were placed on shoulders and upper arms; their positions were recorded by infrared cameras. b) Design of the reaching task. Six targets were equally spaced over a circumference of 5 cm radius, centered in the computer screen. The grey circle corresponds to the central target; this was the initial position from which each movement towards the peripheral targets (empty circles) would start; movements would return to the central target after the peripheral target was reached. The black small circle is the cursor controlled by the subjects. c) Experimental protocol. A calibration phase during which the subject moved freely and aimlessly provided data for the construction of the map H that transformed body movements into cursor movements. This was followed by an adjustment phase in order to match the control space to the computer screen. Training then started, divided in six blocks of nine repetitions of each of the six peripheral targets. https://doi.org/10.1371/journal.pcbi.1007118.g006 To design the matrix H for each subject, we used a standard linear dimensionality reduction method, principal component analysis (PCA) [62], to set up the dimensionality reduction implicit in the q to p map. PCA is based on the decorrelation of the input signals through the diagonalization of their covariance matrix; dimensionality reduction is obtained by keeping only the eigenvectors corresponding to the K largest eigenvalues. Here, these K = 2 eigenvectors became the rows of H. PCA provided us with a computationally straightforward method for identifying those directions that captured the largest range (i.e. the largest variance) of body motions within the space of sensor signals for each subject. The interface map H from the input space of body signals to the output task space was constructed in three steps: Calibration–Subjects were asked to freely explore their range of shoulders and upper arms motion in all possible directions for about 60 seconds. This stage results in an aimless free dance, during which subjects explored their available range of motion. The “calibration” data set QCAL was organized as an SxM matrix, where the number S of rows is the dimensionality of the space of sensor signals (here, S = 8) and the number M of columns is the total number of samples (here M = 5000, collected over about 66 seconds at 75 samples per second). PCA–The principal components of the data set QCAL were extracted using PCA. The S eigenvectors of the covariance matrix of the sensor signals were ordered according to the magnitude of their corresponding eigenvalues, from largest to smallest. The eigenvalues represent the variance of the data along the eigenvector directions; those directions with high signal excursion correspond to larger eigenvalues. We considered those high-variance movement directions to be the user’s best controlled and most used combinations, and thus picked the K leading eigenvectors as rows to construct the KxS matrix H (Eq (5)). In our case the interface map H was a 2x8 matrix; the two first PCs accounted for 73±5% of the variance of the calibration dataset across subjects. Mapping Adjustments–Although the two-dimensional subspace formed by the first two PCs captured a large fraction of the total variance of body motions, it did not necessarily reflect the natural up-down/left-right orientation of the display monitor. Therefore, following calibration and PC extraction, there was a customization phase in which users were allowed to set the origin, orientation, and scaling of the coordinates in task space, based on their preference. After the calibration and mapping adjustments, subjects went through a two-hours training session. During training, the subjects used upper-body motions to control the movement of a cursor on a computer screen in order to reach a set of targets. Each trial started from the central target, which corresponded to the center of the screen; from there the subjects had to reach a peripheral target and return to center. The six peripheral targets were located 5 cm away from the center, equally spaced in directions 60° apart. The targets were presented randomly, with the condition that a given target was not presented until the subject had reached all other targets. The training session was organized into 6 blocks. Each block consisted of 9 sequences of reaches to the 6 targets, for a total of 54 center-out movements and the corresponding 54 returns; this amounts to 324 reaching movements for each session. The subjects had no visual feedback about the cursor position for the first 0.4 seconds following movement onset. The cursor then became visible, and the subjects could use visual feedback to correct for the reaching error if the cursor was not on target. During the initial 0.4 seconds without visual feedback, the subjects executed a ballistic movement based solely on their current estimate of the forward body-to-cursor map. We show in S2 Fig the trajectories described by the cursor in the two-dimensional task space. Data is shown at different stages of learning, for each of the six presented targets. In addition, we show in S3 Fig, for all six subjects enrolled in the study, the time evolution during training of three indices that convey information about cursor movement: i) the time to reach the target, ii) the mean speed, and iii) the linearity index computed as the length of the trajectory from the central target to the external one normalized by the length of the straight line connecting these two targets. Data analysis Estimation of the learning dynamics. To investigate the learning dynamics, we focused on the temporal evolution of two scalar variables in task space: the reaching error (RE) and the inverse model error (IME). These two variables were computed both from the data obtained from the subjects and from the synthetic data generated by simulating each subject-specific model of the individual learning process. The reaching error RE was computed as the norm of the difference between the actual cursor position at the end of the reaching movement and the target position. For the experimental data, we considered as an estimate of the reaching error the distance between target and cursor at the end of the blind phase of the trial, which involved motions that relied only on the subject’s inverse internal model, in absence of visual feedback of the cursor motion. As the cursor reappeared, the subjects performed a corrective movement, bringing the cursor to target. The inverse model error IME was computed as the norm of the difference between the identity matrix IK and the product H G(n) between the actual interface map and the estimate of the inverse map at the end of each trial. The lower dimensionality of the output space for the interface map H causes the problem of finding the inverse map to be ill-posed; the surface defined by the squared reaching error in the state space spanned by the components of G does not exhibit a single global minimum but a flat extended “valley” corresponding to all possible inverses of the interface map H. To circumvent this ambiguity and to monitor whether subjects converged to a stable inverse transformation, we estimated the inverse model matrix G(n) from the subjects’ performance. A typical experimental data set consisted of temporal sequences of reaching movements. At each trial n we considered a movement set, a sequence of r trials that included the n-th trial and the (r-1) trials that preceded it. Here we used r = 12, so that on average each movement set included two trials towards each of the six different targets. The body and target vectors for the n-th movement set were collected in the arrays (16) The matrix G(n) then was obtained from a least-squares estimation based on Q(n) = G(n)U(n) (see Eq (3)): (17) The history of reaching errors for the r trials in the movement set that ended with trial n was computed as (18) A scalar reaching error (RE) was then calculated by taking the spectral norm ∥E(n)∥ of the Kxr matrix in Eq (18). Similarly, we calculated the inverse model error (IME) as the spectral norm ∥H G(n) − IK∥. The IME is expected to approach 0 as learning converges and G(n) approaches a right inverse of H. The convergence to a stable representation of the inverse map was assessed by computing the percentage difference in norm among consecutive estimations of G(n), (19) We defined two additional errors to quantify whether each subject was also forming an estimate of the forward map that converged to the interface map H. We defined the forward model error (FME) as (20) The current estimate affects the prediction error, computed in task space as the difference between the actual position of the cursor and its estimated position based on the current estimate of the forward model. The L2 norm of this difference defines (21) with p(n) = H q(n). No moving window was used to compute these two errors, because both PE and FME could be extracted from the simulated data at each trial. Model parameters. For each subject recruited for the study, we constructed a model that used the same interface map H as used by the subject and was exposed to the same target sequence. To set the individual learning rate η for the learning of the inverse map in Eqs (2) and (14), we fitted the experimentally observed decay of the norm RE of the reaching errors for each subject to an exponential of the form (22) to obtain a value of λRE for each subject. We then set η = λRE in the corresponding subject-specific model. To set values for the parameters ε and σ we adopted a minimum search approach to minimizing a cost function based on the forward model error (FME), as those two parameters mostly influence the evolution of the estimated forward model . The cost function C was defined as (23) where N is the total number of trials. Comparison between simulated and real data. To estimate the similarity between real and model data we used the correlation coefficient R2. By definition, R2 evaluates similarity between the shapes of the compared curves and also provides additional information regarding the amplitude. This metric was applied to the norm RE of the reaching error, the norm IME of the inverse model error, the body coordinates q, and the cursor coordinates p. For each of these quantities, assume that the measured values are {y(n)}, 1 ≤ n ≤ N, where N is the total number of trials. For each one of these values, the model gives a prediction or estimation , 1 ≤ n ≤ N, with residuals , 1 ≤ n ≤ N. The mean of the observed data is given by . The total sum of squares (24) is proportional to the variance of the experimentally observed values. The sum of squares of the residuals is (25) The most general definition of R2, as used here, follows from the ratio between these two sums of squares Ethics statement All experimental procedures were approved by the Northwestern University Institutional Review Board. All subjects signed a consent form prior participating to the study. Computational model Inverse kinematics is a well-known and well-explored computational problem in robotics [54, 55] and human motor control [34, 56]; it refers to finding the configuration of joint angles that results in a desired position of an end effector or of the hand in the operational space [57]. Inverse kinematics problems become ill-posed [58] when there are multiple valid solutions as a consequence of the many-to-one nature of the forward kinematic map. This is the situation considered here, in which the kinematics that the subjects are controlling may be partitioned in a sequence of two maps. In a first map, the subjects control the motions of their bodies by acting on a multitude of muscles and joints. In a second map, the signals triggered by these body motions determine the lower dimensional motion of an external object such as a wheelchair [17], a cursor on a computer screen [16], or a drone [20]. Here we make the critical but reasonable assumption that the subjects have already acquired in a stable form the expertise needed to control the motion of their body, or at least portions of it that were unaffected by injury or disease. Therefore, they only need to acquire the second component. This is the component we focus on, limited here to a body-machine interface whose linear map H transforms, at any given trial n, an S-dimensional vector of body signals q into a K-dimensional control vector p as follows: (5) Here, H is a KxS matrix. Since K<S, this interface map is many-to-one and there is a "null space" of inputs for each value of the output, encompassing all different patterns of body signals that result in the same control signal. This is an important characteristic of the map; earlier work has shown that subjects learn through practice to separate the null space from its orthogonal “potent space” complement [53]. Learning dynamics as first-order state-based model. In a learning experiment where the goal is to reach targets in the control space, the superscript n labels the trials or successive repetitions of a single action; for instance, each trial is a reaching movement in a sequence of such movements. At the end of a trial, the learner observes an error e(n). This error drives the updating of the internal inverse model, which we assume to be a linear map G(n) transforming a goal u(n) into its corresponding body vector (previously Eq (3)): (6) Since the forward map H is linear, the linearity of G is a sufficient but not necessary condition. More complex, nonlinear forms of the inverse model would in principle be admissible. Here, we assume the simplest general form for a linear inverse model of the forward BoMI map; this assumption makes the investigation of learning dynamics tractable. For the n-th reaching trial, u(n) is the position of the target and the reaching error is the K-dimensional vector from the target position to the actual position of the controlled object at the end of the trial: (7) The internal model thus becomes a right-inverse model of the interface map: (8) As learning reaches a steady state, participants are expected to have eliminated this error. At this point, e(n) = 0, which requires H G(n) = IK. Note that the reaching error, a Kx1 vector, can be interpreted as the projection of the KxK matrix (HG(n) − IK) onto the Kx1 vector u(n). The metrics introduced earlier to monitor the acquisition of the inverse model are RE, the L2 norm of the Kx1 reaching error (HG(n) − IK) u(n), and IME, the spectral norm of the KxK matrix (HG(n) − IK). Learning is represented as a dynamical process whose state is the internal inverse model G(n); this state changes after the observation of each reaching error. The targets presented to the learner constitute the external input to this process. To ensure that the change in state leads to a reduction of the error, the learning process drives the state along the gradient of the quadratic error surface in the state space defined by the components of G. The gradient of the squared reaching error with respect to the components of the inverse model is (9) which leads to the update equation (10) or, equivalently (11) Here, η is a learning rate parameter that we model as a scalar. Although in principle there could be a different rate for learning every element of the forward and inverse models, we found that only two learning rates, ε for the forward model and η for the inverse model, sufficed to account for the observed learning behavior. If the interface map H is known, the update Eq (11) provides an estimate of the right inverse of H solely on the basis of control space data, without performing an explicit matrix inversion. Given the targets u(n), the variables of interest are the observed reaching errors e(n) and the estimated inverse model or "state of learning" G(n). As e(n) → 0, G(n) becomes stationary. The gradient of the error involves the actual value of the interface map H. It is not plausible to assume that our subjects had any initial notion of the interface map, let alone an exact representation. In a realistic model of learning, the value of H must be replaced with an evolving estimate . In this scenario, the current state of learning is represented in a higher dimensional space that includes the components of both G(n) and . In the case of S = 8 body signals controlling the location of an object in K = 2 dimensions, the state space for learning is 2x8x2 = 32-dimensional. We follow a concept introduced by Jordan and Rumelhart [14], and represent learning as the parallel development a of forward-inverse model. The forward model leads to a prediction of the controlled device position given the current set of body signals, whereas the inverse model generates the body signals needed to achieve a given target position of the device. In our study, we distinguish between the exploration that takes place when subjects perform tentative movements as they are seeking a goal, and the performance of aimless limb motions, which we refer to as “motor babbling” in accordance with the literature on human and robotic development [59, 60]. In our experiments, babbling takes place during the dance calibration used to establish the forward map. Motor babbling has also been considered as an exploratory behavior leading to the formation of internal dynamical models. However, here we use the concept of “exploration” to refer to the search for alternative solutions to the problem of generating a body configuration q that makes it possible to reach the target u. In principle, motor babbling and goal directed learning could take place in two separate phases. First, a motor babbling phase where freely produced, aimless body motions result in actual object motions that are compared to expected object motions to obtain prediction errors that drive the estimation of the forward model. Second, a phase where the subjects reach for specific targets, and the resulting reaching errors drive the estimation of the inverse model (see S1 Fig). This two-phase mechanism has been suggested as a possible model of motor development in infants [61], who acquire a model of the dynamical properties of limbs by driving them with erratic neuromuscular patterns of activity. However, this is not the case in experiments where subjects are presented with reaching targets from the onset; in this scenario, no aimless motor babbling was observed. It is thus more plausible to model forward and inverse learning as concurrent processes. The prediction error quantifies the difference between actual and predicted positions of the controlled object, without reference to a target. The gradient of the squared prediction error with respect to the components of the forward model H is (12) The update equations for the coupled learning process then are (13) (14) reported earlier as Eqs (1) and (2). Note that Eq (13) contains a term that is quadratic in q; this implies a quadratic dependence on the elements of G(n), since at each step q(n) is derived by applying G(n) to the corresponding target. The update equation for the estimate of the forward model is thus not linear, and the learning process is prone to getting stuck in local minima. The simplest way to avoid this is to add a small amount of noise to the body signals derived from the inverse model, (15) where the noise ξ has the same dimension S as q, and each component of ξ is independently drawn from a Gaussian distribution at each trial. Two important parameters of the combined learning model of Eqs (13) and (14) are the learning rates for the forward (ε) and the inverse (η) models. In the case of eight body signals mapped into a two-dimensional control space, these learning rates apply to the evolution of 16 elements each; their most general form would be a 2x8 matrix for ε and an 8x2 matrix for η. Here, for simplicity and to avoid overfitting, we assume both learning rates to be scalar. A similar assumption is made for the noise amplitude σ, a somewhat less critical parameter whose sole purpose is to add sufficient noise to the learning algorithm so as to avoid getting trapped in local minima. Learning dynamics as first-order state-based model. In a learning experiment where the goal is to reach targets in the control space, the superscript n labels the trials or successive repetitions of a single action; for instance, each trial is a reaching movement in a sequence of such movements. At the end of a trial, the learner observes an error e(n). This error drives the updating of the internal inverse model, which we assume to be a linear map G(n) transforming a goal u(n) into its corresponding body vector (previously Eq (3)): (6) Since the forward map H is linear, the linearity of G is a sufficient but not necessary condition. More complex, nonlinear forms of the inverse model would in principle be admissible. Here, we assume the simplest general form for a linear inverse model of the forward BoMI map; this assumption makes the investigation of learning dynamics tractable. For the n-th reaching trial, u(n) is the position of the target and the reaching error is the K-dimensional vector from the target position to the actual position of the controlled object at the end of the trial: (7) The internal model thus becomes a right-inverse model of the interface map: (8) As learning reaches a steady state, participants are expected to have eliminated this error. At this point, e(n) = 0, which requires H G(n) = IK. Note that the reaching error, a Kx1 vector, can be interpreted as the projection of the KxK matrix (HG(n) − IK) onto the Kx1 vector u(n). The metrics introduced earlier to monitor the acquisition of the inverse model are RE, the L2 norm of the Kx1 reaching error (HG(n) − IK) u(n), and IME, the spectral norm of the KxK matrix (HG(n) − IK). Learning is represented as a dynamical process whose state is the internal inverse model G(n); this state changes after the observation of each reaching error. The targets presented to the learner constitute the external input to this process. To ensure that the change in state leads to a reduction of the error, the learning process drives the state along the gradient of the quadratic error surface in the state space defined by the components of G. The gradient of the squared reaching error with respect to the components of the inverse model is (9) which leads to the update equation (10) or, equivalently (11) Here, η is a learning rate parameter that we model as a scalar. Although in principle there could be a different rate for learning every element of the forward and inverse models, we found that only two learning rates, ε for the forward model and η for the inverse model, sufficed to account for the observed learning behavior. If the interface map H is known, the update Eq (11) provides an estimate of the right inverse of H solely on the basis of control space data, without performing an explicit matrix inversion. Given the targets u(n), the variables of interest are the observed reaching errors e(n) and the estimated inverse model or "state of learning" G(n). As e(n) → 0, G(n) becomes stationary. The gradient of the error involves the actual value of the interface map H. It is not plausible to assume that our subjects had any initial notion of the interface map, let alone an exact representation. In a realistic model of learning, the value of H must be replaced with an evolving estimate . In this scenario, the current state of learning is represented in a higher dimensional space that includes the components of both G(n) and . In the case of S = 8 body signals controlling the location of an object in K = 2 dimensions, the state space for learning is 2x8x2 = 32-dimensional. We follow a concept introduced by Jordan and Rumelhart [14], and represent learning as the parallel development a of forward-inverse model. The forward model leads to a prediction of the controlled device position given the current set of body signals, whereas the inverse model generates the body signals needed to achieve a given target position of the device. In our study, we distinguish between the exploration that takes place when subjects perform tentative movements as they are seeking a goal, and the performance of aimless limb motions, which we refer to as “motor babbling” in accordance with the literature on human and robotic development [59, 60]. In our experiments, babbling takes place during the dance calibration used to establish the forward map. Motor babbling has also been considered as an exploratory behavior leading to the formation of internal dynamical models. However, here we use the concept of “exploration” to refer to the search for alternative solutions to the problem of generating a body configuration q that makes it possible to reach the target u. In principle, motor babbling and goal directed learning could take place in two separate phases. First, a motor babbling phase where freely produced, aimless body motions result in actual object motions that are compared to expected object motions to obtain prediction errors that drive the estimation of the forward model. Second, a phase where the subjects reach for specific targets, and the resulting reaching errors drive the estimation of the inverse model (see S1 Fig). This two-phase mechanism has been suggested as a possible model of motor development in infants [61], who acquire a model of the dynamical properties of limbs by driving them with erratic neuromuscular patterns of activity. However, this is not the case in experiments where subjects are presented with reaching targets from the onset; in this scenario, no aimless motor babbling was observed. It is thus more plausible to model forward and inverse learning as concurrent processes. The prediction error quantifies the difference between actual and predicted positions of the controlled object, without reference to a target. The gradient of the squared prediction error with respect to the components of the forward model H is (12) The update equations for the coupled learning process then are (13) (14) reported earlier as Eqs (1) and (2). Note that Eq (13) contains a term that is quadratic in q; this implies a quadratic dependence on the elements of G(n), since at each step q(n) is derived by applying G(n) to the corresponding target. The update equation for the estimate of the forward model is thus not linear, and the learning process is prone to getting stuck in local minima. The simplest way to avoid this is to add a small amount of noise to the body signals derived from the inverse model, (15) where the noise ξ has the same dimension S as q, and each component of ξ is independently drawn from a Gaussian distribution at each trial. Two important parameters of the combined learning model of Eqs (13) and (14) are the learning rates for the forward (ε) and the inverse (η) models. In the case of eight body signals mapped into a two-dimensional control space, these learning rates apply to the evolution of 16 elements each; their most general form would be a 2x8 matrix for ε and an 8x2 matrix for η. Here, for simplicity and to avoid overfitting, we assume both learning rates to be scalar. A similar assumption is made for the noise amplitude σ, a somewhat less critical parameter whose sole purpose is to add sufficient noise to the learning algorithm so as to avoid getting trapped in local minima. Validation of the model with experimental data To validate the outcomes of the model we recruited six unimpaired subjects (age range 21–40 years old, 3 males and 3 females) in the preliminary study. The subjects practiced the execution of reaching movements via an interface (Fig 6) that mapped an eight-dimensional signal space associated with upper body motions to the two-dimensional task space of a computer cursor. An array of four video cameras (V100, Naturalpoint Inc., OR, USA) was used to track active infrared light sources attached to the subject’s upper-body garments (two for each side of the body, one on the shoulder and one on the upper arm, as shown in Fig 6). Each camera pointed at a single marker, providing two signals defining the coordinates of the marker in the camera’s frame. Collectively, the four sensors provided an eight-dimensional body vector q that was transformed into a two-dimensional command vector p for controlling the position of a cursor on a computer screen, p = Hq. We show in S2 Fig the trajectories described by the each of the body sensors in the two-dimensional plane of its corresponding camera. Data is shown at different stages of learning, for each of the presented targets. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Experimental setup and protocol. a) Schematic overview of the setup of the Body Machine Interface. The subject sat on a chair in front of the computer. Active optical markers were placed on shoulders and upper arms; their positions were recorded by infrared cameras. b) Design of the reaching task. Six targets were equally spaced over a circumference of 5 cm radius, centered in the computer screen. The grey circle corresponds to the central target; this was the initial position from which each movement towards the peripheral targets (empty circles) would start; movements would return to the central target after the peripheral target was reached. The black small circle is the cursor controlled by the subjects. c) Experimental protocol. A calibration phase during which the subject moved freely and aimlessly provided data for the construction of the map H that transformed body movements into cursor movements. This was followed by an adjustment phase in order to match the control space to the computer screen. Training then started, divided in six blocks of nine repetitions of each of the six peripheral targets. https://doi.org/10.1371/journal.pcbi.1007118.g006 To design the matrix H for each subject, we used a standard linear dimensionality reduction method, principal component analysis (PCA) [62], to set up the dimensionality reduction implicit in the q to p map. PCA is based on the decorrelation of the input signals through the diagonalization of their covariance matrix; dimensionality reduction is obtained by keeping only the eigenvectors corresponding to the K largest eigenvalues. Here, these K = 2 eigenvectors became the rows of H. PCA provided us with a computationally straightforward method for identifying those directions that captured the largest range (i.e. the largest variance) of body motions within the space of sensor signals for each subject. The interface map H from the input space of body signals to the output task space was constructed in three steps: Calibration–Subjects were asked to freely explore their range of shoulders and upper arms motion in all possible directions for about 60 seconds. This stage results in an aimless free dance, during which subjects explored their available range of motion. The “calibration” data set QCAL was organized as an SxM matrix, where the number S of rows is the dimensionality of the space of sensor signals (here, S = 8) and the number M of columns is the total number of samples (here M = 5000, collected over about 66 seconds at 75 samples per second). PCA–The principal components of the data set QCAL were extracted using PCA. The S eigenvectors of the covariance matrix of the sensor signals were ordered according to the magnitude of their corresponding eigenvalues, from largest to smallest. The eigenvalues represent the variance of the data along the eigenvector directions; those directions with high signal excursion correspond to larger eigenvalues. We considered those high-variance movement directions to be the user’s best controlled and most used combinations, and thus picked the K leading eigenvectors as rows to construct the KxS matrix H (Eq (5)). In our case the interface map H was a 2x8 matrix; the two first PCs accounted for 73±5% of the variance of the calibration dataset across subjects. Mapping Adjustments–Although the two-dimensional subspace formed by the first two PCs captured a large fraction of the total variance of body motions, it did not necessarily reflect the natural up-down/left-right orientation of the display monitor. Therefore, following calibration and PC extraction, there was a customization phase in which users were allowed to set the origin, orientation, and scaling of the coordinates in task space, based on their preference. After the calibration and mapping adjustments, subjects went through a two-hours training session. During training, the subjects used upper-body motions to control the movement of a cursor on a computer screen in order to reach a set of targets. Each trial started from the central target, which corresponded to the center of the screen; from there the subjects had to reach a peripheral target and return to center. The six peripheral targets were located 5 cm away from the center, equally spaced in directions 60° apart. The targets were presented randomly, with the condition that a given target was not presented until the subject had reached all other targets. The training session was organized into 6 blocks. Each block consisted of 9 sequences of reaches to the 6 targets, for a total of 54 center-out movements and the corresponding 54 returns; this amounts to 324 reaching movements for each session. The subjects had no visual feedback about the cursor position for the first 0.4 seconds following movement onset. The cursor then became visible, and the subjects could use visual feedback to correct for the reaching error if the cursor was not on target. During the initial 0.4 seconds without visual feedback, the subjects executed a ballistic movement based solely on their current estimate of the forward body-to-cursor map. We show in S2 Fig the trajectories described by the cursor in the two-dimensional task space. Data is shown at different stages of learning, for each of the six presented targets. In addition, we show in S3 Fig, for all six subjects enrolled in the study, the time evolution during training of three indices that convey information about cursor movement: i) the time to reach the target, ii) the mean speed, and iii) the linearity index computed as the length of the trajectory from the central target to the external one normalized by the length of the straight line connecting these two targets. Data analysis Estimation of the learning dynamics. To investigate the learning dynamics, we focused on the temporal evolution of two scalar variables in task space: the reaching error (RE) and the inverse model error (IME). These two variables were computed both from the data obtained from the subjects and from the synthetic data generated by simulating each subject-specific model of the individual learning process. The reaching error RE was computed as the norm of the difference between the actual cursor position at the end of the reaching movement and the target position. For the experimental data, we considered as an estimate of the reaching error the distance between target and cursor at the end of the blind phase of the trial, which involved motions that relied only on the subject’s inverse internal model, in absence of visual feedback of the cursor motion. As the cursor reappeared, the subjects performed a corrective movement, bringing the cursor to target. The inverse model error IME was computed as the norm of the difference between the identity matrix IK and the product H G(n) between the actual interface map and the estimate of the inverse map at the end of each trial. The lower dimensionality of the output space for the interface map H causes the problem of finding the inverse map to be ill-posed; the surface defined by the squared reaching error in the state space spanned by the components of G does not exhibit a single global minimum but a flat extended “valley” corresponding to all possible inverses of the interface map H. To circumvent this ambiguity and to monitor whether subjects converged to a stable inverse transformation, we estimated the inverse model matrix G(n) from the subjects’ performance. A typical experimental data set consisted of temporal sequences of reaching movements. At each trial n we considered a movement set, a sequence of r trials that included the n-th trial and the (r-1) trials that preceded it. Here we used r = 12, so that on average each movement set included two trials towards each of the six different targets. The body and target vectors for the n-th movement set were collected in the arrays (16) The matrix G(n) then was obtained from a least-squares estimation based on Q(n) = G(n)U(n) (see Eq (3)): (17) The history of reaching errors for the r trials in the movement set that ended with trial n was computed as (18) A scalar reaching error (RE) was then calculated by taking the spectral norm ∥E(n)∥ of the Kxr matrix in Eq (18). Similarly, we calculated the inverse model error (IME) as the spectral norm ∥H G(n) − IK∥. The IME is expected to approach 0 as learning converges and G(n) approaches a right inverse of H. The convergence to a stable representation of the inverse map was assessed by computing the percentage difference in norm among consecutive estimations of G(n), (19) We defined two additional errors to quantify whether each subject was also forming an estimate of the forward map that converged to the interface map H. We defined the forward model error (FME) as (20) The current estimate affects the prediction error, computed in task space as the difference between the actual position of the cursor and its estimated position based on the current estimate of the forward model. The L2 norm of this difference defines (21) with p(n) = H q(n). No moving window was used to compute these two errors, because both PE and FME could be extracted from the simulated data at each trial. Model parameters. For each subject recruited for the study, we constructed a model that used the same interface map H as used by the subject and was exposed to the same target sequence. To set the individual learning rate η for the learning of the inverse map in Eqs (2) and (14), we fitted the experimentally observed decay of the norm RE of the reaching errors for each subject to an exponential of the form (22) to obtain a value of λRE for each subject. We then set η = λRE in the corresponding subject-specific model. To set values for the parameters ε and σ we adopted a minimum search approach to minimizing a cost function based on the forward model error (FME), as those two parameters mostly influence the evolution of the estimated forward model . The cost function C was defined as (23) where N is the total number of trials. Comparison between simulated and real data. To estimate the similarity between real and model data we used the correlation coefficient R2. By definition, R2 evaluates similarity between the shapes of the compared curves and also provides additional information regarding the amplitude. This metric was applied to the norm RE of the reaching error, the norm IME of the inverse model error, the body coordinates q, and the cursor coordinates p. For each of these quantities, assume that the measured values are {y(n)}, 1 ≤ n ≤ N, where N is the total number of trials. For each one of these values, the model gives a prediction or estimation , 1 ≤ n ≤ N, with residuals , 1 ≤ n ≤ N. The mean of the observed data is given by . The total sum of squares (24) is proportional to the variance of the experimentally observed values. The sum of squares of the residuals is (25) The most general definition of R2, as used here, follows from the ratio between these two sums of squares Estimation of the learning dynamics. To investigate the learning dynamics, we focused on the temporal evolution of two scalar variables in task space: the reaching error (RE) and the inverse model error (IME). These two variables were computed both from the data obtained from the subjects and from the synthetic data generated by simulating each subject-specific model of the individual learning process. The reaching error RE was computed as the norm of the difference between the actual cursor position at the end of the reaching movement and the target position. For the experimental data, we considered as an estimate of the reaching error the distance between target and cursor at the end of the blind phase of the trial, which involved motions that relied only on the subject’s inverse internal model, in absence of visual feedback of the cursor motion. As the cursor reappeared, the subjects performed a corrective movement, bringing the cursor to target. The inverse model error IME was computed as the norm of the difference between the identity matrix IK and the product H G(n) between the actual interface map and the estimate of the inverse map at the end of each trial. The lower dimensionality of the output space for the interface map H causes the problem of finding the inverse map to be ill-posed; the surface defined by the squared reaching error in the state space spanned by the components of G does not exhibit a single global minimum but a flat extended “valley” corresponding to all possible inverses of the interface map H. To circumvent this ambiguity and to monitor whether subjects converged to a stable inverse transformation, we estimated the inverse model matrix G(n) from the subjects’ performance. A typical experimental data set consisted of temporal sequences of reaching movements. At each trial n we considered a movement set, a sequence of r trials that included the n-th trial and the (r-1) trials that preceded it. Here we used r = 12, so that on average each movement set included two trials towards each of the six different targets. The body and target vectors for the n-th movement set were collected in the arrays (16) The matrix G(n) then was obtained from a least-squares estimation based on Q(n) = G(n)U(n) (see Eq (3)): (17) The history of reaching errors for the r trials in the movement set that ended with trial n was computed as (18) A scalar reaching error (RE) was then calculated by taking the spectral norm ∥E(n)∥ of the Kxr matrix in Eq (18). Similarly, we calculated the inverse model error (IME) as the spectral norm ∥H G(n) − IK∥. The IME is expected to approach 0 as learning converges and G(n) approaches a right inverse of H. The convergence to a stable representation of the inverse map was assessed by computing the percentage difference in norm among consecutive estimations of G(n), (19) We defined two additional errors to quantify whether each subject was also forming an estimate of the forward map that converged to the interface map H. We defined the forward model error (FME) as (20) The current estimate affects the prediction error, computed in task space as the difference between the actual position of the cursor and its estimated position based on the current estimate of the forward model. The L2 norm of this difference defines (21) with p(n) = H q(n). No moving window was used to compute these two errors, because both PE and FME could be extracted from the simulated data at each trial. Model parameters. For each subject recruited for the study, we constructed a model that used the same interface map H as used by the subject and was exposed to the same target sequence. To set the individual learning rate η for the learning of the inverse map in Eqs (2) and (14), we fitted the experimentally observed decay of the norm RE of the reaching errors for each subject to an exponential of the form (22) to obtain a value of λRE for each subject. We then set η = λRE in the corresponding subject-specific model. To set values for the parameters ε and σ we adopted a minimum search approach to minimizing a cost function based on the forward model error (FME), as those two parameters mostly influence the evolution of the estimated forward model . The cost function C was defined as (23) where N is the total number of trials. Comparison between simulated and real data. To estimate the similarity between real and model data we used the correlation coefficient R2. By definition, R2 evaluates similarity between the shapes of the compared curves and also provides additional information regarding the amplitude. This metric was applied to the norm RE of the reaching error, the norm IME of the inverse model error, the body coordinates q, and the cursor coordinates p. For each of these quantities, assume that the measured values are {y(n)}, 1 ≤ n ≤ N, where N is the total number of trials. For each one of these values, the model gives a prediction or estimation , 1 ≤ n ≤ N, with residuals , 1 ≤ n ≤ N. The mean of the observed data is given by . The total sum of squares (24) is proportional to the variance of the experimentally observed values. The sum of squares of the residuals is (25) The most general definition of R2, as used here, follows from the ratio between these two sums of squares Supporting information S1 Fig. Modeling human learning as sequential learning of forward and inverse models. Data for the six subjects enrolled in the study (S1-S6). Model parameters in this sequential scenario were independently estimated for each subject as discussed in Methods. (a) Temporal evolution of the norm RE of the reaching error as a function of trial number n calculated from the experimental data (black) and from the model simulations (blue). (b) Temporal evolution of the norm IME of the inverse model error, the difference between the identity matrix IK and the product of the interface map H and the inverse model G(n), estimated from the experimental data (black) and from the model simulations (blue). Both metrics were calculated over a moving window that includes the current and its 11 preceding trials. https://doi.org/10.1371/journal.pcbi.1007118.s001 (TIF) S2 Fig. Example of body and cursor data. Body signals and cursor trajectories across training, from a representative subject. Body signals during each reach are shown as the motions of the four body markers in the frame of the corresponding cameras; camera labels and scales in pixels along camera axes are shown only for the top left panel. All trajectories are color coded to identify target direction, and shown for the 400 ms following movement onset; the cursor was not visible to the user during this period. Scale for cursor motions is also shown only for the top left panel. Cursor and body-signal trajectories are shown for the first and last set of reaches for block 1 (top panels) and block 6 (bottom panels). https://doi.org/10.1371/journal.pcbi.1007118.s002 (TIF) S3 Fig. Cursor control in the task space. Data for the six subjects enrolled in the study (S1-S6). Temporal evolution of the (a) time to reach the target, (b) mean speed, and (c) linearity index, defined as the length of the trajectory from the central target to the peripheral target divided by the length of the straight line connecting the two. All three indices were calculated over a moving window that includes the current and its 11 preceding trials. https://doi.org/10.1371/journal.pcbi.1007118.s003 (TIF) Acknowledgments The authors are grateful to Prof. Marco Baglietto for illuminating discussions about the modeling aspects of this work.

Yan, Jinyuan;Estanbouli, Henri;Liao, Chen;Kim, Wook;Monk, Jonathan M.;Rahman, Rayees;Kamboj, Mini;Palsson, Bernhard O.;Qiu, Weigang;Xavier, Joao B.

doi: 10.1371/journal.pcbi.1007562pmid: 31860667

Introduction The rise of antibiotic resistant bacteria is a major global problem [1,2]. Predicting, preventing and treating antibiotic resistant infections present challenges that are best addressed with multidisciplinary approaches combining evolutionary, molecular and computational biology [3]. Bacteria can acquire resistance through horizontal gene transfer, but they can also repurpose mechanisms they already possess. Chromosomal point mutations, in particular, enable rapid rewiring of bacterial regulatory networks [4] and provide means to evolve antibiotic resistance rapidly—a major risk for patients receiving therapy. P. aeruginosa is a Gram-negative pathogen and a main cause of hospital-acquired infections [5]. This pathogen is often studied in the context of chronic lung infections of cystic fibrosis patients where infections can last decades; during that time patients receive frequent and aggressive treatments that select for antibiotic resistance [6] and biofilm formation [7]. On the other hand, P. aeruginosa causes acute infections in immune-compromised patients where everything happens quicker [8,9]. Acute P. aeruginosa infection of cancer patients receiving immuno-suppressive therapy for bone marrow transplantation, for example, has the highest 7-day mortality rate among all infections that afflict these patients [10]. The mechanisms driving evolution of P. aeruginosa resistance in vivo in acutely-infected patients—even as they receive treatment—has arguably received less attention. Aztreonam, a monobactam derivative of beta-lactams with low susceptibility to beta-lactamases, is an important defense against Gram-negative bacteria including P. aeruginosa [11]. Its use in cystic fibrosis patients started in 2010 and has increased steadily since then [12]. But recent work has shown that P. aeruginosa can acquire rapid resistance against aztreonam in vitro, often through chromosomal mutations in one of 19 genes linked to overexpression of efflux systems or on the cellular target of aztreonam [13]. These mutations reportedly decreased in vitro growth rates in the absence of antibiotics, indicating an associated fitness cost. Similar chromosomal mutations were also found in isolates from cystic fibrosis, which highlights their clinical relevance for the treatment of P. aeruginosa chronic infections [13]. Could the same type of chromosomal mutations drive a rapid evolution of antibiotic resistance in acutely infected patients, even as they receive antibiotic treatment? And if so, what system-level changes enable the pathogen to thrive? Here we present a comparative analysis across dozens of clinical isolates to show that NalD—a transcriptional repressor of the MexAB-OprM efflux system—has the strongest association with aztreonam resistance in isolates from acutely infected patients. Then, we dissect the case of one particular acutely-infected patient in whom aztreonam resistance evolved in vivo during aztreonam therapy. We demonstrate that the resistance was acquired due to a loss-of-function mutation in NalD and caused overexpression of the MexAB-OprM efflux pump consistent with the known mechanism of resistance. However, we found no fitness cost in the aztreonam-resistant strain in the absence of the drug comparing to its closest susceptible isolate. By integrating a genome-scale metabolic model with transcriptomics data, we explored whether the resistant strain has developed metabolic adaptations to compensate for the resistance. The model revealed system-level changes beyond the primary mechanism of resistance that included adaptations in major metabolic pathways, which may explain the lack of a fitness cost. We discuss how understanding the metabolic adaptations that offset the fitness cost of resistance may pave the way to future therapies against antibiotic resistant infections. Results Aztreonam resistance is associated with NalD mutation in acutely-infected patients To identify genomic features related to aztreonam resistance in patients acutely infected with P. aeruginosa, we started by measuring the minimum inhibitory concentration (MIC) of aztreonam in 31 P. aeruginosa isolates from cancer patients that we had previously sequenced [14]. Plotting the MIC levels next to a phylogenetic tree constructed from the core genome of the 31 isolates showed no discernable association between aztreonam MIC and phylogeny (Fig 1A). It is possible that the large genome-scale differences among the clinical isolates obscured the relationship between causal genetic variants and the desired phenotype. Therefore, we narrowed down the analysis to a smaller set of genes by focusing on the 19 genes where mutations emerged recurrently in vitro under aztreonam selection [13]. We tested the association between aztreonam MIC and the variation in the protein sequence coded by each of the 19 genes using a rank sum test (S1 Table). To our surprise, only one—NalD—passed the significance test (p = 0.0046), and associated with a >2-fold increase in average MIC. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Aztreonam resistance is associated with variation in NalD across independent clinical isolates from acute P. aeruginosa infection. (A) Phylogenetic tree of isolates from acute infections of cancer patients reconstructed from core genes, including the type strains PA14, PAO1 and PA7. The minimal inhibitory concertation (MIC) of the aztreonam varies significantly across the phylogenetic tree, showing it is not a phylogenetically conserved trait. (B) NalD protein is the only protein in the mexAB-oprM efflux pathway that is strongly associated with aztreonam MIC in a rank sum test (***, p = 0.005). The table on the bottom shows the p-values for rank sum tests conducted on other proteins known from the mexAB-oprM efflux system and its regulatory pathway. (C) Expanding the analysis to all the transcriptional regulators encoded by the P. aeruginosa genome revealed 30 candidates whose protein sequences variation were associated with aztreonam MIC (see S2 Table), but NalD remained the strongest correlate. https://doi.org/10.1371/journal.pcbi.1007562.g001 NalD is a transcriptional repressor of the efflux system MexAB-OprM and mutations in NalD have been linked to multi-drug resistance, including to aztreonam [15,16]. However, the other two regulators of MexAB-OprM mutated in experimental evolution [13], NalC and MexR, were not significantly associated with aztreonam MIC in our clinical isolates (both with p>0.5). We then tested all the known proteins in the MexAB-OprM pathway [15,17] including the efflux pump coding proteins themselves (Fig 1B). Again, of the 7 proteins only NalD passed the association test. To confirm the association further, we downloaded NalD sequences of 126 P. aeruginosa isolates which had published aztreonam MIC values from the PATRIC database [18]. This collection, which has isolates from many sources including acute and chronic infections, showed again that NalD is significantly associated with aztreonam resistance (p<0.01). This robust association suggested that mutations in NalD are main drivers of parallel evolution of aztreonam resistance in multiple lineages of P. aeruginosa. We compared the NalD protein sequences among our 31 clinical isolates including three type strains of P. aeruginosa (PA14, PAO1 and PA7). These NalD sequences are highly conserved, with only a few variations from the consensus (S1 Fig). Nonetheless, the strains that do vary from the consensus tend to rank high in terms of aztreonam MIC. One of the most resistant isolates, X9820, carries a copy of NalD with a deletion of residues 1~134 (>60% of the full length NalD) which plausibly causes loss of NalD function. Three other strains tested (W70322, W60856 and the type strain PA7) have mutations also in the 10th alpha helix of the protein’s structure, and two strains (H27930, F23197) carry point mutation close to the C-terminus. Four strains (M55212, F30658, W91452, W25637) have mutation T11N located in the first residue of the first alpha helix, which likely impairs the DNA binding function of NalD. Still, NalD variation alone explains only part of the aztreonam MIC. For example, two isolates that have the top aztreonam MIC, T38079 and T6313, have the same NalD sequences as the consensus (S1 Fig). Could variation in other transcriptional regulators explain aztreonam MIC? To address this question we comprehensively examined all annotated transcriptional regulators (>300) in the P. aeruginosa genome [18]. Thirty-one of these regulators were significantly associated with aztreonam resistance according to the rank sum test, but NalD still topped the list (Fig 1C). Evolution of P. aeruginosa aztreonam resistance within a patient To detail the drastic effects of aztreonam resistance in vivo we analyzed isolates obtained from a patient who died with an aztreonam resistant P. aeruginosa. The patient had been diagnosed with pre B cell acute lymphoblastic leukemia and was admitted (day -10 relative to day of transplantation) to Memorial Hospital for hematopoietic cell transplantation after undergoing first chemo remission (Fig 2A). As standard of care, the intense conditioning regimen compromises the patient’s immunity and can lead to life-threatening complications [19,20]. Therefore routine antibiotic prophylaxis with vancomycin, ciprofloxacin and pip-tazobactam was administered. This particular patient developed tachycardia on the day of stem cell infusion (day 0), followed by fever one day later (day +1). Blood cultures were drawn and cefepime and imipenem were administered to treat a plausible bacterial infection. On day +4 the patient worsened and developed sepsis, requiring transfer to the intensive care unit (ICU). On day +5, antimicrobials were changed to meropenem, amikacin and polymyxin. Blood cultures at this time tested positive for P. aeruginosa with resistance to multiple antipseudomonal agents but sensitive to aztreonam (Fig 2B). On day +6 the patient received aztreonam in addition to meropenem and avibactam as a last resort attempt to control the worsening infection. The patient’s clinical condition deteriorated and the patient eventually expired from sepsis on day +8. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. P. aeruginosa infection of a cancer patient hospitalized to receive hematopoietic cell transplantation evolved resistance to aztreonam in the course of therapy. (A) Timeline shows clinical events: conditioning regimen (myeloablation), hematopoietic cell infusion, location in the hospital (bone marrow transplantation unit [BMT] or intensive care unit [ICU]), the period of neutropenia, antibiotics administered, the day (relative to the day of transplant, day 0) and body site of origin (rsw: rectal swab; spt: sputum; bld: blood) of the eight P. aeruginosa isolates analyzed here. (B) Antibiotic resistance profiles of blood isolates measured by the clinical microbiology laboratory as the infection progressed; the profiles informed clinicians that the infection was multi-drug-resistant but also that it was initially sensitive to aztreonam (isolates D+3bld, D+4bld and D+5bld). As the disease progressed to become life-threatening sepsis, the patient was transferred to the ICU and was given aztreonam; however, the isolate D+7bld demonstrated resistance to aztreonam. The patient died on day +8. https://doi.org/10.1371/journal.pcbi.1007562.g002 To better understand the progression of aztreonam resistance, we tracked the origin of the P. aeruginosa infection by retrospectively culturing the initial rectal swab (day -10) and additional swabs taken at days -3 and +4, as well as a sputum sample from day +7. All samples produced P. aeruginosa colonies (Fig 2A). In total, we obtained eight P. aeruginosa isolates from this patient. We named those isolates by the number of days before (D-) or after (D+) the transplantation followed by body sites where they were isolated (Fig 2A). Importantly, the detection of P. aeruginosa on the day -10 rectal swab—obtained at the time of admission to the hospital—indicated that the patient had carried P. aeruginosa asymptomatically in the gut when entering the hospital. Of note, the patient’s pre-transplant care was delivered in another country and no prior rectal swab samples were available to determine the duration of carriage. We sequenced the whole genomes of eight aforementioned P. aeruginosa isolates (hereafter called sepsis isolates as a group). To track whether the infection was originated from the patient or acquired from the hospital, we constructed the phylogenetic tree with the sepsis isolates and isolates from other cancer patients in the same hospital analyzed earlier [14], as well as the three type strains PA14, PAO1 and PA7 (Fig 3A). The tree revealed that the eight sepsis isolates belong to the PAO1 clade and are much more similar to each other than to any other isolates obtained earlier from the same hospital. This supports the notion that the infection progressed from a single clone that the patient harbored at the time of admission and was not acquired after admission to Memorial Hospital. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Whole-genome sequencing of eight P. aeruginosa isolates pinpointed the mutation NalDF198L responsible for aztreonam resistance detected one day before the patient died. (A) A genome-based phylogenetic tree shows that the eight sepsis isolates are highly related to each other compared to other clinical isolates and type strains. (B) Genomic analysis revealed that the aztreonam-resistant isolate (D+7bld) had only two unique variations compared to aztreonam-sensitive isolates. Vertical dashed lines highlight the common presence of a given variation across multiple isolates. (C) Aztreonam MIC confirmed the clinical laboratory results for the eight sepsis isolates. (D) Experimental validation in PA14 showed that mutation in NalD but not in peg.4653 (PA14_50010) confers aztreonam resistance. “-”and “+” denote absence or presence of mutation found in D+7bld; “Δ” denotes deletion of the 10th alpha helix in NalD protein. (**, p-value<0.01). https://doi.org/10.1371/journal.pcbi.1007562.g003 A genome alignment analysis revealed that the eight genomes are remarkably similar to each other (Fig 3B) with only a total of 12 unique allelic differences among them. We confirmed SNPs and small gaps using targeted (Sanger) sequencing (Materials and methods, S3 Table). Isolate D+4rsw is the most phylogenetically distinct strain among the sepsis isolates. Isolates D+5bld and D-3rsw are identical to each other and harbor only 4 variations from the ancestral alleles, which we inferred by using PAO1 as a reference. Among those variations are two discrete insertions greater than 10kb. A BLAST search in NCBI linked one to a transposon insertion (S6 Table) and another to a duplication of a region of its own genome encoding an unclear pathway. There is also a deletion homologous to a phage insertion (S6 Table). Notably, the 12 genetic differences found among the eight isolates showed no pattern of association with either the time or the body site of isolation (Fig 3B), indicating that the P. aeruginosa population had diverged during colonization with multiple sub-clones coexisting at the same time in a single patient. Similar patterns of within host diversification were reported in other host-associated bacteria [21]. Mutation in NalD conferred aztreonam resistance To understand better why many clinical isolates have mutations in NalD, we first confirmed the mechanism of aztreonam resistance in the sepsis isolates. We conducted detailed measurement of aztreonam MIC for the eight sepsis isolates (Fig 3C, S1 Data). The aztreonam-resistant D+7bld isolate displayed a higher MIC (between 8–12μg/mL) than the other 7 sepsis isolates (<8μg/mL) consistent with the clinical report (Fig 2). However, D+7bld was not the isolate with the highest MIC when compared to the expanded collection comprising isolates from other cancer patients and the type strains PAO1, PA14 and PA7 (Fig 3C). Two isolates, X9820 and T38079, had MICs higher than 12μg/mL, the highest concentration tested. PA7, a type strain known for its resistance to a broad spectrum of antibiotics [22], showed similar aztreonam MIC to the 7 sepsis isolates that were considered clinically susceptible to that drug. The two widely used laboratory strains, PAO1 and PA14, had very low MICs. The aztreonam-resistant sepsis isolate D+7bld has only two genetic variations from its most closely related isolates, D+5bld and D-3rsw. One of those two mutations is a 10bp deletion in a dehydrogenase of unclear function. To determine if this mutation alone could have increased P. aeruginosa resistance to aztreonam we introduced the same 10bp deletion in the corresponding dehydrogenase gene (PA14_50010, or peg.4653 in D+7bld) in the laboratory strain PA14 (Materials and methods, S4 Table). This mutation did not increase aztreonam MIC (Fig 3D, S2 Data). The other mutation was a point mutation F198L found in NalD, a mutation that has not been reported nor selected through in vitro experiments before [13]. To confirm that this mutation alone could have caused aztreonam resistance we engineered the same NalDF198L mutation into PA14 and the MIC increased 3-fold from 4μg/mL to 12μg/mL, a MIC similar to the MIC of the terminal sepsis isolate D+7bld (Fig 3D). This confirmed that the NalDF198L mutation alone is sufficient for the observed aztreonam resistance in D+7bld, and is consistent with our finding that NalD mutation can increase aztreonam MIC on average by >2-fold (Fig 1B). NalD variation linked to multi-drug resistance Our data suggests that NalD is a recurrent driver for aztreonam resistance in P. aeruginosa acute infection. NalD is not the only transcriptional regulator where mutations can drive antibiotic resistance. In cystic fibrosis patients treated with ciprofloxacin and azithromycin during chronic P. aeruginosa infection, mutations accumulate in transcriptional regulator NfxB, which negatively regulates another efflux pump, MexCD-OprJ [23]. Can mutations found in transcriptional regulators be used to predict the antibiotic resistance of a P. aeruginosa isolate? To address this question we posed two related but more specific questions: First, is there a way to predict aztreonam resistance from sequence variation in all transcription factors? Second, does NalD variation alone predict resistance to other antibiotics besides aztreonam? To answer the first question, we used a machine learning approach called LASSO (least absolute shrinkage and selection operator) [24]. We checked if this method could select transcriptional regulators based on their sequence variation to explain the aztreonam MIC data in our P. aeruginosa acute infection isolates. The LASSO produced a model where only two transcription factors (out of >200) explained more than 60% of variation in aztreonam MIC (R2 = 0.65, Fig 4A): NalD and PA14_37120, a probable LysR-type transcriptional factor. As expected, the coefficient for NalD was positive, implying that mutations in NalD tend to increase aztreonam MIC and therefore confer aztreonam resistance by >2x. By contrast, PA14_37120 had a negative coefficient, indicating that isolates that contain PA14_37120 different from the consensus have lower aztreonam MIC and, therefore, tend to be more sensitive to aztreonam. The negative relationship seems to be common for LysR-type proteins which are positive regulators of enzymes that degrade antibiotics [25], suggesting that this specific type of transcriptional factors could potentially be explored as a target to sensitize P. aeruginosa to aztreonam. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. NalD mutation contributes to a general signature of antibiotic resistance. (A) A signature of aztreonam resistance obtained with LASSO regression shows only two transcriptional regulators, including NalD, and explains >60% of the variation in aztreonam MIC. The coefficients have units of fold-change. (B) Antibiotic inhibitions zones were measured using the disk assay for 8 antibiotics from several classes. The inhibition zone indices (shows as normalized areas of inhibition disk) show that the 8 sepsis isolates are resistant to multiple antibiotics. A resistance index computed from combining the negative values of the inhibition zone indices shows that the 8 sepsis isolates rank higher in multi-drug resistant than any other isolate tested. (C) The signature of multi-antibiotic resistance has six transcriptional regulators, including NalD, and explains >80% of the variation in the multi-drug resistance index. The coefficients have units of integrated fold-change across all the 8 antibiotics. https://doi.org/10.1371/journal.pcbi.1007562.g004 To answer the second question, we measured the sensitivities of each P. aeruginosa acute infection isolate to a panel of eight antibiotics from several classes (ciprofloxacin, gentamicin, aztreonam, chloramphenicol, ampicillin, tetracycline, meropenem, cefepime) and quantified the degree of multi-drug resistance by combining those sensitivity values into a multi-drug resistance index (Materials and methods, Fig 4B, S3 Data). Notably, the group of eight sepsis isolates showed the highest multi-drug resistance index among all strains. A LASSO analysis identified six transcriptional regulators that combinatorically explain more than 80% of the variation (R2 = 0.85) in the multi-drug resistance index (Fig 4C, S7 Table). Strikingly, NalD arose again as a strong contributor to this signature. To evaluate the contribution of NalD to the resistance to drugs other than aztreonam, we removed aztreonam from the multi-drug resistance index and re-ran LASSO regression (S4 Fig). NalD remained an important contributor to this multi-drug resistance signature, suggesting that mutation in this transcription factor is relevant for general resistance, not just to aztreonam. These results agree with the broad substrate specificity of the MexAB-OprM efflux system [26] and with a previous finding that aztreonam selection can result in collateral resistance to antibiotics including tobramycin, colistin and ciprofloxacin [13]. In addition, a transposon mutant in another regulator identified by our LASSO analysis but with negative coefficient, GlmR, was hypersusceptible to a range of antibiotics in P. aeruginosa strain PAO1 [27], suggesting its important role in the development of multi-drug resistance. NalD structure indicates mechanism of efflux upregulation To investigate how the NalDF198L mutation alters NalD protein function, we studied a high-resolution crystal structure of NalD protein from P. aeruginosa PAO1 (PDB id: 5daj) [28], which has the same sequence as the NalD of D+7bld except for the mutation identified in this study. Structural analysis showed that the residue F198 lies in the 10th alpha helix, which locates in the interface of the NalD dimer (Fig 5A). This residue is close to two other residues in tertiary structure: 205W and 89Y (Fig 5B). All of these three residues have aromatic rings and the interaction between them could be strong, such as pi-stacking, and stabilize the 10th alpha helix facing the dimerization interface of NalD. Changing the residue 198 from F into L (Fig 5C) likely impairs these aromatic interactions and destabilizes the 10th alpha helix, impacting dimerization and further de-repressing mexAB-oprM (Fig 5D–5F). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Molecular details of mechanism of acquired aztreonam resistance in the sepsis patient infected with P. aeruginosa. (A) 3D structure of NalD dimer (PDB id: 5daj) with residue 198F (phenylalanine) shown as stick model. One copy of NalD is rainbow colored, while the other is in gray. The 10th helix is shown in red. (B) A closer look of residue 198F and possible interaction with two nearby aromatic residues, 89Y (tyrosin) and 205W (tryptophan). Those three residues are close together and could have aromatic interactions or a possible hydrogen bond (3.2 Å) between 198F and 89Y. 198F also aligns well with 205W, both of which have ring structure and could form a displaced pi stacking that stabilizes the NalD structure (3). (C) Prediction of mutation effect based on NalD structure. The mutation F198L would widen the distance between carbon groups and lose the pi-pi interaction, which could ultimately destabilize NalD dimerization. (D) Wild-type NalD dimer represses transcription of mexAB-oprM operon. (E) The mutation NalDF198L could interfere with dimerization, and de-repress transcription. (F) MexAB and OprM form an anti-porter system that exports aztreonam, increasing resistance [29]. https://doi.org/10.1371/journal.pcbi.1007562.g005 To validate this model, we deleted the 10th alpha helix of NalD in PA14 without shifting its reading frame (Materials and methods, S4 Table). The deletion increased aztreonam resistance of PA14 to the same level of the PA14 carrying NalDF198L and the sepsis isolate D+7bld (Fig 3D). Therefore, the mutation NalDF198L could indeed have conferred aztreonam resistance by loss of function and release of mexAB-oprM expression. Acquisition of aztreonam resistance shows no fitness cost The D+7bld isolates acquired aztreonam resistance through a point mutation in NalD, which possibly derepressed the expression of an efflux system. Would this mutation carry a fitness cost in the absence of the antibiotic? To answer this question, we first cultured D+7bld and D+5bld individually in vitro without aztreonam, where they showed the same growth rate (S2 Fig). We then asked if the D+7bld would be outcompeted by D+5bld when cultured together. In a competition experiment, we initially mixed D+7bld:D+5bld (1:1000) in a liquid media without aztreonam (S1 Text); we observed no change in that initial frequency, which confirmed that there is no fitness cost in the absence of the antibiotic (S3 Fig). By contrast, in the presence of aztreonam the NalD mutation confers a huge competitive advantage: when 2μg/mL or 4μg/mL aztreonam was added to the mixed population, the frequency of D+7bld increased ~10 fold and >200 fold respectively (S3 Fig). These results suggest that the NalD mutation in the absence of aztreonam either did not have direct fitness cost or the cost has been compensated for by other mechanisms. Possible mechanisms included the secondary 10bp deletion in a dehydrogenase, a non-mutational mechanism that changed bacterial physiology globally or through changes in specific pathways. To understand how the NalD mutation conferred resistance without a fitness cost, we compared the transcriptome of the NalD-mutated D+7bld to the susceptible D-3rsw, which differed from D+7bld by only two mutations. During exponential growth without antibiotics, only four genes were significantly differentially-expressed between those two isolates after multiple hypothesis correction (absolute log2-fold change ≥0.5 and adjusted p-value≤0.05). Two of those genes were mexB and oprM (Fig 6A), the genes coding for the inner and outer membrane components of the efflux pump. We then analyzed the transcriptomes of both isolates in the presence of different concentrations of aztreonam. Hundreds of genes showed significantly differential expression, as expected from the stress of antibiotic exposure [30]. D+7bld had less differentially-expressed genes than D-3rsw (136 compared to 300) at the sub-lethal aztreonam concentration of 2μg/mL. When the concentration of aztreonam increased to 4μg/mL—a level lethal to D-3rsw but not to D+7bld—the differentially-expressed genes in D+7bld increased to 341, a level of response that is similar to D-3rsw at 2μg/mL aztreonam (Fig 6A). A closer examination of the mexAB-oprM operon showed that none of the operon genes changed their expressions in D-3rsw exposed to aztreonam (Fig 6B). The efflux system was, however, over-expressed in D+7bld for all antibiotic concentrations (Fig 6B). Our transcriptomic data support the canonical model whereby the mutation in NalD released the repression of the mexAB-oprM operon regardless whether aztreonam was added to the medium or not [15] (Fig 5E). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. RNA-seq analysis shows that the aztreonam-resistant isolate D+7bld up-regulates the mexAB-oprM efflux system and attenuates response to aztreonam stress. (A) We compared the transcriptomes to reference isolate D-3rsw at 0μg/mL of aztreonam and found hundreds of differentially expressed genes. (B) The up-regulation of the mexAB-oprM efflux system in D+7bld supported that the NalD mutation released the transcriptional repression of mexAB-oprM. (C) The transcriptome of D+7bld at 4μg/mL aztreonam resembled the transcriptome of the aztreonam sensitive D-3rsw at half that dose (2μg/mL), confirming that the aztreonam resistance allows the strain to sustain higher levels of antibiotic challenge. azt, aztreonam. *, p-value<0.05. https://doi.org/10.1371/journal.pcbi.1007562.g006 We clustered expression levels of genes that are differentially expressed in at least one condition relative to the reference (D-3rsw, no aztreonam). The transcriptome of D+7bld was not much different from the D-3rsw in the absence of aztreonam. The overall expression profiles compared between D+7bld at 4μg/mL and D-3rsw at 2μg/mL of aztreonam were indeed similar to each other (Fig 6C), suggesting that the overexpression of mexAB-oprM had a dampening effect on the response to the antibiotic. The profile of D+7bld at 2μg/mL lies in between the profile of no aztreonam and of 4μg/mL aztreonam, further supporting that aztreonam induces a dose-dependent cellular response. Integration of metabolic network model with transcriptomics data accurately predicts bacterial growth Among the differently transcribed genes identified above, the top 3 groups of 107 functionally annotated genes are “transport and metabolism” (73 genes), “energy production and conversion” (54 genes), and “metabolism” (43 genes) (Fig 6C, S4 Data), suggesting a link between bacterial metabolism and aztreonam resistance. Metabolic fluxes can be impacted by the transcription of metabolic genes [31]. Therefore we sought to infer flux changes on the basis of gene expression changes using a computational model of the metabolic network. To study our sepsis isolates, we used a high-quality genome-scale model of Pseudomonas metabolism, iJN1411 [32], which contains 1411 gene products and 2826 reactions. By combining the RNA-seq data with the model, we aimed to explore how differential gene expression redistributed the metabolic fluxes and pathway usages. The method for integrating transcriptomic data with the iJN1411 model involves two major steps (Materials and methods): (1) building a reference model for D-3rsw at 0μg/mL aztreonam using transcriptomics data in the reference condition, and (2) modifying the reference model to accommodate gene expression changes measured for D-3rsw at 2μg/mL and D+7bld at 0, 2, 4μg/mL aztreonam. Under the reference condition, we approximated the flux bounds of reactions in the iJN1411 model by the optimal flux distribution that is most consistent with the mRNA levels in that condition, thereby constraining the metabolic solution space (i.e., range of feasible steady-state fluxes) to represent the actual metabolic behavior implied by data. To build metabolic models in other conditions, we incorporated the transcriptional differences between these conditions and the reference condition by multiplying the reconstructed flux bounds of each reaction in the reference model by expression fold-change values of corresponding genes associated with each reaction. The resulting 5 metabolic models were validated by comparing the growth rates measured experimentally at various aztreonam concentrations (Fig 7A) to model predictions (Fig 7B). Simulations using flux sampling showed that the distributions of biomass flux (i.e., flux through biomass production reaction) between D-3rsw and D+7bld in the absence of aztreonam overlapped, suggesting that the growth capacity of the resistant strain is likely uncompromised by the development of aztreonam resistance. This is consistent with our finding above that the sensitive strain did not outcompete the resistant strain in vitro. However, their biomass flux distributions with aztreonam present were truncated and heavily skewed to the left, indicating that the transcriptomic responses to aztreonam heavily restrict their growth rates. Using biomass as a proxy of bacterial growth, we showed that the ratios of predicted mean biomass flux (Fig 7C, red bars) agree qualitatively with the experimentally measured growth rates (Fig 7C, blue bars). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 7. Validation of metabolic model using experimental growth data. (A) Experimental growth curves of both sensitive (D-3rsw) and resistant (D+7bld) P. aeruginosa strains at various aztreonam concentrations (μg/mL). (B) Steady state distribution of biomass flux predicted from metabolic models for the same experimental conditions (except for D-3rsw at 4μg/mL aztreonam). PDF: probability density function. (C) Comparison of the measured growth rate (blue bars) with the model predictions (red bars). The measured growth rates were obtained by fitting an exponential growth model to the exponential phase of the growth curves shown in (A). The predicted growth rates were approximated from the mean of the biomass flux distributions shown in (B). The growth rates are relative to that of the sensitive strain in the absence of aztreonam (S0). Error bars: standard deviation. https://doi.org/10.1371/journal.pcbi.1007562.g007 Modeling-based analysis reveals metabolic adaptations in the resistant strain Using the validated models, we first assessed the metabolic flux changes across the conditions of different strains and aztreonam concentrations (Fig 8A). For each condition, we calculated the flux through each reaction as the median of its distribution obtained by uniformly sampling the corresponding solution space 100,000 times. This is different from a typical flux balance analysis which optimizes a presumed objective function. We chose this method because biological organisms operate under multiple competing objectives related to fitness (e.g., maximal growth, fast adaptive response) [33]. Antibiotic challenge may introduce new objectives, making any single objective function inappropriate to describe the metabolic goal of bacterial cells. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 8. Evolved metabolic-level adaptations in the aztreonam-resistant strain. (A) Metabolic flux changes relative to the reference condition S0 (D-3rsw, no aztreonam). (B) Venn diagram showing the overlap between reactions whose flux levels significantly altered by aztreonam alone, mutations alone, and their combination. Among all 48 reactions constitutively modulated by mutations (i.e., constitutive mutation effects), 30 are directly related to aztreonam resistance (because aztreonam can induce their responses) and 18 are indirectly related. (C) Grouping of the 48 constitutively modulated reactions by pathways they belong to. (D) Expression of eda serves as a bottleneck to the flux through the Entner-Doudoroff (ED) pathway (red arrows). The connected Embden-Meyerhof-Parnas (EMP) pathway (green arrows) and pentose phosphate (PP) pathway (blue arrows), as well as the relative expression changes of the major genes in the three pathways (heatmap) are also shown. Tpi: triose phosphate isomerase; fba: fructose-1,6-biphosphate aldolase; fbp: fructose-1,6-biphosphatase; pgi: glucose-6-phosphaate isomerase; zwf: glucose-6-phosphate dehydrogenase; pgl: 6-phosphogluconolactonase; edd: phosphogluconate dehydratase; eda: 2-dehydro-3-deoxy-phosphogluconate aldolase; DHAP: dihydroxyacetone phosphate: FDP: D-fructose-1,6-biphosphate; F6P: fructose-6-phosphate; G6P: glucose-6-phosphate; PGL: 6-phospho-D-glucono-1,5-lactone; 6PG: 6-phospho-D-gluconate; KPDG: 2-dehydro-3-deoxy-6-phospho-D-gluconate; G3P: glyceraldehyde 3-phosphate; PYR: pyruvate. https://doi.org/10.1371/journal.pcbi.1007562.g008 Metabolic flux changes shown above can be induced by either acquiring mutations, adding aztreonam to the media, or combination of both. In the absence of aztreonam, the resistant strain only displayed 12% change of metabolic fluxes relative to the sensitive strain (absolute flux value >10−3, absolute log2-fold change ≥0.5 and adjusted p-value <0.05, Materials and methods). Adding aztreonam induced a system-wide flux rearrangement for both strains: over 50% of all 844 reactions with active fluxes were significantly up- or down-regulated. Our result thus suggests a much weaker metabolic effects caused by mutations compared to aztreonam, which agrees with transcriptomic data and could explain the lack of fitness cost of D+7bld observed in experiments. A Venn diagram (Fig 8B) illustrates the overlaps of reactions whose flux levels were significantly changed by mutations alone (flux changes between the sensitive and resistant strain in the absence of aztreonam), by aztreonam alone (flux changes in the sensitive strain between w/ and w/o aztreonam), as well as by their combination (flux changes between the resistant strain with aztreonam and the sensitive strain without aztreonam). We found 403 reactions affected by both factors (i.e., the combination effects), among which 335 can be perturbed by aztreonam as the sole factor, indicating again that aztreonam causes the majority of flux changes when both factors are present. The Venn diagram also reveals how the resistant strain rewired metabolic fluxes as secondary effects of the NalD mutation beyond its primary function that releases MexAB-OprM efflux pump. There are 48 reactions in total (30 constitutive mutation effects, and 18 aztreonam resistance effects) displaying significant flux changes between the resistant and sensitive strain regardless of the presence and concentration of aztreonam (Fig 8B, S5 Data), which indicates that those 48 reactions are not directly related to aztreonam triggered growth defects as the rest reactions do. These constitutive metabolic adaptations include the secondary mutation effects that may or may not be related to aztreonam resistance. 30 reactions that are also affected by aztreonam in the sensitive strain likely provide the mechanisms that enable P. aeruginosa to resist the action of aztreonam. They were all downregulations and found in amino acids, lipid, carbohydrate metabolism as well as membrane transport system (Fig 8C). This finding is consistent with a previous study showing that aztreonam perturbed the metabolite levels in the same pathways as in another Gram-negative, nosocomial pathogen Acinetobacter baumannii [34]. The major identified reactions involved in amino acid metabolism are related to branched-chain (BCAA: leucine, isoleucine and valine) and aromatic amino acids (AAA: phenylalanine, tyrosine). Additionally, 4 out of 6 transport reactions are associated with uptake of valine and phenylalanine, further linking transport and utilization of BCAA and AAA to aztreonam resistance. The other 18 reactions that are not affected by aztreonam in sensitive strain may suggest mechanisms that do not contribute to the mechanism of resistance but compensate for its associated fitness costs. They were all downregulated reactions as well, among which we found two reactions (mediated by EDA and EDD) from the Entner-Doudoroff (ED) pathway (Fig 8D, red arrows) in carbohydrate metabolism. By examining the transcriptional level of enzymes in the central carbon metabolism, we determined that it is eda, a gene encoding KPDG (2-dehydro-3-deoxy-phosphogluconate) aldolase, but not any other enzyme-coding genes, that acts as the bottleneck to the pathway flux in the resistance strain because its expression was constitutively downregulated by mutations regardless of aztreonam (Fig 8D). Aztreonam resistance is associated with NalD mutation in acutely-infected patients To identify genomic features related to aztreonam resistance in patients acutely infected with P. aeruginosa, we started by measuring the minimum inhibitory concentration (MIC) of aztreonam in 31 P. aeruginosa isolates from cancer patients that we had previously sequenced [14]. Plotting the MIC levels next to a phylogenetic tree constructed from the core genome of the 31 isolates showed no discernable association between aztreonam MIC and phylogeny (Fig 1A). It is possible that the large genome-scale differences among the clinical isolates obscured the relationship between causal genetic variants and the desired phenotype. Therefore, we narrowed down the analysis to a smaller set of genes by focusing on the 19 genes where mutations emerged recurrently in vitro under aztreonam selection [13]. We tested the association between aztreonam MIC and the variation in the protein sequence coded by each of the 19 genes using a rank sum test (S1 Table). To our surprise, only one—NalD—passed the significance test (p = 0.0046), and associated with a >2-fold increase in average MIC. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Aztreonam resistance is associated with variation in NalD across independent clinical isolates from acute P. aeruginosa infection. (A) Phylogenetic tree of isolates from acute infections of cancer patients reconstructed from core genes, including the type strains PA14, PAO1 and PA7. The minimal inhibitory concertation (MIC) of the aztreonam varies significantly across the phylogenetic tree, showing it is not a phylogenetically conserved trait. (B) NalD protein is the only protein in the mexAB-oprM efflux pathway that is strongly associated with aztreonam MIC in a rank sum test (***, p = 0.005). The table on the bottom shows the p-values for rank sum tests conducted on other proteins known from the mexAB-oprM efflux system and its regulatory pathway. (C) Expanding the analysis to all the transcriptional regulators encoded by the P. aeruginosa genome revealed 30 candidates whose protein sequences variation were associated with aztreonam MIC (see S2 Table), but NalD remained the strongest correlate. https://doi.org/10.1371/journal.pcbi.1007562.g001 NalD is a transcriptional repressor of the efflux system MexAB-OprM and mutations in NalD have been linked to multi-drug resistance, including to aztreonam [15,16]. However, the other two regulators of MexAB-OprM mutated in experimental evolution [13], NalC and MexR, were not significantly associated with aztreonam MIC in our clinical isolates (both with p>0.5). We then tested all the known proteins in the MexAB-OprM pathway [15,17] including the efflux pump coding proteins themselves (Fig 1B). Again, of the 7 proteins only NalD passed the association test. To confirm the association further, we downloaded NalD sequences of 126 P. aeruginosa isolates which had published aztreonam MIC values from the PATRIC database [18]. This collection, which has isolates from many sources including acute and chronic infections, showed again that NalD is significantly associated with aztreonam resistance (p<0.01). This robust association suggested that mutations in NalD are main drivers of parallel evolution of aztreonam resistance in multiple lineages of P. aeruginosa. We compared the NalD protein sequences among our 31 clinical isolates including three type strains of P. aeruginosa (PA14, PAO1 and PA7). These NalD sequences are highly conserved, with only a few variations from the consensus (S1 Fig). Nonetheless, the strains that do vary from the consensus tend to rank high in terms of aztreonam MIC. One of the most resistant isolates, X9820, carries a copy of NalD with a deletion of residues 1~134 (>60% of the full length NalD) which plausibly causes loss of NalD function. Three other strains tested (W70322, W60856 and the type strain PA7) have mutations also in the 10th alpha helix of the protein’s structure, and two strains (H27930, F23197) carry point mutation close to the C-terminus. Four strains (M55212, F30658, W91452, W25637) have mutation T11N located in the first residue of the first alpha helix, which likely impairs the DNA binding function of NalD. Still, NalD variation alone explains only part of the aztreonam MIC. For example, two isolates that have the top aztreonam MIC, T38079 and T6313, have the same NalD sequences as the consensus (S1 Fig). Could variation in other transcriptional regulators explain aztreonam MIC? To address this question we comprehensively examined all annotated transcriptional regulators (>300) in the P. aeruginosa genome [18]. Thirty-one of these regulators were significantly associated with aztreonam resistance according to the rank sum test, but NalD still topped the list (Fig 1C). Evolution of P. aeruginosa aztreonam resistance within a patient To detail the drastic effects of aztreonam resistance in vivo we analyzed isolates obtained from a patient who died with an aztreonam resistant P. aeruginosa. The patient had been diagnosed with pre B cell acute lymphoblastic leukemia and was admitted (day -10 relative to day of transplantation) to Memorial Hospital for hematopoietic cell transplantation after undergoing first chemo remission (Fig 2A). As standard of care, the intense conditioning regimen compromises the patient’s immunity and can lead to life-threatening complications [19,20]. Therefore routine antibiotic prophylaxis with vancomycin, ciprofloxacin and pip-tazobactam was administered. This particular patient developed tachycardia on the day of stem cell infusion (day 0), followed by fever one day later (day +1). Blood cultures were drawn and cefepime and imipenem were administered to treat a plausible bacterial infection. On day +4 the patient worsened and developed sepsis, requiring transfer to the intensive care unit (ICU). On day +5, antimicrobials were changed to meropenem, amikacin and polymyxin. Blood cultures at this time tested positive for P. aeruginosa with resistance to multiple antipseudomonal agents but sensitive to aztreonam (Fig 2B). On day +6 the patient received aztreonam in addition to meropenem and avibactam as a last resort attempt to control the worsening infection. The patient’s clinical condition deteriorated and the patient eventually expired from sepsis on day +8. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. P. aeruginosa infection of a cancer patient hospitalized to receive hematopoietic cell transplantation evolved resistance to aztreonam in the course of therapy. (A) Timeline shows clinical events: conditioning regimen (myeloablation), hematopoietic cell infusion, location in the hospital (bone marrow transplantation unit [BMT] or intensive care unit [ICU]), the period of neutropenia, antibiotics administered, the day (relative to the day of transplant, day 0) and body site of origin (rsw: rectal swab; spt: sputum; bld: blood) of the eight P. aeruginosa isolates analyzed here. (B) Antibiotic resistance profiles of blood isolates measured by the clinical microbiology laboratory as the infection progressed; the profiles informed clinicians that the infection was multi-drug-resistant but also that it was initially sensitive to aztreonam (isolates D+3bld, D+4bld and D+5bld). As the disease progressed to become life-threatening sepsis, the patient was transferred to the ICU and was given aztreonam; however, the isolate D+7bld demonstrated resistance to aztreonam. The patient died on day +8. https://doi.org/10.1371/journal.pcbi.1007562.g002 To better understand the progression of aztreonam resistance, we tracked the origin of the P. aeruginosa infection by retrospectively culturing the initial rectal swab (day -10) and additional swabs taken at days -3 and +4, as well as a sputum sample from day +7. All samples produced P. aeruginosa colonies (Fig 2A). In total, we obtained eight P. aeruginosa isolates from this patient. We named those isolates by the number of days before (D-) or after (D+) the transplantation followed by body sites where they were isolated (Fig 2A). Importantly, the detection of P. aeruginosa on the day -10 rectal swab—obtained at the time of admission to the hospital—indicated that the patient had carried P. aeruginosa asymptomatically in the gut when entering the hospital. Of note, the patient’s pre-transplant care was delivered in another country and no prior rectal swab samples were available to determine the duration of carriage. We sequenced the whole genomes of eight aforementioned P. aeruginosa isolates (hereafter called sepsis isolates as a group). To track whether the infection was originated from the patient or acquired from the hospital, we constructed the phylogenetic tree with the sepsis isolates and isolates from other cancer patients in the same hospital analyzed earlier [14], as well as the three type strains PA14, PAO1 and PA7 (Fig 3A). The tree revealed that the eight sepsis isolates belong to the PAO1 clade and are much more similar to each other than to any other isolates obtained earlier from the same hospital. This supports the notion that the infection progressed from a single clone that the patient harbored at the time of admission and was not acquired after admission to Memorial Hospital. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Whole-genome sequencing of eight P. aeruginosa isolates pinpointed the mutation NalDF198L responsible for aztreonam resistance detected one day before the patient died. (A) A genome-based phylogenetic tree shows that the eight sepsis isolates are highly related to each other compared to other clinical isolates and type strains. (B) Genomic analysis revealed that the aztreonam-resistant isolate (D+7bld) had only two unique variations compared to aztreonam-sensitive isolates. Vertical dashed lines highlight the common presence of a given variation across multiple isolates. (C) Aztreonam MIC confirmed the clinical laboratory results for the eight sepsis isolates. (D) Experimental validation in PA14 showed that mutation in NalD but not in peg.4653 (PA14_50010) confers aztreonam resistance. “-”and “+” denote absence or presence of mutation found in D+7bld; “Δ” denotes deletion of the 10th alpha helix in NalD protein. (**, p-value<0.01). https://doi.org/10.1371/journal.pcbi.1007562.g003 A genome alignment analysis revealed that the eight genomes are remarkably similar to each other (Fig 3B) with only a total of 12 unique allelic differences among them. We confirmed SNPs and small gaps using targeted (Sanger) sequencing (Materials and methods, S3 Table). Isolate D+4rsw is the most phylogenetically distinct strain among the sepsis isolates. Isolates D+5bld and D-3rsw are identical to each other and harbor only 4 variations from the ancestral alleles, which we inferred by using PAO1 as a reference. Among those variations are two discrete insertions greater than 10kb. A BLAST search in NCBI linked one to a transposon insertion (S6 Table) and another to a duplication of a region of its own genome encoding an unclear pathway. There is also a deletion homologous to a phage insertion (S6 Table). Notably, the 12 genetic differences found among the eight isolates showed no pattern of association with either the time or the body site of isolation (Fig 3B), indicating that the P. aeruginosa population had diverged during colonization with multiple sub-clones coexisting at the same time in a single patient. Similar patterns of within host diversification were reported in other host-associated bacteria [21]. Mutation in NalD conferred aztreonam resistance To understand better why many clinical isolates have mutations in NalD, we first confirmed the mechanism of aztreonam resistance in the sepsis isolates. We conducted detailed measurement of aztreonam MIC for the eight sepsis isolates (Fig 3C, S1 Data). The aztreonam-resistant D+7bld isolate displayed a higher MIC (between 8–12μg/mL) than the other 7 sepsis isolates (<8μg/mL) consistent with the clinical report (Fig 2). However, D+7bld was not the isolate with the highest MIC when compared to the expanded collection comprising isolates from other cancer patients and the type strains PAO1, PA14 and PA7 (Fig 3C). Two isolates, X9820 and T38079, had MICs higher than 12μg/mL, the highest concentration tested. PA7, a type strain known for its resistance to a broad spectrum of antibiotics [22], showed similar aztreonam MIC to the 7 sepsis isolates that were considered clinically susceptible to that drug. The two widely used laboratory strains, PAO1 and PA14, had very low MICs. The aztreonam-resistant sepsis isolate D+7bld has only two genetic variations from its most closely related isolates, D+5bld and D-3rsw. One of those two mutations is a 10bp deletion in a dehydrogenase of unclear function. To determine if this mutation alone could have increased P. aeruginosa resistance to aztreonam we introduced the same 10bp deletion in the corresponding dehydrogenase gene (PA14_50010, or peg.4653 in D+7bld) in the laboratory strain PA14 (Materials and methods, S4 Table). This mutation did not increase aztreonam MIC (Fig 3D, S2 Data). The other mutation was a point mutation F198L found in NalD, a mutation that has not been reported nor selected through in vitro experiments before [13]. To confirm that this mutation alone could have caused aztreonam resistance we engineered the same NalDF198L mutation into PA14 and the MIC increased 3-fold from 4μg/mL to 12μg/mL, a MIC similar to the MIC of the terminal sepsis isolate D+7bld (Fig 3D). This confirmed that the NalDF198L mutation alone is sufficient for the observed aztreonam resistance in D+7bld, and is consistent with our finding that NalD mutation can increase aztreonam MIC on average by >2-fold (Fig 1B). NalD variation linked to multi-drug resistance Our data suggests that NalD is a recurrent driver for aztreonam resistance in P. aeruginosa acute infection. NalD is not the only transcriptional regulator where mutations can drive antibiotic resistance. In cystic fibrosis patients treated with ciprofloxacin and azithromycin during chronic P. aeruginosa infection, mutations accumulate in transcriptional regulator NfxB, which negatively regulates another efflux pump, MexCD-OprJ [23]. Can mutations found in transcriptional regulators be used to predict the antibiotic resistance of a P. aeruginosa isolate? To address this question we posed two related but more specific questions: First, is there a way to predict aztreonam resistance from sequence variation in all transcription factors? Second, does NalD variation alone predict resistance to other antibiotics besides aztreonam? To answer the first question, we used a machine learning approach called LASSO (least absolute shrinkage and selection operator) [24]. We checked if this method could select transcriptional regulators based on their sequence variation to explain the aztreonam MIC data in our P. aeruginosa acute infection isolates. The LASSO produced a model where only two transcription factors (out of >200) explained more than 60% of variation in aztreonam MIC (R2 = 0.65, Fig 4A): NalD and PA14_37120, a probable LysR-type transcriptional factor. As expected, the coefficient for NalD was positive, implying that mutations in NalD tend to increase aztreonam MIC and therefore confer aztreonam resistance by >2x. By contrast, PA14_37120 had a negative coefficient, indicating that isolates that contain PA14_37120 different from the consensus have lower aztreonam MIC and, therefore, tend to be more sensitive to aztreonam. The negative relationship seems to be common for LysR-type proteins which are positive regulators of enzymes that degrade antibiotics [25], suggesting that this specific type of transcriptional factors could potentially be explored as a target to sensitize P. aeruginosa to aztreonam. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. NalD mutation contributes to a general signature of antibiotic resistance. (A) A signature of aztreonam resistance obtained with LASSO regression shows only two transcriptional regulators, including NalD, and explains >60% of the variation in aztreonam MIC. The coefficients have units of fold-change. (B) Antibiotic inhibitions zones were measured using the disk assay for 8 antibiotics from several classes. The inhibition zone indices (shows as normalized areas of inhibition disk) show that the 8 sepsis isolates are resistant to multiple antibiotics. A resistance index computed from combining the negative values of the inhibition zone indices shows that the 8 sepsis isolates rank higher in multi-drug resistant than any other isolate tested. (C) The signature of multi-antibiotic resistance has six transcriptional regulators, including NalD, and explains >80% of the variation in the multi-drug resistance index. The coefficients have units of integrated fold-change across all the 8 antibiotics. https://doi.org/10.1371/journal.pcbi.1007562.g004 To answer the second question, we measured the sensitivities of each P. aeruginosa acute infection isolate to a panel of eight antibiotics from several classes (ciprofloxacin, gentamicin, aztreonam, chloramphenicol, ampicillin, tetracycline, meropenem, cefepime) and quantified the degree of multi-drug resistance by combining those sensitivity values into a multi-drug resistance index (Materials and methods, Fig 4B, S3 Data). Notably, the group of eight sepsis isolates showed the highest multi-drug resistance index among all strains. A LASSO analysis identified six transcriptional regulators that combinatorically explain more than 80% of the variation (R2 = 0.85) in the multi-drug resistance index (Fig 4C, S7 Table). Strikingly, NalD arose again as a strong contributor to this signature. To evaluate the contribution of NalD to the resistance to drugs other than aztreonam, we removed aztreonam from the multi-drug resistance index and re-ran LASSO regression (S4 Fig). NalD remained an important contributor to this multi-drug resistance signature, suggesting that mutation in this transcription factor is relevant for general resistance, not just to aztreonam. These results agree with the broad substrate specificity of the MexAB-OprM efflux system [26] and with a previous finding that aztreonam selection can result in collateral resistance to antibiotics including tobramycin, colistin and ciprofloxacin [13]. In addition, a transposon mutant in another regulator identified by our LASSO analysis but with negative coefficient, GlmR, was hypersusceptible to a range of antibiotics in P. aeruginosa strain PAO1 [27], suggesting its important role in the development of multi-drug resistance. NalD structure indicates mechanism of efflux upregulation To investigate how the NalDF198L mutation alters NalD protein function, we studied a high-resolution crystal structure of NalD protein from P. aeruginosa PAO1 (PDB id: 5daj) [28], which has the same sequence as the NalD of D+7bld except for the mutation identified in this study. Structural analysis showed that the residue F198 lies in the 10th alpha helix, which locates in the interface of the NalD dimer (Fig 5A). This residue is close to two other residues in tertiary structure: 205W and 89Y (Fig 5B). All of these three residues have aromatic rings and the interaction between them could be strong, such as pi-stacking, and stabilize the 10th alpha helix facing the dimerization interface of NalD. Changing the residue 198 from F into L (Fig 5C) likely impairs these aromatic interactions and destabilizes the 10th alpha helix, impacting dimerization and further de-repressing mexAB-oprM (Fig 5D–5F). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Molecular details of mechanism of acquired aztreonam resistance in the sepsis patient infected with P. aeruginosa. (A) 3D structure of NalD dimer (PDB id: 5daj) with residue 198F (phenylalanine) shown as stick model. One copy of NalD is rainbow colored, while the other is in gray. The 10th helix is shown in red. (B) A closer look of residue 198F and possible interaction with two nearby aromatic residues, 89Y (tyrosin) and 205W (tryptophan). Those three residues are close together and could have aromatic interactions or a possible hydrogen bond (3.2 Å) between 198F and 89Y. 198F also aligns well with 205W, both of which have ring structure and could form a displaced pi stacking that stabilizes the NalD structure (3). (C) Prediction of mutation effect based on NalD structure. The mutation F198L would widen the distance between carbon groups and lose the pi-pi interaction, which could ultimately destabilize NalD dimerization. (D) Wild-type NalD dimer represses transcription of mexAB-oprM operon. (E) The mutation NalDF198L could interfere with dimerization, and de-repress transcription. (F) MexAB and OprM form an anti-porter system that exports aztreonam, increasing resistance [29]. https://doi.org/10.1371/journal.pcbi.1007562.g005 To validate this model, we deleted the 10th alpha helix of NalD in PA14 without shifting its reading frame (Materials and methods, S4 Table). The deletion increased aztreonam resistance of PA14 to the same level of the PA14 carrying NalDF198L and the sepsis isolate D+7bld (Fig 3D). Therefore, the mutation NalDF198L could indeed have conferred aztreonam resistance by loss of function and release of mexAB-oprM expression. Acquisition of aztreonam resistance shows no fitness cost The D+7bld isolates acquired aztreonam resistance through a point mutation in NalD, which possibly derepressed the expression of an efflux system. Would this mutation carry a fitness cost in the absence of the antibiotic? To answer this question, we first cultured D+7bld and D+5bld individually in vitro without aztreonam, where they showed the same growth rate (S2 Fig). We then asked if the D+7bld would be outcompeted by D+5bld when cultured together. In a competition experiment, we initially mixed D+7bld:D+5bld (1:1000) in a liquid media without aztreonam (S1 Text); we observed no change in that initial frequency, which confirmed that there is no fitness cost in the absence of the antibiotic (S3 Fig). By contrast, in the presence of aztreonam the NalD mutation confers a huge competitive advantage: when 2μg/mL or 4μg/mL aztreonam was added to the mixed population, the frequency of D+7bld increased ~10 fold and >200 fold respectively (S3 Fig). These results suggest that the NalD mutation in the absence of aztreonam either did not have direct fitness cost or the cost has been compensated for by other mechanisms. Possible mechanisms included the secondary 10bp deletion in a dehydrogenase, a non-mutational mechanism that changed bacterial physiology globally or through changes in specific pathways. To understand how the NalD mutation conferred resistance without a fitness cost, we compared the transcriptome of the NalD-mutated D+7bld to the susceptible D-3rsw, which differed from D+7bld by only two mutations. During exponential growth without antibiotics, only four genes were significantly differentially-expressed between those two isolates after multiple hypothesis correction (absolute log2-fold change ≥0.5 and adjusted p-value≤0.05). Two of those genes were mexB and oprM (Fig 6A), the genes coding for the inner and outer membrane components of the efflux pump. We then analyzed the transcriptomes of both isolates in the presence of different concentrations of aztreonam. Hundreds of genes showed significantly differential expression, as expected from the stress of antibiotic exposure [30]. D+7bld had less differentially-expressed genes than D-3rsw (136 compared to 300) at the sub-lethal aztreonam concentration of 2μg/mL. When the concentration of aztreonam increased to 4μg/mL—a level lethal to D-3rsw but not to D+7bld—the differentially-expressed genes in D+7bld increased to 341, a level of response that is similar to D-3rsw at 2μg/mL aztreonam (Fig 6A). A closer examination of the mexAB-oprM operon showed that none of the operon genes changed their expressions in D-3rsw exposed to aztreonam (Fig 6B). The efflux system was, however, over-expressed in D+7bld for all antibiotic concentrations (Fig 6B). Our transcriptomic data support the canonical model whereby the mutation in NalD released the repression of the mexAB-oprM operon regardless whether aztreonam was added to the medium or not [15] (Fig 5E). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. RNA-seq analysis shows that the aztreonam-resistant isolate D+7bld up-regulates the mexAB-oprM efflux system and attenuates response to aztreonam stress. (A) We compared the transcriptomes to reference isolate D-3rsw at 0μg/mL of aztreonam and found hundreds of differentially expressed genes. (B) The up-regulation of the mexAB-oprM efflux system in D+7bld supported that the NalD mutation released the transcriptional repression of mexAB-oprM. (C) The transcriptome of D+7bld at 4μg/mL aztreonam resembled the transcriptome of the aztreonam sensitive D-3rsw at half that dose (2μg/mL), confirming that the aztreonam resistance allows the strain to sustain higher levels of antibiotic challenge. azt, aztreonam. *, p-value<0.05. https://doi.org/10.1371/journal.pcbi.1007562.g006 We clustered expression levels of genes that are differentially expressed in at least one condition relative to the reference (D-3rsw, no aztreonam). The transcriptome of D+7bld was not much different from the D-3rsw in the absence of aztreonam. The overall expression profiles compared between D+7bld at 4μg/mL and D-3rsw at 2μg/mL of aztreonam were indeed similar to each other (Fig 6C), suggesting that the overexpression of mexAB-oprM had a dampening effect on the response to the antibiotic. The profile of D+7bld at 2μg/mL lies in between the profile of no aztreonam and of 4μg/mL aztreonam, further supporting that aztreonam induces a dose-dependent cellular response. Integration of metabolic network model with transcriptomics data accurately predicts bacterial growth Among the differently transcribed genes identified above, the top 3 groups of 107 functionally annotated genes are “transport and metabolism” (73 genes), “energy production and conversion” (54 genes), and “metabolism” (43 genes) (Fig 6C, S4 Data), suggesting a link between bacterial metabolism and aztreonam resistance. Metabolic fluxes can be impacted by the transcription of metabolic genes [31]. Therefore we sought to infer flux changes on the basis of gene expression changes using a computational model of the metabolic network. To study our sepsis isolates, we used a high-quality genome-scale model of Pseudomonas metabolism, iJN1411 [32], which contains 1411 gene products and 2826 reactions. By combining the RNA-seq data with the model, we aimed to explore how differential gene expression redistributed the metabolic fluxes and pathway usages. The method for integrating transcriptomic data with the iJN1411 model involves two major steps (Materials and methods): (1) building a reference model for D-3rsw at 0μg/mL aztreonam using transcriptomics data in the reference condition, and (2) modifying the reference model to accommodate gene expression changes measured for D-3rsw at 2μg/mL and D+7bld at 0, 2, 4μg/mL aztreonam. Under the reference condition, we approximated the flux bounds of reactions in the iJN1411 model by the optimal flux distribution that is most consistent with the mRNA levels in that condition, thereby constraining the metabolic solution space (i.e., range of feasible steady-state fluxes) to represent the actual metabolic behavior implied by data. To build metabolic models in other conditions, we incorporated the transcriptional differences between these conditions and the reference condition by multiplying the reconstructed flux bounds of each reaction in the reference model by expression fold-change values of corresponding genes associated with each reaction. The resulting 5 metabolic models were validated by comparing the growth rates measured experimentally at various aztreonam concentrations (Fig 7A) to model predictions (Fig 7B). Simulations using flux sampling showed that the distributions of biomass flux (i.e., flux through biomass production reaction) between D-3rsw and D+7bld in the absence of aztreonam overlapped, suggesting that the growth capacity of the resistant strain is likely uncompromised by the development of aztreonam resistance. This is consistent with our finding above that the sensitive strain did not outcompete the resistant strain in vitro. However, their biomass flux distributions with aztreonam present were truncated and heavily skewed to the left, indicating that the transcriptomic responses to aztreonam heavily restrict their growth rates. Using biomass as a proxy of bacterial growth, we showed that the ratios of predicted mean biomass flux (Fig 7C, red bars) agree qualitatively with the experimentally measured growth rates (Fig 7C, blue bars). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 7. Validation of metabolic model using experimental growth data. (A) Experimental growth curves of both sensitive (D-3rsw) and resistant (D+7bld) P. aeruginosa strains at various aztreonam concentrations (μg/mL). (B) Steady state distribution of biomass flux predicted from metabolic models for the same experimental conditions (except for D-3rsw at 4μg/mL aztreonam). PDF: probability density function. (C) Comparison of the measured growth rate (blue bars) with the model predictions (red bars). The measured growth rates were obtained by fitting an exponential growth model to the exponential phase of the growth curves shown in (A). The predicted growth rates were approximated from the mean of the biomass flux distributions shown in (B). The growth rates are relative to that of the sensitive strain in the absence of aztreonam (S0). Error bars: standard deviation. https://doi.org/10.1371/journal.pcbi.1007562.g007 Modeling-based analysis reveals metabolic adaptations in the resistant strain Using the validated models, we first assessed the metabolic flux changes across the conditions of different strains and aztreonam concentrations (Fig 8A). For each condition, we calculated the flux through each reaction as the median of its distribution obtained by uniformly sampling the corresponding solution space 100,000 times. This is different from a typical flux balance analysis which optimizes a presumed objective function. We chose this method because biological organisms operate under multiple competing objectives related to fitness (e.g., maximal growth, fast adaptive response) [33]. Antibiotic challenge may introduce new objectives, making any single objective function inappropriate to describe the metabolic goal of bacterial cells. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 8. Evolved metabolic-level adaptations in the aztreonam-resistant strain. (A) Metabolic flux changes relative to the reference condition S0 (D-3rsw, no aztreonam). (B) Venn diagram showing the overlap between reactions whose flux levels significantly altered by aztreonam alone, mutations alone, and their combination. Among all 48 reactions constitutively modulated by mutations (i.e., constitutive mutation effects), 30 are directly related to aztreonam resistance (because aztreonam can induce their responses) and 18 are indirectly related. (C) Grouping of the 48 constitutively modulated reactions by pathways they belong to. (D) Expression of eda serves as a bottleneck to the flux through the Entner-Doudoroff (ED) pathway (red arrows). The connected Embden-Meyerhof-Parnas (EMP) pathway (green arrows) and pentose phosphate (PP) pathway (blue arrows), as well as the relative expression changes of the major genes in the three pathways (heatmap) are also shown. Tpi: triose phosphate isomerase; fba: fructose-1,6-biphosphate aldolase; fbp: fructose-1,6-biphosphatase; pgi: glucose-6-phosphaate isomerase; zwf: glucose-6-phosphate dehydrogenase; pgl: 6-phosphogluconolactonase; edd: phosphogluconate dehydratase; eda: 2-dehydro-3-deoxy-phosphogluconate aldolase; DHAP: dihydroxyacetone phosphate: FDP: D-fructose-1,6-biphosphate; F6P: fructose-6-phosphate; G6P: glucose-6-phosphate; PGL: 6-phospho-D-glucono-1,5-lactone; 6PG: 6-phospho-D-gluconate; KPDG: 2-dehydro-3-deoxy-6-phospho-D-gluconate; G3P: glyceraldehyde 3-phosphate; PYR: pyruvate. https://doi.org/10.1371/journal.pcbi.1007562.g008 Metabolic flux changes shown above can be induced by either acquiring mutations, adding aztreonam to the media, or combination of both. In the absence of aztreonam, the resistant strain only displayed 12% change of metabolic fluxes relative to the sensitive strain (absolute flux value >10−3, absolute log2-fold change ≥0.5 and adjusted p-value <0.05, Materials and methods). Adding aztreonam induced a system-wide flux rearrangement for both strains: over 50% of all 844 reactions with active fluxes were significantly up- or down-regulated. Our result thus suggests a much weaker metabolic effects caused by mutations compared to aztreonam, which agrees with transcriptomic data and could explain the lack of fitness cost of D+7bld observed in experiments. A Venn diagram (Fig 8B) illustrates the overlaps of reactions whose flux levels were significantly changed by mutations alone (flux changes between the sensitive and resistant strain in the absence of aztreonam), by aztreonam alone (flux changes in the sensitive strain between w/ and w/o aztreonam), as well as by their combination (flux changes between the resistant strain with aztreonam and the sensitive strain without aztreonam). We found 403 reactions affected by both factors (i.e., the combination effects), among which 335 can be perturbed by aztreonam as the sole factor, indicating again that aztreonam causes the majority of flux changes when both factors are present. The Venn diagram also reveals how the resistant strain rewired metabolic fluxes as secondary effects of the NalD mutation beyond its primary function that releases MexAB-OprM efflux pump. There are 48 reactions in total (30 constitutive mutation effects, and 18 aztreonam resistance effects) displaying significant flux changes between the resistant and sensitive strain regardless of the presence and concentration of aztreonam (Fig 8B, S5 Data), which indicates that those 48 reactions are not directly related to aztreonam triggered growth defects as the rest reactions do. These constitutive metabolic adaptations include the secondary mutation effects that may or may not be related to aztreonam resistance. 30 reactions that are also affected by aztreonam in the sensitive strain likely provide the mechanisms that enable P. aeruginosa to resist the action of aztreonam. They were all downregulations and found in amino acids, lipid, carbohydrate metabolism as well as membrane transport system (Fig 8C). This finding is consistent with a previous study showing that aztreonam perturbed the metabolite levels in the same pathways as in another Gram-negative, nosocomial pathogen Acinetobacter baumannii [34]. The major identified reactions involved in amino acid metabolism are related to branched-chain (BCAA: leucine, isoleucine and valine) and aromatic amino acids (AAA: phenylalanine, tyrosine). Additionally, 4 out of 6 transport reactions are associated with uptake of valine and phenylalanine, further linking transport and utilization of BCAA and AAA to aztreonam resistance. The other 18 reactions that are not affected by aztreonam in sensitive strain may suggest mechanisms that do not contribute to the mechanism of resistance but compensate for its associated fitness costs. They were all downregulated reactions as well, among which we found two reactions (mediated by EDA and EDD) from the Entner-Doudoroff (ED) pathway (Fig 8D, red arrows) in carbohydrate metabolism. By examining the transcriptional level of enzymes in the central carbon metabolism, we determined that it is eda, a gene encoding KPDG (2-dehydro-3-deoxy-phosphogluconate) aldolase, but not any other enzyme-coding genes, that acts as the bottleneck to the pathway flux in the resistance strain because its expression was constitutively downregulated by mutations regardless of aztreonam (Fig 8D). Discussion P. aeruginosa is a major pathogen with a large genome, and extensive genomic variation among the strains in the same species. The high genomic diversity challenges our ability to predict clinically important phenotypes, particularly antibiotic resistance. Whole genome sequencing of P. aeruginosa isolates from cystic fibrosis patients had already revealed adaptations to the pressures experienced in the chronically-infected lung [6] and antibiotic therapy [23] but the adaptations to the pressures experienced in acute infection remained less clear. Acute infections may start when P. aeruginosa translocate from the environment, from another patient or, as we have seen here, from asymptomatic colonization in the patient’s own microbiome. Systems-level analyses can help us understand how these transitions shape P. aeruginosa physiology and impact its broad response to antibiotics. Here we used mathematical models to assist in the interpretation of antibiotic resistance from sequenced genomes. Application of these methods to predict antibiotic resistance in a clinical setting will likely require a better understanding of genetic function and gene interaction networks beyond our present knowledge. Evolution experiments conducted in the laboratory can help uncover some of these mechanisms, but such conditions are perhaps drastically different to those experienced in human infection [35]. Here we show that specific clinical cases can help bridge the gap between laboratory insights and clinical relevance. We identified that recurrent mutation in NalD is associated with resistance to aztreonam and to other antibiotics in patients acutely infected with P. aeruginosa. We dissected the case of a multi-drug resistant strain that escaped from the patient’s gut microbiota into their bloodstream, acquired the NalD mutation and ultimately killed the patient. The patient came to Memorial Sloan Kettering (located in the USA) from another country, where use of antibiotics without prescription is more common. Prior use of antibiotics likely explains the unusually high antibiotic resistance of all isolates obtained from this patient (Fig 4B). The translocation of the P. aeruginosa from an asymptomatic gut colonizer to the bloodstream agrees with previous studies showing that disruption of the commensal gut microbiome with antibiotics increases the chance of bloodstream infections by antibiotic resistant bacteria residing in the gut [36,37]. The molecular events we uncovered agree with the known mechanism where a NalD loss-of-function mutation releases mexAB-oprM expression and confers resistance. The wild type NalD responds to inducers such as novobiocin [28]. The strain D-3rsw, which carries the most common (wild type) NalD sequence, did not increase mexAB-oprM expression in 2μg/mL of aztreonam (Fig 6B), which suggests that NalD does not respond to aztreonam. It makes sense in light of P. aeruginosa evolutionary history that the wild type NalD is unadapted to respond to this antibiotic, which is a relatively recent synthetic drug [11–13] that was probably absent in the evolutionary history of P. aeruginosa. Previous experiments had shown that loss of function in transcriptional regulators offers a quick way for bacteria to adapt to such new challenges [38]. Our data shows that the NalD point mutation can occur in a patient and cause a rapid increase in drug resistance, even while a patient receives treatment. The rapid adaptation of nosocomial pathogens often results from mutations in transcriptional regulators [6,39–44]. This is perhaps expected: mutations in transcriptional regulators provide the most dramatic and rapid means to change bacterial physiology [38]. Their relationship to antibiotic resistance may be less well understood [45], especially in acute infections where disease progression and transmission can happen quickly, and the secondary effects of mutations are often obscured by the primary effects and making them barely detectable. However, secondary effects can be critical: they can reduce the fitness cost of resistance mutations and can even help provide collateral resistance to other antibiotics [29]. In this study, we investigated the secondary effects of the NalD mutation by integrating metabolic modeling and transcriptomics data. Over the last decade, metabolic network analysis that combines genome-scale metabolic models and omics data have been applied to study antibiotic resistance in bacteria and to suggest therapeutic targets [46–50]. Although the molecular (primary) function of the NalD mutation has been widely studied, our work adds to our limited understanding of its secondary effects. Our method indicated 48 reactions that may be constitutively downregulated in the aztreonam resistant strain. This is consistent with a general notion that drug resistance is associated with reduced, rather than enhanced, cell metabolism. We predicted that metabolic changes in the membrane transport and metabolism of BCAA and AAA are directly connected to the development of aztreonam resistance. Previous studies have suggested that the carbon catabolite control system CbrAB/Crc regulates BCAA uptake and utilization [51] as well as antibiotic resistance in P. aeruginosa [52–54]. Since channels that actively uptake amino acids can also transport antibiotics with sufficient structural similarity (e.g., Escherichia coli glycine transport system can also uptake the antibiotic D-cycloserine [55]), aztreonam resistance can be possibly potentiated by decreasing drug uptake through BCAA transporters via the CbrAB/Crc system, in addition to the efflux provided by the upregulated MexAB-OprM. Using a defined synthetic cystic fibrosis sputum medium, AAA were reported to induce biosynthesis of the Pseudomonas quinolone signal (PQS) [56], a quorum-sensing signaling molecule that regulates up to 12% of the P. aeruginosa genome [57]. A PQS mutant is more tolerant to ciprofloxacin than its wild-type [58], which supports that downregulating the AAA pathway may protect P. aeruginosa from aztreonam by reducing its PQS level. We also predicted metabolic changes in ED pathway as a potential compensatory mechanism that reduces costs associated with the NalD mutation. Normally, the ED pathway is alternative to glycolysis and catabolizes glucose to pyruvate. However, when growing on casamino acids, P. aeruginosa must operate through gluconeogenesis to produce several essential metabolite precursors such as fructose-6-phosphate (FBP) and glucose-6-phosphate (G6P) for biomass production. The gluconeogenic flux is funneled into the oxidative branch of the pentose phosphate pathway and the ED pathway, forming a cyclic loop (known as the EDEMP cycle [59] that starts and ends with pyruvate) (Fig 8D). The recycling of hexoses back to trioses through the ED pathway can provide two potential compensatory mechanisms. First, it provides a reservoir flux and its downregulations can redirect the flux towards desired pathways. For example, reduced flux through the ED pathway can compensate for decreased flux from F6P to pentose phosphate pathway and biosynthesis of peptidoglycan, where the latter is the direct target for aztreonam. However, this compensatory effect may be limited if the reservoir flux is small. From another angle, a small ED pathway flux can rapidly become depleted in adverse environmental conditions and thus possibly acts as a sensor to indicate the hardship of the environment that P. aeruginosa faces. The functioning of the environment sensor will require the cooperation from a flux-signaling metabolite, which translates the flux change to change in metabolite level and stimulates specific pathways to combat the hardship [60]. The potential distant regulatory role of ED pathway has been implicated in another human pathogen, Vibrio cholerae, where activation of the ED pathway leads to higher transcriptional levels of the prime virulence genes [61]. Our computational investigation of P. aeruginosa metabolism generates new hypothesis for future research but has noteworthy limitations. First, we used the metabolic model iJN1411 which was developed for P. putida, a species very close to P. aeruginosa. We chose this model because of its outstanding quality: the model has 2826 reactions constructed from 409 citations and 72% of the reactions are supported by at least one reference [26]. It is important to keep in mind that the non-conserved pathways between those two species may lead to different metabolic flux distribution. Nonetheless, we expect the effect of metabolism difference to be minor because about 80% of the 1411 genes in model iJN1411 were covered by our transcriptomics. Second, no matter how many times we sample the metabolic space in the reference model, it is always possible that the optimal solution of highest consistency with gene expression may not be unique and alternative solutions that are equally optimal can exist. Third, all methods that use transcriptomics for metabolic modeling have a major limitation: metabolic flux and transcriptomics are only loosely correlated. Metabolic fluxes depend not only on the mRNA levels of the enzyme that catalyzes each reaction, but also on many factors including post-transcriptional modulations and allosteric regulations [62,63]. In the future, these limitations could be overcomed by a high-quality P. aeruginosa-specific metabolic model, assisted by metabolic flux data to constrain solution space. Materials and methods Ethics statement According to the NIH guide for Human Subjects Research, this work is “exempt from the human subject’s regulations, category 4 (Exemption 4)” because it involves “only the use of secondary analysis of biological material/tissue/specimens or data not collected specifically for this study” and “the specimens or data previously collected are de-identified for the purpose of this study by someone involved in the research study. For example, your collaborator will provide you with aliquots of specimen that are no longer linked to the subject identifiers or you are extracting clinical data from medical records without retaining the subject name or medical record number.” Microbiological methods Bacterial culture, gene mutagenesis and genomic sequencing were performed as previously described [14] and more details are given in S1 Text. Primers are listed in S3 and S4 Tables. Other detailed experimental methods including antibiotic resistance assay, bioinformatics and transcriptomic assay are included in SI as well. Aztreonam susceptibility test In clinical lab, phenotypic antimicrobial susceptibility testing (AST) was performed by broth microdilution using the Gram-Negative MIC Panel type 43 on the MicroScan WalkAway system (Beckman Coulter) following overnight incubation and photometric determination of bacterial growth. AST results of aztreonam for P. aeruginosa were interpreted using the Clinical and Laboratory Standards Institute (CLSI) M100-S24 standards (MIC ug/ml ≤ 8 susceptible; 16 intermediate; ≥ 32 resistant). Association between NalD variation and aztreonam MIC The ranksum statistic test measures if strains with high variable NalD protein sequence would have higher MIC than the strains with NalD similar to consensus sequence. NalD protein sequences from P. aeruginosa isolates are aligned and consensus sequence is obtained using Matlab bioinformatics toolbox. Protein variation is calculated by comparing each NalD to the consensus protein sequence built from the collection. A median value of the sequence variation was used as a cutoff to group the strains into high various and low various group. The MIC values in each group were then compared using ranksum test. Overall there is no cutoff drawn for the MIC value. Structural analysis of NalD The 3D structure was obtained from protein data bank (PDB) [64] with ID 5daj [28]. The structure analysis was done in Pymol (The PyMOL Molecular Graphics System, Version 2.0 Schrödinger, LLC.) with educational-Use license. Statistical analysis and machine learning All analyses were carried out using Matlab™ with the Statistics and Machine Learning toolbox. Aztreonam MIC and standard errors of different strains were estimated using function fitlm as: Wilcoxon rank sum test was performed using function ranksum. Antibiotic disk assay data were clustered using seqlinkage function based on pairwise distance (pdist). Elastic net regularization (in lasso function) was used to select for transcriptional regulators to predict antibiotic resistance with cross validation (cvpartition) and later to calculate the coefficient of selected transcriptional regulators using fitlm. For RNAseq analysis, we use the DESeq2 package to call the differentially expressed genes by p-value adjusted with multiple hypotheses (Benjamini-Hochberg method). For analyzing antibiotic disk diffusion, the diameter (D) of cleared zone caused by each antibiotic was measured using imageJ. The inhibition zone index (I) for each antibiotic was calculated across the strains as: where S is the standard deviation of the data, X is the square of measured diameter D to represent the inhibition area. Antibiotic resistance index (R) for strain j was calculated as: Rj = −ΣIi where i refers to antibiotics used in disk diffusion assay. Integrative metabolic flux analysis The boundary fluxes of the iJN1411 model were set to mimic environmental conditions in the experiment: the uptake fluxes of all 20 amino acids except tryptophan were set to 1 (in any unit) so all intracellular reactions have normalized fluxes relative to nutrient uptake. To find the flux solution that has the highest consistency with gene expression data, we implemented the iMAT algorithm [65], which formulated a mixed integer linear programming (MILP) problem to maximize the total number of active reactions associated with highly expressed genes (denoted by RH) and inactive reactions associated with lowly expressed genes (denoted by RL) under a biomass constraint (1) s.t. (2) (3) (4) (5) (6) (7) S is the stoichiometric coefficient matrix of the iJN1411 model. v is a vector of metabolic flux, and vmin and vmax are their lower and upper bounds obtained by flux variability analysis. As suggested by [66], we used the top 25% and 75% gene expression thresholds to determine the set of lowly (<25% quantile) and highly (>75% quantile) expressed reactions. For reactions associated with multiple isozymes or one enzyme with multiple subunits, we determined their corresponding transcription levels by replacing “and” and “or” operators with “min” and “max” respectively in their gene-protein-reaction (GPR) rules. and are Boolean variables to indicate the flux activity of the reaction i in its forward and reverse direction respectively: highly expressed reactions are active if or and lowly expressed reactions are inactive if . We chose ∊ = 0.1, which is a positive threshold for flux activity of highly expressed reactions: active reactions carry fluxes with absolute values equal or above ∊. vbio and vmax,bio are the biomass flux and its maximum possible value respectively, and f is a parameter that tunes the rigidity of the biomass constraint. We determined f = 0.95 from a trade-off analysis (S5 Fig), which chose a large value of f where the objective function remains near-optimal but starts to have diminishing returns by increasing f further. The reference model for D-3rsw in the absence of aztreonam was constructed by constraining the reaction in the iJN1411 model using the iMAT solution (viMAT). For any reaction i, we imposed the following constraints on its flux bounds: 0 ≤ vi ≤ viMAT,i for viMAT,i > 0, −viMAT,i ≤ vi ≤ 0 for viMAT,i < 0, and vi = 0 for viMAT,i = 0. Metabolic models in other conditions were constructed by modifying the flux bounds of reactions in the reference model based on gene expression changes between these conditions and the reference condition, i.e., vmin,i → vmin,i · ci, vmax,i → vmax,i · ci, where ci is the fold change in mRNA levels of genes associated with reaction i. Custom Python codes were developed with the COBRApy package [67] to carry out all metabolic flux modeling and simulations in the paper. Flux variability analysis and flux sampling were performed using the built-in COBRApy function flux_variability_analysis and sample respectively. Ethics statement According to the NIH guide for Human Subjects Research, this work is “exempt from the human subject’s regulations, category 4 (Exemption 4)” because it involves “only the use of secondary analysis of biological material/tissue/specimens or data not collected specifically for this study” and “the specimens or data previously collected are de-identified for the purpose of this study by someone involved in the research study. For example, your collaborator will provide you with aliquots of specimen that are no longer linked to the subject identifiers or you are extracting clinical data from medical records without retaining the subject name or medical record number.” Microbiological methods Bacterial culture, gene mutagenesis and genomic sequencing were performed as previously described [14] and more details are given in S1 Text. Primers are listed in S3 and S4 Tables. Other detailed experimental methods including antibiotic resistance assay, bioinformatics and transcriptomic assay are included in SI as well. Aztreonam susceptibility test In clinical lab, phenotypic antimicrobial susceptibility testing (AST) was performed by broth microdilution using the Gram-Negative MIC Panel type 43 on the MicroScan WalkAway system (Beckman Coulter) following overnight incubation and photometric determination of bacterial growth. AST results of aztreonam for P. aeruginosa were interpreted using the Clinical and Laboratory Standards Institute (CLSI) M100-S24 standards (MIC ug/ml ≤ 8 susceptible; 16 intermediate; ≥ 32 resistant). Association between NalD variation and aztreonam MIC The ranksum statistic test measures if strains with high variable NalD protein sequence would have higher MIC than the strains with NalD similar to consensus sequence. NalD protein sequences from P. aeruginosa isolates are aligned and consensus sequence is obtained using Matlab bioinformatics toolbox. Protein variation is calculated by comparing each NalD to the consensus protein sequence built from the collection. A median value of the sequence variation was used as a cutoff to group the strains into high various and low various group. The MIC values in each group were then compared using ranksum test. Overall there is no cutoff drawn for the MIC value. Structural analysis of NalD The 3D structure was obtained from protein data bank (PDB) [64] with ID 5daj [28]. The structure analysis was done in Pymol (The PyMOL Molecular Graphics System, Version 2.0 Schrödinger, LLC.) with educational-Use license. Statistical analysis and machine learning All analyses were carried out using Matlab™ with the Statistics and Machine Learning toolbox. Aztreonam MIC and standard errors of different strains were estimated using function fitlm as: Wilcoxon rank sum test was performed using function ranksum. Antibiotic disk assay data were clustered using seqlinkage function based on pairwise distance (pdist). Elastic net regularization (in lasso function) was used to select for transcriptional regulators to predict antibiotic resistance with cross validation (cvpartition) and later to calculate the coefficient of selected transcriptional regulators using fitlm. For RNAseq analysis, we use the DESeq2 package to call the differentially expressed genes by p-value adjusted with multiple hypotheses (Benjamini-Hochberg method). For analyzing antibiotic disk diffusion, the diameter (D) of cleared zone caused by each antibiotic was measured using imageJ. The inhibition zone index (I) for each antibiotic was calculated across the strains as: where S is the standard deviation of the data, X is the square of measured diameter D to represent the inhibition area. Antibiotic resistance index (R) for strain j was calculated as: Rj = −ΣIi where i refers to antibiotics used in disk diffusion assay. Integrative metabolic flux analysis The boundary fluxes of the iJN1411 model were set to mimic environmental conditions in the experiment: the uptake fluxes of all 20 amino acids except tryptophan were set to 1 (in any unit) so all intracellular reactions have normalized fluxes relative to nutrient uptake. To find the flux solution that has the highest consistency with gene expression data, we implemented the iMAT algorithm [65], which formulated a mixed integer linear programming (MILP) problem to maximize the total number of active reactions associated with highly expressed genes (denoted by RH) and inactive reactions associated with lowly expressed genes (denoted by RL) under a biomass constraint (1) s.t. (2) (3) (4) (5) (6) (7) S is the stoichiometric coefficient matrix of the iJN1411 model. v is a vector of metabolic flux, and vmin and vmax are their lower and upper bounds obtained by flux variability analysis. As suggested by [66], we used the top 25% and 75% gene expression thresholds to determine the set of lowly (<25% quantile) and highly (>75% quantile) expressed reactions. For reactions associated with multiple isozymes or one enzyme with multiple subunits, we determined their corresponding transcription levels by replacing “and” and “or” operators with “min” and “max” respectively in their gene-protein-reaction (GPR) rules. and are Boolean variables to indicate the flux activity of the reaction i in its forward and reverse direction respectively: highly expressed reactions are active if or and lowly expressed reactions are inactive if . We chose ∊ = 0.1, which is a positive threshold for flux activity of highly expressed reactions: active reactions carry fluxes with absolute values equal or above ∊. vbio and vmax,bio are the biomass flux and its maximum possible value respectively, and f is a parameter that tunes the rigidity of the biomass constraint. We determined f = 0.95 from a trade-off analysis (S5 Fig), which chose a large value of f where the objective function remains near-optimal but starts to have diminishing returns by increasing f further. The reference model for D-3rsw in the absence of aztreonam was constructed by constraining the reaction in the iJN1411 model using the iMAT solution (viMAT). For any reaction i, we imposed the following constraints on its flux bounds: 0 ≤ vi ≤ viMAT,i for viMAT,i > 0, −viMAT,i ≤ vi ≤ 0 for viMAT,i < 0, and vi = 0 for viMAT,i = 0. Metabolic models in other conditions were constructed by modifying the flux bounds of reactions in the reference model based on gene expression changes between these conditions and the reference condition, i.e., vmin,i → vmin,i · ci, vmax,i → vmax,i · ci, where ci is the fold change in mRNA levels of genes associated with reaction i. Custom Python codes were developed with the COBRApy package [67] to carry out all metabolic flux modeling and simulations in the paper. Flux variability analysis and flux sampling were performed using the built-in COBRApy function flux_variability_analysis and sample respectively. Supporting information S1 Fig. Aztreonam resistance is associated with NalD mutation. (A) NalD protein alignments ordered by the value of minimum inhibitory concentration (MIC) of aztreonam obtained for the corresponding isolate. None of the previously collected clinical isolates has the same mutation as in D+7bld NalD (F198L). (B) The protein sequence of NalD is highly conserved. The strains resistant to aztreonam tend to have mutations in NalD compared to the strains susceptible to aztreonam. The one that is most resistant to aztreonam, X9820 has a deletion of 134 amino acid residues at the beginning of NalD. https://doi.org/10.1371/journal.pcbi.1007562.s001 (TIF) S2 Fig. Growth curve synchronization method for precise measurement of growth rate of sepsis isolates D+5bld and D+7bld. The first column shows high-resolution growth curve of serially diluted cell inoculum. The middle column shows aligned curves on the left side. The last column on the right shows the determination of growth rate by linear fitting of time shift against dilution. There is no measurable cost to fitness in vitro for the strain carrying the aztreonam resistance mutation. https://doi.org/10.1371/journal.pcbi.1007562.s002 (TIF) S3 Fig. D+7bld shows no fitness cost measured in direct competition with D+5bld in vitro in the absence of aztreonam. Overnight competition between D+7bld and D+5bld cells in a 1:1000 initial rate is highly impacted by aztreonam concentration. When aztreonam is absent, D+7bld frequency remains unchanged after competition with D+5bld cells. At the sublethal aztreonam concentration of 2μg/mL D+7bld frequency increases, as expected, by ~10 fold. At aztreonam concentration of 4μg/mL, which is above the MIC of D+5bld but not D+7bld, D+7bld frequency increases more than 300 fold. (** p<0.01). https://doi.org/10.1371/journal.pcbi.1007562.s003 (TIF) S4 Fig. Variation in NalD and other transcriptional regulators could explain multiple drug resistance excluding aztreonam. The 13 transcriptional regulators overall explains >90% of the variation in the multi-drug resistance index calculated from 7 antibiotics (excluding aztreonam from Fig 4B). The coefficients have units of summarized fold-change across all the 7 antibiotics. https://doi.org/10.1371/journal.pcbi.1007562.s004 (TIF) S5 Fig. Trade-off between maximizing consistency between metabolic flux and gene expression and maximizing biomass production in the iMAT algorithm. The consistency score in the y axis is equal to the value of the objective function, which is given by Eq (1) in the main text. The parameter f in the x axis imposes a biomass constraint that requires the ratio of biomass flux to its maximum possible value is at least f. f = 0.95 is a point of diminishing return that increasing the minimum biomass flux return will lead to dramatic drop in the consistency score. https://doi.org/10.1371/journal.pcbi.1007562.s005 (TIF) S1 Table. Association between aztreonam resistance and protein sequence variation of the 19 proteins that have recurrent mutations during experimental evolution [13] (Ranksum test, total strain# = 31). https://doi.org/10.1371/journal.pcbi.1007562.s006 (DOCX) S2 Table. Transcriptional regulators associated with aztreonam MIC. https://doi.org/10.1371/journal.pcbi.1007562.s007 (DOCX) S3 Table. Mutations in the eight sepsis isolates are confirmed with Sanger sequencing. https://doi.org/10.1371/journal.pcbi.1007562.s008 (DOCX) S4 Table. Primers used to generate mutations in PA14. https://doi.org/10.1371/journal.pcbi.1007562.s009 (DOCX) S5 Table. Summary of small indels and SNPs comparing to PAO1. https://doi.org/10.1371/journal.pcbi.1007562.s010 (DOCX) S6 Table. Blast result of the three big insertions. https://doi.org/10.1371/journal.pcbi.1007562.s011 (DOCX) S7 Table. Transcriptional regulators associated with overall antibiotic resistance. https://doi.org/10.1371/journal.pcbi.1007562.s012 (DOCX) S1 Data. Minimum inhibitory concentration of aztreonam for all clinical isolates. https://doi.org/10.1371/journal.pcbi.1007562.s013 (XLSX) S2 Data. Experimental validation that NalDF198L is associated with increase of aztreonam minimum inhibitory concentration. https://doi.org/10.1371/journal.pcbi.1007562.s014 (XLSX) S3 Data. Inhibition zone measurement using antibiotic disk diffusion assay for clinical isolates. https://doi.org/10.1371/journal.pcbi.1007562.s015 (XLSX) S4 Data. Genes change expression in the presence of aztreonam. https://doi.org/10.1371/journal.pcbi.1007562.s016 (XLSX) S5 Data. Constitutive mutation effects. Forty-eight reactions constitutively affected by mutations in D+7bld regardless of the presence and concentration of aztreonam. https://doi.org/10.1371/journal.pcbi.1007562.s017 (XLSX) S1 Text. Supplementary materials and methods. https://doi.org/10.1371/journal.pcbi.1007562.s018 (DOCX)

Reischl, Markus;Jouda, Mazin;MacKinnon, Neil;Fuhrer, Erwin;Bakhtina, Natalia;Bartschat, Andreas;Mikut, Ralf;Korvink, Jan G.

doi: 10.1371/journal.pcbi.1006997pmid: 31856159

Introduction A major challenge in biological science is to relate molecular regulation at the cellular level to response and behaviour at the organism level. Knowing this relationship lies at the foundation of every disease, and indeed also in understanding the healthy organism. An experiment establishing this relationship, as schematically shown in Fig 1, requires i) in vivo cellular-level detection of regulation-relevant molecules, such as metabolites and their production rates, ii) precise access to and application of biological perturbation mechanisms, and iii) an unburdened (naturally responding) organism. Clearly, even taken individually, these items are very hard to achieve in general, and correspond to major research areas in their own right. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. A freely moving organism is subject to a stimulus (mechanical, chemical, light, etc.) causing a response at all levels of detail (metabolomic, behaviour, etc.). The experiment designer relates the correlated output to a biological hypothesis and may adapt the stimulus. https://doi.org/10.1371/journal.pcbi.1006997.g001 Small organisms such as C. elegans are biological model organisms useful for studying many human disorders, including neurodegenerative diseases [1, 2]. Model organisms have been the mainstay of biological sciences for decades, and thus a broad knowledge base already exists starting from the genome level, through developmental cycles, and up to behavioural response to applied stress. These platforms offer the opportunity to address the connection between molecular phenotype, which can be conveniently implemented due to the very rapid yet standardized life cycle of C. elegans, to behaviour. What remains is the technological challenge of satisfying the three requirements for robustly linking phenotype to behaviour. Nuclear magnetic resonance (NMR), which is a noninvasive and non-destructive technique, together with its imaging modality (MRI), is a strong candidate as the analytical method of choice, towards the ultimate goal of in vivo measurement of the molecular response. NMR is based on exciting the spin-active nuclei of the magnetized organism with radio frequency (RF) signals, and detecting their response via induced RF signals. Many nuclei are NMR sensitive, but in vivo molecular concentrations of metabolites are typically below millimolar levels, requiring high sensitivity to detect. NMR is a non-ionizing technique, making it superior to computed tomography (CT) based on X-rays. Although MRI microscopy currently only achieves spatial resolutions down to 4 μm, new techniques are partially overcoming these limits, such as the use of nitrogen vacancy centers in diamond, or the use of hyperpolarisation techniques. MRI is a Fourier imaging technique that uses magnetic field gradients to selectively excite parts of the object, and perform consecutive phase and frequency encoding of the excited spins. This so-called spatial encoding makes MRI a relatively slow imaging technique, as a single MRI image requires multiple consecutive signal acquisitions from the object that is being imaged. In conventional MRI, it is assumed that the organism (or object) being imaged is fixed in space, so that spatio-temporally varying magnetic fields are only due to the technical system. Any movement of the organism results in measurements from shifted spatio-temporal positions resulting in image artifacts and ensuing difficulty in data interpretation (see Fig 2). Motion-induced artifacts are the challenge since spatially localized spectroscopy (required for molecular profiling) necessarily requires repeated measurements to bring the signal level above the noise. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Concept of moving frame imaging: An object of interest is placed close to an NMR sensor (coil). For conventional cartesian acquisition of an MR image, N repetitions are required to fill the k-space matrix. The signal of one repetition is used to fill one k-space line. This acquisition scheme results in a total acquisition time of TR ⋅ N. The time scales of worm motion and repetition time TR are of the same magnitude, and with a standard Eulerian or fixed frame MR procedure, strong motion blurring occurs. To enable imaging free of blurring of a freely moving worm, the concept of a Lagrangian moving frame is required. https://doi.org/10.1371/journal.pcbi.1006997.g002 In clinical applications of MRI, free body motion remains a challenge and there is a concerted effort underway to i) collect the MRI data faster than the motion; ii) collect the MRI data at moments when motion is minimal (triggering); iii) track patient motion and correct the MRI data during post processing; iv) track the motion and guide the spatial encoding to reflect the instantaneous geometrical configuration [3–6]; v) predict the motion and adjust the spatial encoding system in real-time, e.g., the prospective motion correction based on respiratory motion prediction [7]. Whilst these methods have been highly successful to control artefacts due to breathing, heartbeat, and low amplitude head movement, their assumptions for the kinematics of the underlying movement or the periodicity of motion are limiting. For example, the head is motion-captured in a model that assumes rigid six-degree-of-freedom body movement involving translations and rotations (x, y, z, θx, θy, θz) along the three orthogonal Cartesian axes. The direct translation of these techniques to MR measurements of small samples is not straightforward primarily because of the reduced size, more complex organism motion including writhing and wiggling, and rapid displacement across the detector’s sensitive region. Methods used to immobilise an organism can be considered to avoid these motion artifacts, such as clamping or freezing; however, such drastic measures typically introduce an undesired stress response into the molecular profile. The challenge of molecular measurement of non-stressed, small model organisms therefore still remains open. Given the advances in image processing, we believe there is an opportunity to address this challenge computationally. Advances in computer vision have revolutionized the speed and accuracy with which image analyses can be performed, and aim to reduce the need for expert knowledge e.g. in medical image analysis [8] and automated experiment handling [9]. These technologies are entering public awareness through the automation of highly complex processes, such as trajectory generation for self-driving vehicles (road, aerial) and surveillance for public safety. This is accomplished by real-time processing of dynamic images, in which accuracy and speed are of paramount importance. To name an example, advances in image and data processing algorithms are expected to make real-time dynamic spectrum imaging (achieving a hyperspectral imaging cube or hypercube) possible at all electromagnetic wavelengths. For instance, recording image and spectral data over 500 × 500 pixels, with a spectral resolution of 5nm over the visible spectrum, at 5 f s−1 has already been demonstrated [10]. Enormous data storage and processing power is required to perform such operations with sufficient spectral, spatial, and temporal resolution, and has motivated an increased effort in sparse imaging modalities. MRI also suffers from limitations in acquisition speed, which in this case are due to physical constraints such as the relaxation times. This renders MRI a relatively slow imaging technique compared to techniques that are based on the visual range of wavelengths. In MRI, data is acquired in -space ( being the spatial frequency), and the observer requires the transformed data in space. Hence, a large number of acquired data is associated with each final image pixel or voxel, and motion or geometrical warping during imaging will introduce errors into the reconstruction algorithms. To obtain sufficient acquired signal power per voxel volume, object tracking must be implemented in order to ensure correct spatial and spectral co-localization over time. This is especially important in MRI microscopy, which depends on accumulated sampling for sufficient image resolution. With the novel capabilities of image and data processing algorithms, real-time image processing becomes feasible. This enables the extraction of information and motion prediction based on a conventional video stream which then can be used to steer the MR sequence in real-time. The prediction horizon will therefore be dependent on the signal acquisition time at a given frame rate and voxel spatial resolution. In this contribution we consider the preconditions for performing MRI experiments on unburdened small organisms. We will focus our attention on the nematode C. elegans, mainly because it fits in best with our own efforts towards in vivo metabolomic profiling, but we will address the underlying problem in more generality so that it is relevant also for other organisms. In this contribution we explore the possibility to completely remove the requirement of organism immobilisation, by providing the host observing technical system (e.g., a spectrometer or microscope) with a real time co-evolving Lagrangian coordinate system centered in the organism that provides the organism’s current center-of-gravity position and shape, paired with a robust prediction of the organism’s future position and deformed shape. There is a multitude of publications of so-called motion-trackers, which mainly quantify rigid body motions and ignore strain fields. Important ones are [11] which categorises behavior and morphology features out of C. elegans-videos by applying segmentation and tracking algorithms and [12], which applies methods of machine vision, data processing and tracking to evaluate drug assays and [13] to follow multiple worm within the same environment. Furthermore, motion decomposition methods using so-called Eigenworms apply methods from oscillation analysis to divide the worm motion into so called eigenmodes [14]. There are automated software-packages being able to quantitatively assess movement parameters (e.g. Track-A-Worm [15], Parallel Worm Tracker [16] etc.). However, there is a lack of an easy-to-use and sufficiently fast prediction algorithm for individually chosen positions within a single worm to be used to adjust the gradient system following a body part to be imaged. The prediction horizon available for the host system depends on the speed of image processing and latency of the capturing hardware (essentially the frame rate), as well as the smoothness of the organism’s motion. To gain insight into how movement prediction can enhance MRI signal detection, the paper introduces: Data of a virtual phantom, combining high resolution electron microscope slice images [17] with conventional video-recordings of C. elegans moving in a Petri dish (Section Phantom Generation); A new concept for location prediction in C. elegans, and detailed movement prediction, exploiting characteristic worm movements (Section MRI Simulation); A computational platform, being able to evaluate the outcome of MR imaging with and without adapting the imaging parameters based on the movement prediction (Section MRI Simulation). A measure for simulation success that compares prediction to the ‘true’ image through a similarity measure sxy, a technique first introduced in [18] and described in detail in the supporting information. Using this simulation model, we demonstrate the capabilities of the motion-prediction algorithm in MRI. Materials and methods Phantom generation The study is based on eight 10 s duration AVI-videos (sampling frequency 12 Hz) of C. elegans. The recording was done in a controlled environment with a fixed camera and constant illumination over time. The worm was enclosed in a technical setup including a microfluidic channel and a Petri dish of the host system, suitable for optical recording and other real-time measurements. Typically, if natural state studies are relevant, the organism can also be provided with an optically transparent nutritional substrate such as a gel containing E. coli bacteria. To simulate MR imaging of the moving worm, we artificially linked transmission electron microscope (TEM) images to the video of the worm, such that virtually scanning each location within the worm would deliver a simulated high-resolution MR-image. At each time-frame, the 1.2mm worm was segmented into 50 slices perpendicular to its center line. This resulted in a reasonable slice thickness of roughly 24μm. Subsequently, we assigned voxel MR signals adapted from the TEM images to each slice as follows: first, we removed the background from the TEM images and scaled them such that the dimensions of the body part in each slice were realistic. Second, we inverted the color map of the images, since the white regions of the TEM images corresponded to low density material, and therefore would appear dark in MR imaging due to the low proton signal intensity. This color inversion is just to make the sample images look like MR images. The reader should bear in mind that TEM and MRI use completely different imaging mechanisms, and therefore converting one to the other is not a straightforward task. In fact, having true MRI sample images is not essential to prove the efficiency of motion prediction in imaging enhancement, and indeed any arbitrary set of morphologically correct sample images would work. Finally, we reduced the number of pixels of the TEM images to 64 × 64, which corresponded to an MRI in-plane resolution of approximately 1.6 μm, and then assigned the voxel signals to the virtual MR slices. Although such a high volumetric resolution lies beyond the capabilities of currently available MRI scanners, it was chosen on purpose to allow more accurate assessment of the prediction algorithm. It is thus a good compromise between being close to a realistically achievable resolution, and being high enough for the structural similarity measure to work more accurately. Furthermore, considering the rapid advances in the field of MRI including higher fields, stronger gradients, and hyperpolarization techniques, the mentioned resolution of 1.6μm is believed to be achievable in the near future. Motion prediction New concept. In order to adjust the gradient system of the MRI, the worm needs to be detected and its future positions need to be predicted. Therefore, we introduced a new concept of real-time image processing of the worm, suitable for any video format, regardless of color model, and also independent of background structures. Fig 3a shows a block diagram of the basic steps of the concept. Starting with a raw image matrix Xraw, a robust estimation of the worm and its position Xworm within the video were determined. Based on the segmented worm, characteristic points such as center of gravity (COG) and position of the head region were determined. To introduce a coordinate system along the center line of the worm, a skeletonization was applied, see Fig 3b. Assuming that body regions excluding the head would move along the center line, the worm velocity was used to predict positions xc,p of arbitrary selected points xc within the worm. A detailed description is given in the supporting information. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. a: Concept for the prediction of defined locations within a worm, b: Parametrization of the worm, c-j: Snapshot of a sample video. c: grayscale image/video Xraw[k], d: Image with subtracted background, e: difference image, f: overlay with worm, g: skeletonization, h-j: Point of interest (POI, gray marker) and predictions for Δk = 30 (= 2.5 s) time samples (white marker). h-j: Time samples k = 50, k = 80, k = 110. Figures show the position every 30 samples and the prediction of the POI 30 samples ahead. Thus, the similarity of POI(80) vs. POIpred(80) and POI(110) vs. POIpred(110) shows the quality of the prediction. https://doi.org/10.1371/journal.pcbi.1006997.g003 Worm detection. This section briefly covers the worm detection and movement prediction. The preprocessing for the worm detection aims to reduce the computational complexity and consists of a grayscale conversion (standard Matlab conversion) of the videos as well as a decrease of the resolution by a factor of 9 (Fig 3c) (Decreasing the resolution proves real time processing to be possible / usage of low-end imaging hardware to be accurate (see Discussion)). For the segmentation of the worm, the background of the video is estimated and removed from the images (Fig 3d). The direction of motion is determined by relying on the position of the head and the COG. The COG is calculated based on the foreground pixels of the segmentation and is smoothed to obtain variations for future predictions (1st order low-pass filtering over time). The position of the head is computed utilizing difference-images (Fig 3e) of the last 10 frames in the video. Fig 3f shows the detected head. Coordinate prediction. The position of the COG moves linearly and is predicted by linear extrapolation based on its past five positions. To predict the position of an arbitrary point of interest (POI) within the worm (which can easily be identified e.g. by a mouse click in the an NMR-simulation), we use the skeleton line as the center line of the segmentation and introduce a normalized coordinate s along it (head: s = 0, tail: s = 1, see Fig 3b and 3g). Assuming that each worm segment moves according to the current shape of the worm, following its predecessor segment with the velocity of the worm (The assumption is not perfectly valid, but it significantly simplifies processing and delivers reasonable results (see also Fig 4)), the velocity and the shape is used to predict the location sc of the POI after Δk time steps using sc = s − vΔk, (Fig 3h–3j) (If the prediction horizon is chosen too high, negative sc are avoided by setting the prediction to the topmost point (s = 0)). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Evaluation (mean values and standard deviation) of the dataset given in section phantom generation. a: fixed location in the worm (s = 0.9), varying prediction horizons, b: fixed prediction horizon (Δk = 12, 1 second, varying locations, prediction horizon too high for s < 0.3). https://doi.org/10.1371/journal.pcbi.1006997.g004 The prediction quality is evaluated using an Euclidean distance. Given the prediction horizon Δk, the predicted position based at time point k + Δk is calculated (x1,c,p[k + Δk], x2,c,p[k + Δk]) and the distance to the true position (x1,c[k + Δk], x2,c[k + Δk]) is measured. The average for all time samples is termed the mean prediction error. The dependency of the prediction accuracy to the prediction horizon, as well as to the predicted position of the worm, is given in Fig 4: The prediction error increases if the prediction horizon becomes larger. Regions near the head cannot be reliably predicted, since the head moves at a much higher frequency and in random patterns while scanning the surroundings and deciding on the moving direction. Furthermore, if the prediction horizon is set too high the predicted point lies outside the current shape of the worm (e.g. for Δk = 12 for all s < 0.3) (depending on the speed of the worm, however wild type worms mostly have the same velocity of approximately 0.13 mm s−1) and the topmost point (s = 0) is chosen as prediction, resulting in deviations). For all other regions, the uncertainty of the prediction stays roughly constant. Phantom generation The study is based on eight 10 s duration AVI-videos (sampling frequency 12 Hz) of C. elegans. The recording was done in a controlled environment with a fixed camera and constant illumination over time. The worm was enclosed in a technical setup including a microfluidic channel and a Petri dish of the host system, suitable for optical recording and other real-time measurements. Typically, if natural state studies are relevant, the organism can also be provided with an optically transparent nutritional substrate such as a gel containing E. coli bacteria. To simulate MR imaging of the moving worm, we artificially linked transmission electron microscope (TEM) images to the video of the worm, such that virtually scanning each location within the worm would deliver a simulated high-resolution MR-image. At each time-frame, the 1.2mm worm was segmented into 50 slices perpendicular to its center line. This resulted in a reasonable slice thickness of roughly 24μm. Subsequently, we assigned voxel MR signals adapted from the TEM images to each slice as follows: first, we removed the background from the TEM images and scaled them such that the dimensions of the body part in each slice were realistic. Second, we inverted the color map of the images, since the white regions of the TEM images corresponded to low density material, and therefore would appear dark in MR imaging due to the low proton signal intensity. This color inversion is just to make the sample images look like MR images. The reader should bear in mind that TEM and MRI use completely different imaging mechanisms, and therefore converting one to the other is not a straightforward task. In fact, having true MRI sample images is not essential to prove the efficiency of motion prediction in imaging enhancement, and indeed any arbitrary set of morphologically correct sample images would work. Finally, we reduced the number of pixels of the TEM images to 64 × 64, which corresponded to an MRI in-plane resolution of approximately 1.6 μm, and then assigned the voxel signals to the virtual MR slices. Although such a high volumetric resolution lies beyond the capabilities of currently available MRI scanners, it was chosen on purpose to allow more accurate assessment of the prediction algorithm. It is thus a good compromise between being close to a realistically achievable resolution, and being high enough for the structural similarity measure to work more accurately. Furthermore, considering the rapid advances in the field of MRI including higher fields, stronger gradients, and hyperpolarization techniques, the mentioned resolution of 1.6μm is believed to be achievable in the near future. Motion prediction New concept. In order to adjust the gradient system of the MRI, the worm needs to be detected and its future positions need to be predicted. Therefore, we introduced a new concept of real-time image processing of the worm, suitable for any video format, regardless of color model, and also independent of background structures. Fig 3a shows a block diagram of the basic steps of the concept. Starting with a raw image matrix Xraw, a robust estimation of the worm and its position Xworm within the video were determined. Based on the segmented worm, characteristic points such as center of gravity (COG) and position of the head region were determined. To introduce a coordinate system along the center line of the worm, a skeletonization was applied, see Fig 3b. Assuming that body regions excluding the head would move along the center line, the worm velocity was used to predict positions xc,p of arbitrary selected points xc within the worm. A detailed description is given in the supporting information. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. a: Concept for the prediction of defined locations within a worm, b: Parametrization of the worm, c-j: Snapshot of a sample video. c: grayscale image/video Xraw[k], d: Image with subtracted background, e: difference image, f: overlay with worm, g: skeletonization, h-j: Point of interest (POI, gray marker) and predictions for Δk = 30 (= 2.5 s) time samples (white marker). h-j: Time samples k = 50, k = 80, k = 110. Figures show the position every 30 samples and the prediction of the POI 30 samples ahead. Thus, the similarity of POI(80) vs. POIpred(80) and POI(110) vs. POIpred(110) shows the quality of the prediction. https://doi.org/10.1371/journal.pcbi.1006997.g003 Worm detection. This section briefly covers the worm detection and movement prediction. The preprocessing for the worm detection aims to reduce the computational complexity and consists of a grayscale conversion (standard Matlab conversion) of the videos as well as a decrease of the resolution by a factor of 9 (Fig 3c) (Decreasing the resolution proves real time processing to be possible / usage of low-end imaging hardware to be accurate (see Discussion)). For the segmentation of the worm, the background of the video is estimated and removed from the images (Fig 3d). The direction of motion is determined by relying on the position of the head and the COG. The COG is calculated based on the foreground pixels of the segmentation and is smoothed to obtain variations for future predictions (1st order low-pass filtering over time). The position of the head is computed utilizing difference-images (Fig 3e) of the last 10 frames in the video. Fig 3f shows the detected head. Coordinate prediction. The position of the COG moves linearly and is predicted by linear extrapolation based on its past five positions. To predict the position of an arbitrary point of interest (POI) within the worm (which can easily be identified e.g. by a mouse click in the an NMR-simulation), we use the skeleton line as the center line of the segmentation and introduce a normalized coordinate s along it (head: s = 0, tail: s = 1, see Fig 3b and 3g). Assuming that each worm segment moves according to the current shape of the worm, following its predecessor segment with the velocity of the worm (The assumption is not perfectly valid, but it significantly simplifies processing and delivers reasonable results (see also Fig 4)), the velocity and the shape is used to predict the location sc of the POI after Δk time steps using sc = s − vΔk, (Fig 3h–3j) (If the prediction horizon is chosen too high, negative sc are avoided by setting the prediction to the topmost point (s = 0)). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Evaluation (mean values and standard deviation) of the dataset given in section phantom generation. a: fixed location in the worm (s = 0.9), varying prediction horizons, b: fixed prediction horizon (Δk = 12, 1 second, varying locations, prediction horizon too high for s < 0.3). https://doi.org/10.1371/journal.pcbi.1006997.g004 The prediction quality is evaluated using an Euclidean distance. Given the prediction horizon Δk, the predicted position based at time point k + Δk is calculated (x1,c,p[k + Δk], x2,c,p[k + Δk]) and the distance to the true position (x1,c[k + Δk], x2,c[k + Δk]) is measured. The average for all time samples is termed the mean prediction error. The dependency of the prediction accuracy to the prediction horizon, as well as to the predicted position of the worm, is given in Fig 4: The prediction error increases if the prediction horizon becomes larger. Regions near the head cannot be reliably predicted, since the head moves at a much higher frequency and in random patterns while scanning the surroundings and deciding on the moving direction. Furthermore, if the prediction horizon is set too high the predicted point lies outside the current shape of the worm (e.g. for Δk = 12 for all s < 0.3) (depending on the speed of the worm, however wild type worms mostly have the same velocity of approximately 0.13 mm s−1) and the topmost point (s = 0) is chosen as prediction, resulting in deviations). For all other regions, the uncertainty of the prediction stays roughly constant. New concept. In order to adjust the gradient system of the MRI, the worm needs to be detected and its future positions need to be predicted. Therefore, we introduced a new concept of real-time image processing of the worm, suitable for any video format, regardless of color model, and also independent of background structures. Fig 3a shows a block diagram of the basic steps of the concept. Starting with a raw image matrix Xraw, a robust estimation of the worm and its position Xworm within the video were determined. Based on the segmented worm, characteristic points such as center of gravity (COG) and position of the head region were determined. To introduce a coordinate system along the center line of the worm, a skeletonization was applied, see Fig 3b. Assuming that body regions excluding the head would move along the center line, the worm velocity was used to predict positions xc,p of arbitrary selected points xc within the worm. A detailed description is given in the supporting information. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. a: Concept for the prediction of defined locations within a worm, b: Parametrization of the worm, c-j: Snapshot of a sample video. c: grayscale image/video Xraw[k], d: Image with subtracted background, e: difference image, f: overlay with worm, g: skeletonization, h-j: Point of interest (POI, gray marker) and predictions for Δk = 30 (= 2.5 s) time samples (white marker). h-j: Time samples k = 50, k = 80, k = 110. Figures show the position every 30 samples and the prediction of the POI 30 samples ahead. Thus, the similarity of POI(80) vs. POIpred(80) and POI(110) vs. POIpred(110) shows the quality of the prediction. https://doi.org/10.1371/journal.pcbi.1006997.g003 Worm detection. This section briefly covers the worm detection and movement prediction. The preprocessing for the worm detection aims to reduce the computational complexity and consists of a grayscale conversion (standard Matlab conversion) of the videos as well as a decrease of the resolution by a factor of 9 (Fig 3c) (Decreasing the resolution proves real time processing to be possible / usage of low-end imaging hardware to be accurate (see Discussion)). For the segmentation of the worm, the background of the video is estimated and removed from the images (Fig 3d). The direction of motion is determined by relying on the position of the head and the COG. The COG is calculated based on the foreground pixels of the segmentation and is smoothed to obtain variations for future predictions (1st order low-pass filtering over time). The position of the head is computed utilizing difference-images (Fig 3e) of the last 10 frames in the video. Fig 3f shows the detected head. Coordinate prediction. The position of the COG moves linearly and is predicted by linear extrapolation based on its past five positions. To predict the position of an arbitrary point of interest (POI) within the worm (which can easily be identified e.g. by a mouse click in the an NMR-simulation), we use the skeleton line as the center line of the segmentation and introduce a normalized coordinate s along it (head: s = 0, tail: s = 1, see Fig 3b and 3g). Assuming that each worm segment moves according to the current shape of the worm, following its predecessor segment with the velocity of the worm (The assumption is not perfectly valid, but it significantly simplifies processing and delivers reasonable results (see also Fig 4)), the velocity and the shape is used to predict the location sc of the POI after Δk time steps using sc = s − vΔk, (Fig 3h–3j) (If the prediction horizon is chosen too high, negative sc are avoided by setting the prediction to the topmost point (s = 0)). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Evaluation (mean values and standard deviation) of the dataset given in section phantom generation. a: fixed location in the worm (s = 0.9), varying prediction horizons, b: fixed prediction horizon (Δk = 12, 1 second, varying locations, prediction horizon too high for s < 0.3). https://doi.org/10.1371/journal.pcbi.1006997.g004 The prediction quality is evaluated using an Euclidean distance. Given the prediction horizon Δk, the predicted position based at time point k + Δk is calculated (x1,c,p[k + Δk], x2,c,p[k + Δk]) and the distance to the true position (x1,c[k + Δk], x2,c[k + Δk]) is measured. The average for all time samples is termed the mean prediction error. The dependency of the prediction accuracy to the prediction horizon, as well as to the predicted position of the worm, is given in Fig 4: The prediction error increases if the prediction horizon becomes larger. Regions near the head cannot be reliably predicted, since the head moves at a much higher frequency and in random patterns while scanning the surroundings and deciding on the moving direction. Furthermore, if the prediction horizon is set too high the predicted point lies outside the current shape of the worm (e.g. for Δk = 12 for all s < 0.3) (depending on the speed of the worm, however wild type worms mostly have the same velocity of approximately 0.13 mm s−1) and the topmost point (s = 0) is chosen as prediction, resulting in deviations). For all other regions, the uncertainty of the prediction stays roughly constant. Results MRI simulation Simulation-platform. The magnetic resonance imaging process is simulated using Matlab. The worm is assumed to reside within a strong and constant magnetic field B = (0, 0, Bz) of the MRI scanner. The software mimics a simplified yet acceptably accurate MR image acquisition process via a standard gradient-echo sequence [19] that takes place during worm motion as follows: An image slice at a parametrised position s along the worm axis is defined as the image plane perpendicular to the worm axis. As previously mentioned, the worm is segmented into 50 slices whose voxels’ intensities are obtained from the TEM images. Thus s is the center of one of those slices. Ideally, to do MRI of a slice s, the center of the gradient system (the field of view FoV), GC, should be exactly at s over the entire signal acquisition process. This is unfortunately not the case as the prediction imposes some error resulting in GC being at another slice, at the desired slice but off-center, or both. In this case the MR excitation is done by calculating the signal intensity of each voxel in the FoV depending on the position of GC. If, for instance, GC is off-center, then some voxels of the true slice will be out of the FoV and the corresponding ones in the FoV will then be filled with zeros. The excitation step is followed by a phase encoding step, in which each voxel of the FoV is given a phase proportional to its position along one cross-sectional axis ξ. After a time delay TE/2, where TE (echo time) is the time from excitation to the center of the MR signal (echo), a frequency encoding gradient is applied, whereby each voxel of the FoV is assigned a frequency proportional to its position along an axis η perpendicular to the phase encoding axis (so that ξ ⋅ η = 0). The superposition of all voxel RF signals are simultaneously detected by a virtual coil. This so-called echo is recorded at instant TE, resulting in one line of k-space. The detection system is assumed to have a uniform spatial sensitivity. After a time delay TR (repetition time), which is the time it takes to repeat the sequence in order to acquire a new line of the k-space, the algorithm calculates the anticipated new position and orientation of the selected slice (as the worm would have already moved to a new position). Now the slice at this new position is excited, phase encoded, and frequency encoded, resulting in a new line of k-space being filled. The entire process is repeated until all the lines of k-space (in our case 64 lines) are complete. Once the imaging procedure is complete, the program reconstructs the anticipated MR image from the k-space data via Fourier transform. Algorithm 1 Gradient-echo MR imaging 1: procedure GRE(TE, TR)      ⊳ Echo time and repetition time. 2:  s = 0      ⊳ s parametrises slice position along worm axis. 3:  while s ≠ 1 do           ⊳ cycle through the slices. 4:   s ← s + δs                 ⊳ Next slice. 5:   t = 0                ⊳ Initialize time counter. 6:   N = 64               ⊳ Number of k-space lines. 7:   k = −π                 ⊳ Set initial phase. 8:   J = 1                     ⊳ Set index. 9:   while t ≠ N ⋅ TR do 10:    Pos. Pred.            ⊳ Predict next position. 11:    Mov. Grad.         ⊳ Update imaging gradient. 12:    Exc.                    ⊳ Excite slice. 13:    t ← t + TE/2      ⊳ Advance time to half echo time. 14:    Ph. Enc. (k + J ⋅ 2π/N)            ⊳ Phase encode. 15:    t ← t + TE/2        ⊳ Advance time to echo time. 16:    Freq. Enc.             ⊳ Frequency encode. 17:    Echo                ⊳ Signal acquisition. 18:    t ← t + TR − TE        ⊳ Advance time to next line. 19:    J ← J + 1 20:   end while 21:  end while 22: end procedure The simulation is integrated into an interactive graphical user interface (GUI) to enable the user to set and change the imaging parameters easily and execute the processing without any programming skills (Fig 5). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Graphical user interface of a Matlab program that simulates the MR imaging experiment of a moving worm. The microscope movie of the worm is shown to the bottom left, and emulates real-time observation. The slice position is also shown. The top right shows the phantom MR image of the worm at the position of the slice. https://doi.org/10.1371/journal.pcbi.1006997.g005 The simulated MRI signals can be chosen by the user and are either MR-like signals translated from real transmission electron microscopy (TEM) images, or any virtual MR-like images provided by the user. To start the simulation, an arbitrary slice of interest (with coordinate s) within the worm’s body is selected. For this point, the simulated MR image of a slice without movement is shown. In the display, the slice position s is denoted by a red line perpendicular to the center line of the worm, while the predicted position to which the virtual imaging gradient coordinate is set is denoted by a green line. Furthermore, the software allows the user to set the TR and TE imaging parameters, and to choose the prediction horizon (a value between 1 and 10 frames), which refers to the number of frames ahead for which the predicted worm position will be calculated. Our algorithm does not correct for the worm motion occurring during recording of a line of k-space. This is equivalent to the assumption that the MRI pulse sequence is based on very short echo times, which is possible, but taken at the expense of increased acquisition bandwidth and thus reduced signal-to-noise ratio (SNR). Simulation paradigms. We performed three MRI simulations using the optical video dataset from Section Phantom Generation to: confirm that the prediction enhances the MR imaging of the moving worm in general (Simulation 1); measure the efficiency of the prediction algorithm as the desired resolution of the MR images increases in comparison to the resolution of the optical video used for the prediction (Simulation 2); and measure the effect of an increased prediction horizon (the number of frames ahead for which the position is predicted, Simulation 3). In all the simulations, the repetition time (TR) was set to 83 ms, which corresponds to the frame rate of the optical video on which the prediction was based. We evaluate the simulation by quantifying the structural similarity between the true image and simulated image as introduced in [18]. Identical images return sxy = 1, whereas structural inequality delivers sxy = 0. Body position (Simulation 1). In Simulation 1, three slices (slice definition from Section Phantom Generation, first slice from the head (s = 0.02), second from the middle body (s = 0.5), third from the tail (s = 1) of the worm) are selected and an MR imaging simulation is performed (prediction horizon Δk = 1 (83 ms)). The results are illustrated in Fig 6a, each row shows the true slice on the left, the simulated MR image based on the prediction algorithm in the middle, and the simulated image when no position prediction is involved on the right. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. a: effect of position prediction on the quality of the MR images in the head (s = 0.02), body (s = 0.5) and tail (s = 1) regions. The prediction horizon in this case is 83 msec. b: dependence of the MR imaging quality (structural similarity) on the position of the slice. The figure displays the simulation results from eight videos of different worms. https://doi.org/10.1371/journal.pcbi.1006997.g006 Clearly, the prediction algorithm significantly reduces the motion artifacts that would otherwise occur if the gradient system did not follow the worm as it moves. Moreover, we observe that prediction quality varies along the worm’s body. More specifically, the prediction performs better for the slices from the body and tail when compared with prediction of the head. This is axiomatic, since the worm rapidly moves its head laterally whilst scavenging for nutrition, thus the motional entropy of the head is higher (see also Fig 4). Fig 6b shows the simulation results of eight videos of different worms. The abscissa represents the slice position starting from the head (s = 0) to the tail (s = 1), while the ordinate shows the structural similarity between the prediction-based MR images and the original slices. To very good agreement with Figs 4 and 6b shows that the quality of the prediction-based MR images are higher for slices from the body and tail than the images taken near the head where the prediction uncertainty is usually higher. Image resolution (Simulation 2). In Simulation 2, a slice from the middle body section of the worm (s = 0.72) was selected and an MRI simulation was performed for different resolution ratios (= MR image: optical video). The idea here is to explore the effect of changing the MRI resolution (voxel size) on the quality of the prediction-based imaging. Imagine, for example, that one pixel of the optical video is 10 μm × 10 μm and the desired MRI isotropic resolution is also 10 μm × μm × 10 μm × 10 μm, then one pixel error in prediction results in one voxel error in the MRI k-space. On the other hand, if higher MRI resolution is wished, then one pixel error in prediction results in more than one voxel error in the MRI k-space. Decreasing the optical resolution can be used to speed up image processing, and thus decreasing the prediction horizon, if needed. The ratios of 4:1 and 2:1 are heuristically chosen, for which each pixel in the optical video respectively corresponds to 4 and 2 pixels in the MR image. Fig 7 demonstrates the results of this simulation—the rows correspond to the different resolution ratios while the columns depict the original slice, the image with prediction, and the image without prediction. The figure shows that the efficiency of the prediction declines as the desired resolution of the MR image increases (pixel size decreases), or alternatively, the optical resolution should be higher or at least equal to the desired MR resolution for a high quality prediction-based MR image. Indeed, the commercial optical imaging solutions can easily meet such demands of high resolution; however, upon implementation, any increase in number of pixels will be at the expense of the prolonged processing time needed for prediction. Nevertheless, this can be overcome by utilizing more powerful processors and by employing techniques such as parallel computing. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 7. Effect of decreasing the ratio of the MR image resolution to the optical video resolution on the imaging quality for a prediction horizon of 1. The rows show the results for MR image:video resolution ratios of 4:1 and 2:1 respectively. Each row displays, from left to right, the reference slice, the MR image with position prediction, and the MR image without position prediction. https://doi.org/10.1371/journal.pcbi.1006997.g007 Prediction horizon (Simulation 3). Simulation 3 varies prediction horizons for one slice (s = 0.52). Fig 8 illustrates the results of this simulation: Fig. (1a–10a) are the prediction-based simulated images for prediction horizons of Δk 1 to 10, respectively. In contrast, Fig. (1b–10b) display the results when prediction is not in action. Because only a few lines of the k-space are collected from the correct slice (depending on TR and the speed of the worm), the images in this figure exhibit a noticeable decrease in quality as the prediction horizon increases, leading to the conclusion that one should, whenever possible, minimize the prediction horizon. Of course, in an actual hardware realization of the system, the choice of the prediction horizon will be bounded by the speed of the image acquisition and processing units. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 8. Effect of enlarging the prediction horizon on the quality of the imaging (s = 0.52, see Fig 6). The images (1a-10a) show the reconstructed MR images based on the prediction algorithm for prediction horizons from 1 to 10 frames. The images (1b-10b) show the reconstructed MR images when no prediction is involved and when the initial position of the imaging gradient is set to the position of the slice after 1 (83 ms) to 10 (830 ms) frames respectively. (c) The similarity measure of the simulated image versus the prediction horizon. https://doi.org/10.1371/journal.pcbi.1006997.g008 The effect of increasing the prediction horizon is described quantitatively in Fig 8c by measuring the structural similarity between the original slice and the simulated image for both cases with and without prediction. The results confirm the variation in predictability along the length of the worm. The prediction horizon degradation is linear for the rear two-thirds of the worm, but falls off more rapidly for the head section, as expected from Fig 4. Moreover, a statistical assessment of the effect of the prediction horizon on the quality of the MR imaging was done: Regarding simulation results of four slices along the worm using the given eight videos it can be shown that the accuracy of prediction and thus the MRI quality decays with the increased prediction horizon but is in mean for all parameter combinations roughly three times better than without prediction. MRI simulation Simulation-platform. The magnetic resonance imaging process is simulated using Matlab. The worm is assumed to reside within a strong and constant magnetic field B = (0, 0, Bz) of the MRI scanner. The software mimics a simplified yet acceptably accurate MR image acquisition process via a standard gradient-echo sequence [19] that takes place during worm motion as follows: An image slice at a parametrised position s along the worm axis is defined as the image plane perpendicular to the worm axis. As previously mentioned, the worm is segmented into 50 slices whose voxels’ intensities are obtained from the TEM images. Thus s is the center of one of those slices. Ideally, to do MRI of a slice s, the center of the gradient system (the field of view FoV), GC, should be exactly at s over the entire signal acquisition process. This is unfortunately not the case as the prediction imposes some error resulting in GC being at another slice, at the desired slice but off-center, or both. In this case the MR excitation is done by calculating the signal intensity of each voxel in the FoV depending on the position of GC. If, for instance, GC is off-center, then some voxels of the true slice will be out of the FoV and the corresponding ones in the FoV will then be filled with zeros. The excitation step is followed by a phase encoding step, in which each voxel of the FoV is given a phase proportional to its position along one cross-sectional axis ξ. After a time delay TE/2, where TE (echo time) is the time from excitation to the center of the MR signal (echo), a frequency encoding gradient is applied, whereby each voxel of the FoV is assigned a frequency proportional to its position along an axis η perpendicular to the phase encoding axis (so that ξ ⋅ η = 0). The superposition of all voxel RF signals are simultaneously detected by a virtual coil. This so-called echo is recorded at instant TE, resulting in one line of k-space. The detection system is assumed to have a uniform spatial sensitivity. After a time delay TR (repetition time), which is the time it takes to repeat the sequence in order to acquire a new line of the k-space, the algorithm calculates the anticipated new position and orientation of the selected slice (as the worm would have already moved to a new position). Now the slice at this new position is excited, phase encoded, and frequency encoded, resulting in a new line of k-space being filled. The entire process is repeated until all the lines of k-space (in our case 64 lines) are complete. Once the imaging procedure is complete, the program reconstructs the anticipated MR image from the k-space data via Fourier transform. Algorithm 1 Gradient-echo MR imaging 1: procedure GRE(TE, TR)      ⊳ Echo time and repetition time. 2:  s = 0      ⊳ s parametrises slice position along worm axis. 3:  while s ≠ 1 do           ⊳ cycle through the slices. 4:   s ← s + δs                 ⊳ Next slice. 5:   t = 0                ⊳ Initialize time counter. 6:   N = 64               ⊳ Number of k-space lines. 7:   k = −π                 ⊳ Set initial phase. 8:   J = 1                     ⊳ Set index. 9:   while t ≠ N ⋅ TR do 10:    Pos. Pred.            ⊳ Predict next position. 11:    Mov. Grad.         ⊳ Update imaging gradient. 12:    Exc.                    ⊳ Excite slice. 13:    t ← t + TE/2      ⊳ Advance time to half echo time. 14:    Ph. Enc. (k + J ⋅ 2π/N)            ⊳ Phase encode. 15:    t ← t + TE/2        ⊳ Advance time to echo time. 16:    Freq. Enc.             ⊳ Frequency encode. 17:    Echo                ⊳ Signal acquisition. 18:    t ← t + TR − TE        ⊳ Advance time to next line. 19:    J ← J + 1 20:   end while 21:  end while 22: end procedure The simulation is integrated into an interactive graphical user interface (GUI) to enable the user to set and change the imaging parameters easily and execute the processing without any programming skills (Fig 5). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Graphical user interface of a Matlab program that simulates the MR imaging experiment of a moving worm. The microscope movie of the worm is shown to the bottom left, and emulates real-time observation. The slice position is also shown. The top right shows the phantom MR image of the worm at the position of the slice. https://doi.org/10.1371/journal.pcbi.1006997.g005 The simulated MRI signals can be chosen by the user and are either MR-like signals translated from real transmission electron microscopy (TEM) images, or any virtual MR-like images provided by the user. To start the simulation, an arbitrary slice of interest (with coordinate s) within the worm’s body is selected. For this point, the simulated MR image of a slice without movement is shown. In the display, the slice position s is denoted by a red line perpendicular to the center line of the worm, while the predicted position to which the virtual imaging gradient coordinate is set is denoted by a green line. Furthermore, the software allows the user to set the TR and TE imaging parameters, and to choose the prediction horizon (a value between 1 and 10 frames), which refers to the number of frames ahead for which the predicted worm position will be calculated. Our algorithm does not correct for the worm motion occurring during recording of a line of k-space. This is equivalent to the assumption that the MRI pulse sequence is based on very short echo times, which is possible, but taken at the expense of increased acquisition bandwidth and thus reduced signal-to-noise ratio (SNR). Simulation paradigms. We performed three MRI simulations using the optical video dataset from Section Phantom Generation to: confirm that the prediction enhances the MR imaging of the moving worm in general (Simulation 1); measure the efficiency of the prediction algorithm as the desired resolution of the MR images increases in comparison to the resolution of the optical video used for the prediction (Simulation 2); and measure the effect of an increased prediction horizon (the number of frames ahead for which the position is predicted, Simulation 3). In all the simulations, the repetition time (TR) was set to 83 ms, which corresponds to the frame rate of the optical video on which the prediction was based. We evaluate the simulation by quantifying the structural similarity between the true image and simulated image as introduced in [18]. Identical images return sxy = 1, whereas structural inequality delivers sxy = 0. Body position (Simulation 1). In Simulation 1, three slices (slice definition from Section Phantom Generation, first slice from the head (s = 0.02), second from the middle body (s = 0.5), third from the tail (s = 1) of the worm) are selected and an MR imaging simulation is performed (prediction horizon Δk = 1 (83 ms)). The results are illustrated in Fig 6a, each row shows the true slice on the left, the simulated MR image based on the prediction algorithm in the middle, and the simulated image when no position prediction is involved on the right. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. a: effect of position prediction on the quality of the MR images in the head (s = 0.02), body (s = 0.5) and tail (s = 1) regions. The prediction horizon in this case is 83 msec. b: dependence of the MR imaging quality (structural similarity) on the position of the slice. The figure displays the simulation results from eight videos of different worms. https://doi.org/10.1371/journal.pcbi.1006997.g006 Clearly, the prediction algorithm significantly reduces the motion artifacts that would otherwise occur if the gradient system did not follow the worm as it moves. Moreover, we observe that prediction quality varies along the worm’s body. More specifically, the prediction performs better for the slices from the body and tail when compared with prediction of the head. This is axiomatic, since the worm rapidly moves its head laterally whilst scavenging for nutrition, thus the motional entropy of the head is higher (see also Fig 4). Fig 6b shows the simulation results of eight videos of different worms. The abscissa represents the slice position starting from the head (s = 0) to the tail (s = 1), while the ordinate shows the structural similarity between the prediction-based MR images and the original slices. To very good agreement with Figs 4 and 6b shows that the quality of the prediction-based MR images are higher for slices from the body and tail than the images taken near the head where the prediction uncertainty is usually higher. Image resolution (Simulation 2). In Simulation 2, a slice from the middle body section of the worm (s = 0.72) was selected and an MRI simulation was performed for different resolution ratios (= MR image: optical video). The idea here is to explore the effect of changing the MRI resolution (voxel size) on the quality of the prediction-based imaging. Imagine, for example, that one pixel of the optical video is 10 μm × 10 μm and the desired MRI isotropic resolution is also 10 μm × μm × 10 μm × 10 μm, then one pixel error in prediction results in one voxel error in the MRI k-space. On the other hand, if higher MRI resolution is wished, then one pixel error in prediction results in more than one voxel error in the MRI k-space. Decreasing the optical resolution can be used to speed up image processing, and thus decreasing the prediction horizon, if needed. The ratios of 4:1 and 2:1 are heuristically chosen, for which each pixel in the optical video respectively corresponds to 4 and 2 pixels in the MR image. Fig 7 demonstrates the results of this simulation—the rows correspond to the different resolution ratios while the columns depict the original slice, the image with prediction, and the image without prediction. The figure shows that the efficiency of the prediction declines as the desired resolution of the MR image increases (pixel size decreases), or alternatively, the optical resolution should be higher or at least equal to the desired MR resolution for a high quality prediction-based MR image. Indeed, the commercial optical imaging solutions can easily meet such demands of high resolution; however, upon implementation, any increase in number of pixels will be at the expense of the prolonged processing time needed for prediction. Nevertheless, this can be overcome by utilizing more powerful processors and by employing techniques such as parallel computing. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 7. Effect of decreasing the ratio of the MR image resolution to the optical video resolution on the imaging quality for a prediction horizon of 1. The rows show the results for MR image:video resolution ratios of 4:1 and 2:1 respectively. Each row displays, from left to right, the reference slice, the MR image with position prediction, and the MR image without position prediction. https://doi.org/10.1371/journal.pcbi.1006997.g007 Prediction horizon (Simulation 3). Simulation 3 varies prediction horizons for one slice (s = 0.52). Fig 8 illustrates the results of this simulation: Fig. (1a–10a) are the prediction-based simulated images for prediction horizons of Δk 1 to 10, respectively. In contrast, Fig. (1b–10b) display the results when prediction is not in action. Because only a few lines of the k-space are collected from the correct slice (depending on TR and the speed of the worm), the images in this figure exhibit a noticeable decrease in quality as the prediction horizon increases, leading to the conclusion that one should, whenever possible, minimize the prediction horizon. Of course, in an actual hardware realization of the system, the choice of the prediction horizon will be bounded by the speed of the image acquisition and processing units. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 8. Effect of enlarging the prediction horizon on the quality of the imaging (s = 0.52, see Fig 6). The images (1a-10a) show the reconstructed MR images based on the prediction algorithm for prediction horizons from 1 to 10 frames. The images (1b-10b) show the reconstructed MR images when no prediction is involved and when the initial position of the imaging gradient is set to the position of the slice after 1 (83 ms) to 10 (830 ms) frames respectively. (c) The similarity measure of the simulated image versus the prediction horizon. https://doi.org/10.1371/journal.pcbi.1006997.g008 The effect of increasing the prediction horizon is described quantitatively in Fig 8c by measuring the structural similarity between the original slice and the simulated image for both cases with and without prediction. The results confirm the variation in predictability along the length of the worm. The prediction horizon degradation is linear for the rear two-thirds of the worm, but falls off more rapidly for the head section, as expected from Fig 4. Moreover, a statistical assessment of the effect of the prediction horizon on the quality of the MR imaging was done: Regarding simulation results of four slices along the worm using the given eight videos it can be shown that the accuracy of prediction and thus the MRI quality decays with the increased prediction horizon but is in mean for all parameter combinations roughly three times better than without prediction. Simulation-platform. The magnetic resonance imaging process is simulated using Matlab. The worm is assumed to reside within a strong and constant magnetic field B = (0, 0, Bz) of the MRI scanner. The software mimics a simplified yet acceptably accurate MR image acquisition process via a standard gradient-echo sequence [19] that takes place during worm motion as follows: An image slice at a parametrised position s along the worm axis is defined as the image plane perpendicular to the worm axis. As previously mentioned, the worm is segmented into 50 slices whose voxels’ intensities are obtained from the TEM images. Thus s is the center of one of those slices. Ideally, to do MRI of a slice s, the center of the gradient system (the field of view FoV), GC, should be exactly at s over the entire signal acquisition process. This is unfortunately not the case as the prediction imposes some error resulting in GC being at another slice, at the desired slice but off-center, or both. In this case the MR excitation is done by calculating the signal intensity of each voxel in the FoV depending on the position of GC. If, for instance, GC is off-center, then some voxels of the true slice will be out of the FoV and the corresponding ones in the FoV will then be filled with zeros. The excitation step is followed by a phase encoding step, in which each voxel of the FoV is given a phase proportional to its position along one cross-sectional axis ξ. After a time delay TE/2, where TE (echo time) is the time from excitation to the center of the MR signal (echo), a frequency encoding gradient is applied, whereby each voxel of the FoV is assigned a frequency proportional to its position along an axis η perpendicular to the phase encoding axis (so that ξ ⋅ η = 0). The superposition of all voxel RF signals are simultaneously detected by a virtual coil. This so-called echo is recorded at instant TE, resulting in one line of k-space. The detection system is assumed to have a uniform spatial sensitivity. After a time delay TR (repetition time), which is the time it takes to repeat the sequence in order to acquire a new line of the k-space, the algorithm calculates the anticipated new position and orientation of the selected slice (as the worm would have already moved to a new position). Now the slice at this new position is excited, phase encoded, and frequency encoded, resulting in a new line of k-space being filled. The entire process is repeated until all the lines of k-space (in our case 64 lines) are complete. Once the imaging procedure is complete, the program reconstructs the anticipated MR image from the k-space data via Fourier transform. Algorithm 1 Gradient-echo MR imaging 1: procedure GRE(TE, TR)      ⊳ Echo time and repetition time. 2:  s = 0      ⊳ s parametrises slice position along worm axis. 3:  while s ≠ 1 do           ⊳ cycle through the slices. 4:   s ← s + δs                 ⊳ Next slice. 5:   t = 0                ⊳ Initialize time counter. 6:   N = 64               ⊳ Number of k-space lines. 7:   k = −π                 ⊳ Set initial phase. 8:   J = 1                     ⊳ Set index. 9:   while t ≠ N ⋅ TR do 10:    Pos. Pred.            ⊳ Predict next position. 11:    Mov. Grad.         ⊳ Update imaging gradient. 12:    Exc.                    ⊳ Excite slice. 13:    t ← t + TE/2      ⊳ Advance time to half echo time. 14:    Ph. Enc. (k + J ⋅ 2π/N)            ⊳ Phase encode. 15:    t ← t + TE/2        ⊳ Advance time to echo time. 16:    Freq. Enc.             ⊳ Frequency encode. 17:    Echo                ⊳ Signal acquisition. 18:    t ← t + TR − TE        ⊳ Advance time to next line. 19:    J ← J + 1 20:   end while 21:  end while 22: end procedure The simulation is integrated into an interactive graphical user interface (GUI) to enable the user to set and change the imaging parameters easily and execute the processing without any programming skills (Fig 5). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Graphical user interface of a Matlab program that simulates the MR imaging experiment of a moving worm. The microscope movie of the worm is shown to the bottom left, and emulates real-time observation. The slice position is also shown. The top right shows the phantom MR image of the worm at the position of the slice. https://doi.org/10.1371/journal.pcbi.1006997.g005 The simulated MRI signals can be chosen by the user and are either MR-like signals translated from real transmission electron microscopy (TEM) images, or any virtual MR-like images provided by the user. To start the simulation, an arbitrary slice of interest (with coordinate s) within the worm’s body is selected. For this point, the simulated MR image of a slice without movement is shown. In the display, the slice position s is denoted by a red line perpendicular to the center line of the worm, while the predicted position to which the virtual imaging gradient coordinate is set is denoted by a green line. Furthermore, the software allows the user to set the TR and TE imaging parameters, and to choose the prediction horizon (a value between 1 and 10 frames), which refers to the number of frames ahead for which the predicted worm position will be calculated. Our algorithm does not correct for the worm motion occurring during recording of a line of k-space. This is equivalent to the assumption that the MRI pulse sequence is based on very short echo times, which is possible, but taken at the expense of increased acquisition bandwidth and thus reduced signal-to-noise ratio (SNR). Simulation paradigms. We performed three MRI simulations using the optical video dataset from Section Phantom Generation to: confirm that the prediction enhances the MR imaging of the moving worm in general (Simulation 1); measure the efficiency of the prediction algorithm as the desired resolution of the MR images increases in comparison to the resolution of the optical video used for the prediction (Simulation 2); and measure the effect of an increased prediction horizon (the number of frames ahead for which the position is predicted, Simulation 3). In all the simulations, the repetition time (TR) was set to 83 ms, which corresponds to the frame rate of the optical video on which the prediction was based. We evaluate the simulation by quantifying the structural similarity between the true image and simulated image as introduced in [18]. Identical images return sxy = 1, whereas structural inequality delivers sxy = 0. Body position (Simulation 1). In Simulation 1, three slices (slice definition from Section Phantom Generation, first slice from the head (s = 0.02), second from the middle body (s = 0.5), third from the tail (s = 1) of the worm) are selected and an MR imaging simulation is performed (prediction horizon Δk = 1 (83 ms)). The results are illustrated in Fig 6a, each row shows the true slice on the left, the simulated MR image based on the prediction algorithm in the middle, and the simulated image when no position prediction is involved on the right. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. a: effect of position prediction on the quality of the MR images in the head (s = 0.02), body (s = 0.5) and tail (s = 1) regions. The prediction horizon in this case is 83 msec. b: dependence of the MR imaging quality (structural similarity) on the position of the slice. The figure displays the simulation results from eight videos of different worms. https://doi.org/10.1371/journal.pcbi.1006997.g006 Clearly, the prediction algorithm significantly reduces the motion artifacts that would otherwise occur if the gradient system did not follow the worm as it moves. Moreover, we observe that prediction quality varies along the worm’s body. More specifically, the prediction performs better for the slices from the body and tail when compared with prediction of the head. This is axiomatic, since the worm rapidly moves its head laterally whilst scavenging for nutrition, thus the motional entropy of the head is higher (see also Fig 4). Fig 6b shows the simulation results of eight videos of different worms. The abscissa represents the slice position starting from the head (s = 0) to the tail (s = 1), while the ordinate shows the structural similarity between the prediction-based MR images and the original slices. To very good agreement with Figs 4 and 6b shows that the quality of the prediction-based MR images are higher for slices from the body and tail than the images taken near the head where the prediction uncertainty is usually higher. Image resolution (Simulation 2). In Simulation 2, a slice from the middle body section of the worm (s = 0.72) was selected and an MRI simulation was performed for different resolution ratios (= MR image: optical video). The idea here is to explore the effect of changing the MRI resolution (voxel size) on the quality of the prediction-based imaging. Imagine, for example, that one pixel of the optical video is 10 μm × 10 μm and the desired MRI isotropic resolution is also 10 μm × μm × 10 μm × 10 μm, then one pixel error in prediction results in one voxel error in the MRI k-space. On the other hand, if higher MRI resolution is wished, then one pixel error in prediction results in more than one voxel error in the MRI k-space. Decreasing the optical resolution can be used to speed up image processing, and thus decreasing the prediction horizon, if needed. The ratios of 4:1 and 2:1 are heuristically chosen, for which each pixel in the optical video respectively corresponds to 4 and 2 pixels in the MR image. Fig 7 demonstrates the results of this simulation—the rows correspond to the different resolution ratios while the columns depict the original slice, the image with prediction, and the image without prediction. The figure shows that the efficiency of the prediction declines as the desired resolution of the MR image increases (pixel size decreases), or alternatively, the optical resolution should be higher or at least equal to the desired MR resolution for a high quality prediction-based MR image. Indeed, the commercial optical imaging solutions can easily meet such demands of high resolution; however, upon implementation, any increase in number of pixels will be at the expense of the prolonged processing time needed for prediction. Nevertheless, this can be overcome by utilizing more powerful processors and by employing techniques such as parallel computing. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 7. Effect of decreasing the ratio of the MR image resolution to the optical video resolution on the imaging quality for a prediction horizon of 1. The rows show the results for MR image:video resolution ratios of 4:1 and 2:1 respectively. Each row displays, from left to right, the reference slice, the MR image with position prediction, and the MR image without position prediction. https://doi.org/10.1371/journal.pcbi.1006997.g007 Prediction horizon (Simulation 3). Simulation 3 varies prediction horizons for one slice (s = 0.52). Fig 8 illustrates the results of this simulation: Fig. (1a–10a) are the prediction-based simulated images for prediction horizons of Δk 1 to 10, respectively. In contrast, Fig. (1b–10b) display the results when prediction is not in action. Because only a few lines of the k-space are collected from the correct slice (depending on TR and the speed of the worm), the images in this figure exhibit a noticeable decrease in quality as the prediction horizon increases, leading to the conclusion that one should, whenever possible, minimize the prediction horizon. Of course, in an actual hardware realization of the system, the choice of the prediction horizon will be bounded by the speed of the image acquisition and processing units. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 8. Effect of enlarging the prediction horizon on the quality of the imaging (s = 0.52, see Fig 6). The images (1a-10a) show the reconstructed MR images based on the prediction algorithm for prediction horizons from 1 to 10 frames. The images (1b-10b) show the reconstructed MR images when no prediction is involved and when the initial position of the imaging gradient is set to the position of the slice after 1 (83 ms) to 10 (830 ms) frames respectively. (c) The similarity measure of the simulated image versus the prediction horizon. https://doi.org/10.1371/journal.pcbi.1006997.g008 The effect of increasing the prediction horizon is described quantitatively in Fig 8c by measuring the structural similarity between the original slice and the simulated image for both cases with and without prediction. The results confirm the variation in predictability along the length of the worm. The prediction horizon degradation is linear for the rear two-thirds of the worm, but falls off more rapidly for the head section, as expected from Fig 4. Moreover, a statistical assessment of the effect of the prediction horizon on the quality of the MR imaging was done: Regarding simulation results of four slices along the worm using the given eight videos it can be shown that the accuracy of prediction and thus the MRI quality decays with the increased prediction horizon but is in mean for all parameter combinations roughly three times better than without prediction. Discussion The results demonstrate that our prediction algorithm can markedly improve MR image quality of arbitrarily moving and deforming objects. The complete processing pipeline (including functions which will not be used in real-time processing, e.g. rotation, prediction of COG, bounding boxes etc.) takes 76 msec per frame on an average Laptop PC (core I7, fifth generation, 16 GB RAM) and has various possibilities for optimization (modern hardware, software parallelisation, hardware-filtering etc.). Decreasing the resolution of the optical video offers an additional option to reduce processing time up to a factor of 8 without significant loss of quality. Based on the current implementation, real-time processing is already possible for sample frequencies smaller than 13 Hz and application of the algorithm to a real MRI system is feasible. For the predictive information to be useful, the characteristic time (velocity) of sample motion tmotion and predictive accuracy needs to be determined. This can be done by recording a freely moving sample with the desired optical setup followed by an image analysis routine as described in this report. Models of the motion kinematics can be tested in order to maximize the prediction horizon given a user-specified predictive accuracy threshold. The threshold can be selected based on the expected error introduced within a single voxel with pre-selected dimensions. The predictive horizon together with the calculation time tpred then can be used to determine if the sample is ‘MR imageable’, i.e. by comparing the timescales to those required for MRI. For example, in this report it was observed that a prediction horizon of 83 ms requiring a calculation time of 76 ms yielded a predictive accuracy of approximately 57% as indicated by the structural similarity measure of the prediction-based MRI simulation, Fig 8c. On the other hand, a prediction horizon of 830 ms requiring a calculation time of 76 ms resulted in a prediction accuracy of approximately 30%. With the timescales of motion and predictive calculation defined, one must evaluate whether MRI is possible by comparing to the instrumental timescale (Only a standard cartesian MRI sequence is considered. The authors acknowledge the existence of more advanced sampling schemes, but these are outside the scope of the present discussion). The shortest relevant timescale is the time between spatial encoding steps, which in the case exemplified here is the repetition time TR (TR includes TE and the data acquisition time). It is during TR that prediction and hardware adjustments must be done prior to the subsequent spatial encoding step. In the MRI simulations described here, TR was 83 ms (including a TE of 4 ms) while tpred was 76 ms. Given the organism motion and hardware/experiment timescale regimes, one can estimate the potential for sample imaging with correction, summarized as follows: tmotion < TR: the object is not MR imageable without motion artifacts. Conditions to slow the natural motion of the sample should be identified and implemented. TR < tmotion, tpred: the object is MR imageable. Careful choice of TE and TR must be done so that the prediction calculation is complete before the next spatial encoding period. This places a restriction on the types of contrast that can be implemented. tpred < TR < tmotion: the object is MR imageable. There is no restriction on the contrast weightings that can be implemented. To further improve the predictive quality, it is important to have kinematic models appropriate to the organism to be imaged. C. elegans is a convenient model for this reason given the advanced studies about behavioural phenotypes [11, 12, 20] and motion decomposition using so-called Eigenworms [14]. Extension of these models to organisms featuring similar motion characteristics should be straightforward (i.e. oscillation/undulation along the long axis—worms, snakes, swimming fish). As the kinematics becomes more complicated and/or sporadic, it will become necessary to introduce a method to guide the organism in order to introduce a predictive nature to its motion (food source, temperature gradient, etc.). Alternatively, shorter prediction horizons can be targeted together with faster MR imaging sequences such as the echo planar imaging (EPI), most likely at the expense of spatial resolution. To bring the presented method of in-situ real-time video capturing and motion prediction experiments for MR microscopy into daily routine, we so far lack available MR-compatible optical microscopes for high-field vertical bore magnets. However, there is strong progress to reach this technical integration in the near future. Initial results have already been presented in the early 2010s and a patent describing the adaptive objective to be integrated into an optical system has been granted in 2019 under the title “MR-compatible microscope, EP 2824471B1 (see e.g. https://worldwide.espacenet.com)”. Once the remaining hardware challenges are overcome the presented procedure for a tracking control loop will give the possibility to fully exploit and expedite these novel techniques for conducting MR microscopy of free-moving, undisturbed microscopic organisms. Supporting information S1 Text. This file describes the principal structure of the motion prediction algorithm. https://doi.org/10.1371/journal.pcbi.1006997.s001 (PDF) S1 Fig. Structural similarity versus prediction horizon for slices (s = 0.1, s = 0.3, s = 0.5, and s = 0.9) for eight videos. https://doi.org/10.1371/journal.pcbi.1006997.s002 (TIF) Acknowledgments We sincerely thank Dr. David H. Hall of Albert Einstein College of Medicine in New York for providing the high resolution images of C. elegans that we used as phantoms for high resolution NMR images. These lent our images more realism. The original images are from the MRC/LMB C. elegans Archive now curated by the Hall lab. These were generously donated by John White and Jonathan Hodgkin to the Hall lab.

Yang, Mengyun;Luo, Huimin;Li, Yaohang;Wu, Fang-Xiang;Wang, Jianxin

doi: 10.1371/journal.pcbi.1007541pmid: 31869322

Introduction The development of new drugs is extremely time-consuming and expensive [1]. It is reported that the average time of developing a new drug is more than 13.5 years and the cost exceeds $1.8 billion dollars [2], while only a relatively small number of novel drugs are approved by US Food and Drug Administration (FDA) each year. Identifying new uses of existing drugs, known as drug repositioning, has been popularly used for the pharmaceutical industry and research community. Since the existing drugs have already owned safety, efficacy, and toleration data after numerous experiments and clinical trials, identifying new and reliable indications for commercialized drugs can sharply reduce time and costs. In addition, some successfully repositioned drugs, such as raloxifene, sildenafil, and thalidomide, have produced great revenues for their patent companies. Hence, drug repositioning is an important strategy of drug discovery in pharmaceutical industry. The computational methods for drug repositioning have received much attention recently, as the traditional manual experimental investigation is complicated and inefficient. In recent years, many types of computational approaches have been proposed, including semantic inference, network-based analysis, and machine learning. The network-based methods are one of the popularly-used approaches to identify potential drug–disease associations. Based on the guilt-by-association principle, Wang et al. constructed a heterogenous graph between drug and target and proposed the HGBI (Heterogeneous Graph Based Inference) algorithm to predict potential drug–target interactions [3]. The HGBI algorithm is also used for prediction of drug–disease associations [4]. Based on the propagation flow algorithm, Martinez et al. proposed a network-based prioritization method named DrugNet for drug repositioning [5]. The DrugNet algorithm can perform both disease–drug and drug–disease prioritization by integrating drug, disease, and target information. In [6], the MBiRW method addressed the drug-repositioning problem by applying a bi-random walk algorithm on heterogeneous network with comprehensive similarity measures for drugs and diseases, obtained by utilizing logistic function [7] and ClusterONE [8]. Machine learning methods have attracted a lot of attention in recent years. Based on the common assumption that similar drugs tend to connect with similar diseases, Gottlieb et al. calculated five drug–drug similarity measures and two disease–disease similarity measures for drug-associated indication prediction, and presented a method (PREDICT) to identify potential drug indications for approved drugs [9]. Integrating chemical structure, drug–target interaction, and side-effect data, Wang et al. presented an approach called PreDR for drug–disease association prediction [10]. PreDR treated the prediction problem as a binary classification problem by defining a kernel function and applying an SVM-based learning algorithm. In [11], a matrix factorization model was developed to predict new indications for known drugs by incorporating the interaction network of genes. Luo et al. proposed a drug repositioning recommendation system (DRRS) [12]. Specifically, a heterogeneous network was constructed by integrating drug similarities, disease similarities, and drug–disease associations and the adjacency matrix of the large-scale heterogeneous network was considered as a low-rank matrix. The singular value thresholding algorithm (SVT) [13] was implemented to complete the missing entries of a drug–disease association matrix. Yang et al. further proposed a bounded nuclear norm regularization (BNNR) model [14], not only tolerating the noisy similarities of drugs and diseases by employing regularization, but also ensuring that all predicted values are within the interval of [0, 1]. However, the computational cost of both DRRS and BNNR increases sharply when target (protein/gene) information is incorporated into the heterogeneous drug–disease network. In this study, we propose an overlap matrix completion for bilayer networks (OMC2) and tri-layer networks (OMC3) to predict potential indications for approved and new drugs. We design two different networks from drug-side aspect and disease-side aspect, instead of constructing a large-scale heterogeneous drug–disease network. This can significantly reduce the computational complexity for matrix completion. Meanwhile, a BNNR model [14] developed in our previous work is implemented to fill out the missing entries in the block adjacency matrix of these networks. We evaluate the performance of OMC2 and OMC3 in three different datasets and compare them with five latest approaches for drug repositioning. Our computational results show that our methods yield better accuracy in predicting potential drug–disease associations. Materials and methods In this section, we introduce OMC for bilayer networks (OMC2) and tri-layer networks (OMC3) to identify potential indications for both known and novel drugs. First of all, a concise description of experimental datasets is presented. In bilayer heterogeneous networks, we integrate drug–drug, disease–disease, and drug–disease information. In tri-layer heterogeneous networks, besides the above three kinds of data, drug–protein and disease–protein associations are considered. Then, we present the OMC2 algorithm for drug–disease bilayer networks to predict novel drug–disease associations. Finally, we extend OMC2 to an OMC3 algorithm in handling the tri-layer networks, where the target-related information is also incorporated. Datasets To construct bilayer and tri-layer networks, we collected drug, disease, and target protein information from published literatures and related authoritative databases. The approaches to collect association information and to compute similarity are described below. Drug–disease associations. Confirmed drug–disease associations were obtained from the supplementary material of [9], which was admittedly treated as the gold standard dataset. There were 1, 933 associations between 593 drugs registered from DrugBank [15] and 313 diseases listed in the Online Mendelian Inheritance in Man (OMIM) database [16]. Drug–drug similarity. Drug–drug similarities were calculated based on chemical structures. The Canonical Simplified Molecular Input Line-Entry System (SMILES) [17] of these 593 drugs were downloaded from DrugBank. Then, the Chemical Development Kit (CDK) [18] was utilized to compute hashed fingerprints for each drug with default parameters. Finally, the similarity between two drugs was measured by the Tanimoto score [19] in the range of [0, 1]. Disease–disease similarity. Disease–disease similarities were computed by MimMiner [20], which identifies similarity of appearance of MESH (medical subject headings vocabulary) terms between two diseases in medical descriptions from the OMIM database. In the MimMiner program, the disease–disease similarity was normalized to the interval of [0, 1]. Drug–protein interactions. The interactions between drugs and proteins were collected from DrugBank. We collected 3, 184 drug–target (protein) interactions between 576 relevant drugs of the gold standard dataset and 975 proteins. Disease–protein associations. We collected disease–protein associations in two steps. In the first step, we downloaded the interactions between diseases included in the gold standard dataset and genes from CTD [21], and the total of 475 disease–gene interactions were collected. Secondly, these genes were mapped into 849 proteins in UniprotKB database [22]. There were 1, 066 associations between 166 diseases and 849 proteins at last. OMC algorithm for bilayer networks Two drug–disease bilayer networks and corresponding adjacency matrices. We construct two heterogeneous drug–disease bilayer networks. One is composed of a drug–drug network and a drug–disease network and the other is of a disease–disease network and a drug–disease network. Fig 1 shows the workflow for constructing two bilayer networks and their corresponding block adjacency matrices. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. The workflow of constructing the DrNet-Dis network and the DisNet-Dr network. (a) Drug–drug network and its similarity matrix. (b) Drug–disease associations and KNN preprocessing. (c) Disease–disease network and its similarity matrix. (d) DrNet-Dis network and its block adjacency matrix. (e) DisNet-Dr network and its block adjacency matrix. https://doi.org/10.1371/journal.pcbi.1007541.g001 For the drug–drug network with m drug nodes, let be its adjacency matrix, where element (ARR)ij represents the similarity between drugs ri and rj. Similarly, is the adjacency matrix of the disease–disease network with n disease nodes, where (ADD)ij denotes the similarity between diseases di and dj. For the drug–disease network, let be its adjacency matrix (drug–disease association matrix), where (ADR)ij is set to 1 if there exists an experimentally validated association between di and rj, otherwise 0. DrNet-Dis network. The DrNet-Dis network, illustrated in Fig 1(a), 1(b) and 1(d), is constructed by integrating the drug–drug network and the drug–disease network. For the sake of generality in applications, we take some novel disease nodes into account, which are not associated with any known drug node. For instance, d4 is a new disease node in Fig 1(b), and the corresponding row of ADR is a zero vector, which causes difficulty in matrix completion and affects the performance of prediction. To address this cold-start problem, we conduct a K-Nearest Neighbor (KNN) preprocessing step for these new diseases. Specifically, for each novel disease dp, K nearest neighbor diseases of dp are picked based on their disease similarities in descending order. We update the corresponding row vector of disease dp in the drug–disease association matrix by filling out a part of weighted association information. The detail of the KNN preprocessing algorithm is described by Algorithm 1. After the KNN preprocessing step, an updated drug–disease association matrix ADR1 is obtained and the block adjacency matrix of the DrNet-Dis network is presented as follows, DisNet-Dr network. The DisNet-Dr network, demonstrated by Fig 1(b), 1(c) and 1(e), is constructed by integrating the disease–disease network and the drug–disease network. For some novel drugs (e.g., drug r2 in Fig 1(b)), the corresponding columns of ADR are zero vectors. Similarly, the KNN preprocessing step is also implemented for these new drugs by Algorithm 1, and a new corresponding association matrix ADR2 is developed. Finally, the block adjacency matrix of the DisNet-Dr network is denoted as follows, Actually, the above KNN preprocessing step is not required if there is no novel disease or drug node. M1 and M2 are the to-be-complete matrices. Algorithm 1: KNN Preprocessing Algorithm Input: The drug similarity matrix , the disease similarity matrix , the disease–drug association matrix may contain some zero rows or columns, and the neighborhood size K. Output: Updated ADR1 and ADR2. 1. Initialize ADR1 = ADR and ADR2 = ADR; 2. Find index numbers of all zero rows of the matrix ADR1, which are denoted as {i1, i2, …, is} ⊂ {1, 2, …, m}. represents the corresponding disease set. /* Entries of D0 actually are novel diseases, where represents i1-th disease in all diseases.*/ for each disease dp ∈ D0 do 3. ; /* KNN is a function for finding the K nearest neighbors of disease node dp based on similarity matrix ADD in descending order.*/ 4. ; 5. ; /*ADR1(p, :) notes the p-th row of matrix ADR1 and the denominator is the normalization term.*/ end for 6. Find index numbers of all zero columns of the matrix ADR2, which are denoted as {j1, j2, …, jt} ⊂ {1, 2, …, n}. represents the corresponding drug set. /*Entries of R0 actually are novel drugs, where represents the j1-th drug in all drugs.*/ for each drug rq ∈ R0 do 7. ; /* KNN is a function for finding the K nearest neighbors of drug node rq based on similarity matrix ARR in descending order.*/ 8. ; 9. ; /*ADR2(:, q) notes the q-th column of matrix ADR2 and the denominator is the normalization term.*/ end for 10. return ADR1 and ADR2. BNNR model. Matrix completion, whose goal is to recover the missing elements of matrix from only a few observations, has been widely used in many applications. Under the low-rank assumption, matrix completion is generally formulated as the following nuclear norm minimization problem (1) where ‖X‖* denotes the nuclear norm of X, which is defined as the sum of all singular values of X. M is the incomplete matrix, Ω is a set including index pairs (i, j) of all known elements in M, and is the projection operator projecting matrix X onto Ω, which is defined as In the drug–disease association matrix, the entry value 1 denotes an experimentally validated indication while 0 indicates the association has not been validated yet. As a result, the predicted drug–disease association values are expected to fall in the interval of [0, 1], indicating the likelihood of being a true association. Therefore, a predicted value beyond the [0, 1] range is meaningless in the context of the application. To enforce the predicted values within the interval of [0, 1], a bounded constraint is added into the matrix completion model. In addition, due to the large amount of “noise” when calculating drug similarity and disease similarity, we relax the constraint satisfaction condition by incorporating a regularization term. As a result, we have proposed the bounded nuclear norm regularization (BNNR) described in [14] as follows, (2) where α > 0 is a harmonic parameter to balance the nuclear norm and the error term and 0 ≤ X ≤ 1 represents 0 ≤ Xij ≤ 1 for all i, j. A simple and effective algorithm is designed to solve model (2) by using the alternating direction method of multipliers (ADMM). By introducing a new splitting matrix W, (2) can be formulated as the following equivalent form, (3) The augmented Lagrangian function of model (3) is (4) where Y is the Lagrange multiplier and β > 0 is the penalty parameter. By applying ADMM, we can obtain the following iterative scheme: (5) (6) (7) We use the inverse operator [23] to solve Eq (5) and acquire a closed-form solution W* as follows, where denotes the identity operator. Moreover, to limit the element values of Wk+1 in the interval of [0, 1], we utilize the following projection operator (8) where is defined as By rearranging the terms of (6), we have (9) where is the singular value shrinkage (SVT) operator [13] [24]. Specifically, SVT operator is defined as where σi is the ith singular value of X larger than threshold τ, while ui and vi are the left and right singular vectors corresponding to σi, respectively. Algorithm 2 presents an iterative BNNR scheme for solving the model (2). After performing BNNR algorithm, we can obtain a completed matrix M*, where all the unknown entries of matrix M have been filled out. Algorithm 2: BNNR Algorithm Input: The to-be-complete M, parameters α, and β. Output: Completed matrix M*. 1. initialize X1 = PΩ(M), W1 = X1, Y1 = X1; 2. k ← 1; repeat 3. 4. 5. Yk+1 ← Yk + β(Xk+1 − Wk+1); 6. k ← k + 1; until convergence 7. M* = Wk; 8. return M*. OMC2 algorithm. We propose the OMC algorithm for bilayer networks (OMC2) to predict the potential drug–disease associations, whose goal is to obtain the low-rank matrices of drug–disease relationships from drug-side information and disease-side information. Firstly, we combine the updated disease–drug association matrix with the drug similarity matrix and create a block adjacency matrix M1, as illustrated in Fig 1(d). Meanwhile, from the disease-side, we combine the updated disease–drug association matrix with the disease similarity matrix and generate a block adjacency matrix M2, as illustrated in Fig 1(e). Secondly, the BNNR algorithm is implemented to fill out the unknown entries of M1 and M2. Finally, we calculate the average of two predicted drug–disease association matrices to obtain the final predicted matrix . Each element represents the predicted score between disease di and drug rj. The higher the score, the more likely that the association exists. To identify the promising candidate indicates for a specific drug, we rank all candidates according to their scores in descending order. The detail of the OMC2 algorithm is described in Algorithm 3. Algorithm 3: OMC2 Algorithm Input: The drug similarity matrix , the disease similarity matrix , the disease–drug association matrix , parameters K, α, and β. Outout: Predicted association matrix . 1. ; 2. 3. ; 4. 5. ; 6. ; 7. ; 8. return . OMC algorithm for tri-layer networks OMC can be easily extended from bilayer networks (OMC2) to tri-layer networks (OMC3) algorithm, where the disease–protein and drug–protein association information are incorporated to further improve prediction accuracy. Firstly, we collect drug–protein (target) interactions and disease–protein associations from different databases. This step has been discussed in the previous section. Secondly, based on the two bilayer networks, i.e., the DrNet-Dis network and the DisNet-Dr network, we design two corresponding tri-layer networks. We integrate protein nodes and drug–protein associations into the DrNet-Dis network and construct a drug–protein–disease network called DrNet-Pro-Dis, as showed in Fig 2(e). The block adjacency matrix of this tri-layer network is defined as Similarly, we integrate protein nodes and disease–protein associations into the DisNet-Dr network and create another tri-layer network called DisNet-Pro-Dr, as illustrated in Fig 2(f). The block adjacency matrix of DisNet-Pro-Dr network is defined as Thirdly, the BNNR algorithm is carried out to fill out the missing entries of M1 and M2 to obtain two predicted drug–disease association matrices. Finally, we calculate the average of these two matrices as the final output. The detail of OMC3 the algorithm is described in Algorithm 4. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. The workflow of constructing the DrNet-Pro-Dis network and the DisNet-Pro-Dr network. (a) DrNet-Dis network and its similarity matrix. (b) Drug–protein interactions and corresponding adjacency matrix. (c) Disease–protein associations and corresponding adjacency matrix. (d) DisNet-Dr network and its block adjacency matrix. (e) DrNet-Pro-Dis network and its block adjacency matrix. (f) DisNet-Pro-Dr network and its block adjacency matrix. https://doi.org/10.1371/journal.pcbi.1007541.g002 Algorithm 4: OMC3 Algorithm Input: Drug similarity matrix , disease similarity matrix , protein–drug association matrix , disease–protein association matrix , disease–drug association matrix , parameters K, α, and β. Output: Predicted association matrix . 1. ; 2. 3. ; 4. 5. ; 6. ; 7. ; 8. return . Datasets To construct bilayer and tri-layer networks, we collected drug, disease, and target protein information from published literatures and related authoritative databases. The approaches to collect association information and to compute similarity are described below. Drug–disease associations. Confirmed drug–disease associations were obtained from the supplementary material of [9], which was admittedly treated as the gold standard dataset. There were 1, 933 associations between 593 drugs registered from DrugBank [15] and 313 diseases listed in the Online Mendelian Inheritance in Man (OMIM) database [16]. Drug–drug similarity. Drug–drug similarities were calculated based on chemical structures. The Canonical Simplified Molecular Input Line-Entry System (SMILES) [17] of these 593 drugs were downloaded from DrugBank. Then, the Chemical Development Kit (CDK) [18] was utilized to compute hashed fingerprints for each drug with default parameters. Finally, the similarity between two drugs was measured by the Tanimoto score [19] in the range of [0, 1]. Disease–disease similarity. Disease–disease similarities were computed by MimMiner [20], which identifies similarity of appearance of MESH (medical subject headings vocabulary) terms between two diseases in medical descriptions from the OMIM database. In the MimMiner program, the disease–disease similarity was normalized to the interval of [0, 1]. Drug–protein interactions. The interactions between drugs and proteins were collected from DrugBank. We collected 3, 184 drug–target (protein) interactions between 576 relevant drugs of the gold standard dataset and 975 proteins. Disease–protein associations. We collected disease–protein associations in two steps. In the first step, we downloaded the interactions between diseases included in the gold standard dataset and genes from CTD [21], and the total of 475 disease–gene interactions were collected. Secondly, these genes were mapped into 849 proteins in UniprotKB database [22]. There were 1, 066 associations between 166 diseases and 849 proteins at last. OMC algorithm for bilayer networks Two drug–disease bilayer networks and corresponding adjacency matrices. We construct two heterogeneous drug–disease bilayer networks. One is composed of a drug–drug network and a drug–disease network and the other is of a disease–disease network and a drug–disease network. Fig 1 shows the workflow for constructing two bilayer networks and their corresponding block adjacency matrices. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. The workflow of constructing the DrNet-Dis network and the DisNet-Dr network. (a) Drug–drug network and its similarity matrix. (b) Drug–disease associations and KNN preprocessing. (c) Disease–disease network and its similarity matrix. (d) DrNet-Dis network and its block adjacency matrix. (e) DisNet-Dr network and its block adjacency matrix. https://doi.org/10.1371/journal.pcbi.1007541.g001 For the drug–drug network with m drug nodes, let be its adjacency matrix, where element (ARR)ij represents the similarity between drugs ri and rj. Similarly, is the adjacency matrix of the disease–disease network with n disease nodes, where (ADD)ij denotes the similarity between diseases di and dj. For the drug–disease network, let be its adjacency matrix (drug–disease association matrix), where (ADR)ij is set to 1 if there exists an experimentally validated association between di and rj, otherwise 0. DrNet-Dis network. The DrNet-Dis network, illustrated in Fig 1(a), 1(b) and 1(d), is constructed by integrating the drug–drug network and the drug–disease network. For the sake of generality in applications, we take some novel disease nodes into account, which are not associated with any known drug node. For instance, d4 is a new disease node in Fig 1(b), and the corresponding row of ADR is a zero vector, which causes difficulty in matrix completion and affects the performance of prediction. To address this cold-start problem, we conduct a K-Nearest Neighbor (KNN) preprocessing step for these new diseases. Specifically, for each novel disease dp, K nearest neighbor diseases of dp are picked based on their disease similarities in descending order. We update the corresponding row vector of disease dp in the drug–disease association matrix by filling out a part of weighted association information. The detail of the KNN preprocessing algorithm is described by Algorithm 1. After the KNN preprocessing step, an updated drug–disease association matrix ADR1 is obtained and the block adjacency matrix of the DrNet-Dis network is presented as follows, DisNet-Dr network. The DisNet-Dr network, demonstrated by Fig 1(b), 1(c) and 1(e), is constructed by integrating the disease–disease network and the drug–disease network. For some novel drugs (e.g., drug r2 in Fig 1(b)), the corresponding columns of ADR are zero vectors. Similarly, the KNN preprocessing step is also implemented for these new drugs by Algorithm 1, and a new corresponding association matrix ADR2 is developed. Finally, the block adjacency matrix of the DisNet-Dr network is denoted as follows, Actually, the above KNN preprocessing step is not required if there is no novel disease or drug node. M1 and M2 are the to-be-complete matrices. Algorithm 1: KNN Preprocessing Algorithm Input: The drug similarity matrix , the disease similarity matrix , the disease–drug association matrix may contain some zero rows or columns, and the neighborhood size K. Output: Updated ADR1 and ADR2. 1. Initialize ADR1 = ADR and ADR2 = ADR; 2. Find index numbers of all zero rows of the matrix ADR1, which are denoted as {i1, i2, …, is} ⊂ {1, 2, …, m}. represents the corresponding disease set. /* Entries of D0 actually are novel diseases, where represents i1-th disease in all diseases.*/ for each disease dp ∈ D0 do 3. ; /* KNN is a function for finding the K nearest neighbors of disease node dp based on similarity matrix ADD in descending order.*/ 4. ; 5. ; /*ADR1(p, :) notes the p-th row of matrix ADR1 and the denominator is the normalization term.*/ end for 6. Find index numbers of all zero columns of the matrix ADR2, which are denoted as {j1, j2, …, jt} ⊂ {1, 2, …, n}. represents the corresponding drug set. /*Entries of R0 actually are novel drugs, where represents the j1-th drug in all drugs.*/ for each drug rq ∈ R0 do 7. ; /* KNN is a function for finding the K nearest neighbors of drug node rq based on similarity matrix ARR in descending order.*/ 8. ; 9. ; /*ADR2(:, q) notes the q-th column of matrix ADR2 and the denominator is the normalization term.*/ end for 10. return ADR1 and ADR2. BNNR model. Matrix completion, whose goal is to recover the missing elements of matrix from only a few observations, has been widely used in many applications. Under the low-rank assumption, matrix completion is generally formulated as the following nuclear norm minimization problem (1) where ‖X‖* denotes the nuclear norm of X, which is defined as the sum of all singular values of X. M is the incomplete matrix, Ω is a set including index pairs (i, j) of all known elements in M, and is the projection operator projecting matrix X onto Ω, which is defined as In the drug–disease association matrix, the entry value 1 denotes an experimentally validated indication while 0 indicates the association has not been validated yet. As a result, the predicted drug–disease association values are expected to fall in the interval of [0, 1], indicating the likelihood of being a true association. Therefore, a predicted value beyond the [0, 1] range is meaningless in the context of the application. To enforce the predicted values within the interval of [0, 1], a bounded constraint is added into the matrix completion model. In addition, due to the large amount of “noise” when calculating drug similarity and disease similarity, we relax the constraint satisfaction condition by incorporating a regularization term. As a result, we have proposed the bounded nuclear norm regularization (BNNR) described in [14] as follows, (2) where α > 0 is a harmonic parameter to balance the nuclear norm and the error term and 0 ≤ X ≤ 1 represents 0 ≤ Xij ≤ 1 for all i, j. A simple and effective algorithm is designed to solve model (2) by using the alternating direction method of multipliers (ADMM). By introducing a new splitting matrix W, (2) can be formulated as the following equivalent form, (3) The augmented Lagrangian function of model (3) is (4) where Y is the Lagrange multiplier and β > 0 is the penalty parameter. By applying ADMM, we can obtain the following iterative scheme: (5) (6) (7) We use the inverse operator [23] to solve Eq (5) and acquire a closed-form solution W* as follows, where denotes the identity operator. Moreover, to limit the element values of Wk+1 in the interval of [0, 1], we utilize the following projection operator (8) where is defined as By rearranging the terms of (6), we have (9) where is the singular value shrinkage (SVT) operator [13] [24]. Specifically, SVT operator is defined as where σi is the ith singular value of X larger than threshold τ, while ui and vi are the left and right singular vectors corresponding to σi, respectively. Algorithm 2 presents an iterative BNNR scheme for solving the model (2). After performing BNNR algorithm, we can obtain a completed matrix M*, where all the unknown entries of matrix M have been filled out. Algorithm 2: BNNR Algorithm Input: The to-be-complete M, parameters α, and β. Output: Completed matrix M*. 1. initialize X1 = PΩ(M), W1 = X1, Y1 = X1; 2. k ← 1; repeat 3. 4. 5. Yk+1 ← Yk + β(Xk+1 − Wk+1); 6. k ← k + 1; until convergence 7. M* = Wk; 8. return M*. OMC2 algorithm. We propose the OMC algorithm for bilayer networks (OMC2) to predict the potential drug–disease associations, whose goal is to obtain the low-rank matrices of drug–disease relationships from drug-side information and disease-side information. Firstly, we combine the updated disease–drug association matrix with the drug similarity matrix and create a block adjacency matrix M1, as illustrated in Fig 1(d). Meanwhile, from the disease-side, we combine the updated disease–drug association matrix with the disease similarity matrix and generate a block adjacency matrix M2, as illustrated in Fig 1(e). Secondly, the BNNR algorithm is implemented to fill out the unknown entries of M1 and M2. Finally, we calculate the average of two predicted drug–disease association matrices to obtain the final predicted matrix . Each element represents the predicted score between disease di and drug rj. The higher the score, the more likely that the association exists. To identify the promising candidate indicates for a specific drug, we rank all candidates according to their scores in descending order. The detail of the OMC2 algorithm is described in Algorithm 3. Algorithm 3: OMC2 Algorithm Input: The drug similarity matrix , the disease similarity matrix , the disease–drug association matrix , parameters K, α, and β. Outout: Predicted association matrix . 1. ; 2. 3. ; 4. 5. ; 6. ; 7. ; 8. return . Two drug–disease bilayer networks and corresponding adjacency matrices. We construct two heterogeneous drug–disease bilayer networks. One is composed of a drug–drug network and a drug–disease network and the other is of a disease–disease network and a drug–disease network. Fig 1 shows the workflow for constructing two bilayer networks and their corresponding block adjacency matrices. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. The workflow of constructing the DrNet-Dis network and the DisNet-Dr network. (a) Drug–drug network and its similarity matrix. (b) Drug–disease associations and KNN preprocessing. (c) Disease–disease network and its similarity matrix. (d) DrNet-Dis network and its block adjacency matrix. (e) DisNet-Dr network and its block adjacency matrix. https://doi.org/10.1371/journal.pcbi.1007541.g001 For the drug–drug network with m drug nodes, let be its adjacency matrix, where element (ARR)ij represents the similarity between drugs ri and rj. Similarly, is the adjacency matrix of the disease–disease network with n disease nodes, where (ADD)ij denotes the similarity between diseases di and dj. For the drug–disease network, let be its adjacency matrix (drug–disease association matrix), where (ADR)ij is set to 1 if there exists an experimentally validated association between di and rj, otherwise 0. DrNet-Dis network. The DrNet-Dis network, illustrated in Fig 1(a), 1(b) and 1(d), is constructed by integrating the drug–drug network and the drug–disease network. For the sake of generality in applications, we take some novel disease nodes into account, which are not associated with any known drug node. For instance, d4 is a new disease node in Fig 1(b), and the corresponding row of ADR is a zero vector, which causes difficulty in matrix completion and affects the performance of prediction. To address this cold-start problem, we conduct a K-Nearest Neighbor (KNN) preprocessing step for these new diseases. Specifically, for each novel disease dp, K nearest neighbor diseases of dp are picked based on their disease similarities in descending order. We update the corresponding row vector of disease dp in the drug–disease association matrix by filling out a part of weighted association information. The detail of the KNN preprocessing algorithm is described by Algorithm 1. After the KNN preprocessing step, an updated drug–disease association matrix ADR1 is obtained and the block adjacency matrix of the DrNet-Dis network is presented as follows, DisNet-Dr network. The DisNet-Dr network, demonstrated by Fig 1(b), 1(c) and 1(e), is constructed by integrating the disease–disease network and the drug–disease network. For some novel drugs (e.g., drug r2 in Fig 1(b)), the corresponding columns of ADR are zero vectors. Similarly, the KNN preprocessing step is also implemented for these new drugs by Algorithm 1, and a new corresponding association matrix ADR2 is developed. Finally, the block adjacency matrix of the DisNet-Dr network is denoted as follows, Actually, the above KNN preprocessing step is not required if there is no novel disease or drug node. M1 and M2 are the to-be-complete matrices. Algorithm 1: KNN Preprocessing Algorithm Input: The drug similarity matrix , the disease similarity matrix , the disease–drug association matrix may contain some zero rows or columns, and the neighborhood size K. Output: Updated ADR1 and ADR2. 1. Initialize ADR1 = ADR and ADR2 = ADR; 2. Find index numbers of all zero rows of the matrix ADR1, which are denoted as {i1, i2, …, is} ⊂ {1, 2, …, m}. represents the corresponding disease set. /* Entries of D0 actually are novel diseases, where represents i1-th disease in all diseases.*/ for each disease dp ∈ D0 do 3. ; /* KNN is a function for finding the K nearest neighbors of disease node dp based on similarity matrix ADD in descending order.*/ 4. ; 5. ; /*ADR1(p, :) notes the p-th row of matrix ADR1 and the denominator is the normalization term.*/ end for 6. Find index numbers of all zero columns of the matrix ADR2, which are denoted as {j1, j2, …, jt} ⊂ {1, 2, …, n}. represents the corresponding drug set. /*Entries of R0 actually are novel drugs, where represents the j1-th drug in all drugs.*/ for each drug rq ∈ R0 do 7. ; /* KNN is a function for finding the K nearest neighbors of drug node rq based on similarity matrix ARR in descending order.*/ 8. ; 9. ; /*ADR2(:, q) notes the q-th column of matrix ADR2 and the denominator is the normalization term.*/ end for 10. return ADR1 and ADR2. BNNR model. Matrix completion, whose goal is to recover the missing elements of matrix from only a few observations, has been widely used in many applications. Under the low-rank assumption, matrix completion is generally formulated as the following nuclear norm minimization problem (1) where ‖X‖* denotes the nuclear norm of X, which is defined as the sum of all singular values of X. M is the incomplete matrix, Ω is a set including index pairs (i, j) of all known elements in M, and is the projection operator projecting matrix X onto Ω, which is defined as In the drug–disease association matrix, the entry value 1 denotes an experimentally validated indication while 0 indicates the association has not been validated yet. As a result, the predicted drug–disease association values are expected to fall in the interval of [0, 1], indicating the likelihood of being a true association. Therefore, a predicted value beyond the [0, 1] range is meaningless in the context of the application. To enforce the predicted values within the interval of [0, 1], a bounded constraint is added into the matrix completion model. In addition, due to the large amount of “noise” when calculating drug similarity and disease similarity, we relax the constraint satisfaction condition by incorporating a regularization term. As a result, we have proposed the bounded nuclear norm regularization (BNNR) described in [14] as follows, (2) where α > 0 is a harmonic parameter to balance the nuclear norm and the error term and 0 ≤ X ≤ 1 represents 0 ≤ Xij ≤ 1 for all i, j. A simple and effective algorithm is designed to solve model (2) by using the alternating direction method of multipliers (ADMM). By introducing a new splitting matrix W, (2) can be formulated as the following equivalent form, (3) The augmented Lagrangian function of model (3) is (4) where Y is the Lagrange multiplier and β > 0 is the penalty parameter. By applying ADMM, we can obtain the following iterative scheme: (5) (6) (7) We use the inverse operator [23] to solve Eq (5) and acquire a closed-form solution W* as follows, where denotes the identity operator. Moreover, to limit the element values of Wk+1 in the interval of [0, 1], we utilize the following projection operator (8) where is defined as By rearranging the terms of (6), we have (9) where is the singular value shrinkage (SVT) operator [13] [24]. Specifically, SVT operator is defined as where σi is the ith singular value of X larger than threshold τ, while ui and vi are the left and right singular vectors corresponding to σi, respectively. Algorithm 2 presents an iterative BNNR scheme for solving the model (2). After performing BNNR algorithm, we can obtain a completed matrix M*, where all the unknown entries of matrix M have been filled out. Algorithm 2: BNNR Algorithm Input: The to-be-complete M, parameters α, and β. Output: Completed matrix M*. 1. initialize X1 = PΩ(M), W1 = X1, Y1 = X1; 2. k ← 1; repeat 3. 4. 5. Yk+1 ← Yk + β(Xk+1 − Wk+1); 6. k ← k + 1; until convergence 7. M* = Wk; 8. return M*. OMC2 algorithm. We propose the OMC algorithm for bilayer networks (OMC2) to predict the potential drug–disease associations, whose goal is to obtain the low-rank matrices of drug–disease relationships from drug-side information and disease-side information. Firstly, we combine the updated disease–drug association matrix with the drug similarity matrix and create a block adjacency matrix M1, as illustrated in Fig 1(d). Meanwhile, from the disease-side, we combine the updated disease–drug association matrix with the disease similarity matrix and generate a block adjacency matrix M2, as illustrated in Fig 1(e). Secondly, the BNNR algorithm is implemented to fill out the unknown entries of M1 and M2. Finally, we calculate the average of two predicted drug–disease association matrices to obtain the final predicted matrix . Each element represents the predicted score between disease di and drug rj. The higher the score, the more likely that the association exists. To identify the promising candidate indicates for a specific drug, we rank all candidates according to their scores in descending order. The detail of the OMC2 algorithm is described in Algorithm 3. Algorithm 3: OMC2 Algorithm Input: The drug similarity matrix , the disease similarity matrix , the disease–drug association matrix , parameters K, α, and β. Outout: Predicted association matrix . 1. ; 2. 3. ; 4. 5. ; 6. ; 7. ; 8. return . OMC algorithm for tri-layer networks OMC can be easily extended from bilayer networks (OMC2) to tri-layer networks (OMC3) algorithm, where the disease–protein and drug–protein association information are incorporated to further improve prediction accuracy. Firstly, we collect drug–protein (target) interactions and disease–protein associations from different databases. This step has been discussed in the previous section. Secondly, based on the two bilayer networks, i.e., the DrNet-Dis network and the DisNet-Dr network, we design two corresponding tri-layer networks. We integrate protein nodes and drug–protein associations into the DrNet-Dis network and construct a drug–protein–disease network called DrNet-Pro-Dis, as showed in Fig 2(e). The block adjacency matrix of this tri-layer network is defined as Similarly, we integrate protein nodes and disease–protein associations into the DisNet-Dr network and create another tri-layer network called DisNet-Pro-Dr, as illustrated in Fig 2(f). The block adjacency matrix of DisNet-Pro-Dr network is defined as Thirdly, the BNNR algorithm is carried out to fill out the missing entries of M1 and M2 to obtain two predicted drug–disease association matrices. Finally, we calculate the average of these two matrices as the final output. The detail of OMC3 the algorithm is described in Algorithm 4. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. The workflow of constructing the DrNet-Pro-Dis network and the DisNet-Pro-Dr network. (a) DrNet-Dis network and its similarity matrix. (b) Drug–protein interactions and corresponding adjacency matrix. (c) Disease–protein associations and corresponding adjacency matrix. (d) DisNet-Dr network and its block adjacency matrix. (e) DrNet-Pro-Dis network and its block adjacency matrix. (f) DisNet-Pro-Dr network and its block adjacency matrix. https://doi.org/10.1371/journal.pcbi.1007541.g002 Algorithm 4: OMC3 Algorithm Input: Drug similarity matrix , disease similarity matrix , protein–drug association matrix , disease–protein association matrix , disease–drug association matrix , parameters K, α, and β. Output: Predicted association matrix . 1. ; 2. 3. ; 4. 5. ; 6. ; 7. ; 8. return . Results In this section, we systematically evaluate the performance of our proposed methods (OMC2 and OMC3) for predicting drug-associated indications. First of all, several evaluation metrics are introduced and parameter settings are discussed. In order to compare our methods with several state-of-the-art approaches, we perform 10-fold cross-validation and de novo tests in the gold standard dataset. Case studies are conducted to confirm the reliability of OMC3 in practical applications. Then, the performance of OMC and comparison on bilayer and tri-layer networks are discussed. Finally, we perform the same experiments on two other datasets to further illustrate the effectiveness and robustness of OMC2 and OMC3. Evaluation metrics To evaluate the performance of our approaches, a 10-fold cross-validation experiment is conducted to identify candidate diseases for specific drugs. In the gold standard dataset, all approved drug–disease associations are randomly divided into ten parts with approximately equal sizes. Each part is treated as the testing set in turn, and the training set is comprised of the remaining nine parts. To obtain convincing results, the 10-fold cross-validation is repeated 10 times and the final result is showed by the average value of the 10 folds. After the performing prediction, all candidate diseases associating with the test drug are ranked by their predicted scores in descending order. For a given rank threshold, the candidate disease is considered as a True Positive (TP) if its rank is above the threshold; otherwise, it is treated as a False Negative (FN). On the other hand, if the rank of a candidate disease had no association with the test drug is greater than the threshold, it is considered as a False Positive (FP), otherwise, it is treated as a True Negative (TN). Based on varying ranking thresholds, we can calculate True Positive Rate (TPR) and False Positive Rate (FPR) by and draw a Receiver Operating Characteristic (ROC) curve. Meanwhile, the area under the ROC curve (AUC) is utilized to evaluate the overall performance of a method. Precision and recall (equivalent to TPR) could be obtained to plot the precision-recall (PR) curve [25]. Due to the fact that the top-ranked result is a more important measurement in real-life drug-repositioning applications, the number of the retrieved correct associations is reported under different top ranking values. Parameter settings In OMC2 and OMC3 algorithms, there are three hyper parameters to be determined, including α, β, and K. In this subsection, using the OMC2 algorithm as an example, we explain the procedure of determining these parameters. The similar parameter determination procedure can be extended to the OMC3 algorithm. For α and β, we perform a 10-fold cross-validation to find the most appropriate values by the grid search, which are chosen from {0.1, 1, 10, 100}. When the neighborhood size K is fixed to 1, S1 Table shows the AUC values of OMC2 under different values of α and β on the gold standard dataset. Our results show that the best performance is achieved by α = 1 and β = 10. For K, we firstly assign 1 and 10 to α and β, respectively and then use cross validation to pick an appropriate K value from {1, 5, 10, 15, 20, 25, 30}. S1 Fig shows the AUC values of OMC2 under this setting. When K is 10, the best AUC value is achieved. Since the values of K have little effect on AUC values, we can treat K = 10 as a prior knowledge in other datasets for simplicity. Actually, We fixed the neighborhood size K to 10, the optimal values of α and β are also equal to 1 and 10, respectively. The results are shown in S2 Table and it could further illustrate the stability of the parameter values. Based on the above analysis, we finally choose α = 1, β = 10, and K = 10 for the gold standard dataset as the default parameters. Comparison with other methods In order to obtain convincing and fair comparison results, OMC2 and OMC3 are compared with the five state-of-the-art approaches: BNNR [14], DRRS [12], MBiRW [6], DrugNet [5], and HGBI [3]. The parameters in the compared approaches are set to either the default values in their papers or the best value by the grid search, if the default values are not provided. We rank the predicted indications and plot the ROC curves and PR curves to analyze the 10-fold cross-validation results. As shown in Fig 3, OMC2 and OMC3 outperform the other methods in ROC curves, PR curves, and top-ranked results. More specifically, OMC2 and OMC3 obtain AUC values of 0.939 and 0.945, while BNNR, DRRS, MBiRW, DrugNet, and HGBI yield AUC values of 0.932, 0.930, 0.917, 0.868, and 0.829, respectively. In real-life drug-repositioning applications, researchers particularly care about precision, because the precise prediction can significantly reduce experimental cost and time. The PR curves show OMC2 and OMC3 achieve the second best and the best precisions of 0.449 and 0.461, while BNNR, DRRS, MBiRW, DrugNet, and HGBI have the precisions of 0.440, 0.375, 0.304, 0.192, and 0.130, respectively. It is important to note that OMC3 can successfully prioritize 46.1% true drug–disease associations at top rank. A true drug–disease association is treated as the retrieved correct association when its predicted rank is higher than the specified ranking threshold. The numbers of correct associations predicted by all methods under different top ranking values are shown in Fig 3(c). The numbers of retrieved associations of both OMC2 and OMC3 exceed those of the other competing approaches. Specifically, among 1, 933 true drug–disease associations, 1, 493(77.2%) and 1, 529(79.1%) associations are correctly predicted at top 10 by OMC2 and OMC3, while in comparison, 1, 475(76.3%), 1, 413(73.1%), 1, 232(63.7%), 900(46.6%), and 752(38.9%) associations are identified by BNNR, DRRS, MBiRW, DrugNet, and HGBI, respectively. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. The performance of all methods for predicting drug–disease associations in the 10-fold cross-validation. (a) ROC curves of prediction results. (b) PR curves of predicting candidate diseases for drugs. (c) The number of correctly retrieved drug–disease associations for various rank thresholds. https://doi.org/10.1371/journal.pcbi.1007541.g003 Prediction of potential indications for new drugs To evaluate the performance of OMC2 and OMC3 for identifying indications of novel drugs, we conduct a de novo test, where these drugs with only one known drug–disease association are chosen. For each of these drugs, the unique disease association is removed in turn as the test sample, and other known drug–disease associations are used as the training samples. There are totally 171 drugs with only one known associated disease in gold standard dataset. As shown in Fig 4, OMC2 and OMC3 achieve the AUC values of 0.851 and 0.871, while BNNR, DRRS, MBiRW, DrugNet, and HGBI have inferior results with the AUC values of 0.830, 0.824, 0.818, 0.782, and 0.746, respectively. OMC3 has demonstrated its advantages measured by PR curves. For top-ranked results, OMC3 outperforms all methods at all ranking thresholds. Meanwhile, OMC2 surpasses the compared approaches at top 5, 10, 30, 50 and 100, except for being inferior to DRRS at top 1. Specifically, 74 and 88 drugs are identified correctly at top 5 by OMC2 and OMC3, respectively. In comparison, 73, 62, 71, 52, and 36 drugs are predicted by BNNR, DRRS, MBiRW, DrugNet, and HGBI, respectively. Summarizing the above results, one can find that our OMC methods are effective to address the cold-start problem to identify potential indications for novel drugs. In particular, OMC3 yields further improvement over OMC2, indicating the effectiveness of incorporating target association information in the tri-layer network. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Performance of all methods in predicting potential diseases for new drugs. (a) ROC curves of prediction results. (b) PR curves of predicting candidate diseases for drugs. (c) Number of correctly retrieved drug–disease associations for various rank thresholds. https://doi.org/10.1371/journal.pcbi.1007541.g004 Case studies We apply OMC3 to predict new uses for already approved drugs in real applications. To predict novel indications for existing drugs in the gold standard dataset, we consider all known associations between drugs and diseases as the training samples and the unknown drug–disease pairs as the candidate samples. By carrying out the OMC3 algorithm, the predicted scores of all candidate pairs are obtained and sorted for each specific drug. In order to verify the predicted diseases, we choose three representative drugs: Doxorubicin, Flecainide, and Levodopafour. We confirm the potential diseases associated with the given drug by retrieving authoritative public databases, such as CTD [21], DrugBank, and KEGG [26]. The newly predicted indications and their supporting evidences are listed in Table 1. One can find that more than three novel indications are validated on top-5 for each representative drug. As shown in this case study, OMC3 can be used as an effective method for identifying new indications for specific drugs in practical applications. In order to provide more helpful references for medical researchers, the top-30 candidate indications of each drug are listed in S3 Table. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. The top-5 candidate diseases for Doxorubicin, Flecainide, and Levodopa. https://doi.org/10.1371/journal.pcbi.1007541.t001 Effectiveness of OMC on performance In order to evaluate the effectiveness of OMC, we compare OMC2 with algorithms using only drug- or disease-side information in 10-fold cross-validation. The first algorithm, called OMC-drug, is to obtain ADR1 by BNNR in DrNet-Dis network, while the other one, called OMC-disease, is to recover ADR2 by BNNR in DisNet-Dr network. As shown in S2 Fig, both OMC-drug and OMC-disease are inferior to OMC2 in each fold in terms of AUC. In conclusion, consolidating drug- and disease-side associations in OMC2 is a better way to predict drug–disease associations than just using one-side information. Comparison on tri-layer networks In this subsection, we illustrate the performance and the computational efficiency of different approaches in tri-layer networks. BNNR, DRRS, DrugNet, and HGBI algorithms are taken into account for extending from bilayer networks into tri-layer networks, in comparison with OMC3. Since the protein association information is incorporated, the resulted affinity matrix of the tri-layer network is significantly enlarged. This also poses computational challenges in the factorization algorithms in matrix completion, which often grow cubically. The running time of each approach is obtained on a Linux server with CPU 2.30 GHz and 128 GB memory. As described in our previous works, BNNR and DRRS constructed the same bilayer networks between drugs and diseases. In order to construct a tri-layer heterogeneous network, we integrate protein-related information into the network, including protein–protein similarities, drug–protein interactions, and disease–protein associations. Accordingly, we get the corresponding square, symmetric adjacency matrix defined as follows, where APP represents the protein–protein similarity matrix, which is calculated based on the amino acid sequence alignment by Rcpi [27]. The programs for completing the matrix M by BNNR and DRRS are called BNNR3 and DRRS3, respectively. For DrugNet, it is also applied to tri-layer networks by integrating target-related information [5], which is denoted as DrugNet3 here. DrugNet3 can predict drug–disease relationships by propagating information in the drug–target–disease network. Based on the guilt-by-association principle, the authors of HGBI had extended bilayer networks into tri-layer networks by integrating drug, target, and disease information [4], which was called TL-HGBI (denotes HGBI3 here). The 10-fold cross-validation is uniformly conducted in the same gold standard dataset for OMC3, BNNR3, DRRS3, DrugNet3, and HGBI3. As shown in Fig 5(a) and 5(b), OMC3 outperforms the other approaches measured by the AUC values of the ROC curves and the precision. Specifically, OMC3 obtains the best AUC value of 0.945, while BNNR3, DRRS3, DrugNet3, and HGBI3 have the AUC values of 0.932, 0.932, 0.835, and 0.855, respectively. The PR curves show that OMC3 obtains the best precision with 0.460, while BNNR3, DRRS3, DrugNet3, and HGBI3 have the precision values of 0.431, 0.329, 0.093, and 0.227, respectively. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Method comparison in bilayer networks and tri-layer networks. (a) ROC curves of prediction results. (b) PR curves of prediction results. (c) The average running time of each fold in the 10-fold cross-validation. https://doi.org/10.1371/journal.pcbi.1007541.g005 Surprisingly, a method extended from bilayer networks into tri-layer networks does not necessary improve the prediction performance. In fact, only OMC and HGBI obtain performance improvement in tri-layer networks over bilayer ones. BNNR3, DRRS3, and DrugNet3 yield even worse performance when tri-layer networks are used compared to the corespondent algorithms on bilayer networks. This is due to the fact that protein–protein similarities calculated by the algorithm contain a large amount of “noise”, which causes BNNR3, DRRS3, and DrugNet3 to degrade their prediction performance. In contrast, OMC3 avoids the use of protein–protein similarities and the information OMC3 used is experimentally proven, such as drug–protein interactions and disease–protein associations, which in turn leads to performance improvement over OMC2 on bilayer networks. As shown in Fig 5(c), the average running time of BNNR and DRRS increase sharply from bilayer networks (BNNR2 and DRRS2) to tri-layer networks (BNNR3 and DRRS3), due to the increase of the affinity matrix. Nevertheless, this does not have such a significant impact on OMC, DrugNet, and HGBI. For OMC, this is because OMC keeps the matrix completion computation at the bilayer network level. As a result, OMC is not only better in terms of the prediction performance, but is also computational efficiency. Experiments on the other datasets We apply OMC2 and OMC3 to two other datasets, including Cdataset [6] and DNdataset [5], to demonstrate their robustness. Cdataset contains 663 drugs collected in DrugBank, 409 diseases obtained in OMIM database, and 2, 352 known drug–disease associations. In addition, we have collected drug–protein interactions related to drugs of Cdataset from DrugBank and retrieved a total of 3, 251 associations between 637 drugs and 891 proteins. For disease–protein associations, we download disease–gene interactions related to diseases of Cdataset from CTD database, and map genes into proteins in the UniprotKB database. There are 1, 280 associations between 226 diseases and 1, 002 proteins. The drug similarity and disease similarity are calculated in the same way as described in the previous section. DNdataset includes 1, 490 drugs registered in DrugBank, 4, 516 diseases annotated by Disease Ontology (DO) terms, 18, 107 proteins extracted from BioGRID, 11, 658 disease–protein associations directly extracted from the disease and gene annotations (DGA), 4, 026 drug–protein interactions collected in DrugBank, and 1, 008 known drug–disease associations. We evaluate the performance of our methods on Cdataset and DNdataset by performing a 10-fold cross-validation and de novo experiments. For Cdataset, as shown in S3(a)–S3(c) Fig, OMC2 and OMC3 demonstrate superior performance in terms of ROC curve, PR curve, and top-ranked results in the 10-fold cross-validation. Specifically, OMC2 and OMC3 obtain the AUC values of 0.953 and 0.957 in the ROC curves, while BNNR, DRRS, MBiRW, DrugNet, and HGBI have 0.948, 0.947, 0.933, 0.903, and 0.858, respectively. The PR curves indicate that OMC2 and OMC3 achieve the second best precision of 0.476 and the best precision of 0.489, while the precision values in BNNR, DRRS, MBiRW, DrugNet, and HGBI are 0.471, 0.403, 0.351, 0.239, and 0.168, respectively. In addition, OMC2 and OMC3 outperform the other methods in the top-ranked results with respect to different ranking thresholds. In the de novo test, there are 177 drugs with only one known associated disease in Cdataset. As shown in S4(a)–S4(c) Fig, OMC2 and OMC3 obtain the AUC values of 0.830 and 0.846, respectively, while BNNR, DRRS, MBiRW, DrugNet, and HGBI have the AUC values of 0.812, 0.819, 0.804, 0.785, and 0.732, respectively. Both OMC2 and OMC3 exceed the other methods in terms of AUC values as well. For top-ranked results, among 177 test drugs, 100 (56.5%) drugs are correctly identified at top 10 by OMC3, while only 87 (49.2%), 78 (44.1%), 80 (45.2%), 61 (34.5%), and 48 (27.1%) drugs are predicted by BNNR, DRRS, MBiRW, DrugNet, and HGBI, respectively. For DNdataset, in the 10-fold cross-validation results shown in S5(a)–S5(c) Fig, OMC2 and OMC3 obtain the AUC values of 0.957 and 0.965, while BNNR, DRRS, MBiRW, DrugNet, and HGBI yield the AUC values of 0.955, 0.934, 0.956, 0.950, and 0.921, respectively. Similar to that of Cdataset, OMC2 obtains the second best precision of 0.360 and OMC3 obtains the best precision of 0.369 in PR curves. Moreover, OMC2 and OMC3 outperform the other methods on top-ranked results at different ranking thresholds. In the de novo test, OMC3 also outperforms the other methods. As shown in S6(a)–S6(c) Fig, OMC2 and OMC3 obtain the AUC values of 0.963 and 0.972, while BNNR, DRRS, MBiRW, DrugNet, and HGBI have the AUC values of 0.956, 0.946, 0.970, 0.969, and 0.928, respectively. For top-ranked results, among 347 test drugs, 228 (65.7%) and 231 (66.6%) drugs are correctly identified at top 1 by OMC2 and OMC3, while only 218 (62.8%), 213 (61.4%), 219 (63.1%), 156 (45.0%), and 150 (43.2%) drugs are predicted by BNNR, DRRS, MBiRW, DrugNet, and HGBI, respectively. In summary, the above results on Cdataset and DNdataset demonstrate the robustness and generalization of OMC. Evaluation metrics To evaluate the performance of our approaches, a 10-fold cross-validation experiment is conducted to identify candidate diseases for specific drugs. In the gold standard dataset, all approved drug–disease associations are randomly divided into ten parts with approximately equal sizes. Each part is treated as the testing set in turn, and the training set is comprised of the remaining nine parts. To obtain convincing results, the 10-fold cross-validation is repeated 10 times and the final result is showed by the average value of the 10 folds. After the performing prediction, all candidate diseases associating with the test drug are ranked by their predicted scores in descending order. For a given rank threshold, the candidate disease is considered as a True Positive (TP) if its rank is above the threshold; otherwise, it is treated as a False Negative (FN). On the other hand, if the rank of a candidate disease had no association with the test drug is greater than the threshold, it is considered as a False Positive (FP), otherwise, it is treated as a True Negative (TN). Based on varying ranking thresholds, we can calculate True Positive Rate (TPR) and False Positive Rate (FPR) by and draw a Receiver Operating Characteristic (ROC) curve. Meanwhile, the area under the ROC curve (AUC) is utilized to evaluate the overall performance of a method. Precision and recall (equivalent to TPR) could be obtained to plot the precision-recall (PR) curve [25]. Due to the fact that the top-ranked result is a more important measurement in real-life drug-repositioning applications, the number of the retrieved correct associations is reported under different top ranking values. Parameter settings In OMC2 and OMC3 algorithms, there are three hyper parameters to be determined, including α, β, and K. In this subsection, using the OMC2 algorithm as an example, we explain the procedure of determining these parameters. The similar parameter determination procedure can be extended to the OMC3 algorithm. For α and β, we perform a 10-fold cross-validation to find the most appropriate values by the grid search, which are chosen from {0.1, 1, 10, 100}. When the neighborhood size K is fixed to 1, S1 Table shows the AUC values of OMC2 under different values of α and β on the gold standard dataset. Our results show that the best performance is achieved by α = 1 and β = 10. For K, we firstly assign 1 and 10 to α and β, respectively and then use cross validation to pick an appropriate K value from {1, 5, 10, 15, 20, 25, 30}. S1 Fig shows the AUC values of OMC2 under this setting. When K is 10, the best AUC value is achieved. Since the values of K have little effect on AUC values, we can treat K = 10 as a prior knowledge in other datasets for simplicity. Actually, We fixed the neighborhood size K to 10, the optimal values of α and β are also equal to 1 and 10, respectively. The results are shown in S2 Table and it could further illustrate the stability of the parameter values. Based on the above analysis, we finally choose α = 1, β = 10, and K = 10 for the gold standard dataset as the default parameters. Comparison with other methods In order to obtain convincing and fair comparison results, OMC2 and OMC3 are compared with the five state-of-the-art approaches: BNNR [14], DRRS [12], MBiRW [6], DrugNet [5], and HGBI [3]. The parameters in the compared approaches are set to either the default values in their papers or the best value by the grid search, if the default values are not provided. We rank the predicted indications and plot the ROC curves and PR curves to analyze the 10-fold cross-validation results. As shown in Fig 3, OMC2 and OMC3 outperform the other methods in ROC curves, PR curves, and top-ranked results. More specifically, OMC2 and OMC3 obtain AUC values of 0.939 and 0.945, while BNNR, DRRS, MBiRW, DrugNet, and HGBI yield AUC values of 0.932, 0.930, 0.917, 0.868, and 0.829, respectively. In real-life drug-repositioning applications, researchers particularly care about precision, because the precise prediction can significantly reduce experimental cost and time. The PR curves show OMC2 and OMC3 achieve the second best and the best precisions of 0.449 and 0.461, while BNNR, DRRS, MBiRW, DrugNet, and HGBI have the precisions of 0.440, 0.375, 0.304, 0.192, and 0.130, respectively. It is important to note that OMC3 can successfully prioritize 46.1% true drug–disease associations at top rank. A true drug–disease association is treated as the retrieved correct association when its predicted rank is higher than the specified ranking threshold. The numbers of correct associations predicted by all methods under different top ranking values are shown in Fig 3(c). The numbers of retrieved associations of both OMC2 and OMC3 exceed those of the other competing approaches. Specifically, among 1, 933 true drug–disease associations, 1, 493(77.2%) and 1, 529(79.1%) associations are correctly predicted at top 10 by OMC2 and OMC3, while in comparison, 1, 475(76.3%), 1, 413(73.1%), 1, 232(63.7%), 900(46.6%), and 752(38.9%) associations are identified by BNNR, DRRS, MBiRW, DrugNet, and HGBI, respectively. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. The performance of all methods for predicting drug–disease associations in the 10-fold cross-validation. (a) ROC curves of prediction results. (b) PR curves of predicting candidate diseases for drugs. (c) The number of correctly retrieved drug–disease associations for various rank thresholds. https://doi.org/10.1371/journal.pcbi.1007541.g003 Prediction of potential indications for new drugs To evaluate the performance of OMC2 and OMC3 for identifying indications of novel drugs, we conduct a de novo test, where these drugs with only one known drug–disease association are chosen. For each of these drugs, the unique disease association is removed in turn as the test sample, and other known drug–disease associations are used as the training samples. There are totally 171 drugs with only one known associated disease in gold standard dataset. As shown in Fig 4, OMC2 and OMC3 achieve the AUC values of 0.851 and 0.871, while BNNR, DRRS, MBiRW, DrugNet, and HGBI have inferior results with the AUC values of 0.830, 0.824, 0.818, 0.782, and 0.746, respectively. OMC3 has demonstrated its advantages measured by PR curves. For top-ranked results, OMC3 outperforms all methods at all ranking thresholds. Meanwhile, OMC2 surpasses the compared approaches at top 5, 10, 30, 50 and 100, except for being inferior to DRRS at top 1. Specifically, 74 and 88 drugs are identified correctly at top 5 by OMC2 and OMC3, respectively. In comparison, 73, 62, 71, 52, and 36 drugs are predicted by BNNR, DRRS, MBiRW, DrugNet, and HGBI, respectively. Summarizing the above results, one can find that our OMC methods are effective to address the cold-start problem to identify potential indications for novel drugs. In particular, OMC3 yields further improvement over OMC2, indicating the effectiveness of incorporating target association information in the tri-layer network. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Performance of all methods in predicting potential diseases for new drugs. (a) ROC curves of prediction results. (b) PR curves of predicting candidate diseases for drugs. (c) Number of correctly retrieved drug–disease associations for various rank thresholds. https://doi.org/10.1371/journal.pcbi.1007541.g004 Case studies We apply OMC3 to predict new uses for already approved drugs in real applications. To predict novel indications for existing drugs in the gold standard dataset, we consider all known associations between drugs and diseases as the training samples and the unknown drug–disease pairs as the candidate samples. By carrying out the OMC3 algorithm, the predicted scores of all candidate pairs are obtained and sorted for each specific drug. In order to verify the predicted diseases, we choose three representative drugs: Doxorubicin, Flecainide, and Levodopafour. We confirm the potential diseases associated with the given drug by retrieving authoritative public databases, such as CTD [21], DrugBank, and KEGG [26]. The newly predicted indications and their supporting evidences are listed in Table 1. One can find that more than three novel indications are validated on top-5 for each representative drug. As shown in this case study, OMC3 can be used as an effective method for identifying new indications for specific drugs in practical applications. In order to provide more helpful references for medical researchers, the top-30 candidate indications of each drug are listed in S3 Table. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. The top-5 candidate diseases for Doxorubicin, Flecainide, and Levodopa. https://doi.org/10.1371/journal.pcbi.1007541.t001 Effectiveness of OMC on performance In order to evaluate the effectiveness of OMC, we compare OMC2 with algorithms using only drug- or disease-side information in 10-fold cross-validation. The first algorithm, called OMC-drug, is to obtain ADR1 by BNNR in DrNet-Dis network, while the other one, called OMC-disease, is to recover ADR2 by BNNR in DisNet-Dr network. As shown in S2 Fig, both OMC-drug and OMC-disease are inferior to OMC2 in each fold in terms of AUC. In conclusion, consolidating drug- and disease-side associations in OMC2 is a better way to predict drug–disease associations than just using one-side information. Comparison on tri-layer networks In this subsection, we illustrate the performance and the computational efficiency of different approaches in tri-layer networks. BNNR, DRRS, DrugNet, and HGBI algorithms are taken into account for extending from bilayer networks into tri-layer networks, in comparison with OMC3. Since the protein association information is incorporated, the resulted affinity matrix of the tri-layer network is significantly enlarged. This also poses computational challenges in the factorization algorithms in matrix completion, which often grow cubically. The running time of each approach is obtained on a Linux server with CPU 2.30 GHz and 128 GB memory. As described in our previous works, BNNR and DRRS constructed the same bilayer networks between drugs and diseases. In order to construct a tri-layer heterogeneous network, we integrate protein-related information into the network, including protein–protein similarities, drug–protein interactions, and disease–protein associations. Accordingly, we get the corresponding square, symmetric adjacency matrix defined as follows, where APP represents the protein–protein similarity matrix, which is calculated based on the amino acid sequence alignment by Rcpi [27]. The programs for completing the matrix M by BNNR and DRRS are called BNNR3 and DRRS3, respectively. For DrugNet, it is also applied to tri-layer networks by integrating target-related information [5], which is denoted as DrugNet3 here. DrugNet3 can predict drug–disease relationships by propagating information in the drug–target–disease network. Based on the guilt-by-association principle, the authors of HGBI had extended bilayer networks into tri-layer networks by integrating drug, target, and disease information [4], which was called TL-HGBI (denotes HGBI3 here). The 10-fold cross-validation is uniformly conducted in the same gold standard dataset for OMC3, BNNR3, DRRS3, DrugNet3, and HGBI3. As shown in Fig 5(a) and 5(b), OMC3 outperforms the other approaches measured by the AUC values of the ROC curves and the precision. Specifically, OMC3 obtains the best AUC value of 0.945, while BNNR3, DRRS3, DrugNet3, and HGBI3 have the AUC values of 0.932, 0.932, 0.835, and 0.855, respectively. The PR curves show that OMC3 obtains the best precision with 0.460, while BNNR3, DRRS3, DrugNet3, and HGBI3 have the precision values of 0.431, 0.329, 0.093, and 0.227, respectively. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Method comparison in bilayer networks and tri-layer networks. (a) ROC curves of prediction results. (b) PR curves of prediction results. (c) The average running time of each fold in the 10-fold cross-validation. https://doi.org/10.1371/journal.pcbi.1007541.g005 Surprisingly, a method extended from bilayer networks into tri-layer networks does not necessary improve the prediction performance. In fact, only OMC and HGBI obtain performance improvement in tri-layer networks over bilayer ones. BNNR3, DRRS3, and DrugNet3 yield even worse performance when tri-layer networks are used compared to the corespondent algorithms on bilayer networks. This is due to the fact that protein–protein similarities calculated by the algorithm contain a large amount of “noise”, which causes BNNR3, DRRS3, and DrugNet3 to degrade their prediction performance. In contrast, OMC3 avoids the use of protein–protein similarities and the information OMC3 used is experimentally proven, such as drug–protein interactions and disease–protein associations, which in turn leads to performance improvement over OMC2 on bilayer networks. As shown in Fig 5(c), the average running time of BNNR and DRRS increase sharply from bilayer networks (BNNR2 and DRRS2) to tri-layer networks (BNNR3 and DRRS3), due to the increase of the affinity matrix. Nevertheless, this does not have such a significant impact on OMC, DrugNet, and HGBI. For OMC, this is because OMC keeps the matrix completion computation at the bilayer network level. As a result, OMC is not only better in terms of the prediction performance, but is also computational efficiency. Experiments on the other datasets We apply OMC2 and OMC3 to two other datasets, including Cdataset [6] and DNdataset [5], to demonstrate their robustness. Cdataset contains 663 drugs collected in DrugBank, 409 diseases obtained in OMIM database, and 2, 352 known drug–disease associations. In addition, we have collected drug–protein interactions related to drugs of Cdataset from DrugBank and retrieved a total of 3, 251 associations between 637 drugs and 891 proteins. For disease–protein associations, we download disease–gene interactions related to diseases of Cdataset from CTD database, and map genes into proteins in the UniprotKB database. There are 1, 280 associations between 226 diseases and 1, 002 proteins. The drug similarity and disease similarity are calculated in the same way as described in the previous section. DNdataset includes 1, 490 drugs registered in DrugBank, 4, 516 diseases annotated by Disease Ontology (DO) terms, 18, 107 proteins extracted from BioGRID, 11, 658 disease–protein associations directly extracted from the disease and gene annotations (DGA), 4, 026 drug–protein interactions collected in DrugBank, and 1, 008 known drug–disease associations. We evaluate the performance of our methods on Cdataset and DNdataset by performing a 10-fold cross-validation and de novo experiments. For Cdataset, as shown in S3(a)–S3(c) Fig, OMC2 and OMC3 demonstrate superior performance in terms of ROC curve, PR curve, and top-ranked results in the 10-fold cross-validation. Specifically, OMC2 and OMC3 obtain the AUC values of 0.953 and 0.957 in the ROC curves, while BNNR, DRRS, MBiRW, DrugNet, and HGBI have 0.948, 0.947, 0.933, 0.903, and 0.858, respectively. The PR curves indicate that OMC2 and OMC3 achieve the second best precision of 0.476 and the best precision of 0.489, while the precision values in BNNR, DRRS, MBiRW, DrugNet, and HGBI are 0.471, 0.403, 0.351, 0.239, and 0.168, respectively. In addition, OMC2 and OMC3 outperform the other methods in the top-ranked results with respect to different ranking thresholds. In the de novo test, there are 177 drugs with only one known associated disease in Cdataset. As shown in S4(a)–S4(c) Fig, OMC2 and OMC3 obtain the AUC values of 0.830 and 0.846, respectively, while BNNR, DRRS, MBiRW, DrugNet, and HGBI have the AUC values of 0.812, 0.819, 0.804, 0.785, and 0.732, respectively. Both OMC2 and OMC3 exceed the other methods in terms of AUC values as well. For top-ranked results, among 177 test drugs, 100 (56.5%) drugs are correctly identified at top 10 by OMC3, while only 87 (49.2%), 78 (44.1%), 80 (45.2%), 61 (34.5%), and 48 (27.1%) drugs are predicted by BNNR, DRRS, MBiRW, DrugNet, and HGBI, respectively. For DNdataset, in the 10-fold cross-validation results shown in S5(a)–S5(c) Fig, OMC2 and OMC3 obtain the AUC values of 0.957 and 0.965, while BNNR, DRRS, MBiRW, DrugNet, and HGBI yield the AUC values of 0.955, 0.934, 0.956, 0.950, and 0.921, respectively. Similar to that of Cdataset, OMC2 obtains the second best precision of 0.360 and OMC3 obtains the best precision of 0.369 in PR curves. Moreover, OMC2 and OMC3 outperform the other methods on top-ranked results at different ranking thresholds. In the de novo test, OMC3 also outperforms the other methods. As shown in S6(a)–S6(c) Fig, OMC2 and OMC3 obtain the AUC values of 0.963 and 0.972, while BNNR, DRRS, MBiRW, DrugNet, and HGBI have the AUC values of 0.956, 0.946, 0.970, 0.969, and 0.928, respectively. For top-ranked results, among 347 test drugs, 228 (65.7%) and 231 (66.6%) drugs are correctly identified at top 1 by OMC2 and OMC3, while only 218 (62.8%), 213 (61.4%), 219 (63.1%), 156 (45.0%), and 150 (43.2%) drugs are predicted by BNNR, DRRS, MBiRW, DrugNet, and HGBI, respectively. In summary, the above results on Cdataset and DNdataset demonstrate the robustness and generalization of OMC. Discussion In this study, we have proposed a novel OMC method for predicting drug-associated indications, which can effectively integrate multiple types of drug and disease information. In addition, our method can be simply extended from bilayer networks to tri-layer networks by incorporating drug-target associations. Furthermore, OMC effectively avoids the use of noisy data in tri-layer networks. The performance of our methods (OMC2 and OMC3) are validated by the cross validation, de novo experiments, and case studies. The experimental results indicate that our methods are effective compared with the latest approaches, particularly for de novo drugs. However, OMC has two potential limitations. First, the drug and disease similarity computations in this work may be not optimal. More reliable similarity measures, for example consensus integrating multiple similarities computations from different aspects could improve the performance of OMC. Second, OMC must perform matrix completion twice from both drug-side and disease-side before the final predicted score is obtained. OMC can actually be used on other drug-related predictions, such as synergistic drug combination and small molecule–miRNA association prediction. The synergistic drug combination is based on the assumption that principal drugs which obtain the synergistic effect with similar adjuvant drugs are often similar and vice versa [28]. That means the drug combination matrix is also of low-rank. Therefore, OMC can be applied to predict potential synergistic drug combinations by integrating the drug similarity matrix and the drug–target interaction matrix. In addition, it may avoid classifying principal drugs and adjunct drugs before obtaining the final score of drug combinations. MiRNAs play an important role in the initiation and development of various human diseases. Several drug-like compound libraries targeting different miRNAs have been successfully screened in cell assays, further demonstrating the possibility of targeting miRNAs with small molecules. Hence, it is very meaningful and promising to develop computational models for drug repositioning based on drug related miRNA. Some original and novel methods have been proposed in recent years [29]. Especially, based on tri-layer heterogeneous networks, more prior information is used to obtain better prediction performance [30]. In the future, we plan to extend our OMC method to explore drug combinations and miRNA-small molecule associations for drug repositioning. Supporting information S1 Fig. The AUC values are indicated by the OMC2 algorithm when the neighborhood size K is chosen from {1, 5, 10, 15, 20, 25, 30} in the cross validation. https://doi.org/10.1371/journal.pcbi.1007541.s001 (TIF) S2 Fig. Performance comparison of the OMC2, OMC-drug and OMC-disease in the 10-fold cross-validation in terms of AUC. The result of each fold is presented. https://doi.org/10.1371/journal.pcbi.1007541.s002 (TIF) S3 Fig. (a) ROC curves of prediction results. (b) PR curves of predicting candidate diseases for drugs. (c) The number of correctly retrieved drug–disease associations for various rank thresholds. The performance of all methods for predicting drug–disease associations in the 10-fold cross-validation on CDataset. https://doi.org/10.1371/journal.pcbi.1007541.s003 (TIF) S4 Fig. (a) ROC curves of prediction results. (b) PR curves of predicting candidate diseases for drugs. (c) The number of correctly retrieved drug–disease associations for various rank threshold. The performance of all methods in predicting potential diseases for new drugs on CDataset. https://doi.org/10.1371/journal.pcbi.1007541.s004 (TIF) S5 Fig. (a) ROC curves of prediction results. (b) PR curves of predicting candidate diseases for drugs. (c) The number of correctly retrieved drug–disease associations for various rank thresholds. The performance of all methods for predicting drug–disease associations in the 10-fold cross-validation on DNdataset. https://doi.org/10.1371/journal.pcbi.1007541.s005 (TIF) S6 Fig. (a) ROC curves of prediction results. (b) PR curves of predicting candidate diseases for drugs. (c) The number of correctly retrieved drug–disease associations for various rank threshold. The performance of all methods in predicting potential diseases for new drugs on DNdataset. https://doi.org/10.1371/journal.pcbi.1007541.s006 (TIF) S1 Table. The AUC values under different values of α and β in the 10-fold cross-validation for the gold standard dataset. https://doi.org/10.1371/journal.pcbi.1007541.s007 (DOC) S2 Table. The AUC values based on K = 10 for α and β in the 10-fold cross-validation for the gold standard dataset. https://doi.org/10.1371/journal.pcbi.1007541.s008 (DOC) S3 Table. The top-30 candidate indications of all drugs listed by OMC3. https://doi.org/10.1371/journal.pcbi.1007541.s009 (XLSX)

Codutti, Agnese;Bente, Klaas;Faivre, Damien;Klumpp, Stefan

doi: 10.1371/journal.pcbi.1007548pmid: 31856155

Introduction The motion of many microorganisms as well as synthetic and biohybrid microswimmers is based on a directed self-propulsion over short time and length scale. On larger scales, however, these swimmers typically perform (persistent) random walks due to either the Brownian rotation of their direction of propulsion, which is unavoidable due to their small size, or due to active mechanisms of re-orientation such as the tumbling of bacteria [1–3]. Biasing this random motion is key to directional motion on large scales. The paradigm for such directional guidance is the chemotaxis of bacteria such as Escherichia coli, which alternate between straight runs and abrupt changes of direction called tumbles. Chemosensing and intracellular signaling to the flagella bias the resulting random walk by prolonging runs in the direction up a gradient of a chemoattractant [3, 4]. Other mechanisms of steering microswimmers make use of external fields that orient the swimming. Specifically, homogeneous magnetic fields are a promising steering mechanism for microswimmers that are equipped with a magnetic moment, a situation that occurs naturally in magnetotactic bacteria [5–7], but is also used in a variety of synthetic and biohybrid swimmers [8–10]. A number of interesting questions relate to the combination of two mechanisms of directional guidance: Which strategies do microorganisms use to resolve conflicts between different directional inputs such as chemotaxis and external forces? Which strategies could be implemented in synthetic systems? Magnetotactic bacteria are a natural example of bacteria undergoing both chemotaxis and magnetic interactions: the passive alignment to the Earth magnetic field of the intracellular magnetic chain of these bacteria is usually intertwined with aerotaxis, i.e. chemotaxis for oxygen [5–7, 11, 12]. Still, the interaction with such a weak field does not provide perfect alignment. What is the advantage given by magnetic fields to the magnetotactic bacteria? Previous studies approached these questions at the population level, both from an experimental and computational point of view, and suggested that the magnetic orientation may provide an advantage for the tactic dynamics of the bacteria [13–15]. These studies however, do not answer the question whether this is always the case, e.g. if the chemical gradient and the magnetic field steer the bacteria in very different directions, and have not addressed the dependence on the chemotactic strategy. The latter question is also important for biohybrids, where a chemotactic strategy that is not adapted to work with a magnetic torque is combined with magnetic steering. Such chemotactic biohybrid swimmer have been realized very recently, by functionalizing E. coli bacteria with magnetic beads [16]. Magnetically steerable biohybrids or functionalized magnetotactic bacteria are envisioned for biomedical applications such as drug delivery and cancer or biofilms targeting [7, 17, 18]. So far, these applications are at the proof-of-principle state [7, 17, 18], but eventually, understanding how magnetic fields or other external forces interact with chemotaxis may help in the rational design of such systems. In this work, we introduce a theoretical approach to address these questions at the micrometer scale. Two types of theoretical approaches have been used to describe the persistent random walks of microswimmers: bacterial chemotaxis has been described by random walk models, which provide a rather accurate description of the bacterial trajectories with abrupt turns due to tumbling [3, 19, 20]. Tumbling is usually not included in the active Brownian particle models commonly used to describe self-propelled particles. Active Brownian particle models, however allow for the straightforward incorporation of external forces and have therefore been used extensively to study interactions between active particles and the resulting collective effects [2, 21]. Here, we combine features of both approaches into a multi-state active Brownian particle model for chemotactic motion. Our approach uses a Langevin equation of motion, in which external forces or torques are easily included (See Fig 1a), but also accounts for the abrupt changes of motion (run and tumble, run and reverse, etc.) characteristic for the motion of many microorganism. Similar approaches have occasionally been used [22, 23], including one study incorporating external forces [24] and one incorporating surface attachment as an additional state of the swimmer [25]. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Model for swimming and chemotaxis in external fields. a) The model describes a bacterium that actively swims with velocity v, and that is subject to an external force F and a magnetic field B (measured in microTesla, μT). Its random walk may also be biased to perform chemotaxis in a concentration gradient ∇C thanks to active changes of direction such as tumbles or reversals. b) Examples trajectories for run and tumble motion in the presence of a magnetic field. The magnetic interaction helps to reduce the motion of the bacteria to an effectively one-dimensional motion and directs it in the direction of the magnetic field. Active directional changes (tumbles) result in swimming in other directions, an effect that is increasingly suppressed with increasing intensity of the magnetic field. https://doi.org/10.1371/journal.pcbi.1007548.g001 Here, we use this model as a general approach to chemotaxis under the influence of external forces and torques and determine how such forces and torques influence the chemotactic velocity. Specifically, we apply it to the magneto-aerotaxis of magnetotactic bacteria and study the competition of orientation changes with the alignment with the field. Moreover, we compare the ‘run and reverse’ and ‘run and tumble’ strategies for chemotaxis. We show that run and reverse allows for magneto-aerotaxis at arbitrary orientations of the magnetic field relative to the gradient, while a higher speed-up of chemotaxis by the field can be reached for run and tumble, but only for limited configuration of field to gradient orientations. In particular, the presence of a magnetic field can speed up the formation of a magneto-aerotactic band around a preferred oxygen concentration in the case of run and reverse, while band formation is prevented by a magnetic field for run and tumble. Our results indicate that magnetic orientation of chemotactic cells is beneficial for the cells under a wide range of conditions, but not universally so, and that the choice of chemotactic strategy is crucial to benefit from a magnetic orientation. Materials and methods Theoretical model and simulation To describe bacterial chemotaxis subject to external fields, we use an active Brownian particle model with multiple internal states (e.g., “run” and “change of direction”). The bacteria are described as point particles with position r and a direction vector e that determines the direction of self-propulsion. In each state, their dynamics is given by Langevin equations: (1) In these equations, kB is the Boltzmann constant, γt and γr are the translational and rotational friction coefficients, respectively, v is the speed of self propulsion, σ is the sign of the self-propulsion velocity (±1 for parallel or antiparallel to e, 0 for no self-propulsion), Fext and Text describe external forces and torques and ξt and ξr describe uncorrelated white noise in the translational and rotational degrees of freedom. The translational noise is purely thermal with temperature T, but the rotational noise may have additional contributions. Specifically, we consider the case of run and tumble motion. In this case, σ = 1 during runs and σ = 0 during tumbles, corresponding to self-propulsion during runs and no self-propulsion during tumbles, respectively (see S1 Fig). Moreover, the rapid reorientation during tumbles is described by a noise strength that exceeds the thermal noise. We adopt a description of the orientation changes during tumbles by rotational diffusion with an effective temperature Ttumble > T. This temperature approximates the distribution of orientation changes [26] (but not the correlations of angular change and tumble duration [27]). Matching that distribution to the observed distribution for E. coli (with an average reorientation by 68°) [28] results in an effective temperature of Ttumble = 4.2 × 104 K (see S2 Fig). Transitions between the run and tumble states are stochastic with exponentially distributed dwell times and occur with rates and [28]. The average tumble time τtumble is rather short (≃ 0.1 s) and constant, the average run time τrun is longer (≃ 1 s) [28] and is modulated by a chemical gradient for chemotaxis. Without loss of generality, these chosen values are based on the widely-studied E. coli behavior [28]. Results can be easily scaled to other parameter choices. For a chemoattractant, we use the following simple implementation of that modulation (other functional dependencies can also be used [29, 30]): (2) where τ0 indicates the mean run time in absence of gradients, tup and tdown represent the positive-biased times and the negative one, ∇C∥ indicates the projection of the chemical gradient onto the direction of motion and ∇C0 is a threshold gradient for which the maximal run time is reached. τrun can be interpreted as a response function to the gradient and concentration. See S1 Text for more details. Run and reverse motion is described in a completely analogous way with three states, run (σ = 1), reversal pause (mimicking the slowing observed before reversals [13, 31, 32], with σ = 0), and reverse run (σ = −1) (see S1 Fig). Transitions occur in a cyclic fashion from run to pause, to reverse to pause to run with average durations τrun, τpause and τreverse respectively. In contrast to run and tumble, the orientational noise is thermal in all three states. Due to thermal rotational diffusion during the pause, the orientation of the cells changes on average by 170° in a reversal. Chemotaxis modulates τrun and τreverse in the same way as in the case of run and tumble. We emphasize that the models we use here do not include detailed descriptions of the chemotactic response. Specifically, signaling and adaptation are not described explicitly. Rather the model is based on a coarse-grained description of the random walk that arises from that signaling. Such an approach, while clearly being simplified and not accounting for all features of chemotactic behavior, has two advantages: On the one hand, such a coarse-grained description can also be used when the underlying chemotactic mechanisms are unknown. On the other hand, the coarse-grained description is also computationally less expensive when simulating large numbers of cells or trajectories over long time scales as we do here, in particular when we study the formation of magneto-aerotactic bands. For most simulations with either scenario, we use the parameters estimated from the experimental data for E. coli from the literature [3, 28] (see S1 Table). Parameters are adjusted to match our simulation results to the experiments for magnetoaerotactic band formation of M. gryphiswaldense as described below/in Results (see S2 Table). The simulations are performed with a custom written code in Fortran 90 (the S1 Source Code is included in the supplementary materials). The stochastic equation are integrated through a Euler method [33, 34], while the vectors ξt and ξr (representing the uncorrelated white noise) are three-dimensional vectors of Gaussian random numbers [33, 35], numerically obtained by a Box Muller method [34]. Simulation of magneto-aerotactic band formation To compare our simulations to the capillary experiments performed for M. gryphiswaldense (see below), the model is extended to include interactions with the capillary walls and a dynamic oxygen gradient: Upon contact with a capillary wall, an active reversal of the direction of motion is simulated, as observed experimentally for flat surfaces [36]. The dimensions of the capillary are matched to the experimental ones as far as possible, with the same length of 40 mm, and a section of 0.1 × 0.1 mm2 (real capillary: 40 mm × 2 mm × 0.2 mm), with a average bacterial density as in the experiment (25000 bacteria are simulated, 6.25 × 107 cell mL−1 corresponding to an optical density OD = 0.18). The oxygen gradient builds up dynamically due to diffusion into the capillary from the capillary’s open end and consumption of oxygen by the bacteria in its interior. Therefore, alongside with the integration of the bacterial equation of motion, the equation for oxygen is integrated in three dimensions, as well [12–14]: (3) Here C is the oxygen concentration, the oxygen diffusion constant, k the consumption rate, ρ the local number of bacteria and Ca a cutoff to avoid negative concentration. The boundary conditions on the plane x = 0 is set to C = 216 μM. The starting condition is completely anoxic (the oxygen is absent), as in the experiments. The density of bacteria and the oxygen concentration and gradient are constant within bins of the discretized space (dimension 20 μm × 20 μm × 20 μm). Since our simulation indicate that robust formation of a band requires a sufficiently steep response to the chemical gradient, i.e. a small value of ∇C0 in Eq 2 (see S3a–S3d Fig) and the value of ∇C0 has only a small effect on the chemotactic velocity (S3e and S3f Fig), we used the limiting case of τrun for ∇C0 ≈ 0 in these simulations. Finally, the run times change along the course of a bacterial trajectory, because the run times (or the reversal rates) are dependent on the oxygen gradient as well as the oxygen concentration, the latter because in (magneto-)aerotaxis, oxygen acts as an attractant at low concentrations but as a repellent at high concentrations (see ref. [12]). In our simulations, the duration of a run is decided by the local oxygen concentration and oxygen gradient at the beginning of a run, thus changes within a run are neglected. This is unproblematic, because the oxygen profile changes slowly on the length scale of a run. There is however, one exception: In our simulations of magneto-aerotactic band formation, when a bacterium crosses the preferred oxygen concentration, the run times change drastically because of the reversed bias as oxygen switches from an attractant to a repellent. Therefore, in this situation the change of the run time during the run has to be included. This is implemented by starting a new run in the same direction when crossing the preferred concentration (implementing an instantaneous change in the reversal rate). Without this, unrealistically broad bands form (see S4 Fig), as the bacteria ‘overshoot’ in the unfavorable direction, while the change in the reversal rate results in shorter runs, reducing band width. Capillary experiments The experiments are performed following Bennet et al. [12, 13]. Briefly, MSR-1 bacteria cultures are grown in the cultivation medium reported by Heyen and Schüler [37]. A motile population is obtained using a procedure reported by Bennet et al. [13]. After growing in the agar semisolid medium, they are grown to mid-exponential phase in homogeneous oxygen conditions in the tubes at 1 mT homogenous magnetic field. 1 ml of motile cells is harvested during the cell’s mid-exponential growth phase. After harvesting, the cells show axial magnetotactic behavior, meaning that under homogeneous oxygen conditions they swim in both directions in a strong external magnetic field. An optical density (OD) of 0.1 at 565 nm is adjusted, which corresponds to 3.4 × 107 cells ml−1. Two other OD are considered: 0.05 and 0.2. The sample is degassed using nitrogen for 15 minutes and the sample is introduced into a rectangular micro-capillary (VitroTubes, #3520–050,) by capillary forces. One end of the capillary is sealed with petroleum jelly and the capillary is glued onto a microscope slide and mounted onto the microscope stage. The microscope features 3 orthogonal Helmholtz coils, which are used to cancel Earths magnetic field with a precision of 0.2 μT and to create a homogeneous 50 μT magnetic field antiparallel to the oxygen gradient inside the capillary at the sample position [13]. A 60x objective (Nikon, Plan Apo VC, 1.2 NA, WI) is used to image cell motility and localize the band position. A 10x objective is used to image the band and measure its size. The images are processed with ImageJ (Version 1.50h, Wayne Rasband National Institutes of Health, USA): from 100 frames (fps = 100 s−1) the standard deviation is calculated. The standard deviation image is a quantification of the movement of the bacteria, where white areas indicate high movement and black areas low movement. The gray-intensity profile of the standard deviation image is then plotted as function of the position in the frame, with the vertical values averaged, with ImageJ ‘Plot Profile’ function. in this way, the band profile is obtained and then it is fitted with Matlab (Version R2017a, Mathworks) with a Laplace distribution. The 10x magnification videos are pre-processed with ImageJ for background subtraction and quality improvement, and then they are tracked with a custom-made tracking algorithm [13]. The 4th-order velocities are extracted from the trajectories with a custom-made Matlab program. Theoretical model and simulation To describe bacterial chemotaxis subject to external fields, we use an active Brownian particle model with multiple internal states (e.g., “run” and “change of direction”). The bacteria are described as point particles with position r and a direction vector e that determines the direction of self-propulsion. In each state, their dynamics is given by Langevin equations: (1) In these equations, kB is the Boltzmann constant, γt and γr are the translational and rotational friction coefficients, respectively, v is the speed of self propulsion, σ is the sign of the self-propulsion velocity (±1 for parallel or antiparallel to e, 0 for no self-propulsion), Fext and Text describe external forces and torques and ξt and ξr describe uncorrelated white noise in the translational and rotational degrees of freedom. The translational noise is purely thermal with temperature T, but the rotational noise may have additional contributions. Specifically, we consider the case of run and tumble motion. In this case, σ = 1 during runs and σ = 0 during tumbles, corresponding to self-propulsion during runs and no self-propulsion during tumbles, respectively (see S1 Fig). Moreover, the rapid reorientation during tumbles is described by a noise strength that exceeds the thermal noise. We adopt a description of the orientation changes during tumbles by rotational diffusion with an effective temperature Ttumble > T. This temperature approximates the distribution of orientation changes [26] (but not the correlations of angular change and tumble duration [27]). Matching that distribution to the observed distribution for E. coli (with an average reorientation by 68°) [28] results in an effective temperature of Ttumble = 4.2 × 104 K (see S2 Fig). Transitions between the run and tumble states are stochastic with exponentially distributed dwell times and occur with rates and [28]. The average tumble time τtumble is rather short (≃ 0.1 s) and constant, the average run time τrun is longer (≃ 1 s) [28] and is modulated by a chemical gradient for chemotaxis. Without loss of generality, these chosen values are based on the widely-studied E. coli behavior [28]. Results can be easily scaled to other parameter choices. For a chemoattractant, we use the following simple implementation of that modulation (other functional dependencies can also be used [29, 30]): (2) where τ0 indicates the mean run time in absence of gradients, tup and tdown represent the positive-biased times and the negative one, ∇C∥ indicates the projection of the chemical gradient onto the direction of motion and ∇C0 is a threshold gradient for which the maximal run time is reached. τrun can be interpreted as a response function to the gradient and concentration. See S1 Text for more details. Run and reverse motion is described in a completely analogous way with three states, run (σ = 1), reversal pause (mimicking the slowing observed before reversals [13, 31, 32], with σ = 0), and reverse run (σ = −1) (see S1 Fig). Transitions occur in a cyclic fashion from run to pause, to reverse to pause to run with average durations τrun, τpause and τreverse respectively. In contrast to run and tumble, the orientational noise is thermal in all three states. Due to thermal rotational diffusion during the pause, the orientation of the cells changes on average by 170° in a reversal. Chemotaxis modulates τrun and τreverse in the same way as in the case of run and tumble. We emphasize that the models we use here do not include detailed descriptions of the chemotactic response. Specifically, signaling and adaptation are not described explicitly. Rather the model is based on a coarse-grained description of the random walk that arises from that signaling. Such an approach, while clearly being simplified and not accounting for all features of chemotactic behavior, has two advantages: On the one hand, such a coarse-grained description can also be used when the underlying chemotactic mechanisms are unknown. On the other hand, the coarse-grained description is also computationally less expensive when simulating large numbers of cells or trajectories over long time scales as we do here, in particular when we study the formation of magneto-aerotactic bands. For most simulations with either scenario, we use the parameters estimated from the experimental data for E. coli from the literature [3, 28] (see S1 Table). Parameters are adjusted to match our simulation results to the experiments for magnetoaerotactic band formation of M. gryphiswaldense as described below/in Results (see S2 Table). The simulations are performed with a custom written code in Fortran 90 (the S1 Source Code is included in the supplementary materials). The stochastic equation are integrated through a Euler method [33, 34], while the vectors ξt and ξr (representing the uncorrelated white noise) are three-dimensional vectors of Gaussian random numbers [33, 35], numerically obtained by a Box Muller method [34]. Simulation of magneto-aerotactic band formation To compare our simulations to the capillary experiments performed for M. gryphiswaldense (see below), the model is extended to include interactions with the capillary walls and a dynamic oxygen gradient: Upon contact with a capillary wall, an active reversal of the direction of motion is simulated, as observed experimentally for flat surfaces [36]. The dimensions of the capillary are matched to the experimental ones as far as possible, with the same length of 40 mm, and a section of 0.1 × 0.1 mm2 (real capillary: 40 mm × 2 mm × 0.2 mm), with a average bacterial density as in the experiment (25000 bacteria are simulated, 6.25 × 107 cell mL−1 corresponding to an optical density OD = 0.18). The oxygen gradient builds up dynamically due to diffusion into the capillary from the capillary’s open end and consumption of oxygen by the bacteria in its interior. Therefore, alongside with the integration of the bacterial equation of motion, the equation for oxygen is integrated in three dimensions, as well [12–14]: (3) Here C is the oxygen concentration, the oxygen diffusion constant, k the consumption rate, ρ the local number of bacteria and Ca a cutoff to avoid negative concentration. The boundary conditions on the plane x = 0 is set to C = 216 μM. The starting condition is completely anoxic (the oxygen is absent), as in the experiments. The density of bacteria and the oxygen concentration and gradient are constant within bins of the discretized space (dimension 20 μm × 20 μm × 20 μm). Since our simulation indicate that robust formation of a band requires a sufficiently steep response to the chemical gradient, i.e. a small value of ∇C0 in Eq 2 (see S3a–S3d Fig) and the value of ∇C0 has only a small effect on the chemotactic velocity (S3e and S3f Fig), we used the limiting case of τrun for ∇C0 ≈ 0 in these simulations. Finally, the run times change along the course of a bacterial trajectory, because the run times (or the reversal rates) are dependent on the oxygen gradient as well as the oxygen concentration, the latter because in (magneto-)aerotaxis, oxygen acts as an attractant at low concentrations but as a repellent at high concentrations (see ref. [12]). In our simulations, the duration of a run is decided by the local oxygen concentration and oxygen gradient at the beginning of a run, thus changes within a run are neglected. This is unproblematic, because the oxygen profile changes slowly on the length scale of a run. There is however, one exception: In our simulations of magneto-aerotactic band formation, when a bacterium crosses the preferred oxygen concentration, the run times change drastically because of the reversed bias as oxygen switches from an attractant to a repellent. Therefore, in this situation the change of the run time during the run has to be included. This is implemented by starting a new run in the same direction when crossing the preferred concentration (implementing an instantaneous change in the reversal rate). Without this, unrealistically broad bands form (see S4 Fig), as the bacteria ‘overshoot’ in the unfavorable direction, while the change in the reversal rate results in shorter runs, reducing band width. Capillary experiments The experiments are performed following Bennet et al. [12, 13]. Briefly, MSR-1 bacteria cultures are grown in the cultivation medium reported by Heyen and Schüler [37]. A motile population is obtained using a procedure reported by Bennet et al. [13]. After growing in the agar semisolid medium, they are grown to mid-exponential phase in homogeneous oxygen conditions in the tubes at 1 mT homogenous magnetic field. 1 ml of motile cells is harvested during the cell’s mid-exponential growth phase. After harvesting, the cells show axial magnetotactic behavior, meaning that under homogeneous oxygen conditions they swim in both directions in a strong external magnetic field. An optical density (OD) of 0.1 at 565 nm is adjusted, which corresponds to 3.4 × 107 cells ml−1. Two other OD are considered: 0.05 and 0.2. The sample is degassed using nitrogen for 15 minutes and the sample is introduced into a rectangular micro-capillary (VitroTubes, #3520–050,) by capillary forces. One end of the capillary is sealed with petroleum jelly and the capillary is glued onto a microscope slide and mounted onto the microscope stage. The microscope features 3 orthogonal Helmholtz coils, which are used to cancel Earths magnetic field with a precision of 0.2 μT and to create a homogeneous 50 μT magnetic field antiparallel to the oxygen gradient inside the capillary at the sample position [13]. A 60x objective (Nikon, Plan Apo VC, 1.2 NA, WI) is used to image cell motility and localize the band position. A 10x objective is used to image the band and measure its size. The images are processed with ImageJ (Version 1.50h, Wayne Rasband National Institutes of Health, USA): from 100 frames (fps = 100 s−1) the standard deviation is calculated. The standard deviation image is a quantification of the movement of the bacteria, where white areas indicate high movement and black areas low movement. The gray-intensity profile of the standard deviation image is then plotted as function of the position in the frame, with the vertical values averaged, with ImageJ ‘Plot Profile’ function. in this way, the band profile is obtained and then it is fitted with Matlab (Version R2017a, Mathworks) with a Laplace distribution. The 10x magnification videos are pre-processed with ImageJ for background subtraction and quality improvement, and then they are tracked with a custom-made tracking algorithm [13]. The 4th-order velocities are extracted from the trajectories with a custom-made Matlab program. Results Run and tumble chemotaxis in an external field To study the interplay of chemotaxis and external fields guiding the swimming of a microorganism, we introduced a model that combines the features of an active Brownian particle, specifically a Langevin equation, into which external forces and torques are easily introduced, with mechanisms for active directional changes such as tumbling and reversals (see Methods and Fig 1a). The latter are modulated by chemical gradients in order to implement a coarse-grained description of chemotaxis that does not explicitly describe the underlying signaling. Example trajectories for run and tumble in the presence of an external magnetic field without chemical gradients are shown in Fig 1b. The magnetic interaction helps to reduce the motion of the bacteria to an effectively one-dimensional motion and directs it in the direction of the magnetic field. Using this model, we first simulate run and tumble chemotaxis. Fig 2a shows representative trajectories and the one-dimensional mean square displacement. In the absence of a chemoattractant gradient, a bacterium performs a persistent random walk with directed motion with velocity v on small time scales and random diffusive motion on long time scales. The latter is characterized by the effective three-dimensional diffusion coefficient , with α = (1 − 〈cos θtumble〉) [38]. A gradient of a chemoattractant biases this random walk up the gradient with velocity vtaxis ≃ 1.08 μms−1 (Fig 2a). As a first simple example for the influence of an external field, we consider a constant external force, which might represent a force applied by the flow of the fluid in which the swimmer moves, or by optical or magnetic tweezers onto the swimmer [39]. Under a constant force, the run and tumble trajectories become biased (stretched out) in the direction of the force. The directionalities imposed by the force and by chemotaxis are linearly superimposed, thus forces with a component parallel to the gradient enhance/promote chemotaxis, while forces with an antiparallel component reduce it (Fig 2b). Chemotactic swimming up the gradient is impossible for opposing forces exceeding a threshold value F* = −γt vtaxis(F = 0)/cos θF,∇C, where θF,∇C is the angle between the gradient and the force (see S2 Text for the calculation of the taxis velocities). We note that the dependence on the force may be more complex than linear if chemotactic signaling is in a recently described nonlinear regime, where the change of the concentration during a run causes a positive feedback in the signaling system [40]. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Chemotaxis without/with forces. a) Comparison between the mean square displacement with (red) and without chemotaxis (blue). Inset: three different realizations of trajectories with chemotaxis towards an attractant with gradient directed along . b) Taxis velocities in the presence of a constant force at different angles θF,∇C relative to the concentration gradient and with different absolute values Fext. The dashed line corresponds to the case without force. For the used parameters, see S1 Table. https://doi.org/10.1371/journal.pcbi.1007548.g002 Run and tumble under the influence of a magnetic torque We next use the model to describe the interaction of a bacterium with a magnetic moment (which could be a magnetotactic bacterium or a magnetically functionalized E. coli) with a homogeneous magnetic field B in the absence of chemical gradients. In this case, we have a torque Text = M × B, where M = M e is the magnetic moment of the bacterium, but no force, Fext = 0. The magnetic torque tends to align the direction vector e of the bacterium with the magnetic field and thereby biases the motion in the direction of the field. However, tumbling perturbs this alignment, so run and tumble motion is characterized by strong perturbations of alignment due to tumbles, followed by relaxation to the aligned state during the runs (Fig 3a) (with a characteristic relaxation time [11, 41], see S3 Text and S5 Fig for more details). Thus, there is a competition between two time scales, the relaxation time characteristic of alignment , and the mean run time τrun, after which a tumble perturbs the alignment. For , the magnetic field does not have enough time to re-align the bacterium before the next tumble, resulting in larger fluctuations similar to an increased temperature. Indeed, the alignment of the swimming direction of magnetotactic bacteria has been described by an effective temperature [42, 43]. To test this concept in our model, we fit the cosine of the angle between the field and the orientation vector θe,B with the Langevin function [11, 42] (see also S4 Text and S6 Fig) (4) using the effective temperature Teff as a fit parameter (Fig 3b). The effective temperature depends on the run time (Fig 3c), reflecting the competition of the two time scales, and interpolates between the noise strength of tumbling and the actual temperature. While the Langevin function with an effective temperature gives a good fit for the order parameter 〈cos(θ)〉, the distribution of the angle deviates clearly from a thermal distribution [11] (see also S4 Text) (5) with that effective temperature (Fig 3d). The simulated distribution presents a peak due to the thermal distribution during runs after relaxation and a broad tail that depends on tumbling and relaxation and is not explainable with a thermal distribution. This observation suggests that measuring the distribution of the alignment angle might provide a way to distinguish the non-thermal noise due to discrete tumble events from non-thermal noise that might be present continuously due to the active swimming motion. The observation of a peak at low angles is consistent with a recent report, where this peak is interpreted as a velocity condensation effect [44]; however, our peak is not centered around zero as in that report, but has its maximum at a finite value. We show in S7 Fig that this difference arises from the presence of noise and from considering trajectories in a three-dimensional space rather than two-dimensional projections, both in contrast to ref. [44]. Finally we note that the non-thermal fluctuations that are induced at discrete time points by tumbling are not observed in the run and reverse scenario, where the fluctuations of orientation are thermal in all states of the particle. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Motion with magnetic torque. a) Alignment angle vs. time for run and tumble (in purple) and run and reverse (in green) for a mean run time of 0.86 s at 500 μT. b) Langevin plots for run and tumble compared to theoretical Langevin functions at T and Ttumble (dashed lines). Each line corresponds to a different mean run time. c) The effective temperature obtained by the Langevin fit in b) is plotted against the mean run time. d) The distribution of the alignment angle for run and tumble with a mean run time of 0.86 s at 500 μT (data points) deviates from the thermal distribution with the corresponding effective temperature, 2896K (solid line). https://doi.org/10.1371/journal.pcbi.1007548.g003 Chemotaxis with magnetic torque The magnetoaerotaxis of magnetotatic bacteria provides an example of chemotaxis influenced by a magnetic torque. The magnetotatic bacteria perform aerotaxis, a chemotactic motion towards micro-aerobic conditions, i.e. towards a preferred (low) oxygen concentration, while being passively oriented by the magnetic field of the Earth. In the natural environment, the inclination of the magnetic field of the Earth with respect to the vertical direction typically results in an angle between the directions defined by the magnetic field and the oxygen gradient. Here we test this scenario with our model for a gradient constant in time and space. In particular, we compare different taxis strategies in the presence of magnetic fields (reverses appear typical in magnetotactic bacteria [13], but there have also been reports of tumbling [45]), with the aim of explaining why natural magnetotactic bacteria adopt some strategies and not others. To this extent, we investigate the effect of a magnetic field on chemotaxis towards an attractant for run and tumble as well as run and reverse motion. Our simulations show that a magnetic field parallel to the gradient is beneficial to both scenarios, since it increases the chemotactic velocity, but that an antiparallel field can be overcome only by run and reverse (Fig 4), suggesting that the natural behavior of magnetotactic bacteria is dominated by reversals rather than by tumbling, consistently with experimental observations [13, 45]. The price for the ability to swim against the direction of the magnetic field is an overall lower velocity (compared to tumble) due to the greater contributions of backward motion. Nevertheless, the magnetic field enhances the chemotactic motion up to rather large angles (approximately 60° in Fig 4). For angles close to 90°, the chemotactic velocity is lower than without the field for both reverse and tumble, therefore rising the question whether magnetotaxis is beneficial at such high field inclinations. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Chemotaxis with magnetic torque. Taxis velocity up the gradient for run and reverse (solid lines) and run and tumble (dotted lines) at B = 0 μT (green line) and B = 50 μT (purple line). In black, the theoretical predictions. In the presence of a field at a small angle θB,∇C relative to the concentration gradient, run and tumble performs better than run and reverse (the pink area shows the gap between the tumble curve and the reverse curve). Reverse plus magnetic field outperforms reverse without the field except for angles around 90° (the green area shows the gap between the curves with and without field for reverse). Tumble with field, on the other hand, hinders chemotaxis at large angles (the red area shows the gap between zero velocity and the tumble curve). https://doi.org/10.1371/journal.pcbi.1007548.g004 An analytical expression that approximately predicts the taxis velocity as function of the magnetic field intensity and orientation can be obtained in the following way: We first consider the case of run and tumble. Neglecting noise, the bacteria are forced to swim in the direction of the magnetic field. Thus, their motion up the gradient is given by the projection of their velocity in the direction of the field, which is given by the cosine of the angle between the magnetic field and the gradient, in the direction of the gradient, cos θe,B. The projection effect is seen directly by the collapse of the velocity data for different magnetic field shown in S8 Fig. The velocity along the field, however, is not the swimming velocity v, but reduced by orientation fluctuations, which include both thermal noise and the those arising form tumbles and subsequent relaxation. The average alignment and thus the average velocity in the direction of the field can be described by an effective temperature, as shown above (Fig 3c). Combining these two consideration, we arrive at the following expression for the chemotactic velocity, (6) For run and reverse, these considerations have to be modified, because the bacteria may swim both up and down the gradient. Thus, the cosine of the projection is replaced by its absolute value and the direction is included explicitly by a factor that accounts for the fraction of time of up-gradient swimming minus the corresponding fraction for down-gradient swimming (R ≃ 0.31 for our parameters). The full expression for the taxis velocity up a constant gradient for run and reverse is thus (7) The predictions obtained from Eqs (6) and (7) are included in Fig 4 and seen to provide a good approximation to the simulation result. The agreement gets even better for stronger fields, where the role of noise is minor (S8 Fig). Our comparison between different strategies for changes of the swimming direction, which identifies reversals as crucial for scenarios with a magnetic field antiparallel to a chemical gradient, might be dependent on choices in our modeling that therefore deserve additional investigation. In our model, we have assumed that runs or reverse runs down the gradient have the same duration as runs in the absence of a gradient, which need not be the case [46, 47]. Therefore we tested whether shortening runs down the gradient changes our results. The corresponding simulations are shown in S9 Fig and show that the advantage of tumbling over reversals in the case of a parallel field is also seen with shortened runs down the gradient, but is less pronounced than without the shortening. Moreover, in the run and tumble motion of peritrichously flagellated bacteria such as E. coli and B. subtilis, the tumbles effectively include a mechanism for the reversal of the direction of motion, as the flagellar bundle opens during tumbles and can subsequently form on either side of the cell body, as demonstrated for the swimming of B. subtilis in a liquid crystal [48]. We therefore tested how choosing the direction of motion randomly as either parallel or antiparallel to the body orientation after a tumble affects the motion. In that case, motion up the gradient under an antiparallel magnetic field is indeed possible, but slower than in the parallel case and slower than in the absence of the field (S10a Fig). Moreover, such a high probability of changing the direction of motion relative to the orientation of the cell body is not realistic given the distributions of reorientation angles during tumbles cf. S2 Fig, as it would result in a distribution that is symmetric around 90°. To test the influence of the fraction of reversals, we also simulated a combination of reversals and tumbles that interpolates smoothly between the two types of behaviors. We varied the fraction of reversals and found that swimming up the gradient against a magnetic field requires more than 50% reversals (S10b Fig), strengthening the point that a mechanism of reversal is needed for effective motion in the scenario of a gradient opposed by a magnetic field. We also note that to our knowledge, none of the magnetotactic bacteria that were described so far are peritrichously flagellated. If tumbling occurs, as reported in some cases [13, 45], the mechanism for tumbling is likely different from the one in E. coli. Finally, the already mentioned nonlinear regime of the chemotactic signaling system [40] results in a scenario where runs up the gradient are very long and cells spend almost all time running. One might expect that in this case, the difference between the tumbling and reversal strategies disappears. This is indeed true if the magnetic field is parallel to the gradient. However, if the field and the gradient are antiparallel, reversals are still seen to be more efficient than tumbles (S11 Fig). In that case, the tumbling bacteria cannot make full use of the very long runs up the gradient, because immediately after a tumble, the magnetic torque reorients them to align them with the magnetic field and thereby forces them to swim down the gradient with very short runs; in this way, they are never able to access the long runs up the gradient. For the case of run and reverse chemotaxis towards an attractant, we quantify how the mean run time influences the taxis velocity with and without the magnetic field. Magnetotactic bacteria can perform very long runs compared to the runs of E. coli [32]. Without a magnetic field, very long runs would be detrimental for gradient sensing, since thermal noise results in a re-orientation of the cell within a time t given by < cos2 θ >= 6Drt. Without a magnetic field (B = 0), we indeed see that the maximal taxis velocity is reached for a run time of about 2s (red curve of Fig 5), for which the average thermal reorientation is ≃ 50°. When a weak magnetic field of 50 μT is turned on (blue curve of Fig 5), the taxis velocity is higher than without the field, as shown in the previous sections. Moreover, the velocity reaches a plateau for run times exceeding 2 s, showing how long runs benefit from the presence of magnetic fields. The advantage persists for the inclination of the magnetic field of the Earth relative to a vertical gradient in a stratified aqueous environment (with an angle of 157° as in Berlin [49], see the purple line in Fig 5). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Influence of the mean run time. For run-and-reverse chemotaxis towards an attractant, we study the influence of the mean run time, during which the bacterium swims up the gradient, on the taxis velocity. We notice that for B = 0, a peak is present around 2 s. For B = 50 μT (parallel to the gradient or at an inclination of 157° (inclination of the Earth magnetic field in Berlin to a downward gradient in a stratified aqueous environment), a plateau is reached instead. https://doi.org/10.1371/journal.pcbi.1007548.g005 Magnetoaerotactic band formation with a constant gradient In the presence of an oxygen gradient and of a magnetic field, magnetotactic bacteria accumulate in regions of their preferred oxygen concentration C*, which leads to the formation of a band of high bacterial density in a quasi-one dimensional geometry such as a capillary [12–14]. This behavior is recovered in our model, when we treat oxygen as a chemoattractant at concentrations C < C* and as a chemorepellent for C > C* in the run and reverse scenario (see S1 Text) (Fig 6a). In the run and tumble scenario, formation of a band is only seen in the absence of a magnetic field, as band formation in the presence of a magnetic field requires motion against that field, which is prohibitively unlikely in run and tumble, as discussed above. However, as aerotaxis without a magnetic field would be sufficient for band formation (and indeed also occurs in non-magnetic bacteria [50]), the question arises what the advantage of magnetically assisted aerotaxis might be. A possible answer is that with magnetic assistance, the band forms more rapidly. Thus, we simulate the formation of a magneto-aerotactic band in a fixed linear oxygen gradient. We estimate the width of the band by the standard deviation σeq of the swimmer position in the direction of the gradient (Fig 6b) (for an alternative derivation, see S12 Fig). This quantity relaxes quickly as the band is formed. The relaxation time of band formation and, to a lesser extent, the steady-state width of the band are seen to depend on the intensity of the magnetic field and the angle of application (Fig 6b–6d). The formation of the band is indeed sped up at low inclination of the field relative to the gradient. For inclinations approaching 90°, however, band formation is strongly slowed down, in particular for high field strength for which it is inhibited, consistent with our observation of a reduced chemotactic velocity above. Notably, for field strengths comparable to the Earth’s magnetic field, a band still forms in less than an hour, indicating that magneto-aerotaxis (based on run and reverse) remains functional even at 90°. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Band formation in constant concentration gradients. a) 100 example trajectories are shown for chemotaxis towards a preferred concentration at the position indicated by the black line. The bacteria concentrate there and form a band. b) Width of the band as a function of time, estimated by the standard deviation of the position of the bacteria in the direction of the gradient, σeq, for scenarios with and without a magnetic field parallel to the gradient. c) Equilibration time teq (i.e. time at which the band stop growing in amplitude and reaches steady state) and d) width of the band at equilibrium, plotted as a function of the angle θB,∇C between the gradient and the magnetic field, for tumble (t.) and reverse (r.) with different intensities of the magnetic field. While run-and-reverse behavior results in band formation for almost any magnetic condition, run-and-tumble bacteria only forma a band without magnetic fields. https://doi.org/10.1371/journal.pcbi.1007548.g006 As already discussed above, run and tumble motion of peritrichous bacteria includes a mechanism for reversals by re-forming the flagellar bundle on the opposite side of the cell body. One would therefore expect that this scenario allows for band formation. This is indeed the case as shown in S13 Fig, but the band is wider than for run and reverse. We tested again combinations of tumbling and reversals and found that a large fraction of reversals (>50%, in excess of what is consistent with the reorientation distribution of E. coli) is needed for robust band formation. Thus, while not all run and tumble mechanism prohibit band formation, it is the reversals included in an E.coli-type tumbling mechanism that are crucial and allow the required motion against the magnetic field. Together, our observations of band formation support the following picture: a mechanism for swimming in the direction imposed by the magnetic field and well as against that direction is required for magneto-aerotaxis. Such a mechanism is provided by reversals. For reversing bacteria, a magnetic field can speed up the formation of an aerotactic band for low inclinations, but does not prohibit it at inclinations close to 90°, consistent with the presence of magnetotactic bacteria close to the Earth’s equator [51]. Magnetoaerotactic band formation with a dynamic gradient To compare our simulations results with experimental data on the magnetically assisted aerotaxis, we perform capillary experiments for the magnetotactic bacterium Magnetospirillum gryphiswaldense MSR-1 and follow the formation of the magneto-aerotactic band. The bacteria, which perform axial magnetotaxis, i.e. swim along the field line without a directional preference, are placed in capillary with one open an one closed end. The interior of the capillary is initially anoxic and oxygen diffuses in from the open end, resulting in an oxygen gradient. A homogeneous magnetic field of 50 μT is applied antiparallel to the oxygen gradient. As a magnetoaerotactic band forms, its position is recorded over time (green data points of Fig 7b). Moreover, we record videos of the stationary band from which the band profile can be extracted as described in Methods (green data points in Fig 7b). From this image, the gray-scale intensity is extracted, which is proportional to the bacterial distribution in the band (green data points of Fig 7d). The bacterial distribution can be fitted by a symmetric Laplace distribution (green line), which gives a characteristic size of the band of 73 μm. Moreover, by tracking individual bacteria, their mean velocity is determined to be 50 μms−1. These parameters are used in the following simulations since we now focus on reproducing a specific system. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 7. Capillary experiments and simulations. a) Scheme of a capillary experiment: a capillary is filled with bacteria; one end of the capillary is sealed, the other is open to the flow of oxygen. The bacteria accumulate at the preferred oxygen concentration, forming a band. b) Experimental band position over time (green data points) for wild type (WT) axial bacteria (optical density OD = 0.1, 5 repetitions, two different days—triangles and circles), with an antiparallel magnetic field of 50 μT, compared to the simulated band position (blue data points) under the same conditions. The red line represents the position of the preferred concentration position as obtained from the simulation. c) Standard deviation of 100 frames (1 s) of a video of the band recorded at 100 fps and 10x magnification (WT OD = 0.1 antiparallel 50 μT). d) The gray-scale values are averaged along the y coordinate of Fig. c), giving the experimental density profile of the band (green data points). The green line represents the Laplace fit (decay length 73 μm). The simulated probability density function (pdf) at equilibrium is represented by the blue data points (decay length 75 μm). https://doi.org/10.1371/journal.pcbi.1007548.g007 We compare these experimental data to our simulations to test our aerotactic model. Compared to the simulations above, where a constant gradient is considered, the simulations are extended to include the dynamics of the oxygen gradient (due to oxygen diffusion and consumption by the bacteria) and the interactions of the bacteria with the walls, see Methods. Interactions between bacteria (specifically excluded volume) are neglected, since the system remains rather dilute even within the band. The run times in the favored and unfavored directions, and the oxygen consumption rate k are used as free fitting parameters. We vary these parameters to match the stationary band size and the position of the band over time. In general, reducing the difference between the two run times and reducing the consumption constant make the dynamics slower; the band size decreases with larger differences between the run times; and the oxygen consumption rate has a strong effect on the stationary position of the band (see S14 Fig S14 for more details on the influence of the parameters on the final outcome). The best match in Fig 7 is obtained with run times of 2 s and 0.9 s and k = 0.005 mol min−1 cell−1. The run times are comparable with the ones of E. coli [28], while they seem to underestimate the mean values for polar MSR1 [32], even though further experiments are needed to measure the biased run times up and down the gradient for magnetotactic bacteria. As it can be seen from Fig 7b and 7d, the simulated data (blue data points) show the same behavior as the experiments (green data points). The band forms close to the open end and moves further into the capillary reaching a stationary position such that the consumption of oxygen (mostly) in the band balances the diffusive flow of oxygen to the band. The density profile of the band is compatible between simulations and experiments, with a band width of 75 μm (Fig 7d). Likewise, the band position is well reproduced by the simulation, at least up to 40 min (data points in Fig 7b); small deviations between the two at larger times likely reflect the fact that the number of bacteria in the band keeps increasing slowly in the simulations due to bacteria arriving from the very far anoxic end of the capillary, while in the experiments many of these bacteria are likely inactive or dead and never reach the band. In the simulation, the band position tracks the location of the preferred oxygen concentration (indicated by the red line), indicating the aerotaxis of the simulated bacteria. Thus, our magneto-aerotactic model gives an effective representation of the system. More simulation at different magnetic fields intensities are show in S15 Fig, which confirm the results obtained in the constant gradient simulations where the band formation is speed up by the magnetic interaction. Run and tumble chemotaxis in an external field To study the interplay of chemotaxis and external fields guiding the swimming of a microorganism, we introduced a model that combines the features of an active Brownian particle, specifically a Langevin equation, into which external forces and torques are easily introduced, with mechanisms for active directional changes such as tumbling and reversals (see Methods and Fig 1a). The latter are modulated by chemical gradients in order to implement a coarse-grained description of chemotaxis that does not explicitly describe the underlying signaling. Example trajectories for run and tumble in the presence of an external magnetic field without chemical gradients are shown in Fig 1b. The magnetic interaction helps to reduce the motion of the bacteria to an effectively one-dimensional motion and directs it in the direction of the magnetic field. Using this model, we first simulate run and tumble chemotaxis. Fig 2a shows representative trajectories and the one-dimensional mean square displacement. In the absence of a chemoattractant gradient, a bacterium performs a persistent random walk with directed motion with velocity v on small time scales and random diffusive motion on long time scales. The latter is characterized by the effective three-dimensional diffusion coefficient , with α = (1 − 〈cos θtumble〉) [38]. A gradient of a chemoattractant biases this random walk up the gradient with velocity vtaxis ≃ 1.08 μms−1 (Fig 2a). As a first simple example for the influence of an external field, we consider a constant external force, which might represent a force applied by the flow of the fluid in which the swimmer moves, or by optical or magnetic tweezers onto the swimmer [39]. Under a constant force, the run and tumble trajectories become biased (stretched out) in the direction of the force. The directionalities imposed by the force and by chemotaxis are linearly superimposed, thus forces with a component parallel to the gradient enhance/promote chemotaxis, while forces with an antiparallel component reduce it (Fig 2b). Chemotactic swimming up the gradient is impossible for opposing forces exceeding a threshold value F* = −γt vtaxis(F = 0)/cos θF,∇C, where θF,∇C is the angle between the gradient and the force (see S2 Text for the calculation of the taxis velocities). We note that the dependence on the force may be more complex than linear if chemotactic signaling is in a recently described nonlinear regime, where the change of the concentration during a run causes a positive feedback in the signaling system [40]. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Chemotaxis without/with forces. a) Comparison between the mean square displacement with (red) and without chemotaxis (blue). Inset: three different realizations of trajectories with chemotaxis towards an attractant with gradient directed along . b) Taxis velocities in the presence of a constant force at different angles θF,∇C relative to the concentration gradient and with different absolute values Fext. The dashed line corresponds to the case without force. For the used parameters, see S1 Table. https://doi.org/10.1371/journal.pcbi.1007548.g002 Run and tumble under the influence of a magnetic torque We next use the model to describe the interaction of a bacterium with a magnetic moment (which could be a magnetotactic bacterium or a magnetically functionalized E. coli) with a homogeneous magnetic field B in the absence of chemical gradients. In this case, we have a torque Text = M × B, where M = M e is the magnetic moment of the bacterium, but no force, Fext = 0. The magnetic torque tends to align the direction vector e of the bacterium with the magnetic field and thereby biases the motion in the direction of the field. However, tumbling perturbs this alignment, so run and tumble motion is characterized by strong perturbations of alignment due to tumbles, followed by relaxation to the aligned state during the runs (Fig 3a) (with a characteristic relaxation time [11, 41], see S3 Text and S5 Fig for more details). Thus, there is a competition between two time scales, the relaxation time characteristic of alignment , and the mean run time τrun, after which a tumble perturbs the alignment. For , the magnetic field does not have enough time to re-align the bacterium before the next tumble, resulting in larger fluctuations similar to an increased temperature. Indeed, the alignment of the swimming direction of magnetotactic bacteria has been described by an effective temperature [42, 43]. To test this concept in our model, we fit the cosine of the angle between the field and the orientation vector θe,B with the Langevin function [11, 42] (see also S4 Text and S6 Fig) (4) using the effective temperature Teff as a fit parameter (Fig 3b). The effective temperature depends on the run time (Fig 3c), reflecting the competition of the two time scales, and interpolates between the noise strength of tumbling and the actual temperature. While the Langevin function with an effective temperature gives a good fit for the order parameter 〈cos(θ)〉, the distribution of the angle deviates clearly from a thermal distribution [11] (see also S4 Text) (5) with that effective temperature (Fig 3d). The simulated distribution presents a peak due to the thermal distribution during runs after relaxation and a broad tail that depends on tumbling and relaxation and is not explainable with a thermal distribution. This observation suggests that measuring the distribution of the alignment angle might provide a way to distinguish the non-thermal noise due to discrete tumble events from non-thermal noise that might be present continuously due to the active swimming motion. The observation of a peak at low angles is consistent with a recent report, where this peak is interpreted as a velocity condensation effect [44]; however, our peak is not centered around zero as in that report, but has its maximum at a finite value. We show in S7 Fig that this difference arises from the presence of noise and from considering trajectories in a three-dimensional space rather than two-dimensional projections, both in contrast to ref. [44]. Finally we note that the non-thermal fluctuations that are induced at discrete time points by tumbling are not observed in the run and reverse scenario, where the fluctuations of orientation are thermal in all states of the particle. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Motion with magnetic torque. a) Alignment angle vs. time for run and tumble (in purple) and run and reverse (in green) for a mean run time of 0.86 s at 500 μT. b) Langevin plots for run and tumble compared to theoretical Langevin functions at T and Ttumble (dashed lines). Each line corresponds to a different mean run time. c) The effective temperature obtained by the Langevin fit in b) is plotted against the mean run time. d) The distribution of the alignment angle for run and tumble with a mean run time of 0.86 s at 500 μT (data points) deviates from the thermal distribution with the corresponding effective temperature, 2896K (solid line). https://doi.org/10.1371/journal.pcbi.1007548.g003 Chemotaxis with magnetic torque The magnetoaerotaxis of magnetotatic bacteria provides an example of chemotaxis influenced by a magnetic torque. The magnetotatic bacteria perform aerotaxis, a chemotactic motion towards micro-aerobic conditions, i.e. towards a preferred (low) oxygen concentration, while being passively oriented by the magnetic field of the Earth. In the natural environment, the inclination of the magnetic field of the Earth with respect to the vertical direction typically results in an angle between the directions defined by the magnetic field and the oxygen gradient. Here we test this scenario with our model for a gradient constant in time and space. In particular, we compare different taxis strategies in the presence of magnetic fields (reverses appear typical in magnetotactic bacteria [13], but there have also been reports of tumbling [45]), with the aim of explaining why natural magnetotactic bacteria adopt some strategies and not others. To this extent, we investigate the effect of a magnetic field on chemotaxis towards an attractant for run and tumble as well as run and reverse motion. Our simulations show that a magnetic field parallel to the gradient is beneficial to both scenarios, since it increases the chemotactic velocity, but that an antiparallel field can be overcome only by run and reverse (Fig 4), suggesting that the natural behavior of magnetotactic bacteria is dominated by reversals rather than by tumbling, consistently with experimental observations [13, 45]. The price for the ability to swim against the direction of the magnetic field is an overall lower velocity (compared to tumble) due to the greater contributions of backward motion. Nevertheless, the magnetic field enhances the chemotactic motion up to rather large angles (approximately 60° in Fig 4). For angles close to 90°, the chemotactic velocity is lower than without the field for both reverse and tumble, therefore rising the question whether magnetotaxis is beneficial at such high field inclinations. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Chemotaxis with magnetic torque. Taxis velocity up the gradient for run and reverse (solid lines) and run and tumble (dotted lines) at B = 0 μT (green line) and B = 50 μT (purple line). In black, the theoretical predictions. In the presence of a field at a small angle θB,∇C relative to the concentration gradient, run and tumble performs better than run and reverse (the pink area shows the gap between the tumble curve and the reverse curve). Reverse plus magnetic field outperforms reverse without the field except for angles around 90° (the green area shows the gap between the curves with and without field for reverse). Tumble with field, on the other hand, hinders chemotaxis at large angles (the red area shows the gap between zero velocity and the tumble curve). https://doi.org/10.1371/journal.pcbi.1007548.g004 An analytical expression that approximately predicts the taxis velocity as function of the magnetic field intensity and orientation can be obtained in the following way: We first consider the case of run and tumble. Neglecting noise, the bacteria are forced to swim in the direction of the magnetic field. Thus, their motion up the gradient is given by the projection of their velocity in the direction of the field, which is given by the cosine of the angle between the magnetic field and the gradient, in the direction of the gradient, cos θe,B. The projection effect is seen directly by the collapse of the velocity data for different magnetic field shown in S8 Fig. The velocity along the field, however, is not the swimming velocity v, but reduced by orientation fluctuations, which include both thermal noise and the those arising form tumbles and subsequent relaxation. The average alignment and thus the average velocity in the direction of the field can be described by an effective temperature, as shown above (Fig 3c). Combining these two consideration, we arrive at the following expression for the chemotactic velocity, (6) For run and reverse, these considerations have to be modified, because the bacteria may swim both up and down the gradient. Thus, the cosine of the projection is replaced by its absolute value and the direction is included explicitly by a factor that accounts for the fraction of time of up-gradient swimming minus the corresponding fraction for down-gradient swimming (R ≃ 0.31 for our parameters). The full expression for the taxis velocity up a constant gradient for run and reverse is thus (7) The predictions obtained from Eqs (6) and (7) are included in Fig 4 and seen to provide a good approximation to the simulation result. The agreement gets even better for stronger fields, where the role of noise is minor (S8 Fig). Our comparison between different strategies for changes of the swimming direction, which identifies reversals as crucial for scenarios with a magnetic field antiparallel to a chemical gradient, might be dependent on choices in our modeling that therefore deserve additional investigation. In our model, we have assumed that runs or reverse runs down the gradient have the same duration as runs in the absence of a gradient, which need not be the case [46, 47]. Therefore we tested whether shortening runs down the gradient changes our results. The corresponding simulations are shown in S9 Fig and show that the advantage of tumbling over reversals in the case of a parallel field is also seen with shortened runs down the gradient, but is less pronounced than without the shortening. Moreover, in the run and tumble motion of peritrichously flagellated bacteria such as E. coli and B. subtilis, the tumbles effectively include a mechanism for the reversal of the direction of motion, as the flagellar bundle opens during tumbles and can subsequently form on either side of the cell body, as demonstrated for the swimming of B. subtilis in a liquid crystal [48]. We therefore tested how choosing the direction of motion randomly as either parallel or antiparallel to the body orientation after a tumble affects the motion. In that case, motion up the gradient under an antiparallel magnetic field is indeed possible, but slower than in the parallel case and slower than in the absence of the field (S10a Fig). Moreover, such a high probability of changing the direction of motion relative to the orientation of the cell body is not realistic given the distributions of reorientation angles during tumbles cf. S2 Fig, as it would result in a distribution that is symmetric around 90°. To test the influence of the fraction of reversals, we also simulated a combination of reversals and tumbles that interpolates smoothly between the two types of behaviors. We varied the fraction of reversals and found that swimming up the gradient against a magnetic field requires more than 50% reversals (S10b Fig), strengthening the point that a mechanism of reversal is needed for effective motion in the scenario of a gradient opposed by a magnetic field. We also note that to our knowledge, none of the magnetotactic bacteria that were described so far are peritrichously flagellated. If tumbling occurs, as reported in some cases [13, 45], the mechanism for tumbling is likely different from the one in E. coli. Finally, the already mentioned nonlinear regime of the chemotactic signaling system [40] results in a scenario where runs up the gradient are very long and cells spend almost all time running. One might expect that in this case, the difference between the tumbling and reversal strategies disappears. This is indeed true if the magnetic field is parallel to the gradient. However, if the field and the gradient are antiparallel, reversals are still seen to be more efficient than tumbles (S11 Fig). In that case, the tumbling bacteria cannot make full use of the very long runs up the gradient, because immediately after a tumble, the magnetic torque reorients them to align them with the magnetic field and thereby forces them to swim down the gradient with very short runs; in this way, they are never able to access the long runs up the gradient. For the case of run and reverse chemotaxis towards an attractant, we quantify how the mean run time influences the taxis velocity with and without the magnetic field. Magnetotactic bacteria can perform very long runs compared to the runs of E. coli [32]. Without a magnetic field, very long runs would be detrimental for gradient sensing, since thermal noise results in a re-orientation of the cell within a time t given by < cos2 θ >= 6Drt. Without a magnetic field (B = 0), we indeed see that the maximal taxis velocity is reached for a run time of about 2s (red curve of Fig 5), for which the average thermal reorientation is ≃ 50°. When a weak magnetic field of 50 μT is turned on (blue curve of Fig 5), the taxis velocity is higher than without the field, as shown in the previous sections. Moreover, the velocity reaches a plateau for run times exceeding 2 s, showing how long runs benefit from the presence of magnetic fields. The advantage persists for the inclination of the magnetic field of the Earth relative to a vertical gradient in a stratified aqueous environment (with an angle of 157° as in Berlin [49], see the purple line in Fig 5). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Influence of the mean run time. For run-and-reverse chemotaxis towards an attractant, we study the influence of the mean run time, during which the bacterium swims up the gradient, on the taxis velocity. We notice that for B = 0, a peak is present around 2 s. For B = 50 μT (parallel to the gradient or at an inclination of 157° (inclination of the Earth magnetic field in Berlin to a downward gradient in a stratified aqueous environment), a plateau is reached instead. https://doi.org/10.1371/journal.pcbi.1007548.g005 Magnetoaerotactic band formation with a constant gradient In the presence of an oxygen gradient and of a magnetic field, magnetotactic bacteria accumulate in regions of their preferred oxygen concentration C*, which leads to the formation of a band of high bacterial density in a quasi-one dimensional geometry such as a capillary [12–14]. This behavior is recovered in our model, when we treat oxygen as a chemoattractant at concentrations C < C* and as a chemorepellent for C > C* in the run and reverse scenario (see S1 Text) (Fig 6a). In the run and tumble scenario, formation of a band is only seen in the absence of a magnetic field, as band formation in the presence of a magnetic field requires motion against that field, which is prohibitively unlikely in run and tumble, as discussed above. However, as aerotaxis without a magnetic field would be sufficient for band formation (and indeed also occurs in non-magnetic bacteria [50]), the question arises what the advantage of magnetically assisted aerotaxis might be. A possible answer is that with magnetic assistance, the band forms more rapidly. Thus, we simulate the formation of a magneto-aerotactic band in a fixed linear oxygen gradient. We estimate the width of the band by the standard deviation σeq of the swimmer position in the direction of the gradient (Fig 6b) (for an alternative derivation, see S12 Fig). This quantity relaxes quickly as the band is formed. The relaxation time of band formation and, to a lesser extent, the steady-state width of the band are seen to depend on the intensity of the magnetic field and the angle of application (Fig 6b–6d). The formation of the band is indeed sped up at low inclination of the field relative to the gradient. For inclinations approaching 90°, however, band formation is strongly slowed down, in particular for high field strength for which it is inhibited, consistent with our observation of a reduced chemotactic velocity above. Notably, for field strengths comparable to the Earth’s magnetic field, a band still forms in less than an hour, indicating that magneto-aerotaxis (based on run and reverse) remains functional even at 90°. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Band formation in constant concentration gradients. a) 100 example trajectories are shown for chemotaxis towards a preferred concentration at the position indicated by the black line. The bacteria concentrate there and form a band. b) Width of the band as a function of time, estimated by the standard deviation of the position of the bacteria in the direction of the gradient, σeq, for scenarios with and without a magnetic field parallel to the gradient. c) Equilibration time teq (i.e. time at which the band stop growing in amplitude and reaches steady state) and d) width of the band at equilibrium, plotted as a function of the angle θB,∇C between the gradient and the magnetic field, for tumble (t.) and reverse (r.) with different intensities of the magnetic field. While run-and-reverse behavior results in band formation for almost any magnetic condition, run-and-tumble bacteria only forma a band without magnetic fields. https://doi.org/10.1371/journal.pcbi.1007548.g006 As already discussed above, run and tumble motion of peritrichous bacteria includes a mechanism for reversals by re-forming the flagellar bundle on the opposite side of the cell body. One would therefore expect that this scenario allows for band formation. This is indeed the case as shown in S13 Fig, but the band is wider than for run and reverse. We tested again combinations of tumbling and reversals and found that a large fraction of reversals (>50%, in excess of what is consistent with the reorientation distribution of E. coli) is needed for robust band formation. Thus, while not all run and tumble mechanism prohibit band formation, it is the reversals included in an E.coli-type tumbling mechanism that are crucial and allow the required motion against the magnetic field. Together, our observations of band formation support the following picture: a mechanism for swimming in the direction imposed by the magnetic field and well as against that direction is required for magneto-aerotaxis. Such a mechanism is provided by reversals. For reversing bacteria, a magnetic field can speed up the formation of an aerotactic band for low inclinations, but does not prohibit it at inclinations close to 90°, consistent with the presence of magnetotactic bacteria close to the Earth’s equator [51]. Magnetoaerotactic band formation with a dynamic gradient To compare our simulations results with experimental data on the magnetically assisted aerotaxis, we perform capillary experiments for the magnetotactic bacterium Magnetospirillum gryphiswaldense MSR-1 and follow the formation of the magneto-aerotactic band. The bacteria, which perform axial magnetotaxis, i.e. swim along the field line without a directional preference, are placed in capillary with one open an one closed end. The interior of the capillary is initially anoxic and oxygen diffuses in from the open end, resulting in an oxygen gradient. A homogeneous magnetic field of 50 μT is applied antiparallel to the oxygen gradient. As a magnetoaerotactic band forms, its position is recorded over time (green data points of Fig 7b). Moreover, we record videos of the stationary band from which the band profile can be extracted as described in Methods (green data points in Fig 7b). From this image, the gray-scale intensity is extracted, which is proportional to the bacterial distribution in the band (green data points of Fig 7d). The bacterial distribution can be fitted by a symmetric Laplace distribution (green line), which gives a characteristic size of the band of 73 μm. Moreover, by tracking individual bacteria, their mean velocity is determined to be 50 μms−1. These parameters are used in the following simulations since we now focus on reproducing a specific system. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 7. Capillary experiments and simulations. a) Scheme of a capillary experiment: a capillary is filled with bacteria; one end of the capillary is sealed, the other is open to the flow of oxygen. The bacteria accumulate at the preferred oxygen concentration, forming a band. b) Experimental band position over time (green data points) for wild type (WT) axial bacteria (optical density OD = 0.1, 5 repetitions, two different days—triangles and circles), with an antiparallel magnetic field of 50 μT, compared to the simulated band position (blue data points) under the same conditions. The red line represents the position of the preferred concentration position as obtained from the simulation. c) Standard deviation of 100 frames (1 s) of a video of the band recorded at 100 fps and 10x magnification (WT OD = 0.1 antiparallel 50 μT). d) The gray-scale values are averaged along the y coordinate of Fig. c), giving the experimental density profile of the band (green data points). The green line represents the Laplace fit (decay length 73 μm). The simulated probability density function (pdf) at equilibrium is represented by the blue data points (decay length 75 μm). https://doi.org/10.1371/journal.pcbi.1007548.g007 We compare these experimental data to our simulations to test our aerotactic model. Compared to the simulations above, where a constant gradient is considered, the simulations are extended to include the dynamics of the oxygen gradient (due to oxygen diffusion and consumption by the bacteria) and the interactions of the bacteria with the walls, see Methods. Interactions between bacteria (specifically excluded volume) are neglected, since the system remains rather dilute even within the band. The run times in the favored and unfavored directions, and the oxygen consumption rate k are used as free fitting parameters. We vary these parameters to match the stationary band size and the position of the band over time. In general, reducing the difference between the two run times and reducing the consumption constant make the dynamics slower; the band size decreases with larger differences between the run times; and the oxygen consumption rate has a strong effect on the stationary position of the band (see S14 Fig S14 for more details on the influence of the parameters on the final outcome). The best match in Fig 7 is obtained with run times of 2 s and 0.9 s and k = 0.005 mol min−1 cell−1. The run times are comparable with the ones of E. coli [28], while they seem to underestimate the mean values for polar MSR1 [32], even though further experiments are needed to measure the biased run times up and down the gradient for magnetotactic bacteria. As it can be seen from Fig 7b and 7d, the simulated data (blue data points) show the same behavior as the experiments (green data points). The band forms close to the open end and moves further into the capillary reaching a stationary position such that the consumption of oxygen (mostly) in the band balances the diffusive flow of oxygen to the band. The density profile of the band is compatible between simulations and experiments, with a band width of 75 μm (Fig 7d). Likewise, the band position is well reproduced by the simulation, at least up to 40 min (data points in Fig 7b); small deviations between the two at larger times likely reflect the fact that the number of bacteria in the band keeps increasing slowly in the simulations due to bacteria arriving from the very far anoxic end of the capillary, while in the experiments many of these bacteria are likely inactive or dead and never reach the band. In the simulation, the band position tracks the location of the preferred oxygen concentration (indicated by the red line), indicating the aerotaxis of the simulated bacteria. Thus, our magneto-aerotactic model gives an effective representation of the system. More simulation at different magnetic fields intensities are show in S15 Fig, which confirm the results obtained in the constant gradient simulations where the band formation is speed up by the magnetic interaction. Discussion In this paper, we propose an effective model for bacterial motility that couples chemotaxis, swimming strategies and external forces and torques. We show that the model reproduces the experimental data for magnetotactic bacteria, and therefore can be used to explore the effect of external forces and torques on the chemotactic behavior of microswimmers, in particular highlighting the effects on the chemotactic behavior of bacteria and biohybrids. We show that forces improve chemotaxis for angles smaller than 90° and totally hinder it for values higher than a threshold value (Fig 2b), for both run and reverse and run and tumble motions. This could be of particular interest when considering bacteria or biohybrids actuated and directed by external forces, such as magnetic swimmers in a magnetic gradient [17] and under the influence of magnetic tweezers [39], or when considering chemotactic swimmers inside a fluid flow [36, 52], for which previous studies ignored the complexity given by chemotactic behaviors. While the effect of the forces does not depend on the strategy of motion of the bacteria, we show that bacteria performing ‘run and reverse’ or bacteria performing ‘run and tumble’ behave differently in the presence of magnetic torques. In particular, even though bacteria performing tumbles have an advantage compared to bacteria performing reversals under a parallel magnetic field, the presence of an antiparallel field completely inhibits chemotaxis in the case of tumbling, and thus tumbling bacteria cannot form a chemotactic/ aerotactic band in the presence of a magnetic field. If the mechanism of tumbling, as the opening and reformation of a flagellar bundle in peritrichously flagellated bacteria, includes a mechanism of reversals (by forming the bundle on the opposite pole of the cell [48], tumbling allows for band formation, but less efficiently and only if such reversals are more probable than what is consistent with observed reorientation distributions. Moreover, it is not clear whether this applies to magnetotactic bacteria, as no peritrichous magnetotactic bacteria are known. On the one hand, this observation could explain why magnetotactic bacteria do not rely on tumbles as the main mechanism to actively change direction and instead use reversals, or 90° changes, or mixed behaviors [31, 32, 45, 53]); on the other hand, this observation provides a crucial design constraint in planning future biomedical applications with biohybrids, magnetically functionalized, but naturally non-magnetotactic organisms. Until now, the swimming and taxis strategies have not been taken into account in detail and the design of such applications only considered the general chemotactic characteristic of the species [16]. Our results indicate that a mechanism for the reversal of motion is needed for chemotaxis in the presence of magnetic guidance, so suitable non-magnetic bacteria must be used; in laternative, flicks with mean angles of 90° could also be suitable (see MC-1 behavior [53]). If reversals due to tumbling with formation of the flagellar bundle on the opposite side of the cell body turn out to be sufficient, the biohybrid design must ensure that this remains possible when the cells are attached to large beads [16] or partly encapsulated [54]. The crucial difference between the two chemotactic strategies is one of directionality more than a difference in the underlying signaling. For both run and reverse and run and tumble, a magnetic field reduces the aerotactic search from a three-dimensional search to one-dimensional one, confined (up to fluctuations in alignment) along the direction of the field. For tumbling bacteria, directed swimming occurs only in the direction of the field, while reversing bacteria can swim in both directions and are thus able to perform one-dimensional chemotactic motion even against the field direction. On top of this directionality effect, there could be more detailed interactions between the chemotactic response and external fields that are mediated by the signaling apparatus. For example, it was shown for E. coli that the swimming response to a chemical gradient results in a behavioral feedback because the signal received by the cell is affected by the cell’s motion [40, 47]. Since the latter motion is modified by the external field, one can expect that under strong gradients the field will impact the chemotactic response also in more subtle ways mediated by the signaling system. A related question is to what extent the impact of the external field is dependent on how close the system is to optimal parameters for the given signaling system, as the chemotactic performance is rather sensitive to the phenotypic parameters [47, 55]. Unfortunately, for magnetotatic bacteria, the chemotactic signaling mechanisms are not very well understood, as only few studies have addressed the underlying molecular signaling system [32]. At a coarse-grained level, however, the rates of directional changes (tumbles or reversals) have to depend on both the local oxygen concentration and the local oxygen gradient [12]. The dependence on the oxygen gradient determines the direction of motion (although a recent theoretical study has proposed a mechanism where a dependence on the local oxygen concentration may be sufficient [56]), while a dependence on the oxygen concentration is needed to switch between oxygen as an attractant and a repellent. Our work can also throw some light on why magnetotactic bacteria produce the intracellular magnetic chain: we demonstrate from the microscopic point of view that parallel or antiparallel magnetic fields speed up chemotaxis considerably, even for weak fields such as the Earth’s magnetic field, confirming and extending previous work at the population level [13–15]. We also show that this advantage is strongly dependent on the angle between magnetic fields and chemical gradients: for high angles, chemotaxis is actually slowed down, but not completely hindered, in accordance with the finding that magnetotactic bacteria exist at the Equator [51]. Moreover, we show that magnetic fields allow the bacteria to perform very long runs without losing the orientation, indeed tracking of single bacteria shows that runs can extend up to 20 s [32]. Why these long runs are useful for the bacteria is still an open question. They could be useful to bacteria in the oxic zone of water above the sediment, which try to return rapidly to the preferred microoxic conditions in the sediment. Future studies will have to explore what happens to the bacteria once they swim inside the sediments. Does the magnetic field still provide an advantage in such porous media? Some research has been done towards this end [15, 36], but more studies are needed. This would also be of great importance for future biomedical applications where the environments are usually crowded, complex and porous [18, 36]. In summary, we propose and study a model for chemotactic motility that can incorporate external forces and torques due to, e.g., tweezers, fluid flow or magnetic fields. These scenarios are experimentally accessible and our model may inform the design of such experiments. Specifically, we apply the model to describe magnetotactic bacteria, but other magnetically guided microswimmers such as biohybrids can be treated in a similar fashion. The model demonstrates benefits and disadvantages of the magnetic guidance of aerotaxis or other chemotaxis. In particular, the functionality of a run and tumble mechanism under magnetic guidance is limited, which may be important for the design of biohybrid microswimmers that are often based on run-and-tumble bacteria such as E. coli [16]. By contrast, a run and reverse mechanism is beneficial in the presence of a magnetic field at moderate inclinations, speeding up chemotaxis/aerotaxis and remains non-prohibitive at high inclinations of the magnetic field relative to the gradient. Beyond the aspects studied here, our model provides a starting point to address the collective behavior of magnetotactic bacteria and other magnetic microswimmers as well as the interplay of the dynamics of taxis with complex confining geometries. Supporting information S1 Text. Implementation of chemotaxis. Implementation of chemotaxis for attractants, reppellents and preferred concentration, available as an attachment. https://doi.org/10.1371/journal.pcbi.1007548.s001 (PDF) S2 Text. Chemotactic velocity for an attractant. Method to calculate the chemotactic velocity for an attractant, available as an attachment. https://doi.org/10.1371/journal.pcbi.1007548.s002 (PDF) S3 Text. Alignment time in the presence of magnetic fields. Derivation of the alignment angle in the presence of magnetic fields, available as an attachment. https://doi.org/10.1371/journal.pcbi.1007548.s003 (PDF) S4 Text. The alignment angle distribution and the mean cosine of the alignment angle. Derivation of the of the alignment angle distribution and the mean cosine of the alignment angle, available as an attachment. https://doi.org/10.1371/journal.pcbi.1007548.s004 (PDF) S1 Source Code. Available as an attachment. https://doi.org/10.1371/journal.pcbi.1007548.s005 (PDF) S1 Fig. Changes of direction. The left column shows a cartoon of the mechanism for the change of direction. The change of direction of bacteria can be performed in different ways. Here in this work we considered tumble (upper row) or reverse (lower row). For example, E. coli performs tumbles with a mean angle of 68° [28]. In our model, tumbling is implemented as rotational diffusion with a high noise strength, which is chosen such as to match this mean angle. This matching is described in S2 Fig. Other bacteria perform run and reverse motion. During a reversal, the body of the bacteria does not re-orient but the bacteria just inverts the direction of velocity. The mean angle is close to 180°, but no exactly, because of thermal noise. In our model, reverse is implemented as a pause mimicking the slowing of motion during a reversal, so there is a time window without active propulsion, during which rotational diffusion due to thermal noise can reorient the cells direction of motion. The middle column shows three example trajectories for run and tumble and for run and reverse, in absence of external forces, torques or gradients. Both run and reverse and run and tumble nicely explore the three dimensional space thanks to thermal noise and thanks to the active changes of direction. A magnetic field provides a torque aligning the direction of motion with the direction of the field and thus, when the field is turned on, the observed behavior changes. Some examples trajectories in the presence of the magnetic field are shown in the right column (blue is without field, red for B = 50 μT -corresponding to the magnetic field of the Earth-, and yellow for B = 500 μT. The field is directed along ). For run and tumble, the trajectories are stretched in the direction of the magnetic field and motion is unidirectional parallel to the field, with excursions away from that direction. These are due to the tumbles that kick the trajectories out of alignment, followed by re-alignment during the runs, provided those are long enough. For run and reverse, the trajectories show bidirectional motion parallel and antiparallel to the magnetic field. The bacteria remain aligned with the field and have the same probability of swimming up or down. Changes of the direction only occur via rotational diffusion which is suppressed by the field, effectively confining the trajectories to a one-dimensional persistent random walk along the axis of the field. https://doi.org/10.1371/journal.pcbi.1007548.s006 (TIFF) S2 Fig. Tumbling effective temperature. Tumbling has been implemented as an effective rotational diffusion with enhanced noise. The strength of that noise can formally be described by an effective ‘tumbling temperature’, Ttumble. This parameter is chosen such that the resulting mean tumbling angle matches the measured tumbling angle of 〈θtumble〉 = 68° observed for E. coli [28]. To do so we generated 50000 tumbling angles from a simulation of our model for different noise strengths and calculated the mean and standard deviation for each noise strength to find a parameter value that matches the experimental angle. The normalized distribution of the tumbling angle obtained with the best fit parameter Ttumble = 4.2 × 104 K is shown here as black bars, and it is seen to be very close to the experimental distribution of Berg and Brown [28] (blue points), with the same mean and standard deviation (〈θtumble〉 = (68 ± 40)° compared to 〈θtumble〉 = (68 ± 36)° [28]) and a very similar skewness to the right. https://doi.org/10.1371/journal.pcbi.1007548.s007 (TIFF) S3 Fig. Dependence of the chemotactic response on the parameter ∇C0. Effect in capillaries. Band at 1 min (blue), 10 min (yellow) and 20 min (light blue) and the corresponding oxygen concentration (filled line, dashed line and dotted line) for a) ∇C0 = 100 μMmm−1, b)∇C0 = 1 μMmm−1, c) ∇C0 = 0.01 μMmm−1, d) Step-function. The black arrow shows the time direction. The inset shows the zoom of the band in d). Here the oxygen concentration is integrated only along the long axis of the capillary and it is considered constant in the cross section of the capillary. The band forms only for values of ∇C0 close to 0, so for simplicity the response function τrun is chosen to be a step function, in accordance with previous works [12, 13]. Effect of ∇C0 on the chemotactuc velocity for a constant gradient. Mean position of 1000 bacteria performing attractive chemotaxis with a constant gradient. The slope of the line gives the chemotactic velocity. The results are obtained in green for 0 magnetic field, and in blue for a parallel magnetic field of 50 μT. Filled lines correspond to ∇C0 = 0 (equivalent to considering a step-function for τrun), dashed lines correspond to ∇C0 = ∇C/10, and dotted lines to ∇C0 = ∇C. For both tumble e) and reverse f) and in all magnetic conditions, it can be seen that chosing ∇C0 = 0 or ∇C0 = ∇C/10 gives the same result, while chosing ∇C0 = ∇C slows down chemotaxis of 15% for tumble and 0 μT, 9% for tumble and 50 μT, 20% for reverse and 0 μT and 8% for reverse and 50 μT. In conclusion, changing the cutoff ∇C0 (thus changing the linear behavior range in trun) affects in minimum part the chemotactic velocity. The effect of ∇C0 for a dynamic gradient in a capillary is explored in the supplementary S3 Fig. Again, changing the gradient cut slows down or speeds up the chemotaxis; the effect is more pronounced for this system. https://doi.org/10.1371/journal.pcbi.1007548.s008 (TIFF) S4 Fig. Band size in the modified model. Band size as function of time for the models with and without resetting of the run (yellow and blue, respectively) after crossing the preferred concentration. Without resetting the bacterium is allowed to finish its run after crossing the preferred concentration, while with resetting a new run is started (likely with a smaller run time) when the bacterium crosses the preferred concentration and runs in the unfavorable direction (thus shorter). As it can be seen, resetting results in a smaller band size, which is also necessary to match the experimental data. For these simulations, tup = 6 s, tdown = 1 s and k = 0.01 fmol min−1 cell−1. https://doi.org/10.1371/journal.pcbi.1007548.s009 (TIFF) S5 Fig. Alignment time in the presence of magnetic fields. The relaxation of the magnetic alignment after a tumble is described by , see S3 Text. As an example, we fit (case a), in red) the cosine of the alignment angle θe,B after a tumble for 500 μt without noise (case a), in blue), the case for which the theory was derived. With the parameters of our simulation, the expected value for is 4.4 s−1. This value is recovered by fitting the simulation data, giving β = 4.1s−1. The fit has been performed with the function , where b evaluates , a evaluates ez0 and d is needed to re-scale the times to 0. The data have been fitted with f(x) as described in the S3 Text (in red). Then we consider the cosine of the angle θe,B after a tumble event at B = 500 μT in the presence of thermal noise (case b), blue curve). The data have been fitted with g(x) = f(x) + g0 (in red). The constant g0 = 〈cosθe,B〉 reflects the nonzero mean value of the cosine in thermal equilibrium. The fit in this case leads to β ≃ 4.0 s−1, still in good agreement with the theory derived in absence of noise. For a magnetic field of B = 50 μT, the strength of the magnetic field of the Earth, the relaxation constant is β = 0.44 s−1, corresponding to a decay time of . However, in that case, the fluctuations around that decay are considerably more pronounced when the thermal noise is present. https://doi.org/10.1371/journal.pcbi.1007548.s010 (TIFF) S6 Fig. Data collapse for the mean cosine. The Langevin curves for the cosine of the alignment angle can be plotted as function of MB/kBTeff, with the effective temperature determined by fitting the Langevin function to the individual curves. In this case, all data for different run times collapse on one Langevin curve, when the cosine of the alignment angle is plotted as function of the effective Langevin parameter MB/kBTeff with a run-time dependent effective temperature. Only small deviations from the theoretical curve are observed, but not as strong as the deviation seen for the histogram of the alignment angle. We emphasize that the effective temperature which we obtain by fitting the Langevin plots is different from the tumbling temperature, it depends on the mean tumble time and on the rotational friction coefficient and generally satisfies T < Teff < Ttumble. https://doi.org/10.1371/journal.pcbi.1007548.s011 (TIFF) S7 Fig. Velocity condensation. Alignment angle for run and tumble at B = 500 μT, for 3D motion (purple curve) with and without thermal noise during runs (diamonds/stars) compared to a 2D projection (blue curve) with and without thermal noise (diamonds stars). While 3D and the 2D projections differ from each other when thermal noise during runs is present, when thermal noise is taken away the two cases coincide, and they show the velocity condensation phenomenon described by Rupprecht et al. [44]. In dark blue, the theoretical curve at small angles Pθ θ−1+KT/mB [44]. The effect they describe results true only without thermal noise. https://doi.org/10.1371/journal.pcbi.1007548.s012 (TIFF) S8 Fig. Chemotaxis up a gradient in presence of a magnetic field. Chemotactic velocity as a function of the field inclination relative to the gradient for tumble (a) and reverse (b) at different magnetic field velocities. For a very strong magnetic field and tumble, we see that in the parallel case, the self velocity of 14.2 μm is reached. For run and reverse, the velocities are always smaller than for the corresponding tumble case at 0°. The same plots are presented in (c) (tumble) and (d) (reverse), but here they are collapsed since now the velocity is normalized to 1. In black, the cosine of the magnetic angle for the tumble and of the absolute value of the cosine for reverse. Higher magnetic field follow the theoretical cosine better. https://doi.org/10.1371/journal.pcbi.1007548.s013 (TIFF) S9 Fig. Shortened runs down the gradient. We tested whether shortening runs down the gradient changes our results. When the bacterium is running down the gradient, in the model used in the main text the runs have a fixed duration (left Fig), in this case of 0.5 s, while the maximal run time up the gradient is of 1.5 s. In the right Fig instead, the runs down the gradient are shortened following a linear decrease up to a value of 0.5 s, reached at −∇C0 (right). The advantage of tumbling over reversals in the case of a parallel field is also seen with shortened runs down the gradient, but is less pronounced than without the shortening. https://doi.org/10.1371/journal.pcbi.1007548.s014 (TIFF) S10 Fig. Chemotactic velocities for tumbling with reversals. We tested two different mechanisms that combine tumbles with reversals. (a) In the first case, after each tumble the direction of motion is chosen randomly as either parallel or antiparallel to the body orientation mimicking the formation of a flagellar bundle on either side of the body (data in purple). In that case, motion up the gradient under an antiparallel magnetic field is possible, but slower than in the parallel case and slower than in the absence of the field. For comparison, tumble without the randomization of the direction of motion is shown in orange. (b) In the second case, the bacterium performs tumbles or reversals with certain probabilities at the end of each run. We simulated a combination of reversals and tumbles that interpolates smoothly between the two types of behaviors. We varied the fraction of reversals and found that swimming up the gradient against a magnetic field requires more than 50% reversals. https://doi.org/10.1371/journal.pcbi.1007548.s015 (TIFF) S11 Fig. Non-linear regime with antiparallel magnetic fields. We tested a scenario where runs up the gradient are very long and cells spend almost all time running as expected for a nonlinear chemotactic regime. Here, the run times up the gradient are 20 s, while the run times downward are 1 s. The averaged position of the bacteria as function of time is plotted for reverse and tumble. Positive values correspond to bacteria climbing up the gradient. Here, an antiparallel magnetic field is used and 100 cycles of run and change of directions are plotted. Reversing bacteria practically run always up the gradient with very long runs, while tumbling bacteria are forced to follow the antiparallel magnetic field, and are thus forced to swim down the gradient with very short runs. https://doi.org/10.1371/journal.pcbi.1007548.s016 (TIFF) S12 Fig. Band width estimation. The size of the band is estimated in the main paper as the standard deviation of the position of the bacteria along the gradient. An alternative way to estimate it is the following: the density profile of the band after the equilibrium time (in this case for run and reverse of 477 s) is obtained (Fig (a)), integrating over all time-steps after the equilibration and normalizing it by the number of time-steps and number of bacteria. The profile can be well fitted with a Laplace distribution , where the free fitting parameters are m, the position of the preferred concentration and L, the decay length of the curve. The curve is symmetrical with respect to the preferred concentration (which is situated at 2000 μm and indicated by the red line for this case without magnetic field). L is thus a good estimator of the size of the band, since 68% of the density is situated in [m − L, m + L]. The bands seen in the simulations have a symmetric shape and their density profiles decay on both sides with the same decay length (this has been tested by fitting both sides separately with an exponential). Comparing the two approaches, we notice that the results show the same trends, but using the Laplace distribution fit leads to systematically smaller values. This is due to the fact that the standard deviation is influenced by the long tails of bacteria that are not in the band, which increase the estimate of the band width compared to L. In Fig (b), we show the band width obtained by the fit with the Laplace distribution for different magnetic field intensities, as a function of the magnetic field orientation. https://doi.org/10.1371/journal.pcbi.1007548.s017 (TIFF) S13 Fig. Band formation for tumble with reverse. Band formation in a constant gradient for reverse, tumble, tumble with random change in orientation and mix of 50% reverse and 50% tumble at different magnetic field intensities. In the plot, the position of 100 bacteria is shown after 500 cycles of run and change of direction. The bacteria are initial distributed randomly within the black box at [1600, 2400] μm. The preferred concentration is situated at 2000 μm (red dotted vertical line). The x axis is in μm, y axis has size of 800 μm. While reverse and tumble with random orientation can form the band in any condition, pure tumble and a mixture of tumble and reverse perform chemotaxis poorly and do not form a band that is well localized around the preferred concentration. https://doi.org/10.1371/journal.pcbi.1007548.s018 (TIFF) S14 Fig. Choice of the free parameters for the capillary assay. The model of the capillary assay presents three free parameters that could not be determined by the experiments: tup the run time towards a favorable direction, tdown towards an unfavorable direction, and the oxygen consumption k. First we varied the run times to get the desired band size (Fig a). We plot here the band size as function of the two run times for an antiparallel field of 50 μT; the green plane corresponds to the desired experimental value. The blue points were obtained keeping tdown constant; the green points, keeping tup constant; and the red points, keeping the ration between the two times constant. The winning point is the bright red case, obtained for tup = 2 s and tdown = 0.9 s with k = 0.01 fmol min−1 cell−1 (see [12, 13]). The point size is proportional to the ratio of the times. Then, to match the band position dynamics (Fig b), the consumption constant k is varied. Smaller consumption constants make the dynamics slower. To match the experimental data points in green, we chose k = 0.005 fmol min−1 cell−1. https://doi.org/10.1371/journal.pcbi.1007548.s019 (TIFF) S15 Fig. Capillary assay at other magnetic field intensities. Since the model provides an effective description of the system, the effect of the magnetic field can be explored also in this capillary assay. The dynamics of the band position depends on the magnetic field intensity (a): stronger magnetic fields stabilize the band closer to the air-water interface thanks to the interplay between aerotaxis, oxygen consumption and magnetic fields. The band position is represented by the data points and the preferred concentration (C*) position by the red lines. Magnetic fields do not influence the band position at small times (below 5 min), when the leading dynamics is the oxygen flow as it can be seen from Fig (a), where pure oxygen flow -dashed red curve- matches the curves in the presence of bacteria. On the contrary, the magnetic field speeds the formation of the band up at small times before reaching the equilibrium: the band formed with the Earth magnetic field (blue curve of Fig (b) and with 500 μT (yellow curve) is already symmetrical at 5 min and more dense compared to the band formed without magnetic field (in red). Therefore, in agreement with the results for a constant gradient, antiparallel magnetic fields help the band formation. https://doi.org/10.1371/journal.pcbi.1007548.s020 (TIFF) S1 Table. Simulation parameters for free bacteria. Simulation parameters for free swimming bacteria. Some comments are in order with respect to these parameters. (i) The friction coefficients are calculated assuming a spherical shape of the bacterium/microswimmer. The radius of the sphere was chosen to be 1μm, a values compatible with the dimension of E. coli bacteria [28], as well as of some magnetotactic bacteria, in particular strain MSR-1 [11]. (ii) The magnetic moment was chosen to match the one of MSR-1 [57]. Here, a magnetic crystal with size 25 nm possesses a magnetization of ≃ 3.1 × 10−17 Am2. Considering a typical crystal of MSR-1 [58, 59] with radius 50nm, and considering a mean of 20-30 crystals per bacterium [11], we obtain the final value of 0.6 × 10−3 A μm2, in good agreement with the magnetic moment calculated in [43]. (iii) Regarding the swimming speed, the mean run time with and without chemical gradients, and the mean tumble time, the parameters were chosen as measured for E. coli [28]. The parameters for magnetotactic bacteria have not been measured at the same level of detail, but swimming speeds tend to be larger and to vary greatly between studied strains [12]. (iv) The gradient was chosen constant in space and time, with a value similar to the serine gradient experienced by the bacteria in the experiments of Berg and Brown [28]. Oxygen gradients that built up dynamically in experiments with magnetotactic bacteria were seen to have similar values [12]. The preferred concentration was chosen in the range of oxygen concentration preferred by microaerophilic bacteria [13, 60]. https://doi.org/10.1371/journal.pcbi.1007548.s021 (PDF) S2 Table. Simulation parameters for bacteria in capillaries. Simulation parameters for capillary assays. If not stated here, the parameters are unchanged with respect to S1 Table. For the capillary assay, the oxygen concentration and gradient are dynamic and not constant. The run-time response function to the gradient is a step function, thus the reference gradient ∇C0 is 0. https://doi.org/10.1371/journal.pcbi.1007548.s022 (PDF)

Mair, Colette;Nickbakhsh, Sema;Reeve, Richard;McMenamin, Jim;Reynolds, Arlene;Gunson, Rory N.;Murcia, Pablo R.;Matthews, Louise

doi: 10.1371/journal.pcbi.1007492pmid: 31834896

Introduction Animals and plants are exposed to a wide range of pathogenic organisms that co-circulate in time and space. When multiple pathogens infect the same tissue, they form diverse communities, effectively sharing an ecological niche that provides the opportunity for interspecific interactions [1–3]. It is known that pathogen interactions may alter the within-host dynamics of infection with consequences for the population transmission of some common infections. Interactions among microorganisms include the promoting or inhibiting effects of gut microbiota on invading pathogenic bacteria in the gastrointestinal tract [4]; the enhanced carriage of pneumococcal bacteria following influenza infection in the respiratory tract [5]; the rise in human monkeypox after eradication of smallpox [6]; and immune-driven enhancement of Zika virus infection following Dengue virus exposure [7]. The complex ecology of pathogen communities therefore has potentially important implications for the epidemiology and control of infectious diseases. Pathogens that act non-independently and their health implications is an actively growing and important area of research [8]. Pathogen interactions can be cooperative or competitive and can occur within a host or in a population where pathogens co-circulate [9]. While some evidence of population-level interactions between pathogens exists, statistical support for the occurrence of pathogen-pathogen interactions from multiple non-stationary time series independent of prior biological or ecological knowledge is lacking. This is due in part to a paucity of appropriate long-term time series data that describe infection frequencies for multiple pathogens simultaneously, allowing such interactions to be identified, but also due to statistical techniques that are limited in their ability to handle such complex datasets [10, 11]. Various statistical methods are available to analyse health-related time series data. Statistical methods for handling non-stationary time series data include multiple regression and generalised additive models, which are able to capture non-linear trends and explanatory factors such as seasonality and climate as well as other confounders and typically model a univariate health outcome as opposed to a multivariate distribution of several non-independent outcomes [12–18]. Consequently, they do not necessarily focus on estimating pathogen-pathogen interactions. More specialist techniques that focus on decomposition of the time series include singular spectrum analysis and wavelet analysis. Singular spectrum analysis has been used to model interactions between a pathogen and an environmental factor [19], whilst wavelet decomposition has been used to infer pathogen-pathogen interactions [20] and virus-virus interactions [21]. These techniques only capture pairwise relationships between time series (for example pathogen-pathogen or pathogen-environment) although in principal singular spectrum analysis can be extended to multiple time series [22]. Moreover these methods do not account for or adjust the data for potential confounders. Another recent approach is to use mechanistic stochastic models to estimate time varying parameters (e.g. a transmission rate) and then employ wavelet analysis to compare with potential weather or climatic drivers [11], but again in a pairwise manner. Alternative approaches that focus specifically on identifying interactions include confirmatory analyses that fit observed time series data from two pathogens to models containing hypothesised interactions [9, 23]. Extending to multiple pathogens increases the complexity of this approach [9]. Confirmatory analyses rely on prior biological and ecological knowledge in order to hypothesize an appropriate model with interpretable parameters. Specifically, the ‘true’ interaction needs to be modelled and therefore such analyses cannot capture unexpected or unknown interactions [10]. In contrast, exploratory approaches such as Granger-Causality and Transfer Entropy can provide robust statistical evidence for unknown interactions from multiple time series whilst accounting for confounding variables [24], and have been used to detect virus-virus interactions [10]. However, they rely on stationarity of the times series, and non-stationarity can generate spurious results [25]. This limits the applicability of this approach to many epidemiological time series since seasonality and long term trends (and therefore non-stationarity) is a long-recognised attribute of many infectious diseases [26]. A framework that can infer unknown interactions from multiple pathogens incorporating non-stationary time series data whilst adjusting for confounding factors will advance this important research area [10]. Here, we construct just such a robust framework, which is able to identify pathogen-pathogen interactions from multiple non-stationary time series at the population scale independent of prior biological or ecological knowledge. The conceptual framework for our new approach derives from Bayesian disease mapping models—a class of regression model that has received much attention in recent years for the analysis of spatial distributions of incidence data routinely collected by public health bodies [27, 28]. These models are typically applied to incidence data to estimate spatial patterns of disease risk over a geographical region—with several models proposed to capture spatial autocorrelations [19] using conditional autoregressive priors [29, 30]. While some extensions to disease mapping models have been made to include temporal patterns [29, 31] and space-time interactions [32, 33], most disease mapping applications focus on spatial structures [34] with temporal dependencies in disease incidence often being overlooked [35, 36]. Modelling multiple pathogens simultaneously allows assessment of related patterns and non-independence of infection risk. Multivariate forms of disease mapping models provide a suitable framework for estimating temporal dependencies between pathogens as they naturally incorporate a between-disease (or pathogen) covariance matrix [37]. In this paper, we construct a framework for time series data analysis that allows the estimation of covariances among temporal disease datasets. Because the approach accounts for confounding variables and sources of non-stationarity such as seasonally varying infection risk, the resulting statistical framework now enables the joint estimation of pathogen dependencies on the temporal dimension whilst, crucially, distinguishing genuine pathogen-pathogen interactions from simple correlations. To validate our method we conducted extensive simulation studies using synthetic data. We then applied the method to diagnostic data on five respiratory viruses (adenovirus [AdV], coronavirus [Cov], human metapneumovirus [MPV], influenza B virus [IBV] and respiratory syncytial virus [RSV]) from the patient population of a major urban UK population (Glasgow, United Kingdom) over a period of nine years. We chose this particular group of pathogens because i) respiratory viruses are obligate intracellular pathogens that have a strong predilection for the cells of the respiratory tract (i.e. they share the same ecological niche); ii) contemporary diagnostic tests based on multiplex real-time PCR (qPCR) technology allow the simultaneous detection of multiple respiratory viruses from the same patient; and iii) multiplex qPCR was routinely used to diagnose respiratory viruses in the patient population of Glasgow during the 2005-2013 period. Modelling approach The framework presented infers unknown pathogen interactions adjusted for confounding factors such as seasonality, demographics and testing frequencies using time series data from multiple contemporaneous pathogens (Fig 1). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Model used to estimate pairwise relative risk covariances. The diagram should be read from the bottom (starting with Ymtv) to the top. All prior choices have been fully specified. Numbers indicate hyperparameter choices, for instance, mean and variance in the normal distribution, lower and upper bound in the uniform distributions and shape and rate in the gamma distribution. Numbers in red indicate all relevant subscripts month m = 1, …, 12, year t = 1, …, 9 and virus v = 1, …, 5. Green arrows correspond to the neighbourhood structure and maroon arrows correspond to the autoregressive structure. https://doi.org/10.1371/journal.pcbi.1007492.g001 We used Ymtv to denote the observed count of pathogen v during the mth month of year t conditional on expected count Emtv and relative risk RRmtv with αv an intercept term specific to virus v and ϕ.t. = {ϕ.t1, …, ϕ.tV} a vector of random effects modelled conditionally through a MCAR prior Estimating expected counts enables us to adjust for potential and established confounding factors. For instance, the virus diagnostic data allowed us to use age, sex, whether the patient had attended a general practice or hospital (as a proxy for infection severity), month of year and testing frequencies. Therefore, expected counts explained a proportion of the variation in the observed counts and we attributed the remaining unexplained variation to temporal autocorrelation, virus-virus interactions and residual random variation. The temporal autocorrelation is handled by adapting the approach from MCAR (Multivariate Conditional Autoregressive) models, designed to model spatially autocorrelated data based on neighbourhood relationships. Here, the parameterisation of a MCAR model captured both the seasonal trends of each pathogen via precision matrix Ω and non-independence between pathogens via Λ. Temporal effects ϕ.t. captured long term temporal tends with smoothing parameters s1, …, sV. Dependency structures between neighbouring months accounted for seasonality in pathogen infection frequencies. Two such structures were considered, namely the neighbourhood structure (Fig 1 green arrows), where all neighbouring months are equally correlated to the month in question, and the autoregressive structure (Fig 1 maroon arrows), where there is a power law weighting the correlation between related months and the month in question. This method focuses primarily on the estimation of pathogen covariance matrix Λ−1. By formally testing which off-diagonal entries of are significantly different from zero, we can explicitly provide statistical support for pathogen interactions. Results Simulation study In order to validate the proposed method, we performed an extensive simulation study using synthetic virus diagnostic data with a wide range of time series structures and estimated the power and type 1 error rate (i.e. rejection of the true null hypothesis) of this method for a range of correlations between viruses. Individual level data of age, sex and general practice versus hospital attendance (a proxy for infection severity) were simulated to reflect the real virus diagnostic data, and the probabilities of infection for each virus within each month were estimated. For a full data description, we refer readers to Nickbakhsh et al [38]. Within each year, the number of samples tested for each virus per month ranged from 20 to 200 to reflect variable testing frequencies. Expected counts were calculated through standardised infection probabilities and testing frequencies. Generation of the matrix Ω depended on the choice of correlation structure (either neighbourhood or autoregressive). Relative risks were calculated from the virus specific intercept term αv, simulated uniformly, and monthly effect terms ϕ..v. Monthly effect sizes were simulated without constraining the nature of time series data in order to illustrate the flexibility of this framework (Fig 2). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Examples of simulated temporal effects (ϕ..v) for three viruses. Illustrations of seasonal autoregressive integrated moving average time series data simulated under parameter settings used in simulation study. https://doi.org/10.1371/journal.pcbi.1007492.g002 An example of data simulated under the neighbourhood structure is present in Fig 3. A full description of the simulation setup and parameter choices is given in the material and methods. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Example of simulated observed and expected counts. An example of observed and expected counts simulated from three viruses using the method described in the simulation study section. https://doi.org/10.1371/journal.pcbi.1007492.g003 Since our approach incorporated two structures that captured monthly autocorrelations (the neighbourhood structure (N) and autoregressive structure (A) either adjusting for multiple comparisons (post-mcc) or not (pre-mcc)), four possible combinations of simulation (Sim) and estimation (Est) are reported (Table 1). A range of correlations between two viruses were considered from weakly related viruses (correlation = 0.2) to a moderately strong correlation (correlation = 0.5) based on data simulated from three viruses over five years with two viruses correlated and the remaining virus independent. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Simulation structure. https://doi.org/10.1371/journal.pcbi.1007492.t001 Power and type 1 error control. Without correcting for multiple comparisons (pre-mcc) the power of detecting a moderately strong correlation of 0.5 was greater than 0.8 under each of the four scenarios (Fig 4, power pre-mcc). As expected, as the strength of the relationship between viruses increased, the power also increased. On the other hand, this test was unable to adequately control the type 1 error rate at a 5% significance level without correcting for multiple comparisons (Fig 4, Type 1 error pre-mcc). Therefore, as the number of related viruses increased, we were more likely to infer false relationships between viruses. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Power and type 1 error rate. Estimated power (top) and type 1 error (bottom) based on analysis of synthetic data for three viruses. Data were simulated (Sim) under one of two structures, neighbourhood (N) and autoregressive (A) and parameters estimated (Est) under one of the two structures. Results shown for no multiple comparison correction (pre-mcc), left, and with a multiple comparison correction (post-mcc), right. https://doi.org/10.1371/journal.pcbi.1007492.g004 After correcting for multiple comparisons, the power of the test ranged from around 0.2 in the case of weakly correlated viruses (Fig 4, power post mcc). As expected, power decreased after correcting for multiple comparisons. We were able to precisely and accurately estimate, and generally found better control of, the type 1 error rate after correcting for multiple comparisons. However, we found no significant difference in the type 1 error rate pre and post multiple comparison correction (Fig 4, type 1 error pre and post mcc). Overall, we found the autoregressive model to be more powerful in inferring correlations between viruses (Fig 4, power post mcc, purple line) with the least amount of success inferring correlations with the neighbourhood model (Fig 4, power post mcc black, lines). For instance the autoregressive model had an estimated power of 0.9 when λ = 0.5 whereas the neighbourhood model had an estimated power of 0.68. Virus diagnostic data From the 28,647 patient episodes, defined as aggregated samples taken from each patient over a 30-day window, 4,759 were positive to at least one virus group and detection was most common in children aged between 1 and 5 years. Detection of any virus in a given episode was most common in December and least common in August. We observed differing patterns between the five viruses (Fig 5, black lines). IBV, RSV and CoV were more prevalent in winter months (November, December and January), AdV was generally less common with a slight increase in prevalence in spring months (April, May and June) and MPV shifts from winter peaks (January and February) to spring peaks (March, April and May) after 2010. IBV was the only virus not to display a regular seasonal pattern. This virus peaked in winter during 2005/2006, 2007/2008, 2008/2009, 2010/2011 and 2012/2013 but failed to peak during the winter periods 2006/2007, 2009/2010 and 2011/2012. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Observed, expected and fitted counts of AdV, hCov, hMPV, IBV and RSV. Observed (black), expected (purple) and fitted (light blue) counts of the five groups of respiratory viruses between January 2005 and December 2013. A full description of the estimated expected counts is given in the expected count section. Fitted values are based on autoregressive model. https://doi.org/10.1371/journal.pcbi.1007492.g005 Estimated infection expected counts. Expected counts were estimated for each virus and shown in Fig 5 (purple lines). The expected number of infections of AdV infection remained relatively high between 2005 to 2010 but decreased during the summer and autumn months of 2011, 2012 and 2013. We found an increased expected number of IBV infection during the autumn and winter periods of 2005/2006, 2010/2011 and 2012/2013. During the second half of 2009, we found a heightened risk of RSV and MPV infections. More generally, the risk of RSV infection peaked during late summer through to autumn from 2008 onwards whereas the risk of MPV infection shifted from winter, between 2005 and 2008, to summer, from 2011 onwards. Virus-virus interactions For comparison, we first fitted a null model that assumed all five viruses to be independent by setting Λ−1 = I5 (the identity matrix of dimension 5 × 5). Under the neighbourhood structure, we found that allowing dependencies between viruses (Λ−1 ≠ I5) provided a better fit to the data (DIC = 2795.6 versus DIC = 3583.8 for the null model). However, the autoregressive structure with Λ−1 ≠ I5 minimised DIC (DIC = 2686.4). Comparing observed values to fitted values under the autoregressive model fit (Fig 5, black and light blue lines respectively for each virus) to informally check model fit, we were able to accurately and precisely estimate observed counts of each virus across the nine year time period. Correlations between observed and fitted values ranged from 0.96 (p-value < 0.001) for AdV and 0.9997 (p-value < 0.001) for IBV (S2 Appendix). More precisely, our model captured winter peaks in CoV, winter and spring peaks in MPV and irregularities in AdV and IBV validating the model fit to these data. Under the neighbourhood structure, we found a positive covariance between RSV/MPV and negative covariances between IBV/MPV, CoV/MPV and AdV/IBV (Table 2, Wneigh). Under the autoregressive structure, we found a positive covariance between RSV/MPV and a negative covariance between IBV/AdV (Table 2, Wauto), with adjusted p-values for the covariances between IBV/MPV and CoV/MPV of 0.075 and 0.073 respectively. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 2. Estimated covariances between AdV, Cov, MPV, IBV and RSV. https://doi.org/10.1371/journal.pcbi.1007492.t002 Our analysis showed robust statistical evidence of a facilitative form of interaction between RSV and MPV and a competitive form of interaction between IBV and AdV. Simulation study In order to validate the proposed method, we performed an extensive simulation study using synthetic virus diagnostic data with a wide range of time series structures and estimated the power and type 1 error rate (i.e. rejection of the true null hypothesis) of this method for a range of correlations between viruses. Individual level data of age, sex and general practice versus hospital attendance (a proxy for infection severity) were simulated to reflect the real virus diagnostic data, and the probabilities of infection for each virus within each month were estimated. For a full data description, we refer readers to Nickbakhsh et al [38]. Within each year, the number of samples tested for each virus per month ranged from 20 to 200 to reflect variable testing frequencies. Expected counts were calculated through standardised infection probabilities and testing frequencies. Generation of the matrix Ω depended on the choice of correlation structure (either neighbourhood or autoregressive). Relative risks were calculated from the virus specific intercept term αv, simulated uniformly, and monthly effect terms ϕ..v. Monthly effect sizes were simulated without constraining the nature of time series data in order to illustrate the flexibility of this framework (Fig 2). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Examples of simulated temporal effects (ϕ..v) for three viruses. Illustrations of seasonal autoregressive integrated moving average time series data simulated under parameter settings used in simulation study. https://doi.org/10.1371/journal.pcbi.1007492.g002 An example of data simulated under the neighbourhood structure is present in Fig 3. A full description of the simulation setup and parameter choices is given in the material and methods. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Example of simulated observed and expected counts. An example of observed and expected counts simulated from three viruses using the method described in the simulation study section. https://doi.org/10.1371/journal.pcbi.1007492.g003 Since our approach incorporated two structures that captured monthly autocorrelations (the neighbourhood structure (N) and autoregressive structure (A) either adjusting for multiple comparisons (post-mcc) or not (pre-mcc)), four possible combinations of simulation (Sim) and estimation (Est) are reported (Table 1). A range of correlations between two viruses were considered from weakly related viruses (correlation = 0.2) to a moderately strong correlation (correlation = 0.5) based on data simulated from three viruses over five years with two viruses correlated and the remaining virus independent. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Simulation structure. https://doi.org/10.1371/journal.pcbi.1007492.t001 Power and type 1 error control. Without correcting for multiple comparisons (pre-mcc) the power of detecting a moderately strong correlation of 0.5 was greater than 0.8 under each of the four scenarios (Fig 4, power pre-mcc). As expected, as the strength of the relationship between viruses increased, the power also increased. On the other hand, this test was unable to adequately control the type 1 error rate at a 5% significance level without correcting for multiple comparisons (Fig 4, Type 1 error pre-mcc). Therefore, as the number of related viruses increased, we were more likely to infer false relationships between viruses. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Power and type 1 error rate. Estimated power (top) and type 1 error (bottom) based on analysis of synthetic data for three viruses. Data were simulated (Sim) under one of two structures, neighbourhood (N) and autoregressive (A) and parameters estimated (Est) under one of the two structures. Results shown for no multiple comparison correction (pre-mcc), left, and with a multiple comparison correction (post-mcc), right. https://doi.org/10.1371/journal.pcbi.1007492.g004 After correcting for multiple comparisons, the power of the test ranged from around 0.2 in the case of weakly correlated viruses (Fig 4, power post mcc). As expected, power decreased after correcting for multiple comparisons. We were able to precisely and accurately estimate, and generally found better control of, the type 1 error rate after correcting for multiple comparisons. However, we found no significant difference in the type 1 error rate pre and post multiple comparison correction (Fig 4, type 1 error pre and post mcc). Overall, we found the autoregressive model to be more powerful in inferring correlations between viruses (Fig 4, power post mcc, purple line) with the least amount of success inferring correlations with the neighbourhood model (Fig 4, power post mcc black, lines). For instance the autoregressive model had an estimated power of 0.9 when λ = 0.5 whereas the neighbourhood model had an estimated power of 0.68. Power and type 1 error control. Without correcting for multiple comparisons (pre-mcc) the power of detecting a moderately strong correlation of 0.5 was greater than 0.8 under each of the four scenarios (Fig 4, power pre-mcc). As expected, as the strength of the relationship between viruses increased, the power also increased. On the other hand, this test was unable to adequately control the type 1 error rate at a 5% significance level without correcting for multiple comparisons (Fig 4, Type 1 error pre-mcc). Therefore, as the number of related viruses increased, we were more likely to infer false relationships between viruses. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Power and type 1 error rate. Estimated power (top) and type 1 error (bottom) based on analysis of synthetic data for three viruses. Data were simulated (Sim) under one of two structures, neighbourhood (N) and autoregressive (A) and parameters estimated (Est) under one of the two structures. Results shown for no multiple comparison correction (pre-mcc), left, and with a multiple comparison correction (post-mcc), right. https://doi.org/10.1371/journal.pcbi.1007492.g004 After correcting for multiple comparisons, the power of the test ranged from around 0.2 in the case of weakly correlated viruses (Fig 4, power post mcc). As expected, power decreased after correcting for multiple comparisons. We were able to precisely and accurately estimate, and generally found better control of, the type 1 error rate after correcting for multiple comparisons. However, we found no significant difference in the type 1 error rate pre and post multiple comparison correction (Fig 4, type 1 error pre and post mcc). Overall, we found the autoregressive model to be more powerful in inferring correlations between viruses (Fig 4, power post mcc, purple line) with the least amount of success inferring correlations with the neighbourhood model (Fig 4, power post mcc black, lines). For instance the autoregressive model had an estimated power of 0.9 when λ = 0.5 whereas the neighbourhood model had an estimated power of 0.68. Virus diagnostic data From the 28,647 patient episodes, defined as aggregated samples taken from each patient over a 30-day window, 4,759 were positive to at least one virus group and detection was most common in children aged between 1 and 5 years. Detection of any virus in a given episode was most common in December and least common in August. We observed differing patterns between the five viruses (Fig 5, black lines). IBV, RSV and CoV were more prevalent in winter months (November, December and January), AdV was generally less common with a slight increase in prevalence in spring months (April, May and June) and MPV shifts from winter peaks (January and February) to spring peaks (March, April and May) after 2010. IBV was the only virus not to display a regular seasonal pattern. This virus peaked in winter during 2005/2006, 2007/2008, 2008/2009, 2010/2011 and 2012/2013 but failed to peak during the winter periods 2006/2007, 2009/2010 and 2011/2012. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Observed, expected and fitted counts of AdV, hCov, hMPV, IBV and RSV. Observed (black), expected (purple) and fitted (light blue) counts of the five groups of respiratory viruses between January 2005 and December 2013. A full description of the estimated expected counts is given in the expected count section. Fitted values are based on autoregressive model. https://doi.org/10.1371/journal.pcbi.1007492.g005 Estimated infection expected counts. Expected counts were estimated for each virus and shown in Fig 5 (purple lines). The expected number of infections of AdV infection remained relatively high between 2005 to 2010 but decreased during the summer and autumn months of 2011, 2012 and 2013. We found an increased expected number of IBV infection during the autumn and winter periods of 2005/2006, 2010/2011 and 2012/2013. During the second half of 2009, we found a heightened risk of RSV and MPV infections. More generally, the risk of RSV infection peaked during late summer through to autumn from 2008 onwards whereas the risk of MPV infection shifted from winter, between 2005 and 2008, to summer, from 2011 onwards. Estimated infection expected counts. Expected counts were estimated for each virus and shown in Fig 5 (purple lines). The expected number of infections of AdV infection remained relatively high between 2005 to 2010 but decreased during the summer and autumn months of 2011, 2012 and 2013. We found an increased expected number of IBV infection during the autumn and winter periods of 2005/2006, 2010/2011 and 2012/2013. During the second half of 2009, we found a heightened risk of RSV and MPV infections. More generally, the risk of RSV infection peaked during late summer through to autumn from 2008 onwards whereas the risk of MPV infection shifted from winter, between 2005 and 2008, to summer, from 2011 onwards. Virus-virus interactions For comparison, we first fitted a null model that assumed all five viruses to be independent by setting Λ−1 = I5 (the identity matrix of dimension 5 × 5). Under the neighbourhood structure, we found that allowing dependencies between viruses (Λ−1 ≠ I5) provided a better fit to the data (DIC = 2795.6 versus DIC = 3583.8 for the null model). However, the autoregressive structure with Λ−1 ≠ I5 minimised DIC (DIC = 2686.4). Comparing observed values to fitted values under the autoregressive model fit (Fig 5, black and light blue lines respectively for each virus) to informally check model fit, we were able to accurately and precisely estimate observed counts of each virus across the nine year time period. Correlations between observed and fitted values ranged from 0.96 (p-value < 0.001) for AdV and 0.9997 (p-value < 0.001) for IBV (S2 Appendix). More precisely, our model captured winter peaks in CoV, winter and spring peaks in MPV and irregularities in AdV and IBV validating the model fit to these data. Under the neighbourhood structure, we found a positive covariance between RSV/MPV and negative covariances between IBV/MPV, CoV/MPV and AdV/IBV (Table 2, Wneigh). Under the autoregressive structure, we found a positive covariance between RSV/MPV and a negative covariance between IBV/AdV (Table 2, Wauto), with adjusted p-values for the covariances between IBV/MPV and CoV/MPV of 0.075 and 0.073 respectively. Download: PPT PowerPoint slide PNG larger image TIFF original image Table 2. Estimated covariances between AdV, Cov, MPV, IBV and RSV. https://doi.org/10.1371/journal.pcbi.1007492.t002 Our analysis showed robust statistical evidence of a facilitative form of interaction between RSV and MPV and a competitive form of interaction between IBV and AdV. Discussion Humans, animals and plants are exposed to a plethora of co-circulating pathogens, creating frequent opportunity for interactions between them. There is a growing interest in the health implications of interacting pathogens that has led to the development of new research in healthcare [8]. However, robust statistical methods to identify and quantify interactions among multiple pathogens have been lacking. Traditional regression-based approaches can handle confounding variables but do not necessarily infer non-independencies between multiple response variables [12–18]. Time series specific methods (e.g. wavelets or spectral analysis) are powerful but do not handle confounding variables, are limited to pairwise comparison, and may also make assumptions of non-stationarity (e.g. Granger-Causality) [19–22, 24, 25]. Fitting epidemiological models which contain interactions to the data is also possible, but becomes very complex when multiple pathogens are present. This paper addresses the need for a more widely applicable statistical framework that can jointly infer unknown interactions among pathogens for which multiple contemporaneous time series are available. The framework accounts for non-stationarity, confounding variables such as seasonality and patient demographics and requires no prior knowledge or specification of the underlying biological or ecological mechanisms. We presented a conceptual framework derived from Bayesian multivariate disease mapping methods that provides a powerful statistical tool for inferring pathogen-pathogen interactions from diagnostic and/or surveillance time series data. Whilst standard multivariate disease mapping frameworks investigate the joint spatial distribution of multiple diseases coinfecting a population simultaneously, our method instead analyses the joint temporal distribution of multiple infections. Because multivariate disease mapping naturally incorporates a between-disease covariance matrix, these methods conveniently lend themselves to the inference of temporal signatures of pathogen-pathogen interactions when adapted to analyse temporal dependencies. Importantly, because our method accounts for confounding variables as well as the autocorrelation structure, the method distinguishes genuine pathogen-pathogen interactions from simple correlations. By applying our framework to extensive diagnostic data accrued over a nine-year period from a well-defined patient population, our analysis provides evidence of epidemiological interactions among respiratory viruses. Acute respiratory infections are a significant cause of illness and mortality and are primarily attributed to a group of viruses that occupy a shared ecological niche in the respiratory tract. Although observational data [39–42] and univariate response regression models [41, 43–45] indicate the potential for interactions among these common pathogens, limited evidence exists of their impact on epidemiological infection dynamics. Under the autoregressive structure, which provided a better fit to these data, our analysis provides robust evidence of a positive covariance between RSV and MPV and a negative covariance between IBV and AdV. This provides a basis for future work to explore the public health implications of these relationships. We anticipate that this framework will aid in the epidemiological understanding of linked pathogen dynamics. The knowledge that specific pathogen-pathogen interactions exist and of their form (positive or negative) provides an important first step towards improving disease forecasting models. Such models could be adapted for multi-pathogen systems by incorporating pathogen-pathogen interactions through reduced or enhanced transmissibility of secondary/co-infecting pathogens. Ultimately, improved understanding of the impact of coinfections on health outcomes will improve the public health utility of such models by enabling estimation of disease burden and pressures on different sections of the healthcare system, for instance the numbers of hospital beds needed at different times of the year. In summary, we have developed a new and robust method of inferring interactions from multiple pathogen time series. Applying this approach to time series data of pathogens that co-circulate in a given population allows quantification of interactions that will lead to a better understanding of the joint epidemiological dynamics of diseases. These inferences, in combination with laboratory experiments to further elucidate the underlying mechanisms, will enhance the understanding of linked pathogen dynamics, inform the forecasting of disease incidence and improve public health preparedness. In addition, they will result in better ways to evaluate the impact of public health interventions, thus aiding the design of better measures to control infectious diseases. Materials and methods Respiratory virus infection time series data Our dataset derives from routinely collected clinical samples tested for respiratory viruses by the West of Scotland Specialist Virology Center (WoSSVC) for Greater Glasgow and Clyde Health Board between January 2005 and December 2013. Each sample was tested by multiplex real-time RT-PCR and test results (virus positive or negative) were available for five groups of respiratory viruses: adenovirus [AdV]; coronavirus [CoV]; human metapneumovirus [MPV]; influenza B virus [IBV]; and respiratory syncytial virus [RSV] [46]. Sampling date, patient age, patient gender and sample origin (hospital or general practice submission that we used as a proxy for infection severity) were recorded. Multiple samples from the same patient received within a 30-day period were aggregated into a single episode of respiratory illness resulting in 28,647 patient episodes. A patient was considered virus-positive during an episode if at least one clinical sample was positive during the 30-day window. Ethical approval was not required here since samples were collected as part of routine diagnostic work. Information from NHS Scotland [47–49] informed participating patients of the use of their data. We refer the reader to Nickbakhsh et al. [38] for a full description of these data. Whilst data are available at the individual level, we are predominantly interested in estimating correlations in temporal patterns between the five viruses at the population level. Therefore, for each virus, data were aggregated into monthly infection counts across the time frame of this study. Relative risks identify time points where observed counts are higher or lower than expected, with expected counts accounting for expected seasonality and risk factors associated with respiratory infection [38]. We note that this differs from the conventional measure which compares exposed and unexposed groups. We used the relative risk to measure the excess risk of viral infection that cannot be explained by seasonality or patient demographics. By inferring dependencies between viral species in terms of excess risks, we can directly infer viral interactions. Multivariate spatio-temporal model Conditional autoregressive models are extensively used in the analysis of spatial data to model the relative risk of a virus or more generally a disease [50, 51]. The class of Bayesian model typically used in this context is given by where Yi, Ei and RRi are the observed count, expected count, derived from available patient demographic data (refer to expected counts section), and relative risk for some index i (for example, location or time interval) [30] and ϕ = {ϕ1, …, ϕI} spatial random effects modelled jointly through a conditional autoregressive (CAR) distribution [52] Matrix W is a proximity matrix, λ a smoothing parameter, τ a measure of precision and D a diagonal matrix such that Di = ∑i′ Wii′. Extending this model to multiple viruses, or more generally multiple pathogens, then where Yiv, Eiv and RRiv are the observed count, expected count and relative risk of virus v and αv a virus specific intercept term. A multivariate CAR (MCAR) distribution can jointly model ϕ by incorporating a between virus covariance matrix Λ−1 of dimension V × V (where V is the total number of viruses): In this case, Ω = D − λW, ϕ = {ϕ.1, …, ϕ.V} and ϕ.v = {ϕ1v, …, ϕIv} [53, 54]. Temporal autocorrelations may be induced in this model, at time point j, through the conditional expectation of ϕj|ϕj−1 The parameter s controls the level of temporal autocorrelation such that s = 0 implies no autocorrelation whereas s = 1 is equivalent to a first order random walk [32]. Typically, where temporal autocorrelations are modelled through the conditional expectation, spatial autocorrelations are modelled through the precision matrix [32]. Full model We model monthly time series count data from multiple viruses simultaneously over a nine year period. We index over monthly time intervals and so monthly autocorrelations are modelled in terms of the precision matrix and yearly autocorrelations are modelled in terms of the conditional expectation in a similar fashion to the multivariate spatial-temporal model detailed above. The observed count of virus v in month m of year t, Ymtv is modelled in terms of the expected count Emtv and relative risk RRmtv: with αv an intercept term specific to virus v and ϕ.t. = {ϕ.t1, …, ϕ.tV} a vector of random effects modelled conditionally through a MCAR prior This parameterisation of a MCAR model captures both the seasonal trends of each virus via Ω and long-term temporal trends via s1, …, sV. The conditional expectation of ϕ.t. depends on the previous year ϕ.t−1., capturing long term temporal trends. By allowing dependencies between neighbouring months, we account for seasonality in viral infection frequencies. MCAR prior specification. The covariance structure of the MCAR distribution used to model random seasonal-temporal effects is the Kronecker product of precision matrices Ω and Λ. The between-virus precision matrix Λ accounts for dependencies between viral relative risks in terms of monthly trends. Wishart priors can be used for unstructured precision matrices such as Λ [55], however, we employed a modified Cholesky decomposition to estimate covariance matrix Λ−1: where Σ was a diagonal matrix with elements proportional to viral standard deviations and Γ a lower triangular matrix relating to viral correlations [56]. This parameterisation ensured the positive-definiteness of Λ−1, although we note that other parameterisations are available [57]. Matrix Ω captures seasonal trends in terms of monthly dependencies defined through a proximity matrix W. We will consider two possible constructions of W: neighbourhood structure and autoregressive structure. Neighbourhood structure. Assuming neighbouring months are more similar than distant months, W can be defined such that wij = 1 if months i and j are neighbouring months and wij = 0 if months i and j are not neighbouring months. Neighbours were fixed as the previous and subsequent three months. Taking a neighbourhood approach, we set where λ is a smoothing parameter and D a 12 × 12 diagonal matrix with . The total number of nearest neighbours of month i [53, 58]. Autoregressive structure. Under this construction, W was defined through an autoregressive process and the corresponding matrix denoted by Wauto. We set the ijth entry of Wauto (i ≠ j) to be with dij the distance between months i and j and ρ a temporal correlation parameter satisfying ρ < 1. We defined distance as the number of months between i and j. Taking an autoregressive approach, we set with D a diagonal matrix with We note that these formulations can easily be extended to other MCAR structures [53, 59]. Expected counts. We required expected counts of each virus at each time point in this study. Since individual level data were available, a series of logistic regressions were used to estimate the probability of testing positive for a virus at a given time point. For month of the year m, the log odds of virus v, logit(pmv), was estimated through fixed effects of age, sex and severity (estimated by hospital or general practice submission) and a yearly random effect. The standardised probability of virus v in month m, , was estimated as where Naslt was the number of people of age a, sex s and infection severity l in year t; the estimated probability of a person of age a, sex s with infection severity l in year t testing positive for virus v in month m; and Nmv the number of swabs tested for virus v in month m. The estimated probabilities of each virus in each month are therefore standardised for age, sex and severity and account for yearly differences in circulation. The expected count for virus v in month m of year t was then with Nmtv the number of of patient episodes of illness tested for virus v in month m in year t. Estimating model parameters. This model was implemented in jags [60] using the R2jags package [61] in R [62]. All results are averaged across five independent chains. In each chain, we took 50,000 thinned draws across 500,000 iterations after a burn-in period of 300,000 iterations. R code used to fit models is provided (S1 Appendix). We note that the multivariate intrinsic Gaussian CAR prior distribution is fully specified in GeoBUGS [63]. However, our approach allows for other parameterisations of the MCAR distribution providing more flexibility in separating monthly and yearly temporal dependencies. Multiple comparison correction. For each covariance parameter, higher posterior density intervals (HPDI) were estimated. Posterior probabilities were then estimated to assess the probability of zero being included in each interval, synonymous to Bayesian p-values defined in terms of lower tail posterior probabilities [64, 65]. Covariance parameters with a posterior probability less than 0.05 were deemed different from zero [64]. In order to control for multiple comparisons, covariance parameters with an adjusted probability, controlling the false discovery rate [64, 66], less than 0.05 were deemed different from zero and used as support for a significant covariance between the corresponding viruses. Simulation study The specific aim of this paper was to estimate the between-virus covariance matrix Λ−1. We prove the validity of our proposed model (Fig 1) in modelling multivariate time series data through simulating data from three viral infections ranging from independence to moderately high correlations. We illustrate that this method had power to detect dependent time series data whilst controlling the Type 1 error rate. We began by simulating individual level data reflecting the virological diagnostic data. For each sample, an age, sex and severity were drawn from the observed virological diagnostic data distributions [38]. Regression coefficients used to estimate the probability of each virus were drawn such that βintercept = 0, βage ∼ N(0, 0.1), βgender ∼ N(0, 0.1) and βseverity ∼ N(0, 0.1). Within each year, we randomly sampled between 20 and 200 samples per month per virus in order to reflect differing testing frequencies within and between viruses. Standardised probabilities of each virus within each month were then estimated using the methods described in the Expected counts section. Expected counts were taken as the product of the standardised probabilities and the number of samples taken within that month for the corresponding virus. Monthly effect sizes were simulated using the sarima package [67] in R [62]. We choose this package due to its flexibility in simulating seasonal non-stationary time series data. We were able to combine differencing (or order d) with an autoregression (of order p) and a moving average model (of order q) to obtain a non-seasonal ARIMA model. In addition seasonal components were included through seasonal differencing (D), autoregression (P) and a moving average model (Q) over period m therefore simulating from a SARIMA(p, d, q)(P, D, Q)12 with period 12 since we are dealing with monthly data. Within each simulation, we used differencing d = 1 with a second or first order autoregression and moving average p, q ∈ {1, 2}. Likewise, we used either no or a seasonal differencing D ∈ {0, 1} and no or a first order autoregression and moving average p, q ∈ {0, 1}. These parameter settings allowed for a wide range of seasonal and non-stationary time series data. Fig 2 provides examples of simulated time series data under these parameter settings. Random effects ϕ were drawn from multivariate normal distributions with yearly smoothing parameters and monthly smoothing parameter s1, s2, s3 and λ simulated uniformly between 0 and 0.9 and precision matrix equal to the Kronecker product of matrices Ω and Λ. Matrix Ω was dependent on the choice of structure used to simulate data. In this case we simulated from both the neighbourhood and autoregressive structure (Fig 1). In the case of the autoregressive structure, we simulated ρ uniformly between 0 and 0.9 (method described in MCAR prior specification section). Matrix Λ was the virus correlation matrix that we aimed to estimate. We simulated data from three viruses with one virus pair, virus 1 and virus 2, non-independent of each other but both independent of the remaining virus, virus 3. We explored a variety of correlations between virus 1 and virus 2 ranging from 0.2 to 0.5. This range was chosen to reflect weakly related viruses (0.2) to moderate to strongly related viruses (0.5). We anticipated that as the strength of correlation increased, the power would also increase whilst still controlling the type 1 error rate. Relative risks were then taken additively as the exponential of virus intercept terms α1, α2, α2 simulated uniformly and random effects ϕ. Observed counts were the product of expected counts and relative risks. Fig 3 illustrates observed and expected counts from three viruses. We fitted both models (Fig 1, neighbourhood and autoregressive structure) to data simulated through both structures with or without a multiple comparison correction creating eight possible simulation and estimation scenarios (Table 1). In each case we simulated and estimated 100 times. Each model was fitted in jags [60] using the R2jags package [61] in R [62] (S1 Appendix). All results are averaged across two independent chains. In each chain, we took 3000 thinned draws across 300,000 iterations after a burn-in period of 200,000 iterations. Under each scenario we estimated higher posterior density intervals (HPDI) for covariance parameters (, and ). Posterior probabilities were then estimated to assess the probability of zero being included in each interval, synonymous to Bayesian p-values defined in terms of lower tail posterior probabilities [64, 65]. Covariance parameters with a posterior probability less than 0.05 were deemed different from zero [64]. In order to control for multiple comparisons, covariance parameters with an adjusted probability, controlling the false discovery rate [64, 66], less than 0.05 were deemed different from zero and used as support for a significant covariance between the corresponding viruses. Respiratory virus infection time series data Our dataset derives from routinely collected clinical samples tested for respiratory viruses by the West of Scotland Specialist Virology Center (WoSSVC) for Greater Glasgow and Clyde Health Board between January 2005 and December 2013. Each sample was tested by multiplex real-time RT-PCR and test results (virus positive or negative) were available for five groups of respiratory viruses: adenovirus [AdV]; coronavirus [CoV]; human metapneumovirus [MPV]; influenza B virus [IBV]; and respiratory syncytial virus [RSV] [46]. Sampling date, patient age, patient gender and sample origin (hospital or general practice submission that we used as a proxy for infection severity) were recorded. Multiple samples from the same patient received within a 30-day period were aggregated into a single episode of respiratory illness resulting in 28,647 patient episodes. A patient was considered virus-positive during an episode if at least one clinical sample was positive during the 30-day window. Ethical approval was not required here since samples were collected as part of routine diagnostic work. Information from NHS Scotland [47–49] informed participating patients of the use of their data. We refer the reader to Nickbakhsh et al. [38] for a full description of these data. Whilst data are available at the individual level, we are predominantly interested in estimating correlations in temporal patterns between the five viruses at the population level. Therefore, for each virus, data were aggregated into monthly infection counts across the time frame of this study. Relative risks identify time points where observed counts are higher or lower than expected, with expected counts accounting for expected seasonality and risk factors associated with respiratory infection [38]. We note that this differs from the conventional measure which compares exposed and unexposed groups. We used the relative risk to measure the excess risk of viral infection that cannot be explained by seasonality or patient demographics. By inferring dependencies between viral species in terms of excess risks, we can directly infer viral interactions. Multivariate spatio-temporal model Conditional autoregressive models are extensively used in the analysis of spatial data to model the relative risk of a virus or more generally a disease [50, 51]. The class of Bayesian model typically used in this context is given by where Yi, Ei and RRi are the observed count, expected count, derived from available patient demographic data (refer to expected counts section), and relative risk for some index i (for example, location or time interval) [30] and ϕ = {ϕ1, …, ϕI} spatial random effects modelled jointly through a conditional autoregressive (CAR) distribution [52] Matrix W is a proximity matrix, λ a smoothing parameter, τ a measure of precision and D a diagonal matrix such that Di = ∑i′ Wii′. Extending this model to multiple viruses, or more generally multiple pathogens, then where Yiv, Eiv and RRiv are the observed count, expected count and relative risk of virus v and αv a virus specific intercept term. A multivariate CAR (MCAR) distribution can jointly model ϕ by incorporating a between virus covariance matrix Λ−1 of dimension V × V (where V is the total number of viruses): In this case, Ω = D − λW, ϕ = {ϕ.1, …, ϕ.V} and ϕ.v = {ϕ1v, …, ϕIv} [53, 54]. Temporal autocorrelations may be induced in this model, at time point j, through the conditional expectation of ϕj|ϕj−1 The parameter s controls the level of temporal autocorrelation such that s = 0 implies no autocorrelation whereas s = 1 is equivalent to a first order random walk [32]. Typically, where temporal autocorrelations are modelled through the conditional expectation, spatial autocorrelations are modelled through the precision matrix [32]. Full model We model monthly time series count data from multiple viruses simultaneously over a nine year period. We index over monthly time intervals and so monthly autocorrelations are modelled in terms of the precision matrix and yearly autocorrelations are modelled in terms of the conditional expectation in a similar fashion to the multivariate spatial-temporal model detailed above. The observed count of virus v in month m of year t, Ymtv is modelled in terms of the expected count Emtv and relative risk RRmtv: with αv an intercept term specific to virus v and ϕ.t. = {ϕ.t1, …, ϕ.tV} a vector of random effects modelled conditionally through a MCAR prior This parameterisation of a MCAR model captures both the seasonal trends of each virus via Ω and long-term temporal trends via s1, …, sV. The conditional expectation of ϕ.t. depends on the previous year ϕ.t−1., capturing long term temporal trends. By allowing dependencies between neighbouring months, we account for seasonality in viral infection frequencies. MCAR prior specification. The covariance structure of the MCAR distribution used to model random seasonal-temporal effects is the Kronecker product of precision matrices Ω and Λ. The between-virus precision matrix Λ accounts for dependencies between viral relative risks in terms of monthly trends. Wishart priors can be used for unstructured precision matrices such as Λ [55], however, we employed a modified Cholesky decomposition to estimate covariance matrix Λ−1: where Σ was a diagonal matrix with elements proportional to viral standard deviations and Γ a lower triangular matrix relating to viral correlations [56]. This parameterisation ensured the positive-definiteness of Λ−1, although we note that other parameterisations are available [57]. Matrix Ω captures seasonal trends in terms of monthly dependencies defined through a proximity matrix W. We will consider two possible constructions of W: neighbourhood structure and autoregressive structure. Neighbourhood structure. Assuming neighbouring months are more similar than distant months, W can be defined such that wij = 1 if months i and j are neighbouring months and wij = 0 if months i and j are not neighbouring months. Neighbours were fixed as the previous and subsequent three months. Taking a neighbourhood approach, we set where λ is a smoothing parameter and D a 12 × 12 diagonal matrix with . The total number of nearest neighbours of month i [53, 58]. Autoregressive structure. Under this construction, W was defined through an autoregressive process and the corresponding matrix denoted by Wauto. We set the ijth entry of Wauto (i ≠ j) to be with dij the distance between months i and j and ρ a temporal correlation parameter satisfying ρ < 1. We defined distance as the number of months between i and j. Taking an autoregressive approach, we set with D a diagonal matrix with We note that these formulations can easily be extended to other MCAR structures [53, 59]. Expected counts. We required expected counts of each virus at each time point in this study. Since individual level data were available, a series of logistic regressions were used to estimate the probability of testing positive for a virus at a given time point. For month of the year m, the log odds of virus v, logit(pmv), was estimated through fixed effects of age, sex and severity (estimated by hospital or general practice submission) and a yearly random effect. The standardised probability of virus v in month m, , was estimated as where Naslt was the number of people of age a, sex s and infection severity l in year t; the estimated probability of a person of age a, sex s with infection severity l in year t testing positive for virus v in month m; and Nmv the number of swabs tested for virus v in month m. The estimated probabilities of each virus in each month are therefore standardised for age, sex and severity and account for yearly differences in circulation. The expected count for virus v in month m of year t was then with Nmtv the number of of patient episodes of illness tested for virus v in month m in year t. Estimating model parameters. This model was implemented in jags [60] using the R2jags package [61] in R [62]. All results are averaged across five independent chains. In each chain, we took 50,000 thinned draws across 500,000 iterations after a burn-in period of 300,000 iterations. R code used to fit models is provided (S1 Appendix). We note that the multivariate intrinsic Gaussian CAR prior distribution is fully specified in GeoBUGS [63]. However, our approach allows for other parameterisations of the MCAR distribution providing more flexibility in separating monthly and yearly temporal dependencies. Multiple comparison correction. For each covariance parameter, higher posterior density intervals (HPDI) were estimated. Posterior probabilities were then estimated to assess the probability of zero being included in each interval, synonymous to Bayesian p-values defined in terms of lower tail posterior probabilities [64, 65]. Covariance parameters with a posterior probability less than 0.05 were deemed different from zero [64]. In order to control for multiple comparisons, covariance parameters with an adjusted probability, controlling the false discovery rate [64, 66], less than 0.05 were deemed different from zero and used as support for a significant covariance between the corresponding viruses. MCAR prior specification. The covariance structure of the MCAR distribution used to model random seasonal-temporal effects is the Kronecker product of precision matrices Ω and Λ. The between-virus precision matrix Λ accounts for dependencies between viral relative risks in terms of monthly trends. Wishart priors can be used for unstructured precision matrices such as Λ [55], however, we employed a modified Cholesky decomposition to estimate covariance matrix Λ−1: where Σ was a diagonal matrix with elements proportional to viral standard deviations and Γ a lower triangular matrix relating to viral correlations [56]. This parameterisation ensured the positive-definiteness of Λ−1, although we note that other parameterisations are available [57]. Matrix Ω captures seasonal trends in terms of monthly dependencies defined through a proximity matrix W. We will consider two possible constructions of W: neighbourhood structure and autoregressive structure. Neighbourhood structure. Assuming neighbouring months are more similar than distant months, W can be defined such that wij = 1 if months i and j are neighbouring months and wij = 0 if months i and j are not neighbouring months. Neighbours were fixed as the previous and subsequent three months. Taking a neighbourhood approach, we set where λ is a smoothing parameter and D a 12 × 12 diagonal matrix with . The total number of nearest neighbours of month i [53, 58]. Autoregressive structure. Under this construction, W was defined through an autoregressive process and the corresponding matrix denoted by Wauto. We set the ijth entry of Wauto (i ≠ j) to be with dij the distance between months i and j and ρ a temporal correlation parameter satisfying ρ < 1. We defined distance as the number of months between i and j. Taking an autoregressive approach, we set with D a diagonal matrix with We note that these formulations can easily be extended to other MCAR structures [53, 59]. Expected counts. We required expected counts of each virus at each time point in this study. Since individual level data were available, a series of logistic regressions were used to estimate the probability of testing positive for a virus at a given time point. For month of the year m, the log odds of virus v, logit(pmv), was estimated through fixed effects of age, sex and severity (estimated by hospital or general practice submission) and a yearly random effect. The standardised probability of virus v in month m, , was estimated as where Naslt was the number of people of age a, sex s and infection severity l in year t; the estimated probability of a person of age a, sex s with infection severity l in year t testing positive for virus v in month m; and Nmv the number of swabs tested for virus v in month m. The estimated probabilities of each virus in each month are therefore standardised for age, sex and severity and account for yearly differences in circulation. The expected count for virus v in month m of year t was then with Nmtv the number of of patient episodes of illness tested for virus v in month m in year t. Estimating model parameters. This model was implemented in jags [60] using the R2jags package [61] in R [62]. All results are averaged across five independent chains. In each chain, we took 50,000 thinned draws across 500,000 iterations after a burn-in period of 300,000 iterations. R code used to fit models is provided (S1 Appendix). We note that the multivariate intrinsic Gaussian CAR prior distribution is fully specified in GeoBUGS [63]. However, our approach allows for other parameterisations of the MCAR distribution providing more flexibility in separating monthly and yearly temporal dependencies. Multiple comparison correction. For each covariance parameter, higher posterior density intervals (HPDI) were estimated. Posterior probabilities were then estimated to assess the probability of zero being included in each interval, synonymous to Bayesian p-values defined in terms of lower tail posterior probabilities [64, 65]. Covariance parameters with a posterior probability less than 0.05 were deemed different from zero [64]. In order to control for multiple comparisons, covariance parameters with an adjusted probability, controlling the false discovery rate [64, 66], less than 0.05 were deemed different from zero and used as support for a significant covariance between the corresponding viruses. Simulation study The specific aim of this paper was to estimate the between-virus covariance matrix Λ−1. We prove the validity of our proposed model (Fig 1) in modelling multivariate time series data through simulating data from three viral infections ranging from independence to moderately high correlations. We illustrate that this method had power to detect dependent time series data whilst controlling the Type 1 error rate. We began by simulating individual level data reflecting the virological diagnostic data. For each sample, an age, sex and severity were drawn from the observed virological diagnostic data distributions [38]. Regression coefficients used to estimate the probability of each virus were drawn such that βintercept = 0, βage ∼ N(0, 0.1), βgender ∼ N(0, 0.1) and βseverity ∼ N(0, 0.1). Within each year, we randomly sampled between 20 and 200 samples per month per virus in order to reflect differing testing frequencies within and between viruses. Standardised probabilities of each virus within each month were then estimated using the methods described in the Expected counts section. Expected counts were taken as the product of the standardised probabilities and the number of samples taken within that month for the corresponding virus. Monthly effect sizes were simulated using the sarima package [67] in R [62]. We choose this package due to its flexibility in simulating seasonal non-stationary time series data. We were able to combine differencing (or order d) with an autoregression (of order p) and a moving average model (of order q) to obtain a non-seasonal ARIMA model. In addition seasonal components were included through seasonal differencing (D), autoregression (P) and a moving average model (Q) over period m therefore simulating from a SARIMA(p, d, q)(P, D, Q)12 with period 12 since we are dealing with monthly data. Within each simulation, we used differencing d = 1 with a second or first order autoregression and moving average p, q ∈ {1, 2}. Likewise, we used either no or a seasonal differencing D ∈ {0, 1} and no or a first order autoregression and moving average p, q ∈ {0, 1}. These parameter settings allowed for a wide range of seasonal and non-stationary time series data. Fig 2 provides examples of simulated time series data under these parameter settings. Random effects ϕ were drawn from multivariate normal distributions with yearly smoothing parameters and monthly smoothing parameter s1, s2, s3 and λ simulated uniformly between 0 and 0.9 and precision matrix equal to the Kronecker product of matrices Ω and Λ. Matrix Ω was dependent on the choice of structure used to simulate data. In this case we simulated from both the neighbourhood and autoregressive structure (Fig 1). In the case of the autoregressive structure, we simulated ρ uniformly between 0 and 0.9 (method described in MCAR prior specification section). Matrix Λ was the virus correlation matrix that we aimed to estimate. We simulated data from three viruses with one virus pair, virus 1 and virus 2, non-independent of each other but both independent of the remaining virus, virus 3. We explored a variety of correlations between virus 1 and virus 2 ranging from 0.2 to 0.5. This range was chosen to reflect weakly related viruses (0.2) to moderate to strongly related viruses (0.5). We anticipated that as the strength of correlation increased, the power would also increase whilst still controlling the type 1 error rate. Relative risks were then taken additively as the exponential of virus intercept terms α1, α2, α2 simulated uniformly and random effects ϕ. Observed counts were the product of expected counts and relative risks. Fig 3 illustrates observed and expected counts from three viruses. We fitted both models (Fig 1, neighbourhood and autoregressive structure) to data simulated through both structures with or without a multiple comparison correction creating eight possible simulation and estimation scenarios (Table 1). In each case we simulated and estimated 100 times. Each model was fitted in jags [60] using the R2jags package [61] in R [62] (S1 Appendix). All results are averaged across two independent chains. In each chain, we took 3000 thinned draws across 300,000 iterations after a burn-in period of 200,000 iterations. Under each scenario we estimated higher posterior density intervals (HPDI) for covariance parameters (, and ). Posterior probabilities were then estimated to assess the probability of zero being included in each interval, synonymous to Bayesian p-values defined in terms of lower tail posterior probabilities [64, 65]. Covariance parameters with a posterior probability less than 0.05 were deemed different from zero [64]. In order to control for multiple comparisons, covariance parameters with an adjusted probability, controlling the false discovery rate [64, 66], less than 0.05 were deemed different from zero and used as support for a significant covariance between the corresponding viruses. Supporting information S1 Appendix. R code used to fit neighbourhood and autoregressive models. R code used to fit models described in Fig 1. Models were written and fitted in jags. https://doi.org/10.1371/journal.pcbi.1007492.s001 (R) S2 Appendix. Observed values plotted agained fitted values. Fitted values based on the best fitting autoregressive model plotted against observed values with the line of equality (y = x). Correlations and p-values between fitted and observed values are given for each virus. https://doi.org/10.1371/journal.pcbi.1007492.s002 (PDF) Acknowledgments We thank Paul Johnson and Theo Pepler for their helpful comments on the manuscript.

Li, Bowen;Taniguchi, Denis;Gedara, Jayathilake Pahala;Gogulancea, Valentina;Gonzalez-Cabaleiro, Rebeca;Chen, Jinju;McGough, Andrew Stephen;Ofiteru, Irina Dana;Curtis, Thomas P.;Zuliani, Paolo

doi: 10.1371/journal.pcbi.1007125pmid: 31830032

Introduction Microbial communities are groups of microbes that live together in a contiguous environment and interact with each other. The presence of microbial communities on the planet plays an important role in natural processes, as well as in environmental engineering applications such as wastewater treatment [1], waste recycling [2] and the production of alternative energy source [3]. Therefore, studies on how these communities form and behave have become increasingly important over the past few decades [4, 5]. Work on microbial communities has revealed that the community’s emergent behaviour arises from a variety of interactions between microbes and their local environment. In vitro experiments offer a way to gain insights into these complex interactions, but at great expense in time and resources. On the other hand, in silico computational models and numerical simulations could help researchers to investigate and predict how complex processes affect the behaviour of biological systems in an explicit and efficient way. Different approaches have been developed for modelling microbial communities [6–8]. One of the most promising strategies is to develop a mathematical model from the description of the characteristics and behaviour of the individual microbes, usually referred to as Individual-based Models (IbM) [9, 10]. In conventional IbM, the microbes are represented as rigid particles, each of which is associated with a set of properties such as mass, position, and velocity. These properties are affected by internal or external processes (e.g., diffusion), resulting in microbial growth, decay, motility, etc. Therefore, IbM are particularly useful when one is interested in understanding how individual heterogeneity and local interactions influence an emergent behaviour. The development of an IbM solver should focus on the following aspects. First, the solver needs to be flexible. Depending on the purpose of the model, IbM may involve multiple microbial functional groups, nutrients and sophisticated biological, chemical and physical processes, or sometimes it may be a simple model that describes mono-functional group or focuses on a few processes. Thus, it is important for the solver to be highly customisable (for building IbM) and extendible (with new IbM features). Second, the solver should be scalable. Simulation of large microbial communities is difficult since they contain a very high number of individuals. Different modelling strategies have been proposed to overcome this limitation, including using super-individuals and statistically representative volume elements [11]. However, there is very little work on the development of a scalable IbM solver. Parallel computing can help scalability by using multiple computer resources to simulate many individuals simultaneously. This is accomplished by breaking the problem domain into discrete sub-domains which separate out the individuals as much as possible, allowing each processing element to simulate the local interactions between individuals whilst minimizing the interactions between sub-domains. In this way each sub-domain can largely run concurrently with the others. In this work, we present a three-dimensional, open-source, and massively parallel IbM solver called NUFEB that addresses these desired features above. The purpose of NUFEB is to offer a flexible and efficient framework for simulating microbial communities at the micro-scale, with an emphasis on biofilms. A comprehensive IbM is implemented in the solver which explicitly models biological, chemical and physical processes, as well as individual microbes. The present solver supports parallel computing and allows flexible extension and customisation of the model. NUFEB is based on the state-of-the-art software LAMMPS (Large-scale Atomic Molecular Massively Parallel Simulator) [12]. We selected LAMMPS because of its open-source, parallel, and extendible nature. There are several open-source IbM solvers that have been developed over the past decade and widely applied to microbiology research, such as iDynoMiCS [13], SimBiotics [14], BioDynaMo [15], and CellModeller [16]. However, most of them only facilitate serial computing for single simulation, or focus only on biological processes, but do not model mechanical and chemical processes in detail. The NUFEB simulator instead includes all of these features. A prototype NUFEB implementation was used in [17] to study physical behaviour of microbial communities. In this manuscript we focus on a major improvement of the tool, in which new features and enhancements have been developed including three-way coupling with fluid dynamics (two-way coupled fluid-particle interactions plus particle-particle interactions), code parallelisation, chemical processes (pH dynamics, thermodynamics, and gas-liquid transfer), and post-processing routines. Model description In this section, we describe and review the IbM implemented in NUFEB. The model is described through the ODD protocol (Overview, Design concepts and Details) which is a standard structure for clearly and efficiently describing IbM [18]. We build on the ideas described previously in [17] and extend them to cope with hydrodynamics and chemical processes. For the sake of accuracy, in the following we use the term “microbe” when describing biological and chemical processes, and use the term “particle” when describing physical process. However, the two terms effectively mean the same thing—“individual” of the IbM. Model overview The purpose of the IbM implemented in NUFEB is to model a range of microbial systems (including biofilm and flocs) at the micro-scale, in order to study and predict their population-level properties and behaviours emerged from the interactions between individuals and their environment. In the model, microbes are described as soft spheres, with each individual having a set of state variables including position, density, velocity, force, mass, diameter, outer-mass, outer-diameter, growth rate, yield, etc. These attributes vary among individuals and can change through time. Outer-diameter and outer-mass are used to represent an EPS (Extracellular Polymeric Substances) shell: in some microbes EPS is initially accumulated as an extra shell around the particle. Microbial functional groups (types) are groups of one or more individual microbes that share same characteristics or parameters (such as maximum growth rate), which are constant throughout a simulation. The separation of individuals into different functional groups is based on their specific metabolism. The computational domain is the environment where microbes reside and the biological, physical and chemical processes take place. It is defined as a micro-scale 3D rectangular box with dimensions LX × LY × LZ. The size of the domain normally ranges from hundreds to thousands micrometers for micro-scale simulation. Therefore, the computational domain is considered as a sub-space of a macro-scale bioreactor or any other large-scale microbiological system. We assume that the macro-scale ecosystem is made up of replicates of the micro-scale domain if the bioreactor is perfectly mixed [17, 19]. Within the domain, chemical properties such as nutrient concentration, pH, and Gibbs free energies for microbial catabolism and anabolism are represented as continuous fields. To resolve their dynamics over time and space, the domain is discretised into Cartesian grid elements so that the values can be calculated at each discrete voxel on the meshed geometry. The style of domain boundary can be defined as either periodic or fixed. The former allows particles to cross the boundary, and re-appear on the opposite side of the domain, while a fixed wall prevents particles to interact across the boundary. The IbM allows to model a biofilm system where different regions may be defined within the computational domain [20] (Fig 1). The biofilm region is the volume occupied by microbial agents and their EPS, in which the distribution of soluble chemical species is affected by both diffusion, reaction and advection (due to biofilm porosity) processes. The boundary layer region lies over the biofilm, so that nutrient diffusion and advection is resolved in this space. The bulk liquid region is situated at the top of the boundary layer, and nutrients in this area are assumed to be perfectly mixed with the same concentration as in the macro-scale bioreactor. We also assume that the boundary layer region stretches from the maximum biofilm thickness to the bulk liquid, with the height of the region specified by the user. The boundary between the diffusion layer and bulk liquid regions is parallel with the bottom surface (substratum), but its location needs to be updated when the biofilm thickness changes. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. The computational domain with different regions. The top part is the well mixed bulk liquid region, the middle part is the diffusion boundary layer region, and the bottom part represents the biofilm region. https://doi.org/10.1371/journal.pcbi.1007125.g001 Design concepts Here we address several important concepts in the NUFEB model design. Emergence. The morphology of the microbial system, and the spatial distribution of its microbial functional groups, emerge from local interactions due to microbial growth, division, decay and mechanical interaction. Sensing. The growth of each individual depends on a number of chemical properties within the voxel in which the individual resides, including nutrient concentration, pH, and Gibbs free energy. The motion of each individual results from local mechanical interactions and fluid flow. Observation. At each given step, the physical and biological states of each individual (e.g., location, mass, size, type, growth rate) and the chemical state of each voxel (e.g., nutrient concentration, pH) are stored. Interaction. The direct interactions among individuals are driven by mechanical forces, for example, contact force, EPS adhesive force, drag force, etc. Stochasticity. The stochastic processes considered in the model include the size and location of daughter microbes after division, the location of EPS particle after EPS production, and the initial distribution of microbes. Details of the sub-models The processes that influence microbe activities are considered under three main sub-models: biological, physical, and chemical. Biological processes. The NUFEB biological sub-model handles microbial metabolism, growth, decay, and reproduction (cell division and EPS formation). Details of the biological sub-model are given below. Microbe growth and decay. An individual microbe grows and its mass increases by consuming nearby nutrients. The process of growth and decay is described by the following ordinary differential equation: (1) where mi is the biomass of the ith microbe, and μi is the specified growth rate. To determine μi, two growth models are implemented: (i) Monod-based and (ii) energy-based. The user can choose one of the growth models when configuring the simulation to run. For exemplification, the Monod-based growth model implements the work described in [17] and [19]. Three functional groups of microbes and two inert states are considered. They include: active heterotrophs (HET), ammonia oxidizing bacteria (AOB), nitrite oxidizing bacteria (NOB), and inactive EPS and dead cells. Microbial growth is based on Monod kinetics driven by the local concentration of nutrients (Ssubstrate, , , ) at the voxel in which each microbe resides [21]. The decay rate is assumed to be first order. The energy-based growth model implements the work proposed in [22]. In this model, the growth rate of each microbe μi is determined not only by nutrient availability but also by the amount of energy available for its metabolism: (2) where the maximum growth yield Yi is estimated by using the Energy Dissipation Method [23], is the metabolic rate which depends on the availability of nutrients (in particular, their dissociation forms), and is the average maintenance requirement. Thus, a microbe grows if it harvests more energy than the necessary for its maintenance requirement. On the other hand, the microbe decays when the energy requirement is not met. Microbe division and death. Microbe division is the result of biomass growth, while death is the result of biomass decay. Both are considered as instantaneous processes. Division occurs if the diameter of a microbe reaches a user-specified threshold value; the cell then divides into two daughter cells. The total mass of the two daughter cells is always conserved from the parent cell. One daughter cell is (uniformly) randomly assigned 40%–60% of the parent cell’s mass, and the other gets the rest. Also, one daughter cell takes the location of the parent cell while the centre of the other daughter cell is (uniformly) randomly chosen at a distance d (distance between the centres of the two agents) corresponding to the sum of the diameter of the daughters. The size of a microbe decreases when nutrients are limited or the energy available is not sufficient to meet maintenance requirements. Microbes which shrink below a user-specified minimum diameter are considered as dead. Their type then changes to the dead type. Dead cells do not perform biological activities, but their biomass will linearly convert to substrate to be consumed by other microbes. For computational efficiency, decaying cells are removed from the system when their size is sufficiently small (defined as 10 times smaller than the death threshold diameter). EPS production. The Monod-based growth model allows active heterotrophs to secrete EPS into their neighbouring environment. The EPS play an important role in microbial aggregation by offering a protective medium. The production process follows the approach presented in [17] and [24] with the simplification that EPS are secreted by heterotrophs only. Initially, EPS is accumulated as an extra shell around a HET particle (note that EPS density is lower than microbe density). When the relative thickness of the EPS shell of the HET particle exceeds a certain threshold value, around half (uniformly random ratio between 0.4-0.6) of the EPS mass excretes as a separate EPS particle and is (uniformly) randomly placed next to the HET. Physical processes. NUFEB’s physical sub-model includes two key features and the dependencies between them: microbes (particles) and fluid. Microbes interact among themselves and with the ambient fluid. The physics of microbial motion is solved by using the discrete element method (DEM). Fluid momentum and continuity equations are solved based on computational fluid dynamics (CFD) and coupled with particle motion. The model developed in this study allows us to describe biofilm formation, detachment, and deformation based on mechanical interactions between particles. This is more realistic than many conventional IbMs where mechanistic approaches are not considered, for example, implementing detachment as probability or rate functions [24, 25]. Mechanical relaxation. When microbes grow and divide, the system may deviate from mechanical equilibrium (i.e., non-zero net force on particles) due to particle overlap or collision. Hence, mechanical relaxation is required to update the location of the particles and minimise the stored mechanical energy of the system. Mechanical relaxation is carried out using the discrete element method, and the Newtonian equations of motion are solved for each particle in a Lagrangian framework. The equation for the translational and rotational movement of particle i is given by: (3) where mi is the mass, and is the velocity. The type of force acting on the particle varies according to different biological systems. For example, the above equation takes into account three commonly used forces in microbial system. The contact force Fc,i is a pair-wise force exerted on the particles to resolve the overlap problem at the particle level. The force equation is solved based on Hooke’s law, as described in [26]: (4) where Ni is the total number of neighbouring particles of i, Kn is the elastic constant for normal contact, δni,j are overlap distance between the center of particles i and its neighbour particle j, mi,j is the effective mass of particles i and j, γn is the viscoelastic damping constant for normal contact, and vi,j is the relative velocity of the two particles. The EPS adhesive force Fa,i is a pair-wise interaction, which is modelled as a van der Waals force [27]: (5) where Ha is the Hamaker coefficient, ri,j is the effective outer-radius of particles i and j, hmin,i,j is the minimum separation distance of the two particles, and ni,j is the unit vector from particle i to j. The drag force Fd,i is the fluid-particle interaction force due to fluid flow, with direction opposite the microbe motion in a fluid. It is formulated as [27]: (6) where ϵs,i is the particle volume fraction, ϵf,i = 1 − ϵs,i is the fluid volume fraction, Vp,i and up,i are volume and velocity of particle i, respectively, Uf,i is the fluid velocity imposed on particle i, and βi is the drag correction coefficient [28]. Apart from the forces above, the LAMMPS framework offers mechanical interactions that one can apply directly to particles (see LAMMPS’ user manual for more details [29]). In the model, we assume that the mechanical equilibrium is obtained when the average pressure of the microbial community reaches the equilibrium state. The average pressure of the system due to mechanical interactions is calculated as [30]: (7) here V is the sum of the volumes of particles. The first term in the bracket addresses the contribution from kinetic energy, where mi is the mass of particle i, is the velocity. The second term addresses the interaction energy, where and are the distance and force between two interacting particles i and j, respectively. Fluid dynamics. Hydrodynamics is an important factor in microbial community modelling as many microorganisms are found in water where their behaviour is influenced by fluid flow in two ways: transport of nutrients and detachment. Nevertheless, accurate hydrodynamics has rarely been considered in 3D microbial community modelling due to its computational complexity [31]. In NUFEB, with the support of code parallelisation, hydrodynamics is introduced by using the two-way coupled CFD-DEM approach [32, 33]: the fluid flow affects the motion of particles and the particles in turn affect the motion of the fluid. DEM solves the motion of Lagrangian particles based on Newton’s second law, while CFD (computational fluid dynamics) tracks the motion of fluid based on locally averaged Navier-Stokes equations. The particle velocity at each voxel in space is replaced by its average, and the locally averaged incompressible continuity and momentum equations for the fluid phase are given by [34]: (8) and (9) where ϵs, Us, and Ff are the fields of the solid volume fraction, velocity and fluid-particle interaction forces (e.g., drag force) of microbes, respectively. They are obtained by averaging discrete particle data in DEM [35, 36]; ϵf is the fluid volume fraction and Uf is the fluid velocity. Besides the fluid-particle interaction, the terms on the right-hand side of Eq 9 also include the fluid density ρf, the pressure gradient ∇P, the divergence of the stress tensor and the gravitational acceleration g. The fluid momentum equations are discretised and solved on an Eulerian grid by a finite volume method. The results, in particular the velocity field and the particle drag force, are used for solving nutrient transport and mechanical relaxation, respectively. Chemical processes. Nutrient transport is described using the diffusion-advection-reaction equation. To improve the representation of microbial growth, NUFEB also implemented the work proposed in [37] to allow pH dynamics and gas-liquid transfer to be considered. In this section, we briefly address the main ideas of this chemical sub-model. Nutrient consumption. The rate of nutrient consumption (or reaction rate) is calculated at each voxel. The reaction rates in the Monod-based growth model are defined according to Monod kinetics, where the stoichiometric matrix for particulate and soluble components is given in the S1 File. By contrast, in the energy-based growth model the consumption/formation rate of the ith microbe for each soluble species is driven by the microbial growth rate and the stoichiometric coefficients of the overall growth reaction, and is formalised as follows [22]: (10) where Cati is the free energy supplied by the microbial catabolic reactions, Anai is the free energy required by the anabolic reactions, and X is the biomass density. Nutrient mass balance. Nutrient concentration at each point within the computational domain is affected by different processes. For example, nutrient advection due to fluid flow, microbial growth causing nutrient consumption, or solute component transferred into gas. On the other hand, the variation of the concentration also affects biological activities. A typical example would be that aerobes such as AOB can survive and grow in aerobic regions but not in anoxic regions. To solve the nutrient distribution for each soluble component, the following diffusion-advection-reaction equation for the solute concentration S is employed in the model: (11) On the right-hand side of the equation, the first part is the diffusion term which describes nutrient movement from an area of high concentration to an area of low concentration, where ∇ is the gradient operator and D is the diffusion coefficient. The second part is the advection term which describes nutrient motion along the fluid flow, where is the fluid velocity field. Finally, R is the reaction term which is governed by both biological activities and chemical activities. Nutrient concentration in the bulk region is dynamically updated according to the following mass balance equation [20]: (12) The bulk concentration Sb of each soluble component is influenced by the nutrient inflow and outflow in the bioreactor (Q is the volumetric flow rate, V is the bioreactor volume and Sin is the influent nutrient concentration), as well as the total consumption rate in the biofilm volume in the bioreactor (Af is the surface area of the biofilm, and R(x, y, z) is the reaction rate at each voxel). pH dynamics. We assume the influent enters the bulk liquid with a fixed pH value. However, the pH varies in space and time due to change in nutrient concentration as a result of microbial activity. The model considers acid-base reactions as equilibrium processes. The kinetic expressions are amended to consider only the non-charged form of the nutrients (e.g., HNO2 but not NO2−). The concentration of all dissociated and undissociated forms can be expressed as a function of the proton (H+) concentration. Then, the proton concentration can be determined by finding the root between 1 and 10−14 of the following charge balance equation, using an implicit Newton-Raphson approximation: (13) where p and q are the total number of cations and anions contributing to the pH, respectively, m and n are the charges corresponding to the cations and anions considered in the dissociation equilibrium, and S is the concentrations of cations and anions calculated based on the Gibbs free energy adjusted for ambient temperature. Gas-liquid transfer. A gas field can be defined in NUFEB to describe the rate of nutrient mass transfer from gas to liquid or vice-versa. The phase is modelled as a volume fraction inside the biofilm region. The equilibrium between liquid and gas is disturbed by the acid-base reaction and the microbial activity taking place in the biofilm and bulk liquid. On the other hand, mass transfer may also affect microbial growth due to varying nutrient concentration in the liquid phase. The reaction rate from gas to liquid RG→L of a given nutrient can be expressed by the following equation: (14) The rate is determined by the mass transfer coefficient of chemical component KLa, the gas concentration Sgas in the head space, the saturation liquid concentration Sliq, and Henry’s constant KH. Mass transfer from liquid to gas is formalised as follows: (15) where Vgas is the volume of the reactor head space, which is considered of equal size to the computational domain, and Rg and T are the ideal gas constant and temperature, respectively. Model overview The purpose of the IbM implemented in NUFEB is to model a range of microbial systems (including biofilm and flocs) at the micro-scale, in order to study and predict their population-level properties and behaviours emerged from the interactions between individuals and their environment. In the model, microbes are described as soft spheres, with each individual having a set of state variables including position, density, velocity, force, mass, diameter, outer-mass, outer-diameter, growth rate, yield, etc. These attributes vary among individuals and can change through time. Outer-diameter and outer-mass are used to represent an EPS (Extracellular Polymeric Substances) shell: in some microbes EPS is initially accumulated as an extra shell around the particle. Microbial functional groups (types) are groups of one or more individual microbes that share same characteristics or parameters (such as maximum growth rate), which are constant throughout a simulation. The separation of individuals into different functional groups is based on their specific metabolism. The computational domain is the environment where microbes reside and the biological, physical and chemical processes take place. It is defined as a micro-scale 3D rectangular box with dimensions LX × LY × LZ. The size of the domain normally ranges from hundreds to thousands micrometers for micro-scale simulation. Therefore, the computational domain is considered as a sub-space of a macro-scale bioreactor or any other large-scale microbiological system. We assume that the macro-scale ecosystem is made up of replicates of the micro-scale domain if the bioreactor is perfectly mixed [17, 19]. Within the domain, chemical properties such as nutrient concentration, pH, and Gibbs free energies for microbial catabolism and anabolism are represented as continuous fields. To resolve their dynamics over time and space, the domain is discretised into Cartesian grid elements so that the values can be calculated at each discrete voxel on the meshed geometry. The style of domain boundary can be defined as either periodic or fixed. The former allows particles to cross the boundary, and re-appear on the opposite side of the domain, while a fixed wall prevents particles to interact across the boundary. The IbM allows to model a biofilm system where different regions may be defined within the computational domain [20] (Fig 1). The biofilm region is the volume occupied by microbial agents and their EPS, in which the distribution of soluble chemical species is affected by both diffusion, reaction and advection (due to biofilm porosity) processes. The boundary layer region lies over the biofilm, so that nutrient diffusion and advection is resolved in this space. The bulk liquid region is situated at the top of the boundary layer, and nutrients in this area are assumed to be perfectly mixed with the same concentration as in the macro-scale bioreactor. We also assume that the boundary layer region stretches from the maximum biofilm thickness to the bulk liquid, with the height of the region specified by the user. The boundary between the diffusion layer and bulk liquid regions is parallel with the bottom surface (substratum), but its location needs to be updated when the biofilm thickness changes. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. The computational domain with different regions. The top part is the well mixed bulk liquid region, the middle part is the diffusion boundary layer region, and the bottom part represents the biofilm region. https://doi.org/10.1371/journal.pcbi.1007125.g001 Design concepts Here we address several important concepts in the NUFEB model design. Emergence. The morphology of the microbial system, and the spatial distribution of its microbial functional groups, emerge from local interactions due to microbial growth, division, decay and mechanical interaction. Sensing. The growth of each individual depends on a number of chemical properties within the voxel in which the individual resides, including nutrient concentration, pH, and Gibbs free energy. The motion of each individual results from local mechanical interactions and fluid flow. Observation. At each given step, the physical and biological states of each individual (e.g., location, mass, size, type, growth rate) and the chemical state of each voxel (e.g., nutrient concentration, pH) are stored. Interaction. The direct interactions among individuals are driven by mechanical forces, for example, contact force, EPS adhesive force, drag force, etc. Stochasticity. The stochastic processes considered in the model include the size and location of daughter microbes after division, the location of EPS particle after EPS production, and the initial distribution of microbes. Details of the sub-models The processes that influence microbe activities are considered under three main sub-models: biological, physical, and chemical. Biological processes. The NUFEB biological sub-model handles microbial metabolism, growth, decay, and reproduction (cell division and EPS formation). Details of the biological sub-model are given below. Microbe growth and decay. An individual microbe grows and its mass increases by consuming nearby nutrients. The process of growth and decay is described by the following ordinary differential equation: (1) where mi is the biomass of the ith microbe, and μi is the specified growth rate. To determine μi, two growth models are implemented: (i) Monod-based and (ii) energy-based. The user can choose one of the growth models when configuring the simulation to run. For exemplification, the Monod-based growth model implements the work described in [17] and [19]. Three functional groups of microbes and two inert states are considered. They include: active heterotrophs (HET), ammonia oxidizing bacteria (AOB), nitrite oxidizing bacteria (NOB), and inactive EPS and dead cells. Microbial growth is based on Monod kinetics driven by the local concentration of nutrients (Ssubstrate, , , ) at the voxel in which each microbe resides [21]. The decay rate is assumed to be first order. The energy-based growth model implements the work proposed in [22]. In this model, the growth rate of each microbe μi is determined not only by nutrient availability but also by the amount of energy available for its metabolism: (2) where the maximum growth yield Yi is estimated by using the Energy Dissipation Method [23], is the metabolic rate which depends on the availability of nutrients (in particular, their dissociation forms), and is the average maintenance requirement. Thus, a microbe grows if it harvests more energy than the necessary for its maintenance requirement. On the other hand, the microbe decays when the energy requirement is not met. Microbe division and death. Microbe division is the result of biomass growth, while death is the result of biomass decay. Both are considered as instantaneous processes. Division occurs if the diameter of a microbe reaches a user-specified threshold value; the cell then divides into two daughter cells. The total mass of the two daughter cells is always conserved from the parent cell. One daughter cell is (uniformly) randomly assigned 40%–60% of the parent cell’s mass, and the other gets the rest. Also, one daughter cell takes the location of the parent cell while the centre of the other daughter cell is (uniformly) randomly chosen at a distance d (distance between the centres of the two agents) corresponding to the sum of the diameter of the daughters. The size of a microbe decreases when nutrients are limited or the energy available is not sufficient to meet maintenance requirements. Microbes which shrink below a user-specified minimum diameter are considered as dead. Their type then changes to the dead type. Dead cells do not perform biological activities, but their biomass will linearly convert to substrate to be consumed by other microbes. For computational efficiency, decaying cells are removed from the system when their size is sufficiently small (defined as 10 times smaller than the death threshold diameter). EPS production. The Monod-based growth model allows active heterotrophs to secrete EPS into their neighbouring environment. The EPS play an important role in microbial aggregation by offering a protective medium. The production process follows the approach presented in [17] and [24] with the simplification that EPS are secreted by heterotrophs only. Initially, EPS is accumulated as an extra shell around a HET particle (note that EPS density is lower than microbe density). When the relative thickness of the EPS shell of the HET particle exceeds a certain threshold value, around half (uniformly random ratio between 0.4-0.6) of the EPS mass excretes as a separate EPS particle and is (uniformly) randomly placed next to the HET. Physical processes. NUFEB’s physical sub-model includes two key features and the dependencies between them: microbes (particles) and fluid. Microbes interact among themselves and with the ambient fluid. The physics of microbial motion is solved by using the discrete element method (DEM). Fluid momentum and continuity equations are solved based on computational fluid dynamics (CFD) and coupled with particle motion. The model developed in this study allows us to describe biofilm formation, detachment, and deformation based on mechanical interactions between particles. This is more realistic than many conventional IbMs where mechanistic approaches are not considered, for example, implementing detachment as probability or rate functions [24, 25]. Mechanical relaxation. When microbes grow and divide, the system may deviate from mechanical equilibrium (i.e., non-zero net force on particles) due to particle overlap or collision. Hence, mechanical relaxation is required to update the location of the particles and minimise the stored mechanical energy of the system. Mechanical relaxation is carried out using the discrete element method, and the Newtonian equations of motion are solved for each particle in a Lagrangian framework. The equation for the translational and rotational movement of particle i is given by: (3) where mi is the mass, and is the velocity. The type of force acting on the particle varies according to different biological systems. For example, the above equation takes into account three commonly used forces in microbial system. The contact force Fc,i is a pair-wise force exerted on the particles to resolve the overlap problem at the particle level. The force equation is solved based on Hooke’s law, as described in [26]: (4) where Ni is the total number of neighbouring particles of i, Kn is the elastic constant for normal contact, δni,j are overlap distance between the center of particles i and its neighbour particle j, mi,j is the effective mass of particles i and j, γn is the viscoelastic damping constant for normal contact, and vi,j is the relative velocity of the two particles. The EPS adhesive force Fa,i is a pair-wise interaction, which is modelled as a van der Waals force [27]: (5) where Ha is the Hamaker coefficient, ri,j is the effective outer-radius of particles i and j, hmin,i,j is the minimum separation distance of the two particles, and ni,j is the unit vector from particle i to j. The drag force Fd,i is the fluid-particle interaction force due to fluid flow, with direction opposite the microbe motion in a fluid. It is formulated as [27]: (6) where ϵs,i is the particle volume fraction, ϵf,i = 1 − ϵs,i is the fluid volume fraction, Vp,i and up,i are volume and velocity of particle i, respectively, Uf,i is the fluid velocity imposed on particle i, and βi is the drag correction coefficient [28]. Apart from the forces above, the LAMMPS framework offers mechanical interactions that one can apply directly to particles (see LAMMPS’ user manual for more details [29]). In the model, we assume that the mechanical equilibrium is obtained when the average pressure of the microbial community reaches the equilibrium state. The average pressure of the system due to mechanical interactions is calculated as [30]: (7) here V is the sum of the volumes of particles. The first term in the bracket addresses the contribution from kinetic energy, where mi is the mass of particle i, is the velocity. The second term addresses the interaction energy, where and are the distance and force between two interacting particles i and j, respectively. Fluid dynamics. Hydrodynamics is an important factor in microbial community modelling as many microorganisms are found in water where their behaviour is influenced by fluid flow in two ways: transport of nutrients and detachment. Nevertheless, accurate hydrodynamics has rarely been considered in 3D microbial community modelling due to its computational complexity [31]. In NUFEB, with the support of code parallelisation, hydrodynamics is introduced by using the two-way coupled CFD-DEM approach [32, 33]: the fluid flow affects the motion of particles and the particles in turn affect the motion of the fluid. DEM solves the motion of Lagrangian particles based on Newton’s second law, while CFD (computational fluid dynamics) tracks the motion of fluid based on locally averaged Navier-Stokes equations. The particle velocity at each voxel in space is replaced by its average, and the locally averaged incompressible continuity and momentum equations for the fluid phase are given by [34]: (8) and (9) where ϵs, Us, and Ff are the fields of the solid volume fraction, velocity and fluid-particle interaction forces (e.g., drag force) of microbes, respectively. They are obtained by averaging discrete particle data in DEM [35, 36]; ϵf is the fluid volume fraction and Uf is the fluid velocity. Besides the fluid-particle interaction, the terms on the right-hand side of Eq 9 also include the fluid density ρf, the pressure gradient ∇P, the divergence of the stress tensor and the gravitational acceleration g. The fluid momentum equations are discretised and solved on an Eulerian grid by a finite volume method. The results, in particular the velocity field and the particle drag force, are used for solving nutrient transport and mechanical relaxation, respectively. Chemical processes. Nutrient transport is described using the diffusion-advection-reaction equation. To improve the representation of microbial growth, NUFEB also implemented the work proposed in [37] to allow pH dynamics and gas-liquid transfer to be considered. In this section, we briefly address the main ideas of this chemical sub-model. Nutrient consumption. The rate of nutrient consumption (or reaction rate) is calculated at each voxel. The reaction rates in the Monod-based growth model are defined according to Monod kinetics, where the stoichiometric matrix for particulate and soluble components is given in the S1 File. By contrast, in the energy-based growth model the consumption/formation rate of the ith microbe for each soluble species is driven by the microbial growth rate and the stoichiometric coefficients of the overall growth reaction, and is formalised as follows [22]: (10) where Cati is the free energy supplied by the microbial catabolic reactions, Anai is the free energy required by the anabolic reactions, and X is the biomass density. Nutrient mass balance. Nutrient concentration at each point within the computational domain is affected by different processes. For example, nutrient advection due to fluid flow, microbial growth causing nutrient consumption, or solute component transferred into gas. On the other hand, the variation of the concentration also affects biological activities. A typical example would be that aerobes such as AOB can survive and grow in aerobic regions but not in anoxic regions. To solve the nutrient distribution for each soluble component, the following diffusion-advection-reaction equation for the solute concentration S is employed in the model: (11) On the right-hand side of the equation, the first part is the diffusion term which describes nutrient movement from an area of high concentration to an area of low concentration, where ∇ is the gradient operator and D is the diffusion coefficient. The second part is the advection term which describes nutrient motion along the fluid flow, where is the fluid velocity field. Finally, R is the reaction term which is governed by both biological activities and chemical activities. Nutrient concentration in the bulk region is dynamically updated according to the following mass balance equation [20]: (12) The bulk concentration Sb of each soluble component is influenced by the nutrient inflow and outflow in the bioreactor (Q is the volumetric flow rate, V is the bioreactor volume and Sin is the influent nutrient concentration), as well as the total consumption rate in the biofilm volume in the bioreactor (Af is the surface area of the biofilm, and R(x, y, z) is the reaction rate at each voxel). pH dynamics. We assume the influent enters the bulk liquid with a fixed pH value. However, the pH varies in space and time due to change in nutrient concentration as a result of microbial activity. The model considers acid-base reactions as equilibrium processes. The kinetic expressions are amended to consider only the non-charged form of the nutrients (e.g., HNO2 but not NO2−). The concentration of all dissociated and undissociated forms can be expressed as a function of the proton (H+) concentration. Then, the proton concentration can be determined by finding the root between 1 and 10−14 of the following charge balance equation, using an implicit Newton-Raphson approximation: (13) where p and q are the total number of cations and anions contributing to the pH, respectively, m and n are the charges corresponding to the cations and anions considered in the dissociation equilibrium, and S is the concentrations of cations and anions calculated based on the Gibbs free energy adjusted for ambient temperature. Gas-liquid transfer. A gas field can be defined in NUFEB to describe the rate of nutrient mass transfer from gas to liquid or vice-versa. The phase is modelled as a volume fraction inside the biofilm region. The equilibrium between liquid and gas is disturbed by the acid-base reaction and the microbial activity taking place in the biofilm and bulk liquid. On the other hand, mass transfer may also affect microbial growth due to varying nutrient concentration in the liquid phase. The reaction rate from gas to liquid RG→L of a given nutrient can be expressed by the following equation: (14) The rate is determined by the mass transfer coefficient of chemical component KLa, the gas concentration Sgas in the head space, the saturation liquid concentration Sliq, and Henry’s constant KH. Mass transfer from liquid to gas is formalised as follows: (15) where Vgas is the volume of the reactor head space, which is considered of equal size to the computational domain, and Rg and T are the ideal gas constant and temperature, respectively. Biological processes. The NUFEB biological sub-model handles microbial metabolism, growth, decay, and reproduction (cell division and EPS formation). Details of the biological sub-model are given below. Microbe growth and decay. An individual microbe grows and its mass increases by consuming nearby nutrients. The process of growth and decay is described by the following ordinary differential equation: (1) where mi is the biomass of the ith microbe, and μi is the specified growth rate. To determine μi, two growth models are implemented: (i) Monod-based and (ii) energy-based. The user can choose one of the growth models when configuring the simulation to run. For exemplification, the Monod-based growth model implements the work described in [17] and [19]. Three functional groups of microbes and two inert states are considered. They include: active heterotrophs (HET), ammonia oxidizing bacteria (AOB), nitrite oxidizing bacteria (NOB), and inactive EPS and dead cells. Microbial growth is based on Monod kinetics driven by the local concentration of nutrients (Ssubstrate, , , ) at the voxel in which each microbe resides [21]. The decay rate is assumed to be first order. The energy-based growth model implements the work proposed in [22]. In this model, the growth rate of each microbe μi is determined not only by nutrient availability but also by the amount of energy available for its metabolism: (2) where the maximum growth yield Yi is estimated by using the Energy Dissipation Method [23], is the metabolic rate which depends on the availability of nutrients (in particular, their dissociation forms), and is the average maintenance requirement. Thus, a microbe grows if it harvests more energy than the necessary for its maintenance requirement. On the other hand, the microbe decays when the energy requirement is not met. Microbe division and death. Microbe division is the result of biomass growth, while death is the result of biomass decay. Both are considered as instantaneous processes. Division occurs if the diameter of a microbe reaches a user-specified threshold value; the cell then divides into two daughter cells. The total mass of the two daughter cells is always conserved from the parent cell. One daughter cell is (uniformly) randomly assigned 40%–60% of the parent cell’s mass, and the other gets the rest. Also, one daughter cell takes the location of the parent cell while the centre of the other daughter cell is (uniformly) randomly chosen at a distance d (distance between the centres of the two agents) corresponding to the sum of the diameter of the daughters. The size of a microbe decreases when nutrients are limited or the energy available is not sufficient to meet maintenance requirements. Microbes which shrink below a user-specified minimum diameter are considered as dead. Their type then changes to the dead type. Dead cells do not perform biological activities, but their biomass will linearly convert to substrate to be consumed by other microbes. For computational efficiency, decaying cells are removed from the system when their size is sufficiently small (defined as 10 times smaller than the death threshold diameter). EPS production. The Monod-based growth model allows active heterotrophs to secrete EPS into their neighbouring environment. The EPS play an important role in microbial aggregation by offering a protective medium. The production process follows the approach presented in [17] and [24] with the simplification that EPS are secreted by heterotrophs only. Initially, EPS is accumulated as an extra shell around a HET particle (note that EPS density is lower than microbe density). When the relative thickness of the EPS shell of the HET particle exceeds a certain threshold value, around half (uniformly random ratio between 0.4-0.6) of the EPS mass excretes as a separate EPS particle and is (uniformly) randomly placed next to the HET. Physical processes. NUFEB’s physical sub-model includes two key features and the dependencies between them: microbes (particles) and fluid. Microbes interact among themselves and with the ambient fluid. The physics of microbial motion is solved by using the discrete element method (DEM). Fluid momentum and continuity equations are solved based on computational fluid dynamics (CFD) and coupled with particle motion. The model developed in this study allows us to describe biofilm formation, detachment, and deformation based on mechanical interactions between particles. This is more realistic than many conventional IbMs where mechanistic approaches are not considered, for example, implementing detachment as probability or rate functions [24, 25]. Mechanical relaxation. When microbes grow and divide, the system may deviate from mechanical equilibrium (i.e., non-zero net force on particles) due to particle overlap or collision. Hence, mechanical relaxation is required to update the location of the particles and minimise the stored mechanical energy of the system. Mechanical relaxation is carried out using the discrete element method, and the Newtonian equations of motion are solved for each particle in a Lagrangian framework. The equation for the translational and rotational movement of particle i is given by: (3) where mi is the mass, and is the velocity. The type of force acting on the particle varies according to different biological systems. For example, the above equation takes into account three commonly used forces in microbial system. The contact force Fc,i is a pair-wise force exerted on the particles to resolve the overlap problem at the particle level. The force equation is solved based on Hooke’s law, as described in [26]: (4) where Ni is the total number of neighbouring particles of i, Kn is the elastic constant for normal contact, δni,j are overlap distance between the center of particles i and its neighbour particle j, mi,j is the effective mass of particles i and j, γn is the viscoelastic damping constant for normal contact, and vi,j is the relative velocity of the two particles. The EPS adhesive force Fa,i is a pair-wise interaction, which is modelled as a van der Waals force [27]: (5) where Ha is the Hamaker coefficient, ri,j is the effective outer-radius of particles i and j, hmin,i,j is the minimum separation distance of the two particles, and ni,j is the unit vector from particle i to j. The drag force Fd,i is the fluid-particle interaction force due to fluid flow, with direction opposite the microbe motion in a fluid. It is formulated as [27]: (6) where ϵs,i is the particle volume fraction, ϵf,i = 1 − ϵs,i is the fluid volume fraction, Vp,i and up,i are volume and velocity of particle i, respectively, Uf,i is the fluid velocity imposed on particle i, and βi is the drag correction coefficient [28]. Apart from the forces above, the LAMMPS framework offers mechanical interactions that one can apply directly to particles (see LAMMPS’ user manual for more details [29]). In the model, we assume that the mechanical equilibrium is obtained when the average pressure of the microbial community reaches the equilibrium state. The average pressure of the system due to mechanical interactions is calculated as [30]: (7) here V is the sum of the volumes of particles. The first term in the bracket addresses the contribution from kinetic energy, where mi is the mass of particle i, is the velocity. The second term addresses the interaction energy, where and are the distance and force between two interacting particles i and j, respectively. Fluid dynamics. Hydrodynamics is an important factor in microbial community modelling as many microorganisms are found in water where their behaviour is influenced by fluid flow in two ways: transport of nutrients and detachment. Nevertheless, accurate hydrodynamics has rarely been considered in 3D microbial community modelling due to its computational complexity [31]. In NUFEB, with the support of code parallelisation, hydrodynamics is introduced by using the two-way coupled CFD-DEM approach [32, 33]: the fluid flow affects the motion of particles and the particles in turn affect the motion of the fluid. DEM solves the motion of Lagrangian particles based on Newton’s second law, while CFD (computational fluid dynamics) tracks the motion of fluid based on locally averaged Navier-Stokes equations. The particle velocity at each voxel in space is replaced by its average, and the locally averaged incompressible continuity and momentum equations for the fluid phase are given by [34]: (8) and (9) where ϵs, Us, and Ff are the fields of the solid volume fraction, velocity and fluid-particle interaction forces (e.g., drag force) of microbes, respectively. They are obtained by averaging discrete particle data in DEM [35, 36]; ϵf is the fluid volume fraction and Uf is the fluid velocity. Besides the fluid-particle interaction, the terms on the right-hand side of Eq 9 also include the fluid density ρf, the pressure gradient ∇P, the divergence of the stress tensor and the gravitational acceleration g. The fluid momentum equations are discretised and solved on an Eulerian grid by a finite volume method. The results, in particular the velocity field and the particle drag force, are used for solving nutrient transport and mechanical relaxation, respectively. Chemical processes. Nutrient transport is described using the diffusion-advection-reaction equation. To improve the representation of microbial growth, NUFEB also implemented the work proposed in [37] to allow pH dynamics and gas-liquid transfer to be considered. In this section, we briefly address the main ideas of this chemical sub-model. Nutrient consumption. The rate of nutrient consumption (or reaction rate) is calculated at each voxel. The reaction rates in the Monod-based growth model are defined according to Monod kinetics, where the stoichiometric matrix for particulate and soluble components is given in the S1 File. By contrast, in the energy-based growth model the consumption/formation rate of the ith microbe for each soluble species is driven by the microbial growth rate and the stoichiometric coefficients of the overall growth reaction, and is formalised as follows [22]: (10) where Cati is the free energy supplied by the microbial catabolic reactions, Anai is the free energy required by the anabolic reactions, and X is the biomass density. Nutrient mass balance. Nutrient concentration at each point within the computational domain is affected by different processes. For example, nutrient advection due to fluid flow, microbial growth causing nutrient consumption, or solute component transferred into gas. On the other hand, the variation of the concentration also affects biological activities. A typical example would be that aerobes such as AOB can survive and grow in aerobic regions but not in anoxic regions. To solve the nutrient distribution for each soluble component, the following diffusion-advection-reaction equation for the solute concentration S is employed in the model: (11) On the right-hand side of the equation, the first part is the diffusion term which describes nutrient movement from an area of high concentration to an area of low concentration, where ∇ is the gradient operator and D is the diffusion coefficient. The second part is the advection term which describes nutrient motion along the fluid flow, where is the fluid velocity field. Finally, R is the reaction term which is governed by both biological activities and chemical activities. Nutrient concentration in the bulk region is dynamically updated according to the following mass balance equation [20]: (12) The bulk concentration Sb of each soluble component is influenced by the nutrient inflow and outflow in the bioreactor (Q is the volumetric flow rate, V is the bioreactor volume and Sin is the influent nutrient concentration), as well as the total consumption rate in the biofilm volume in the bioreactor (Af is the surface area of the biofilm, and R(x, y, z) is the reaction rate at each voxel). pH dynamics. We assume the influent enters the bulk liquid with a fixed pH value. However, the pH varies in space and time due to change in nutrient concentration as a result of microbial activity. The model considers acid-base reactions as equilibrium processes. The kinetic expressions are amended to consider only the non-charged form of the nutrients (e.g., HNO2 but not NO2−). The concentration of all dissociated and undissociated forms can be expressed as a function of the proton (H+) concentration. Then, the proton concentration can be determined by finding the root between 1 and 10−14 of the following charge balance equation, using an implicit Newton-Raphson approximation: (13) where p and q are the total number of cations and anions contributing to the pH, respectively, m and n are the charges corresponding to the cations and anions considered in the dissociation equilibrium, and S is the concentrations of cations and anions calculated based on the Gibbs free energy adjusted for ambient temperature. Gas-liquid transfer. A gas field can be defined in NUFEB to describe the rate of nutrient mass transfer from gas to liquid or vice-versa. The phase is modelled as a volume fraction inside the biofilm region. The equilibrium between liquid and gas is disturbed by the acid-base reaction and the microbial activity taking place in the biofilm and bulk liquid. On the other hand, mass transfer may also affect microbial growth due to varying nutrient concentration in the liquid phase. The reaction rate from gas to liquid RG→L of a given nutrient can be expressed by the following equation: (14) The rate is determined by the mass transfer coefficient of chemical component KLa, the gas concentration Sgas in the head space, the saturation liquid concentration Sliq, and Henry’s constant KH. Mass transfer from liquid to gas is formalised as follows: (15) where Vgas is the volume of the reactor head space, which is considered of equal size to the computational domain, and Rg and T are the ideal gas constant and temperature, respectively. Design and implementations NUFEB is developed in C++ as a user package within the LAMMPS platform. In this section, we summarise the NUFEB functionalities and give some implementation details. IbM in LAMMPS NUFEB is built on top of LAMMPS and extends it with IbM features. LAMMPS is a classical molecular dynamics simulator and primarily solves particle physics, including a wide range of inter-particle interactions and potentials [12]. In NUFEB, a new sphere-like particle type is defined with additional attributes (e.g., outer-diameter, outer-mass) to represent a microbe. Microbes are grouped into different functional groups, and members of each group share the same biological parameters. The computational domain is restricted to a 3D rectangular box with user-specified dimensions, Cartesian grid size and boundary conditions. In LAMMPS, a “fix” is any operation that applies to the system during time integration. Examples include the updating of particle locations, velocities, forces. NUFEB defines a series of new fixes to perform the IbM related processes previously described. The user can also extend the tool with other processes by implementing new fix commands. During the simulation, fixes are invoked at a user-defined frequency to update field quantities and microbe attributes. For the purpose of computational efficiency and different modelling systems, NUFEB allows users to customise a simulation by enabling/disabling any of the fix commands from the input setting. In order to execute a NUFEB simulation, an input script (a text file) is prepared with certain commands and parameters. NUFEB will read those commands and parameters, one line at a time. Each command causes NUFEB to take some action, such as setting initial conditions, performing IbM processes, exporting results, or running a simulation. In Fig 2, we show an excerpt of an input script used in this work (Case Study 2). The details of the input format are described in S1 Manual. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. An example of (partial) input script for NUFEB simulation. A list of “fix” commands defines IbM processes that apply to the simulation, which includes Monod-based growth (k1), nutrient mass balance (k2), cell division (d1), and EPS production (e1). Each fix command may require one or more parameters for the model specification, such as EPS density (EPSdens), division diameter (divDia), and HET reduction factor in anoxic condition (etaHET). The last parameter is introduced due to the fact that the observed growth rate of HET in anoxic condition was always smaller than that in aerobic condition. https://doi.org/10.1371/journal.pcbi.1007125.g002 The procedure of a classical IbM simulation in NUFEB is presented in Algorithm 1. Note that when solving nutrient mass balance (step 5), if energy-growth model is applied, the computation is accomplished with pH dynamics and gas-liquid transfer to update the concentration of dissociation forms, pH and reaction rate. The Monod-based model, however, does not couple with pH and gas-liquid transfer in the current implementation. The model’s processes can be operated sequentially as they are on different timescales. The mechanical timestep is typically of the order of 10−7s; the diffusion timestep is of the order of 10−4s, while the biological timestep is much larger ranging from minutes to hours. The coupling between multiple timescales relies on the pseudo steady-state approximation and the frozen state [31]. For example, when a steady state solute concentration is reached at each biological timestep, the concentration is assumed to remain unchanged (frozen state) until the next biological step. In this way, the computational load for solving fast dynamic processes can be significantly reduced. Algorithm 1 (IbM simulation procedure) 1: input: initial states of computation domain, microbes and fields 2: output: states of all microbes and fields at each output time step 3: while biological time step tbio < tend do 4:  solve fluid dynamics to update velocity field and particle (drag) force 5:  solve nutrient mass balance to update solute concentration field 6:  update nutrient concentration in bulk based on the total consumption rate in biofilm region 7:  perform microbe growth to update biomass and size 8:  perform microbe division, death and EPS production 9:  perform mechanical relaxation to update microbe position and velocity 10:  update boundary layer location and neighbour list 11:  tbio = tbio + Δt 12: end while Mechanical relaxation is resolved by using the Verlet algorithm provided by LAMMPS [38]. The computation of interaction forces between particles, such as contact force, relies on LAMMPS’ neighbour lists. During the Verlet integration, instead of iterating through every other particle, which would result in a quadratic time complexity algorithm, LAMMPS maintains a list of neighbours for each particle. The computation of the interaction force is only performed between a particle and its corresponding neighbours. The neighbour lists must be updated from time to time depending on the microbe’s displacement and division. The nutrient mass balance equation is discretised on a Marker-And-Cell (MAC) uniform grid and the concentration scalar is defined at the centre of the cubic voxel. The temporal and spatial derivatives of the transport equation are discretised by Forward Euler and Central Finite Differences, respectively. Further details about the calculations are given in the S1 File. Depending on the physical situation, different boundary conditions can be chosen when solving the equation. For instance, a biofilm system would normally uses non-flux Neumann conditions through the bottom surface to model impermeable support material, Dirichlet boundary conditions at the top surface as it is assumed to connect with bulk environment, and periodic boundary conditions for the rest of the four surfaces [20]. Coupling with fluid dynamics NUFEB employs and extends the existing CFD-DEM solver SediFoam for the simulation of hydrodynamics [39]. SediFoam provides a flexible interface between the two open-source solvers LAMMPS and OpenFOAM. LAMMPS aims to simulate particle motions, while OpenFOAM (Open Field Operation and Manipulation) is a parallel CFD solver that can perform three-dimensional fluid flow simulations [40]. Inbetween them, SediFoam offers efficient parallel algorithms that transfers and maps the properties of Lagrangian particles to an Eulerian mesh, and vice versa. In this work, we also extend SediFoam for compatibility with IbM features, in particular, to transfer and map the information of new divided particles and velocity field between the solvers. The schema of the CFD-DEM coupling is shown in Fig 3. At each fluid timestep, particle information (e.g., mass, force, velocity) is transferred from the DEM module to the CFD module. A particles list maintained by OpenFOAM is updated based on the obtained information. An averaging procedure is then performed to convert the properties of discrete particles to the Eulerian CFD mesh. The fluid momentum equations are discretised and solved on the Eulerian mesh by the finite volume method. The PISO (Pressure-Implicit with Splitting of Operators) algorithm is followed to solve the equations [41]. In the DEM module, the solutions of drag force and velocity field obtained from OpenFOAM are assigned to each particle and corresponding mesh grid, respectively. The drag force will be taken into account in evolving the motion of the particles. The velocity field is used for solving the advection-diffusion-reaction equation (Eq 11). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Block diagram of CFD-DEM coupling. Diagram adopted from [39]. Fluid dynamics is solved in the CFD module and particle motion is solved in the DEM module. Particle and field information are transferred between the two modules based on an averaging procedure. https://doi.org/10.1371/journal.pcbi.1007125.g003 Code parallelisation The IbM implementation in NUFEB involves particle-based functions (e.g., contact force, division, and EPS excretion) and continuum-based functions (e.g., nutrient mass balance, and pH). The parallelisation of the former function group is based on LAMMPS’ parallel mechanism. The computational domain is spatially decomposed into multiple MPI (Message Passing Interface) processes (sub-domains). Each sub-domain contains local particles as well as ghost particles. Local particles are those residing in the owned sub-domain, and each process is responsible for updating the status of their local particles. Ghost particles are copies of particles owned by neighbouring processes. During the simulation, the local particles obtain information from their ghost (and neighbour) counterparts for calculating and updating their physical properties (e.g., forces). The neighbour lists require updating when particles are moved due to mechanical interactions, and created/deleted due to microbe division, decay, etc. Continuum-based functions have their variables computed using a uniform grid. Parallelism is achieved by spatially decomposing the computational domain into sub-domains. Computations such as diffusion and advection require solute information from adjacent cells. Therefore, like particle-based functions, grid cells on the boundary of each sub-domain need to be communicated to neighbouring sub-domains and are treated locally as ghost cells. The implementation of grid decomposition also considers the sub-domain box to be always conforming to the uniform grid boundary. Automatic vectorisation was employed to further optimise computation intensive routines, such as pH calculation. The loops in the routines are vectorised and can be performed simultaneously. The vectorisation is achieved together with the use of control directives (i.e., #pragmas) to instruct the compiler on how to handle data dependencies within a loop. Other features During simulation, NUFEB allows to output any state variable of microbes or voxels based on the keywords given in the input script. The output results can be stored into various formats for visualisation or analysis. The supported formats are VTK, POVray and HDF5. The VTK binary format is readable by the VTK visualisation toolkit or other visualisation tools, such as ParaView (all biofilm figures shown in the Results section were produced from ParaView). A post-processing routine is implemented to convert the LAMMPS default format to POVray image, which supports high quality rendering of particles. HDF5 is a hierarchical, filesystem-like data format supported by a number of popular software platforms, including Java, MATLAB and Python. This allows the user to directly import simulation data to any of the platforms for further analysis. To understand the morphological dynamics of microbial systems, characteristics such as biofilm average height, biofilm surface roughness, floc equivalent diameter and floc fractal dimension can be measured and exported during simulation. These aggregated characteristics are essential factors to study and design microbial system [42]. Parallelisation of the characteristics measurements is supported by NUFEB and based on domain decomposition. NUFEB also supports most of the LAMMPS default commands, which can be useful in microbial simulations. For example, the restart and read restart functions write out the current state of a simulation as a binary file, and then start a new simulation with the previously saved system configuration; the lattice and create atoms functions allow to automatically create large numbers of initial microbes based on a user-defined lattice structure; load balance is performed with the objective of maintaining the same number of particles in each sub-domain in parallel runs. During the simulation, this function adjusts the size and shape of the sub-domains to balance the computational cost in the processors. IbM in LAMMPS NUFEB is built on top of LAMMPS and extends it with IbM features. LAMMPS is a classical molecular dynamics simulator and primarily solves particle physics, including a wide range of inter-particle interactions and potentials [12]. In NUFEB, a new sphere-like particle type is defined with additional attributes (e.g., outer-diameter, outer-mass) to represent a microbe. Microbes are grouped into different functional groups, and members of each group share the same biological parameters. The computational domain is restricted to a 3D rectangular box with user-specified dimensions, Cartesian grid size and boundary conditions. In LAMMPS, a “fix” is any operation that applies to the system during time integration. Examples include the updating of particle locations, velocities, forces. NUFEB defines a series of new fixes to perform the IbM related processes previously described. The user can also extend the tool with other processes by implementing new fix commands. During the simulation, fixes are invoked at a user-defined frequency to update field quantities and microbe attributes. For the purpose of computational efficiency and different modelling systems, NUFEB allows users to customise a simulation by enabling/disabling any of the fix commands from the input setting. In order to execute a NUFEB simulation, an input script (a text file) is prepared with certain commands and parameters. NUFEB will read those commands and parameters, one line at a time. Each command causes NUFEB to take some action, such as setting initial conditions, performing IbM processes, exporting results, or running a simulation. In Fig 2, we show an excerpt of an input script used in this work (Case Study 2). The details of the input format are described in S1 Manual. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. An example of (partial) input script for NUFEB simulation. A list of “fix” commands defines IbM processes that apply to the simulation, which includes Monod-based growth (k1), nutrient mass balance (k2), cell division (d1), and EPS production (e1). Each fix command may require one or more parameters for the model specification, such as EPS density (EPSdens), division diameter (divDia), and HET reduction factor in anoxic condition (etaHET). The last parameter is introduced due to the fact that the observed growth rate of HET in anoxic condition was always smaller than that in aerobic condition. https://doi.org/10.1371/journal.pcbi.1007125.g002 The procedure of a classical IbM simulation in NUFEB is presented in Algorithm 1. Note that when solving nutrient mass balance (step 5), if energy-growth model is applied, the computation is accomplished with pH dynamics and gas-liquid transfer to update the concentration of dissociation forms, pH and reaction rate. The Monod-based model, however, does not couple with pH and gas-liquid transfer in the current implementation. The model’s processes can be operated sequentially as they are on different timescales. The mechanical timestep is typically of the order of 10−7s; the diffusion timestep is of the order of 10−4s, while the biological timestep is much larger ranging from minutes to hours. The coupling between multiple timescales relies on the pseudo steady-state approximation and the frozen state [31]. For example, when a steady state solute concentration is reached at each biological timestep, the concentration is assumed to remain unchanged (frozen state) until the next biological step. In this way, the computational load for solving fast dynamic processes can be significantly reduced. Algorithm 1 (IbM simulation procedure) 1: input: initial states of computation domain, microbes and fields 2: output: states of all microbes and fields at each output time step 3: while biological time step tbio < tend do 4:  solve fluid dynamics to update velocity field and particle (drag) force 5:  solve nutrient mass balance to update solute concentration field 6:  update nutrient concentration in bulk based on the total consumption rate in biofilm region 7:  perform microbe growth to update biomass and size 8:  perform microbe division, death and EPS production 9:  perform mechanical relaxation to update microbe position and velocity 10:  update boundary layer location and neighbour list 11:  tbio = tbio + Δt 12: end while Mechanical relaxation is resolved by using the Verlet algorithm provided by LAMMPS [38]. The computation of interaction forces between particles, such as contact force, relies on LAMMPS’ neighbour lists. During the Verlet integration, instead of iterating through every other particle, which would result in a quadratic time complexity algorithm, LAMMPS maintains a list of neighbours for each particle. The computation of the interaction force is only performed between a particle and its corresponding neighbours. The neighbour lists must be updated from time to time depending on the microbe’s displacement and division. The nutrient mass balance equation is discretised on a Marker-And-Cell (MAC) uniform grid and the concentration scalar is defined at the centre of the cubic voxel. The temporal and spatial derivatives of the transport equation are discretised by Forward Euler and Central Finite Differences, respectively. Further details about the calculations are given in the S1 File. Depending on the physical situation, different boundary conditions can be chosen when solving the equation. For instance, a biofilm system would normally uses non-flux Neumann conditions through the bottom surface to model impermeable support material, Dirichlet boundary conditions at the top surface as it is assumed to connect with bulk environment, and periodic boundary conditions for the rest of the four surfaces [20]. Coupling with fluid dynamics NUFEB employs and extends the existing CFD-DEM solver SediFoam for the simulation of hydrodynamics [39]. SediFoam provides a flexible interface between the two open-source solvers LAMMPS and OpenFOAM. LAMMPS aims to simulate particle motions, while OpenFOAM (Open Field Operation and Manipulation) is a parallel CFD solver that can perform three-dimensional fluid flow simulations [40]. Inbetween them, SediFoam offers efficient parallel algorithms that transfers and maps the properties of Lagrangian particles to an Eulerian mesh, and vice versa. In this work, we also extend SediFoam for compatibility with IbM features, in particular, to transfer and map the information of new divided particles and velocity field between the solvers. The schema of the CFD-DEM coupling is shown in Fig 3. At each fluid timestep, particle information (e.g., mass, force, velocity) is transferred from the DEM module to the CFD module. A particles list maintained by OpenFOAM is updated based on the obtained information. An averaging procedure is then performed to convert the properties of discrete particles to the Eulerian CFD mesh. The fluid momentum equations are discretised and solved on the Eulerian mesh by the finite volume method. The PISO (Pressure-Implicit with Splitting of Operators) algorithm is followed to solve the equations [41]. In the DEM module, the solutions of drag force and velocity field obtained from OpenFOAM are assigned to each particle and corresponding mesh grid, respectively. The drag force will be taken into account in evolving the motion of the particles. The velocity field is used for solving the advection-diffusion-reaction equation (Eq 11). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Block diagram of CFD-DEM coupling. Diagram adopted from [39]. Fluid dynamics is solved in the CFD module and particle motion is solved in the DEM module. Particle and field information are transferred between the two modules based on an averaging procedure. https://doi.org/10.1371/journal.pcbi.1007125.g003 Code parallelisation The IbM implementation in NUFEB involves particle-based functions (e.g., contact force, division, and EPS excretion) and continuum-based functions (e.g., nutrient mass balance, and pH). The parallelisation of the former function group is based on LAMMPS’ parallel mechanism. The computational domain is spatially decomposed into multiple MPI (Message Passing Interface) processes (sub-domains). Each sub-domain contains local particles as well as ghost particles. Local particles are those residing in the owned sub-domain, and each process is responsible for updating the status of their local particles. Ghost particles are copies of particles owned by neighbouring processes. During the simulation, the local particles obtain information from their ghost (and neighbour) counterparts for calculating and updating their physical properties (e.g., forces). The neighbour lists require updating when particles are moved due to mechanical interactions, and created/deleted due to microbe division, decay, etc. Continuum-based functions have their variables computed using a uniform grid. Parallelism is achieved by spatially decomposing the computational domain into sub-domains. Computations such as diffusion and advection require solute information from adjacent cells. Therefore, like particle-based functions, grid cells on the boundary of each sub-domain need to be communicated to neighbouring sub-domains and are treated locally as ghost cells. The implementation of grid decomposition also considers the sub-domain box to be always conforming to the uniform grid boundary. Automatic vectorisation was employed to further optimise computation intensive routines, such as pH calculation. The loops in the routines are vectorised and can be performed simultaneously. The vectorisation is achieved together with the use of control directives (i.e., #pragmas) to instruct the compiler on how to handle data dependencies within a loop. Other features During simulation, NUFEB allows to output any state variable of microbes or voxels based on the keywords given in the input script. The output results can be stored into various formats for visualisation or analysis. The supported formats are VTK, POVray and HDF5. The VTK binary format is readable by the VTK visualisation toolkit or other visualisation tools, such as ParaView (all biofilm figures shown in the Results section were produced from ParaView). A post-processing routine is implemented to convert the LAMMPS default format to POVray image, which supports high quality rendering of particles. HDF5 is a hierarchical, filesystem-like data format supported by a number of popular software platforms, including Java, MATLAB and Python. This allows the user to directly import simulation data to any of the platforms for further analysis. To understand the morphological dynamics of microbial systems, characteristics such as biofilm average height, biofilm surface roughness, floc equivalent diameter and floc fractal dimension can be measured and exported during simulation. These aggregated characteristics are essential factors to study and design microbial system [42]. Parallelisation of the characteristics measurements is supported by NUFEB and based on domain decomposition. NUFEB also supports most of the LAMMPS default commands, which can be useful in microbial simulations. For example, the restart and read restart functions write out the current state of a simulation as a binary file, and then start a new simulation with the previously saved system configuration; the lattice and create atoms functions allow to automatically create large numbers of initial microbes based on a user-defined lattice structure; load balance is performed with the objective of maintaining the same number of particles in each sub-domain in parallel runs. During the simulation, this function adjusts the size and shape of the sub-domains to balance the computational cost in the processors. Results We have successfully validated NUFEB against two biofilm benchmark problems BM2 and BM3 proposed by the International Water Association (IWA) task group on biofilm modelling [43, 44]. The validation results can be found in the S1 File. This section shows two further examples implemented using NUFEB. The case studies are based on biofilm systems, which are microbial communities of single or multiple microbial functional groups in which cells stick together and attach to a substratum by means of EPS. Case study 1: Biofilm deformation and detachment One of the outstanding questions in biofilm research is understanding how fluid-biofilm interactions affect the mechanical properties of biofilms. In this section, we describe how to use NUFEB to simulate a biofilm system with fluid dynamics by using three-way fluid and particle coupling. This is different from previous studies, where the coupling was from flow field to biofilm structure but not the other way [17], or assumes biofilm as a collection of 1D springs [45, 46]. To simulate a hydrodynamic biofilm, we apply fluid flow to a pre-grown biofilm that consists of heterotrophs and their EPS production. The biofilm is grown from 40 microbes inoculated on the substratum to a pre-determined height (80 μm) without flow and in an oxygen-limited condition (1 × 10−4 kg m−3). In this way, a mushroom-shaped biofilm structure can be developed to model liquid filled voids and channels (see Fig 4(a) and S1 Video). Then, we impose a fluid flow to the biofilm. During the fluid stage, any biological process is considered to be in the frozen state due to the small time scale of hydrodynamic calculations, and nutrient mass balance is omitted for the sake of simplicity. The motion of microbes is driven by both particle-particle and particle-fluid interactions, including EPS adhesion, contact force and drag force, as described in Eqs (4)–(6). Note that the EPS adhesive force also exists between HETs due to their EPS shells. The physical model parameters are kept constant throughout the simulation and can be found in the S1 File. For boundary conditions, we impose a fixed velocity Uf at the top surface with direction along the x-axis as inlet velocity, no-slip condition at the bottom surface and periodic conditions at other four surfaces. The pressure are enforced as zero gradient at the top and bottom surfaces. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Biofilm deformation and detachment at Uf = 0.2 m s−1. (a) Time = 0; (b) Time = 0.0015s; (c) Time = 0.003s; (d) Time = 0.01s. The model simulates 4 × 104 particles. Particles crossing the domain boundary will be removed from the system. Particle colours are blue for heterotrophs and grey for EPS. CPU time is 8 hours with dual processors, and initial particle number is 41210. https://doi.org/10.1371/journal.pcbi.1007125.g004 Fig 4(b)–4(d) and S2 Video. show the biofilm deformation and detachment at Uf = 0.2 m s−1 (Reynolds number = 20). The biofilm deforms and microbes detach along with the flow direction. In the early stage of the detachment process, the top of the biofilm is highly elongated and forms filamentous streamers. However, most of the microbes are still connected together with cohesion, and there is only a small number of clusters detached from the head of the streamers due to cohesive failures (Fig 4(b)). As the fluid continues to flow, large chunks of microbes detach from the biofilm surface. These detached microbe chunks may also break-up again, re-agglomerate with other clusters or re-attach to the biofilm surface (Fig 4(c)). Such deformation and detachment events observed from our NUFEB simulation show qualitative agreement with both experimental results [47] and other numerical simulations using different methods [17, 46]. The deformation reaches a pseudo-steady-state when the mushroom-shaped biofilm protrusions are removed from the system. As a result, the biofilm morphology changes dramatically from a rough to a flat surface (Fig 4(d)). During the deformation, the fluid, represented as red arrows, travels around the biofilm. Due to the irregular shape of the biofilm and the high fluid velocity, small vortexes can be observed at the biofilm surface on both the upstream and downstream sides. This phenomenon has been observed in previous studies [46]. For a more quantitative measurement of the deforming biofilm, we evaluated the area density and surface roughness of the biofilm at different fluid velocities. The biofilm surface roughness is calculated by [17]: (16) where h(x, y) is the biofilm height in the z direction at location (x, y) on the substratum, and is the average biofilm height: (17) As expected, when the fluid velocity increases the removed biomass also increases. For example, when Uf = 0.4 m s−1 is applied, the area density reaches steady-state after 0.004s, and the value decreases by 13% (Fig 5(a)). Note that the decrease does not take into account the change of basis. By contrast, the area density decreases less than 1% if Uf = 0.1 m s−1 is applied. The biofilm surface roughness shows a similar trend: the roughness decreases with increasing velocities (Fig 5(b)), indicating that biofilm morphology tends to be more flat in high-velocity fluid conditions as most of the mushroom protrusions can be removed. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Effect of emergent properties on biofilm detachment. (a) biomasss area density, and (b) biofilm surface roughness. https://doi.org/10.1371/journal.pcbi.1007125.g005 Case study 2: Biofilm growth with 107 particles In this case study, we first show the development of a large and complex biofilm system and then focus on the parallel efficiency of the simulations. The aim of this case study is to demonstrate the capability and performance of NUFEB in the simulation of larger biological systems. Biofilm development. The system is defined as a multi-functional group and multi-nutrient biofilm. In order to represent a more realistic biofilm, we explicitly consider nitrification as a two-step oxidation process that is performed by different groups of microbes: ammonia oxidizing bacteria (AOB) and nitrite oxidizing bacteria (NOB). In addition, the biofilm includes heterotrophs (HET) and their EPS production. The reaction model contains five soluble species, nutrients and products during microbial metabolism. The catabolic reactions include oxidation of ammonium NH4+ to nitrite NO2− by AOB, oxidation of nitrite to nitrate NO3− by NOB, and HET aerobic and anaerobic growth by consuming organic substrate in oxygenated conditions or nitrate in anoxic denitrifying conditions. The kinetics and reaction stoichiometry of the modelled processes and their corresponding parameters are detailed in the S1 File. The computational domain is divided into three regions. In the bulk region, nutrients are assumed to be completely mixed and their concentration is updated dynamically at each biological timestep, except for oxygen. We also assume that there is sufficient O2 and NH4+ but no NO2− and NO3− in the reactor influent. So the concentration of the two N compounds can only come from the transformation of NH4+. In the boundary layer region, a 20 μm distance from the maximum biofilm thickness to the bulk liquid is defined for solving the nutrient gradient. In the biofilm region, instead of using the super particle method [20], a small division diameter (1.3 μm) is chosen to represent real microbe sizes (on average 1 μm [48]). The simulation is run on an in-house HPC system at Newcastle University. In Fig 6 and S5 Video. we present the biofilm development over time. The system reaches 2.3 × 107 particles after 160 hours (CPU time = 30 hours). The initial particles are randomly placed on the substratum. In the early stages of biofilm formation, due to high growth rate and sufficient supply of substrate from bulk liquid, heterotrophs grow faster than nitrifiers (0, 60 and 120 hours). As the biomass grows, the system turns to substrate-limited condition for the HET group, while there is still sufficient NH4+ due to its high initial concentration that favours nitrifier growth. As a result, biofilm surface coverage of heterotrophs becomes smaller than nitrifiers (160 hours). This phenomenon matches previous experimental results where the nitrifying population can be significantly higher than heterotrophs in a substrate-limited reactor [49]. The biofilm geometry forms a wavy structure after 160 hours. This is because of a self-enhancing process from the non-uniform initial microbial distribution [50]. The spatial distribution of NO2− concentration is shown in Fig 7. It is clear that the NO2− concentration field differs according to the AOB distribution, where areas with high NO2− concentration are the locations where AOB clusters are present. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Biofilm development after 0, 60, 120, and 160 hours. The simulation uses 100 processors and 30 hours CPU time to reach 2.3 × 107 particles. The biological timestep is 0.25 hour. Particle colours are blue for heterotrophs, grey for EPS, light blue for AOB, and green for NOB. https://doi.org/10.1371/journal.pcbi.1007125.g006 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 7. Nitrite concentration field at a small part of the simulated domain after 60 hours. The spatial distribution of NO2− concentration follows the nitrifier distribution. The areas where NO2− accumulates are due to production by AOB. https://doi.org/10.1371/journal.pcbi.1007125.g007 Fig 8(a) shows a quantitative evaluation of the total biomass accumulation over time. The trend shows linear biomass increase which indicates that the total microbial growth rate is not yet balanced by the decay rate. Therefore, a biomass steady state is not achieved after 160 hours. This is due to the high-oxygen environment (1 × 10−2 kg m−3) and the thin biofilm which nutrients can penetrate. However, the concentration of substrate and NH4+ in the bulk liquid relax to steady state before the total biomass concentration relaxes (Fig 8(b)), as bulk concentrations are mainly determined by biomass in the top biofilm layers, which ensures a high growth rate [20]. The NO2− profile is influenced by both AOB synthesis and NOB consumption. Thus, the bulk concentration decreases with the increase of NOB populations. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 8. Quantitative evaluation of Case Study 2. (a) Total biomass of active functional groups over time, and (b) nutrients concentration in bulk liquid. Note that we assume the oxygen concentration in bulk liquid is kept constant by aeration. The slow growth of NOBs is due to the nutrient limitation, their low growth rate, and spatial distribution. At the early stage of the biofilm formation, the system is in NO2− limited condition. As the biofilm growth, the bottom area where NOBs reside turns to oxygen-limited condition. Both conditions inhibit NOBs growth. https://doi.org/10.1371/journal.pcbi.1007125.g008 Parallel performance. Parallel performance is crucial to IbM solvers for simulating large and complex problems. To investigate the performance of NUFEB, a scalability test was performed based on Case Study 2. The test aims at evaluating how simulation time varies with the number of processors for a fixed problem size. The speed-up of the scalability test is defined as tp0/tpn, where tp0 and tpn are the actual CPU times spent on the baseline case and the test case respectively. Then the efficiency is the ratio between the speed-up of the baseline and the test cases obtained when using a given number of processors, i.e., Np0tp0/Npntpn, where Np0 and Npn are the number of processors employed in these cases. In order to reach a considerable number of microbes (4 millions), the baseline case is performed using 4 processors, and the subsequent tests were performed with increasing numbers of processors, ranging from 8 to 256. As mentioned previously, NUFEB implements two distinct solutions for the parallelisation of continuum-based and particle-based processes. The performance of the two solutions are studied separately, and are presented in Fig 9. It can be observed that when employing a small number of processors, both solutions are close to the ideal speed-up (linear increase) and the parallel efficiency is over 90%. As more processor nodes are added, each processing node spends more time doing inter-processor communication than useful processing, and the parallelisation becomes less efficient. In the tests using 128 and 256 processors, the parallel efficiency decreases to around 42%, but it still shows a good scalability. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 9. Performance of Case Study 2 with 4–256 processors. The initial conditions and the model parameters are kept the same in all cases. The average achieved particle number of all cases is 4.77 × 106 with 0.5% standard deviation. The difference is due to the randomness in sub-domains. The simulation with 4 processors is regarded as baseline case for computing speed-up. Doubling the numbers of processors results in almost double speed-up when small processor numbers are employed (e.g., less than 32). However, the speed-up and parallel efficiency decrease with increasing numbers of processors due to inter-processor communication. https://doi.org/10.1371/journal.pcbi.1007125.g009 Case study 1: Biofilm deformation and detachment One of the outstanding questions in biofilm research is understanding how fluid-biofilm interactions affect the mechanical properties of biofilms. In this section, we describe how to use NUFEB to simulate a biofilm system with fluid dynamics by using three-way fluid and particle coupling. This is different from previous studies, where the coupling was from flow field to biofilm structure but not the other way [17], or assumes biofilm as a collection of 1D springs [45, 46]. To simulate a hydrodynamic biofilm, we apply fluid flow to a pre-grown biofilm that consists of heterotrophs and their EPS production. The biofilm is grown from 40 microbes inoculated on the substratum to a pre-determined height (80 μm) without flow and in an oxygen-limited condition (1 × 10−4 kg m−3). In this way, a mushroom-shaped biofilm structure can be developed to model liquid filled voids and channels (see Fig 4(a) and S1 Video). Then, we impose a fluid flow to the biofilm. During the fluid stage, any biological process is considered to be in the frozen state due to the small time scale of hydrodynamic calculations, and nutrient mass balance is omitted for the sake of simplicity. The motion of microbes is driven by both particle-particle and particle-fluid interactions, including EPS adhesion, contact force and drag force, as described in Eqs (4)–(6). Note that the EPS adhesive force also exists between HETs due to their EPS shells. The physical model parameters are kept constant throughout the simulation and can be found in the S1 File. For boundary conditions, we impose a fixed velocity Uf at the top surface with direction along the x-axis as inlet velocity, no-slip condition at the bottom surface and periodic conditions at other four surfaces. The pressure are enforced as zero gradient at the top and bottom surfaces. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Biofilm deformation and detachment at Uf = 0.2 m s−1. (a) Time = 0; (b) Time = 0.0015s; (c) Time = 0.003s; (d) Time = 0.01s. The model simulates 4 × 104 particles. Particles crossing the domain boundary will be removed from the system. Particle colours are blue for heterotrophs and grey for EPS. CPU time is 8 hours with dual processors, and initial particle number is 41210. https://doi.org/10.1371/journal.pcbi.1007125.g004 Fig 4(b)–4(d) and S2 Video. show the biofilm deformation and detachment at Uf = 0.2 m s−1 (Reynolds number = 20). The biofilm deforms and microbes detach along with the flow direction. In the early stage of the detachment process, the top of the biofilm is highly elongated and forms filamentous streamers. However, most of the microbes are still connected together with cohesion, and there is only a small number of clusters detached from the head of the streamers due to cohesive failures (Fig 4(b)). As the fluid continues to flow, large chunks of microbes detach from the biofilm surface. These detached microbe chunks may also break-up again, re-agglomerate with other clusters or re-attach to the biofilm surface (Fig 4(c)). Such deformation and detachment events observed from our NUFEB simulation show qualitative agreement with both experimental results [47] and other numerical simulations using different methods [17, 46]. The deformation reaches a pseudo-steady-state when the mushroom-shaped biofilm protrusions are removed from the system. As a result, the biofilm morphology changes dramatically from a rough to a flat surface (Fig 4(d)). During the deformation, the fluid, represented as red arrows, travels around the biofilm. Due to the irregular shape of the biofilm and the high fluid velocity, small vortexes can be observed at the biofilm surface on both the upstream and downstream sides. This phenomenon has been observed in previous studies [46]. For a more quantitative measurement of the deforming biofilm, we evaluated the area density and surface roughness of the biofilm at different fluid velocities. The biofilm surface roughness is calculated by [17]: (16) where h(x, y) is the biofilm height in the z direction at location (x, y) on the substratum, and is the average biofilm height: (17) As expected, when the fluid velocity increases the removed biomass also increases. For example, when Uf = 0.4 m s−1 is applied, the area density reaches steady-state after 0.004s, and the value decreases by 13% (Fig 5(a)). Note that the decrease does not take into account the change of basis. By contrast, the area density decreases less than 1% if Uf = 0.1 m s−1 is applied. The biofilm surface roughness shows a similar trend: the roughness decreases with increasing velocities (Fig 5(b)), indicating that biofilm morphology tends to be more flat in high-velocity fluid conditions as most of the mushroom protrusions can be removed. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Effect of emergent properties on biofilm detachment. (a) biomasss area density, and (b) biofilm surface roughness. https://doi.org/10.1371/journal.pcbi.1007125.g005 Case study 2: Biofilm growth with 107 particles In this case study, we first show the development of a large and complex biofilm system and then focus on the parallel efficiency of the simulations. The aim of this case study is to demonstrate the capability and performance of NUFEB in the simulation of larger biological systems. Biofilm development. The system is defined as a multi-functional group and multi-nutrient biofilm. In order to represent a more realistic biofilm, we explicitly consider nitrification as a two-step oxidation process that is performed by different groups of microbes: ammonia oxidizing bacteria (AOB) and nitrite oxidizing bacteria (NOB). In addition, the biofilm includes heterotrophs (HET) and their EPS production. The reaction model contains five soluble species, nutrients and products during microbial metabolism. The catabolic reactions include oxidation of ammonium NH4+ to nitrite NO2− by AOB, oxidation of nitrite to nitrate NO3− by NOB, and HET aerobic and anaerobic growth by consuming organic substrate in oxygenated conditions or nitrate in anoxic denitrifying conditions. The kinetics and reaction stoichiometry of the modelled processes and their corresponding parameters are detailed in the S1 File. The computational domain is divided into three regions. In the bulk region, nutrients are assumed to be completely mixed and their concentration is updated dynamically at each biological timestep, except for oxygen. We also assume that there is sufficient O2 and NH4+ but no NO2− and NO3− in the reactor influent. So the concentration of the two N compounds can only come from the transformation of NH4+. In the boundary layer region, a 20 μm distance from the maximum biofilm thickness to the bulk liquid is defined for solving the nutrient gradient. In the biofilm region, instead of using the super particle method [20], a small division diameter (1.3 μm) is chosen to represent real microbe sizes (on average 1 μm [48]). The simulation is run on an in-house HPC system at Newcastle University. In Fig 6 and S5 Video. we present the biofilm development over time. The system reaches 2.3 × 107 particles after 160 hours (CPU time = 30 hours). The initial particles are randomly placed on the substratum. In the early stages of biofilm formation, due to high growth rate and sufficient supply of substrate from bulk liquid, heterotrophs grow faster than nitrifiers (0, 60 and 120 hours). As the biomass grows, the system turns to substrate-limited condition for the HET group, while there is still sufficient NH4+ due to its high initial concentration that favours nitrifier growth. As a result, biofilm surface coverage of heterotrophs becomes smaller than nitrifiers (160 hours). This phenomenon matches previous experimental results where the nitrifying population can be significantly higher than heterotrophs in a substrate-limited reactor [49]. The biofilm geometry forms a wavy structure after 160 hours. This is because of a self-enhancing process from the non-uniform initial microbial distribution [50]. The spatial distribution of NO2− concentration is shown in Fig 7. It is clear that the NO2− concentration field differs according to the AOB distribution, where areas with high NO2− concentration are the locations where AOB clusters are present. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Biofilm development after 0, 60, 120, and 160 hours. The simulation uses 100 processors and 30 hours CPU time to reach 2.3 × 107 particles. The biological timestep is 0.25 hour. Particle colours are blue for heterotrophs, grey for EPS, light blue for AOB, and green for NOB. https://doi.org/10.1371/journal.pcbi.1007125.g006 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 7. Nitrite concentration field at a small part of the simulated domain after 60 hours. The spatial distribution of NO2− concentration follows the nitrifier distribution. The areas where NO2− accumulates are due to production by AOB. https://doi.org/10.1371/journal.pcbi.1007125.g007 Fig 8(a) shows a quantitative evaluation of the total biomass accumulation over time. The trend shows linear biomass increase which indicates that the total microbial growth rate is not yet balanced by the decay rate. Therefore, a biomass steady state is not achieved after 160 hours. This is due to the high-oxygen environment (1 × 10−2 kg m−3) and the thin biofilm which nutrients can penetrate. However, the concentration of substrate and NH4+ in the bulk liquid relax to steady state before the total biomass concentration relaxes (Fig 8(b)), as bulk concentrations are mainly determined by biomass in the top biofilm layers, which ensures a high growth rate [20]. The NO2− profile is influenced by both AOB synthesis and NOB consumption. Thus, the bulk concentration decreases with the increase of NOB populations. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 8. Quantitative evaluation of Case Study 2. (a) Total biomass of active functional groups over time, and (b) nutrients concentration in bulk liquid. Note that we assume the oxygen concentration in bulk liquid is kept constant by aeration. The slow growth of NOBs is due to the nutrient limitation, their low growth rate, and spatial distribution. At the early stage of the biofilm formation, the system is in NO2− limited condition. As the biofilm growth, the bottom area where NOBs reside turns to oxygen-limited condition. Both conditions inhibit NOBs growth. https://doi.org/10.1371/journal.pcbi.1007125.g008 Parallel performance. Parallel performance is crucial to IbM solvers for simulating large and complex problems. To investigate the performance of NUFEB, a scalability test was performed based on Case Study 2. The test aims at evaluating how simulation time varies with the number of processors for a fixed problem size. The speed-up of the scalability test is defined as tp0/tpn, where tp0 and tpn are the actual CPU times spent on the baseline case and the test case respectively. Then the efficiency is the ratio between the speed-up of the baseline and the test cases obtained when using a given number of processors, i.e., Np0tp0/Npntpn, where Np0 and Npn are the number of processors employed in these cases. In order to reach a considerable number of microbes (4 millions), the baseline case is performed using 4 processors, and the subsequent tests were performed with increasing numbers of processors, ranging from 8 to 256. As mentioned previously, NUFEB implements two distinct solutions for the parallelisation of continuum-based and particle-based processes. The performance of the two solutions are studied separately, and are presented in Fig 9. It can be observed that when employing a small number of processors, both solutions are close to the ideal speed-up (linear increase) and the parallel efficiency is over 90%. As more processor nodes are added, each processing node spends more time doing inter-processor communication than useful processing, and the parallelisation becomes less efficient. In the tests using 128 and 256 processors, the parallel efficiency decreases to around 42%, but it still shows a good scalability. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 9. Performance of Case Study 2 with 4–256 processors. The initial conditions and the model parameters are kept the same in all cases. The average achieved particle number of all cases is 4.77 × 106 with 0.5% standard deviation. The difference is due to the randomness in sub-domains. The simulation with 4 processors is regarded as baseline case for computing speed-up. Doubling the numbers of processors results in almost double speed-up when small processor numbers are employed (e.g., less than 32). However, the speed-up and parallel efficiency decrease with increasing numbers of processors due to inter-processor communication. https://doi.org/10.1371/journal.pcbi.1007125.g009 Biofilm development. The system is defined as a multi-functional group and multi-nutrient biofilm. In order to represent a more realistic biofilm, we explicitly consider nitrification as a two-step oxidation process that is performed by different groups of microbes: ammonia oxidizing bacteria (AOB) and nitrite oxidizing bacteria (NOB). In addition, the biofilm includes heterotrophs (HET) and their EPS production. The reaction model contains five soluble species, nutrients and products during microbial metabolism. The catabolic reactions include oxidation of ammonium NH4+ to nitrite NO2− by AOB, oxidation of nitrite to nitrate NO3− by NOB, and HET aerobic and anaerobic growth by consuming organic substrate in oxygenated conditions or nitrate in anoxic denitrifying conditions. The kinetics and reaction stoichiometry of the modelled processes and their corresponding parameters are detailed in the S1 File. The computational domain is divided into three regions. In the bulk region, nutrients are assumed to be completely mixed and their concentration is updated dynamically at each biological timestep, except for oxygen. We also assume that there is sufficient O2 and NH4+ but no NO2− and NO3− in the reactor influent. So the concentration of the two N compounds can only come from the transformation of NH4+. In the boundary layer region, a 20 μm distance from the maximum biofilm thickness to the bulk liquid is defined for solving the nutrient gradient. In the biofilm region, instead of using the super particle method [20], a small division diameter (1.3 μm) is chosen to represent real microbe sizes (on average 1 μm [48]). The simulation is run on an in-house HPC system at Newcastle University. In Fig 6 and S5 Video. we present the biofilm development over time. The system reaches 2.3 × 107 particles after 160 hours (CPU time = 30 hours). The initial particles are randomly placed on the substratum. In the early stages of biofilm formation, due to high growth rate and sufficient supply of substrate from bulk liquid, heterotrophs grow faster than nitrifiers (0, 60 and 120 hours). As the biomass grows, the system turns to substrate-limited condition for the HET group, while there is still sufficient NH4+ due to its high initial concentration that favours nitrifier growth. As a result, biofilm surface coverage of heterotrophs becomes smaller than nitrifiers (160 hours). This phenomenon matches previous experimental results where the nitrifying population can be significantly higher than heterotrophs in a substrate-limited reactor [49]. The biofilm geometry forms a wavy structure after 160 hours. This is because of a self-enhancing process from the non-uniform initial microbial distribution [50]. The spatial distribution of NO2− concentration is shown in Fig 7. It is clear that the NO2− concentration field differs according to the AOB distribution, where areas with high NO2− concentration are the locations where AOB clusters are present. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 6. Biofilm development after 0, 60, 120, and 160 hours. The simulation uses 100 processors and 30 hours CPU time to reach 2.3 × 107 particles. The biological timestep is 0.25 hour. Particle colours are blue for heterotrophs, grey for EPS, light blue for AOB, and green for NOB. https://doi.org/10.1371/journal.pcbi.1007125.g006 Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 7. Nitrite concentration field at a small part of the simulated domain after 60 hours. The spatial distribution of NO2− concentration follows the nitrifier distribution. The areas where NO2− accumulates are due to production by AOB. https://doi.org/10.1371/journal.pcbi.1007125.g007 Fig 8(a) shows a quantitative evaluation of the total biomass accumulation over time. The trend shows linear biomass increase which indicates that the total microbial growth rate is not yet balanced by the decay rate. Therefore, a biomass steady state is not achieved after 160 hours. This is due to the high-oxygen environment (1 × 10−2 kg m−3) and the thin biofilm which nutrients can penetrate. However, the concentration of substrate and NH4+ in the bulk liquid relax to steady state before the total biomass concentration relaxes (Fig 8(b)), as bulk concentrations are mainly determined by biomass in the top biofilm layers, which ensures a high growth rate [20]. The NO2− profile is influenced by both AOB synthesis and NOB consumption. Thus, the bulk concentration decreases with the increase of NOB populations. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 8. Quantitative evaluation of Case Study 2. (a) Total biomass of active functional groups over time, and (b) nutrients concentration in bulk liquid. Note that we assume the oxygen concentration in bulk liquid is kept constant by aeration. The slow growth of NOBs is due to the nutrient limitation, their low growth rate, and spatial distribution. At the early stage of the biofilm formation, the system is in NO2− limited condition. As the biofilm growth, the bottom area where NOBs reside turns to oxygen-limited condition. Both conditions inhibit NOBs growth. https://doi.org/10.1371/journal.pcbi.1007125.g008 Parallel performance. Parallel performance is crucial to IbM solvers for simulating large and complex problems. To investigate the performance of NUFEB, a scalability test was performed based on Case Study 2. The test aims at evaluating how simulation time varies with the number of processors for a fixed problem size. The speed-up of the scalability test is defined as tp0/tpn, where tp0 and tpn are the actual CPU times spent on the baseline case and the test case respectively. Then the efficiency is the ratio between the speed-up of the baseline and the test cases obtained when using a given number of processors, i.e., Np0tp0/Npntpn, where Np0 and Npn are the number of processors employed in these cases. In order to reach a considerable number of microbes (4 millions), the baseline case is performed using 4 processors, and the subsequent tests were performed with increasing numbers of processors, ranging from 8 to 256. As mentioned previously, NUFEB implements two distinct solutions for the parallelisation of continuum-based and particle-based processes. The performance of the two solutions are studied separately, and are presented in Fig 9. It can be observed that when employing a small number of processors, both solutions are close to the ideal speed-up (linear increase) and the parallel efficiency is over 90%. As more processor nodes are added, each processing node spends more time doing inter-processor communication than useful processing, and the parallelisation becomes less efficient. In the tests using 128 and 256 processors, the parallel efficiency decreases to around 42%, but it still shows a good scalability. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 9. Performance of Case Study 2 with 4–256 processors. The initial conditions and the model parameters are kept the same in all cases. The average achieved particle number of all cases is 4.77 × 106 with 0.5% standard deviation. The difference is due to the randomness in sub-domains. The simulation with 4 processors is regarded as baseline case for computing speed-up. Doubling the numbers of processors results in almost double speed-up when small processor numbers are employed (e.g., less than 32). However, the speed-up and parallel efficiency decrease with increasing numbers of processors due to inter-processor communication. https://doi.org/10.1371/journal.pcbi.1007125.g009 Availability and future directions In this paper, we have presented the NUFEB tool for modelling and simulating Individual-based Models. The tool, documentation, and examples are publicly available on the GitHub repository: https://github.com/nufeb/NUFEB. To date, NUFEB has been adopted to model microbial communities in a variety of studies. In [17], we studied the influence of nutrient gradients on biofilm structure formation, and the influence of shear flow on growing biofilm. Bacteria twitching motility under fluid flow was investigated in [51]. We also successfully applied the tool to study the influence of thermodynamics and pH on microbial growth [37]. In [42] and [52], we used micro-scale NUFEB simulations to emulate the behaviour of biofilm/floc in the macro-scale. An emulation strategy for parameter calibration of NUFEB against iDynoMiCS has been applied in [53]. The main advantage of the 3D microbial system model is to study effects of spatial behaviours of microbial systems. Our ongoing work includes modelling biofilm streamer oscillation, modelling nitritation anammox system, and coupling microbial growth with fluid condition. The goal for NUFEB development in the future would be to deliver an even more general, efficient and user-friendly platform. This will include, for example, the development of an intuitive Graphical User Interface which will significantly improve the user experience. In order to make NUFEB available for power-efficient HPC architecture, we will also focus on a Kokkos port for the NUFEB code. This would allow the code to run on different kinds of hardware, such as GPUs (Graphics Processing Units), Intel Xeon Phis, or many-core CPUs. The computational demands of IbM will always place a limit on the scale at which they can be applied. However, it is now evident that this limit can overcome by the use of statistical emulators [42]. A statistical emulator is a computationally efficient mimic of an IbM that can run thousands of times faster. In principle this new approach will allow the output of an IbM to be used at the metre scale and beyond and thus to make predictions about systems level performance in, for example, wastewater treatment or biofilm-fouling and drag on ships. This is a strategically important advance that creates a new impetus for the development of IbM that can credibly combine chemistry, mechanics, biology and hydrodynamics in a computationally efficient framework. NUFEB is, we believe, the first generation of IbM to meet that need and will help IbM transition from a research to application. Supporting information S1 File. Supporting information (SI). https://doi.org/10.1371/journal.pcbi.1007125.s001 (PDF) S1 Manual. User manual. https://doi.org/10.1371/journal.pcbi.1007125.s002 (PDF) S1 Chart. Chart data. https://doi.org/10.1371/journal.pcbi.1007125.s003 (XLSX) S1 Video. Growth of a biofilm without flow. The biofilm is initially grown for 9 days without flow. It forms a mushroom-shaped structure in an oxygen-limited condition (1 × 10−4 kg m−3). https://doi.org/10.1371/journal.pcbi.1007125.s004 (AVI) S2 Video. Biofilm removal at Uf = 0.2 m s−1. Large chunks of microbes detach from the biofilm surface and then remove from the systems. The biofilm morphology changes from a rough to a flat surface. https://doi.org/10.1371/journal.pcbi.1007125.s005 (MP4) S3 Video. Biofilm removal at Uf = 0.1 m s−1. The top of the biofilm is highly elongated. Small clusters erode from the deforming biofilm and the amount of biomass removed is less than the Uf = 0.2 m s−1 case. https://doi.org/10.1371/journal.pcbi.1007125.s006 (MP4) S4 Video. Biofilm deformation and detachment with periodic domain boundary style at Uf = 0.2 m s−1. Microbes crossing the boundary will re-appear on the opposite side of the domain. It can be observed that the detached clusters can re-attached to the biofilm surface or re-agglomerate with other clusters. https://doi.org/10.1371/journal.pcbi.1007125.s007 (MP4) S5 Video. Growth of a large biofilm system. The multiple functional groups biofilm is grown without flow and reaches 2.3 × 107 particles after 160 hours (CPU time = 30 hours). The biofilm forms a wavy structure because of the high nutrient concentration environment and the non-uniform initial microbial distribution. https://doi.org/10.1371/journal.pcbi.1007125.s008 (MP4) Acknowledgments The authors would like to thank Dr. Prashant Gupta and Dr. Curtis Madsen for the initial implementation of NUFEB. We also thank the NUFEB modelling team and Yuqing Xia for their helpful advice.